Loading...

A Hierarchical Approach to Policy Generation in Reinforcement Learning Based on Utilizing Auxiliary Information Sources

Ghandi Jelvani, Ali | 2025

97 Viewed
  1. Type of Document: Ph.D. Dissertation
  2. Language: Farsi
  3. Document No: 58566 (05)
  4. University: Sharif University of Technology
  5. Department: Electrical Engineering
  6. Advisor(s): Bagheri Shouraki, Saeed; Gholampour, Iman
  7. Abstract:
  8. Despite its remarkable successes, Reinforcement Learning (RL) faces challenges of low sample efficiency and difficulty in generalizing knowledge to new tasks, especially when the underlying Markov Decision Processes (MDPs) differ. Inspired by the human ability to understand problems at various levels of abstraction, this dissertation presents a novel paradigm based on two-level learning to address this issue, where 'detail learning' is decoupled from 'concept learning'. To this end, the 'Experience-based Reinforcement Learning' (Ex-RL) framework and its advanced fuzzy version (Fuzzy Ex-RL) were developed. This framework utilizes an 'External Critic' that achieves a conceptual model of optimal strategies by abstracting previous successful experiences and uses this knowledge to intelligently guide a new agent. The results of comprehensive experiments in classic control environments demonstrated that this approach significantly accelerates learning, enables effective knowledge transfer between environments with entirely different dynamics, and reduces the need for random exploration. This research demonstrates that shifting from detail-based learning towards learning and transferring abstract concepts is an effective step toward the development of more general and adaptable intelligent agents
  9. Keywords:
  10. Reinforcement Learning ; Knowledge Transfer ; Hidden Markov Model ; Fuzzy Logic ; Transfer Learning ; Experience Abstraction ; Two Dimensional Learning

 Digital Object List

 Bookmark

...see more