Loading...

Decentralized MARL in Road Traffic Congestion in the Presence of Non-Stationarity

Hokmi, Sadredin | 2024

233 Viewed
  1. Type of Document: M.Sc. Thesis
  2. Language: Farsi
  3. Document No: 58160 (05)
  4. University: Sharif University of Technology
  5. Department: Electrical Engineering
  6. Advisor(s): Haeri, Mohammad
  7. Abstract:
  8. In this project, the primary objective is to propose a method to increase the convergence rate and accelerate the time it takes for agents to reach their destination in a traffic network. This topic serves as an intersection between two main areas: reinforcement learning and game theory. In this context, agents cooperate to achieve a common goal while remaining unaware of each other's strategies. Therefore, a decentralized reinforcement learning algorithm is proposed and implemented in the presence of non-stationarity. The proposed algorithm is based on introducing temporary and controlled deviations in the regular reinforcement learning mechanism, specifically the Q-learning algorithm used in this project. Broadly speaking, the algorithm can be categorized among known approaches that enhance convergence rates through mechanisms such as adjustment terms, modifying update frequencies, noise injection, or changing learning rate coefficients. However, while the proposed algorithm shares similarities with these approaches, it does not fully align with any of them. For instance, it does not involve statistical characteristics or uncertainty, the added terms are not functions of the algorithm's components, and they are temporary. These deviations are controlled and cease to operate once convergence is achieved. Finally, the proposed algorithm has been applied to a traffic network and subsequently validated using two additional case studies. The results, along with comparisons to other existing methods and algorithms, demonstrate that the proposed method maintains its efficiency even under changing agent conditions (e.g., increasing or decreasing the number of agents), dynamic environments, and non-stationary strategies.
  9. Keywords:
  10. Game Theory ; Nonstationarity ; Convergence Rate ; Traffic Network ; Decentralized Multi-Agent Reinforcement Leaning