Loading...

Design and Implementation of a Reinforcement Learning and Muscle Activity-Based Control System for a Lower-Limb Wearable Robot to Reduce the User’s Energy Consumption

Sahandi, Mohammad | 2025

32 Viewed
  1. Type of Document: M.Sc. Thesis
  2. Language: Farsi
  3. Document No: 58866 (08)
  4. University: Sharif University of Technology
  5. Department: Mechanical Engineering
  6. Advisor(s): Vossoughi, Gholamreza; Zohoor, Hassan
  7. Abstract:
  8. Wearable robots, particularly lower-limb exoskeletons, have gained significant attention in recent years as effective tools for rehabilitation, mobility assistance, and enhancement of physical performance. One of the key challenges in designing such robots is achieving natural and low-stress human-robot interaction, enabling users to perform smooth motions without perceiving mechanical resistance or discomfort. In this context, zero-force control has emerged as an effective strategy to minimize contact forces and enhance user comfort. The primary objective of this research is to develop a deep reinforcement learning (DRL)-based control method for a lower-limb wearable robot with two degrees of freedom, capable of robustly and stably tracking natural human walking motions while minimizing interaction forces. The designed controller must also adapt to diverse human user characteristics, such as varying heights and weights. Beyond theoretical goals, practical implementation of this controller on the laboratory’s physical robot prototype was a core priority of this study. To train the learning agent, a simulation environment was first developed in MuJoCo software, where a deep reinforcement learning agent was trained using the TD3 (Twin Delayed Deep Deterministic Policy Gradient) algorithm. The trained controller was then implemented and evaluated on the wearable robot in the Mechatronics Research Laboratory. Three control architectures were investigated: In the first approach, the DRL agent directly determined the reference velocity setpoints for the robot’s motors. In the second approach, the agent adaptively tuned the coefficients of a proportional-derivative (PD) controller online. In the third approach, the agent was responsible for online tuning of the coefficients of a lead compensator. Quantitative experimental results revealed that in the first method, the average interaction forces at the hip joint and knee joint were 6.13 N and 6.68 N, respectively. In the second method, these values decreased to 3.94 N and 4.40 N. In the third method, the average forces reached 1.99 N and 2.21 N. Furthermore, maximum interaction forces in prior laboratory studies were reported at approximately 45 N and 20 N (measured without the robot worn and under manual force application). In contrast, the two proposed methods achieved maximum forces of 18 N and 20 N during walking experiments with the robot, while the third method achieved a maximum force of only 9 N. Overall, the results demonstrated that reinforcement learning–based methods outperformed conventional control strategies. Crucially, integrating this approach with a proportional-derivative controller (second method) and a lead compensator (third method) yielded further improvements—not only in reducing interaction forces but also in maintaining motion stability—compared to using pure reinforcement learning alone
  9. Keywords:
  10. Lower Limb Exoskeleton ; Exoskeleton ; Deep Reinforcement Learning ; Zero-Forcing Sets ; Twin Delayed Deep Deterministic Policy Gradient (TD3)Algorithm ; Multi-Joint Dynamics with Contact (MuJoCo)Simulator

 Digital Object List

 Bookmark

No TOC