Loading...
A reinforcement learning approach to find optimal propulsion strategy for microrobots swimming at low reynolds number
179 viewed

A reinforcement learning approach to find optimal propulsion strategy for microrobots swimming at low reynolds number

Jebellat, I

A reinforcement learning approach to find optimal propulsion strategy for microrobots swimming at low reynolds number

Jebellat, I ; Sharif University of Technology | 2024

179 Viewed
  1. Type of Document: Article
  2. DOI: 10.1016/j.robot.2024.104659
  3. Publisher: 2024
  4. Abstract:
  5. The development of artificial microscopic robots, like synthetic microswimmers, is one of the state-of-the-art research topics due to their promising biomedical applications. The movement of microswimmers is affected by stringent constraints because of the low Reynolds number of the surrounding environment. Researchers have been working on enhancing microrobots’ propulsion tactics and figuring out new approaches to find optimal propulsion strategies. In this research, we employ a Reinforcement Learning (RL) algorithm, specifically Q-Learning, to train linear-shaped microrobots, comprised of spheres and rods, by introducing an innovative and pioneer coding approach, termed “Basic Coding”, which is utilized to specify states and actions within the RL framework. Basic Coding is a powerful general method that can be employed for different agents in any discrete RL environment. We show how to apply Basic Coding to various microrobots with different geometrical configurations, like a triangular one. Our smart microswimmers, with different numbers of spheres, acquire the knowledge of the optimal propulsion cycle to accomplish large net mechanical displacement without relying on any pre-existing locomotion expertise. The three-sphere linear microrobot recovers the cycle Najafi and Golestanian suggested. The N-sphere microrobots with higher degrees of freedom can find the optimal propulsion cycle within a reasonable number of learning steps and low computational cost utilizing our RL and Basic Coding approach, while the learning step number significantly increases using other methods like Brute-force search. For example, we show this number for the 5-sphere microrobot is 1.19E03 and 8.97E10 steps using our methodology and Brute-force, respectively. Furthermore, our intelligent microrobots can successfully and adaptively find new optimal strategies in indeterministic environments in the presence of uncertainty. Moreover, the effects of learning parameters on our RL agents are investigated in this work. © 2024
  6. Keywords:
  7. Machine learning ; States and actions ; Data communication equipment ; Degrees of freedom (mechanics) ; Learning systems ; Medical applications ; Reynolds number ; Robots ; Spheres ; Basic coding ; Low Reynolds number ; Machine-learning ; Micro robots ; Micro-swimmer ; Propulsion strategy ; Reinforcement learning approach ; Reinforcement learnings ; Smart microswimmer ; State and action ; Reinforcement learning
  8. Source: Robotics and Autonomous Systems ; Volume 175 , 2024 ; 09218890 (ISSN)
  9. URL: https://www.sciencedirect.com/science/article/abs/pii/S0921889024000423