| Citation: | ZHANG Lingjun, TANG Liang, LIU Lei. Target Position-guided In-hand Re-orientation for Five-fingered Dexterous Hands[J]. ROBOT, 2025, 47(1): 10-21. DOI: 10.13973/j.cnki.robot.240019 |
Target Position-guided In-hand Re-orientation for Five-fingered Dexterous Hands
-
Abstract
Re-orientation involves rotating an object to a target configuration, with the most challenging case being the rotation from an arbitrary initial configuration to an arbitrary target configuration. To address the challenge of efficiently performing in-hand re-orientation tasks in a more human-like manner by guiding anthropomorphic five-fingered dexterous hands with different degrees of actuation (DoA), a target position-guided in-hand object re-orientation policy generation method is proposed. Firstly, a feasible principle for designing target positions is proposed, inspired by the operation characteristics of human hands during in-hand re-orientation and based on the distribution characteristics of DoA in anthropomorphic five-fingered dexterous hands. The difference between the actual and target positions of the object during re-orientation process is utilized as a component of the immediate reward to guide anthropomorphic five-fingered dexterous hands in maintaining the object near the target. Secondly, a method is developed inspired by the preparatory states of human hands before performing re-orientation tasks, to sample the joint positions of anthropomorphic five-fingered dexterous hands when resetting the state everytime, aiming to enhance manipulation capabilities. Finally, the re-orientation policy is trained using the proximal policy optimization (PPO) algorithm based on the long short-term memory (LSTM) network and asymmetric actor-critic architecture. Simulation results show that the proposed method enables the 9-DoA Schunk SVH dexterous hand, the 13-DoA BICE dexterous hand developed by Beijing Institute of Control Engineering (BICE), and the 18-DoA Shadow dexterous hand to approach the predefined maximum number of consecutive successes when performing re-orientation tasks. Moreover, compared with in-hand object re-orientation policy generation method without target position guidance, the proposed method significantly reduces the average number of steps required to perform re-orientation tasks. The proposed method enables anthropomorphic five-fingered dexterous hands with different DoA to efficiently perform object re-orientation tasks in a human-like manner through coordinated action of the palm and fingers, significantly enhancing operational efficiency.Keywords:
-
References
[1] BICCHI A. Hands for dexterous manipulation and robust grasping: A difficult road toward simplicity[J]. IEEE Transactions on Robotics and Automation, 2000, 16(6): 652-662. doi: 10.1109/70.897777[2] KUMAR V, TODOROV E, LEVINE S. Optimal control with learned local models: Application to dexterous manipulation[C]// IEEE International Conference on Robotics and Automation. Piscataway, USA: IEEE, 2016: 378-383. doi: 10.1109/ICRA.2016.7487156[3] RAJESWARAN A, KUMAR V, GUPTA A, et al. Learning complex dexterous manipulation with deep reinforcement learning and demonstrations[C]//Robotics: Science and Systems XIV. 2018. doi: 10.15607/RSS.2018.XIV.049[4] NAGABANDI A, KONOLIGE K, LEVINE S, et al. Deep dynamics models for learning dexterous manipulation[C]//Proceedings of the Conference on Robot Learning. 2020: 1101-1112. https://proceedings.mlr.press/v100/nagabandi20a.html[5] ANDRYCHOWICZ M, BAKER B, CHOCIEJ M, et al. Learning dexterous in-hand manipulation[J]. International Journal of Robotics Research, 2020, 39(1): 3-20. doi: 10.1177/0278364919887447[6] CHEN T, XU J, AGRAWAL P. A system for general in-hand object re-orientation[C]//Proceedings of the 5th Conference on Robot Learning. 2022: 297-307. https://proceedings.mlr.press/v164/chen22a.html[7] CHEN T, TIPPUR M, WU S, et al. Visual dexterity: In-hand re-orientation of novel and complex object shapes[J]. Science Robotics, 2023, 8(84). doi: 10.1126/scirobotics.adc9244[8] HUANG W, MORDATCH I, ABBEEL P, et al. Generalization in dexterous manipulation via geometry-aware multi-task learning[DB/OL]. (2021-11-04) [2024-01-02]. doi: 10.48550/arXiv.2111.03062[9] PETRENKO A, ALLSHIRE A, STATE G, et al. DexPBT: Scaling up dexterous manipulation for hand-arm systems with population based training[C]//Robotics: Science and Systems XIX. 2023. doi: 10.15607/RSS.2023.XIX.037[10] MA Y J, LIANG W, WANG G, et al. Eureka: Human-level reward design via coding large language models][DB/OL]. (2023-10-06) [2024-03-28]. https://arxiv.org/abs/2310.12931[11] KHANDATE G, HAAS-HEGER M, CIOCARLIE M. On the feasibility of learning finger-gaiting in-hand manipulation with intrinsic sensing[C]//International Conference on Robotics and Automation. Piscataway, USA: IEEE, 2022: 2752-2758. doi: 10.1109/ICRA46639.2022.9812212[12] KHANDATE G, SHANG S, CHANG E T, et al. Sampling-based exploration for reinforcement learning of dexterous manipulation[DB/OL]. (2023-05-23) [2024-01-20]. https://arxiv.org/abs/2303.03486[13] XU Y Z, WAN W K, ZHANG J L, et al. UniDexGrasp: Universal robotic dexterous grasping via learning diverse proposal generation and goal-conditioned policy[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, USA: IEEE, 2023: 4737-4746.[14] Shadow Robot Company. Shadow robot dexterous hand] [EB /OL]. (2011-07-25) [2023-12-18]. https://www.shadowrobot.com/dexterous-hand-series/[15] Schunk SE & Co. KG. SVH 5-finger hand][EB/OL]. (2022-12-01) [2023-12-19]. http://wiki.ros.org/schunk_svh_driver[16] SCHULMAN J, WOLSKI F, DHARIWAL P, et al. Proximal policy optimization algorithms[DB/OL]. (2017-08-28) [2023-12-19]. doi: 10.48550/arXiv.1707.06347[17] HOCHREITER S, SCHMIDHUBER J. Long short-term memory][J]. Neural Computation, 1997, 9(8): 1735-1780. doi: 10.1162/neco.1997.9.8.1735[18] MAKOVIYCHUK V, WAWRZYNIAK L, GUO Y, et al. Isaac Gym: High performance GPU-based physics simulation for robot learning[DB/OL]. (2021-08-24)[2023-12-19]. https://arxiv.org/abs/2108.10470[19] NAIR V, HINTON G E. Rectified linear units improve restricted Boltzmann machines][C]//International Conference on Machine Learning. 2010: 807-814. https://api.semanticscholar.org/CorpusID: 15539264 -
Related Articles