Robbyant Open Sources LingBot-Touch Model to Enhance Robotic Tactile Perception and Dexterity

Robbyant, as a company under Ant Group specializing in embodied artificial intelligence, has open-sourced LingBot-Touch, a tactile foundation model aimed at enhancing the tactile perception capabilities of robots. The model has been pre-trained on nearly 5,000 hours of real tactile data and supports three sensory modalities: magnetic touch, visual touch, and piezoresistive sensing. It converts signals from different sensors into a unified tactile representation, enabling various robotic hands and action models to share and reuse tactile-related information. The company states that this capability could improve fine manipulation tasks, including turning pages, opening bottle caps, and identifying Mahjong tiles.

LingBot-Touch is an open-source tactile foundation model designed to help robots interpret physical contact from different types of sensors. The model addresses a major challenge in robotic operations, which is enabling robots to retain and reuse tactile perception capabilities when switching sensors, dexterous hands, and control models. Existing commercial tactile sensors utilize different technologies, including magnetic touch, visual touch, and piezoresistive sensing. Each method has its own advantages, disadvantages, costs, and application scenarios, making it unlikely for a single sensory technology to dominate the market. However, these different sensing principles can produce incompatible raw signals even when detecting the same physical contact.

Models trained on a single sensor type may encounter difficulties in transferring knowledge to another type, requiring developers to collect new datasets, retrain models, and adjust systems each time there is a hardware change.

Overcoming Sensor Incompatibility Issues

LingBot-Touch attempts to overcome this limitation by converting heterogeneous tactile signals into a shared representation. Its architecture employs independent encoding channels to process data from different sensing technologies, then maps contact points from various sensor layouts into a common fingertip coordinate space. The shared model subsequently learns to interpret contact locations, force distributions, and how these properties change over time. The resulting unified tactile labels allow different robotic hands and action models to access and reuse the same learned tactile information. The architecture also enables the integration of human data collected from wearing tactile gloves and robotic hand data from remote operations during the training process, assisting developers in transferring tactile knowledge across hardware platforms without needing to establish independent perceptual systems for each sensor configuration.

The company states that this architecture can also accommodate emerging tactile sensing technologies.

Real-World Operational Tests Show Effectiveness

Robbyant evaluated the model through 26 real-world operational tests involving two dexterous robotic hands, Sharpa Wave and XHand. Four action models—LingBot-VLA, π0.5, Diffusion Policy, and TRex—demonstrated an average increase in task success rates of approximately 14 to 26 percentage points when using LingBot-Touch. One test involved using the T-Rex action model to control Sharpa Wave to open a bottle cap.

Without tactile input, the robot's success rate was 56.25%. When the raw sensor signals were directly input into the action model, the success rate increased to 62.5%. When LingBot-Touch processed the tactile information before passing it to the policy, the success rate reached 90%. The team also tested whether the model could transfer its capabilities to different hardware. The magnetic tactile sensors used by XHand were not included in LingBot-Touch's pre-training data but learned magnetic tactile information through human-collected glove data.

Robbyant kept the parameters of the base model unchanged and only used XHand demonstration data to train the action model, assessing the reuse across hardware.

The company offers two model sizes to balance perceptual capabilities and computational demands. All 26 real-world tests utilized LingBot-Touch-Tiny, which contains approximately 5.3 million parameters. When integrated with LingBot-VLA, π0.5, and T-Rex, the total model size increased by only about 0.13 to 0.16%. Robbyant states that with its pre-training dataset nearing 5,000 hours, the larger model also shows potential for further improvements.

Technical reports and source code are available through Hugging Face and GitHub.