Project Overview

Leveraging Demonstration Learning and Invariant Representations for Modular Assembly and Cable Manipulation

This master's thesis focuses on developing an autonomous robotic system capable of performing precise manipulation of rigid and deformable objects. The system utilizes kinesthetic demonstration, invariant trajectory representations, and perception-driven planning to insert Ethernet cables and interact with modular components under real-world constraints.

Key Features

  • Invariant Representation (DHB): Encodes task-relevant motion in a bidirectional Denavit-Hartenberg space for consistent reproduction in different spatial conditions.
  • Vision-Based Perception: Uses ArUco markers and Intel RealSense depth sensing for object pose detection and 3D environment awareness.
  • Modular Environment: Flexible and reconfigurable board inspired by NIST benchmarks enables evaluation of task generalization.
  • Learning from Demonstration: Kinesthetic demonstrations allow intuitive programming of cable manipulation skills.

System Architecture

The robotic platform integrates a Franka Emika Panda arm, Intel RealSense D435i camera, and ROS-based control system. A perception module estimates the 6D pose of target objects, while DHB encodes the trajectory in a space-invariant form. Execution is coordinated through behavior trees and ROS action nodes.

System Architecture Setup

Scenarios

Three experimental scenarios were conducted:

  • Clip Insertion: High-precision cable insertion into modular clips using learned trajectories.
  • Full Modular Path: Partial success in executing complex sequences across multiple modules, limited by joint constraints and cable slack.
  • Board Reconfiguration: The system successfully adapted to new board layouts, confirming DHB invariance properties.

Results indicate the system performs well in structured environments and highlights the challenges of manipulating flexible elements such as cables in unstructured settings.

Clip Insertion Full Path

Experimental Results

Process

  • Recording demonstrations of task execution via kinesthetic teaching.
  • Filtering trajectory data to reduce noise and enhance stability.
  • Encoding demonstrations using the DHB invariant representation model.
  • Generalizing trajectories to support flexible adaptation across configurations.

Advantage

A single demonstration can be reused in various spatial setups, enabling robust task execution in different configurations thanks to the invariance properties of DHB.

Looking for Clip
Grabbing Clip
Inserting Cable
Closing Clip
Unlocking Clip

Technologies Used

  • Robotics

    Franka Emika Panda robotic arm for precise motion execution and kinesthetic teaching.

  • Vision

    Intel RealSense D435i depth camera and ArUco markers for 6D pose estimation.

  • Software

    ROS, Python, OpenCV, and custom Python DHB libraries for planning and control.

  • Modular Hardware

    3D-printed task boards and inserts designed for flexible and reconfigurable testing.

Conclusions and Future Work

This thesis demonstrates the feasibility of applying invariant representations to complex robotic tasks such as cable insertion. The DHB-based approach facilitates generalization across configurations, but future improvements are needed for robustness under deformability and occlusion.

  • Explore bimanual coordination for complex wire routing.
  • Enhance visual tracking and tactile feedback for tighter control in cluttered environments.
  • Integrate real-time pose refinement and object learning for novel parts.
Download Thesis PDF