Project Overview

Leveraging Demonstration Learning and Invariant Representations for Modular Assembly and Cable Manipulation

This master’s thesis develops an autonomous robotic system able to manipulate rigid and deformable objects. The approach combines kinesthetic demonstration, invariant trajectory representations (DHB), and vision-based perception to insert Ethernet cables and interact with modular components under real-world constraints.

Key Features

Invariant Representation (DHB)

Encodes motion in a bidirectional Denavit–Hartenberg space, enabling robust reproduction across spatial configurations.

Vision-based Perception

Intel RealSense depth sensing and ArUco markers provide 6D pose estimation and 3D environment awareness.

Modular Environment

Board inspired by NIST benchmarks to evaluate task generalization under reconfigurable layouts.

Learning from Demonstration

Kinesthetic teaching for intuitive programming of cable manipulation skills.

System Architecture

The robotic platform integrates a Franka Emika Panda arm, Intel RealSense D435i camera, and ROS-based control system. A perception module estimates the 6D pose of target objects, while DHB encodes the trajectory in a space-invariant form. Execution is coordinated through behavior trees and ROS action nodes.

System block architecture
Experimental setup configuration

Scenarios

Three experimental scenarios were conducted:

  • Clip Insertion: High-precision cable insertion into modular clips using learned trajectories.
  • Full Modular Path: Partial success in executing complex sequences across multiple modules, limited by joint constraints and cable slack.
  • Board Reconfiguration: The system successfully adapted to new board layouts, confirming DHB invariance properties.

Results indicate the system performs well in structured environments and highlights the challenges of manipulating flexible elements such as cables in unstructured settings.

Clip insertion scenario Full modular path scenario

Experimental Results

Process

  • Recording demonstrations of task execution via kinesthetic teaching.
  • Filtering trajectory data to reduce noise and enhance stability.
  • Encoding demonstrations using the DHB invariant representation model.
  • Generalizing trajectories to support flexible adaptation across configurations.

Advantage

A single demonstration can be reused in various spatial setups, enabling robust task execution in different configurations thanks to the invariance properties of DHB.

Looking for Clip
Grabbing Clip
Inserting Cable
Closing Clip
Unlocking Clip

Technologies Used

Robotics

Franka Emika Panda arm for precise motion execution and kinesthetic teaching.

Vision

Intel RealSense D435i depth camera + ArUco markers for 6D pose estimation.

Software

ROS, Python, OpenCV, and custom DHB libraries for planning and control.

Modular Hardware

3D-printed boards and inserts designed for flexible, reconfigurable testing.

Conclusions & Future Work

The thesis shows that invariant representations can support complex tasks such as cable insertion and modular assembly. DHB facilitates generalization across configurations; future work targets robustness with deformable objects and occlusions.

  • Bimanual coordination for advanced wire routing.
  • Enhanced visual tracking and tactile feedback.
  • Real-time pose refinement and learning for novel parts.

Download Thesis PDF