Project Overview

Leveraging Demonstration Learning and Invariant Representations for Modular Assembly and Cable Manipulation

This master’s thesis develops a documented robotic research prototype for manipulating rigid and deformable objects. The approach combines kinesthetic demonstration, invariant trajectory representations (DHB), and vision-based perception to insert Ethernet cables and interact with modular components under real-world constraints. The public summary should be read as partially reproducible: it depends on local hardware, calibration, and recorded data.

Status: Completed

Key Features

Invariant Representation (DHB)

Encodes motion in a bidirectional Denavit–Hartenberg space, enabling robust reproduction across spatial configurations.

Vision-based Perception

Intel RealSense depth sensing and ArUco markers provide 6D pose estimation and 3D environment awareness.

Modular Environment

Board inspired by NIST benchmarks to evaluate task generalization under reconfigurable layouts.

Learning from Demonstration

Kinesthetic teaching for intuitive programming of cable manipulation skills.

System Architecture

The robotic platform integrates a Franka Emika Panda arm, Intel RealSense D435i camera, and ROS-based control system. A perception module estimates the 6D pose of target objects, while DHB encodes the trajectory in a space-invariant form. Execution is coordinated through behavior trees and ROS action nodes.

System block architecture
Experimental setup configuration

Scenarios

Three experimental scenarios were conducted:

  • Clip Insertion: High-precision cable insertion into modular clips using learned trajectories.
  • Full Modular Path: Partial success in executing complex sequences across multiple modules, limited by joint constraints and cable slack.
  • Board Reconfiguration: The system successfully adapted to new board layouts, confirming DHB invariance properties.

The public materials show representative scenarios for clip insertion, longer modular paths, and board reconfiguration. The repository itself remains a research prototype, so results, behavior, and replayability depend on local hardware, calibration, and recorded data.

Clip insertion scenario Full modular path scenario

Experimental Results

Process

  • Recording demonstrations of task execution via kinesthetic teaching.
  • Filtering trajectory data to reduce noise and enhance stability.
  • Encoding demonstrations using the DHB invariant representation model.
  • Generalizing trajectories to support flexible adaptation across configurations.

Advantage

A single demonstration can be reused in various spatial setups, enabling robust task execution in different configurations thanks to the invariance properties of DHB.

Looking for Clip
Grabbing Clip
Inserting Cable
Closing Clip
Unlocking Clip

Technologies Used

Robotics

Franka Emika Panda arm for precise motion execution and kinesthetic teaching.

Vision

Intel RealSense D435i depth camera + ArUco markers for 6D pose estimation.

Software

ROS, Python, OpenCV, and custom DHB libraries for planning and control.

Modular Hardware

3D-printed boards and inserts designed for flexible, reconfigurable testing.

Conclusions & Future Work

The thesis explores whether invariant representations can support tasks such as cable insertion and modular assembly. DHB supports trajectory generalization across configurations; future work would need stronger robustness for deformable objects, occlusions, and calibration changes.

  • Bimanual coordination for advanced wire routing.
  • Enhanced visual tracking and tactile feedback.
  • Real-time pose refinement and learning for novel parts.

Download Thesis PDF GitHub Repository