AI & ML interests

Force/torque data infrastructure for Physical AI and robotics foundation models. Specializing in contact-rich manipulation datasets, sim-to-real calibration, and tactile sensing for Large Behavior Models (LBMs). Building the data engine behind the next generation of humanoid robots.

Recent Activity

Organization Card

EXOKERN — Pre-trained Manipulation Skills for Physical AI

Contact-Foundation Model for Assembly · Force/Torque-validated · Open & Reproducible

We build pre-trained robotic manipulation skills for contact-rich tasks — where vision alone fails and precise force control matters. Every skill ships with full F/T ablation studies, multi-seed evaluation, and standardized benchmarks.

Vision: Building the first open Kontakt-Foundation Model for robotic assembly — multi-task, multi-robot, force-aware.


🎯 The Problem

Over 95% of robotic manipulation approaches are vision-only. They fail at contact-rich tasks like insertion, assembly, and snap-fit — where sub-Newton force control is the difference between success and broken parts.

Training a robot for a new contact task takes weeks of engineering. Validating it works safely takes even longer. And the resulting policy only works on one robot.

💡 What EXOKERN Provides

Skills — Pre-trained policies with force/torque awareness

Skill Version Key Features Status
skill-forge-peginsert-v0 v0 Diffusion Policy, F/T ablation, 3 seeds ✅ Live
skill-forge-peginsert-v0.1.1 v0.1.1 + Domain Randomization (6 DR parameters) 🔄 Training
skill-forge-peginsert-v0.2 v0.2 "See & Feel" + iDP3 Point Cloud + 9D Task-Space 📋 Planned

v0.2 Architecture (in development):

  • iDP3 Egocentric Point Cloud (1024×3) — no camera calibration needed
  • 9D Task-Space Actions (3 Pos + 6D Continuous Rotation) — robot-agnostic, no singularities
  • F/T in EE-Frame — consistent force feedback regardless of arm configuration
  • First system to combine: Point Cloud + F/T + Diffusion Policy

Datasets — Contact-rich training data with F/T annotations

Dataset Episodes Frames Domain Randomization Status
contactbench-forge-peginsert-v0 2,221 264K No ✅ Live
contactbench-forge-peginsert-v0.1 5,000 558K Yes (6 params) ✅ Live
contactbench-forge-peginsert-v0.1.1 5,000 745K Yes (production) ✅ Live

All datasets: LeRobot v3.0 format · 6-axis F/T wrench · Full state + action pairs · CC-BY-NC-4.0

Tools — Evaluate any manipulation policy

pip install exokern-eval
exokern-eval --policy your_checkpoint.pt --env Isaac-Forge-PegInsert-Direct-v0 --episodes 100

📊 Validated Results — Peg Insert v0

Evaluated across 3 random seeds × 2 conditions × 100 episodes = 600 rollouts total:

Metric With F/T (full_ft) Without F/T (no_ft) Improvement
Success Rate 100% 100%
Avg Contact Force 3.2 N 5.2 N −38%
Architecture-independent ✅ MLP, Temporal CNN, Diffusion Policy all show F/T benefit

Key Finding: F/T sensing consistently reduces contact forces across all architectures — critical for real-world deployment where excessive force = broken parts.


🧬 Skill Quality Pyramid

Every skill has a defined quality level — we don't pretend sim-only is production-ready:

Level 3: Real-World Validated     → v2 (2027)     → Production-ready
Level 2: Multi-Robot Sim          → v1 (Q4 2026)  → Cross-platform
Level 1: Sim-Validated + OOD      → v0.2 (Q2 2026) → Community
Level 0: Proof of Concept         → v0 (DONE ✅)  → Research baseline

🔧 What Makes EXOKERN Different

Feature EXOKERN Typical Open-Source
F/T ablation on every skill
Multi-seed evaluation (not cherry-picked) Rare
Open benchmark tool (exokern-eval)
LeRobot v3.0 compatible datasets Varies
Full data provenance (capture method, sensor specs, DR params) Rare
Contact-rich tasks (not just pick-and-place) Rare
v0.2: Point Cloud + F/T + Diffusion Policy 🔄 First to combine

🏗️ Built With

  • NVIDIA Isaac Lab — high-fidelity physics simulation with GPU parallelization
  • Diffusion Policy / iDP3 — 71.3M parameter architecture (v0.2: egocentric 3D)
  • Franka FR3 (7-DOF) — research-grade robot platform
  • LeRobot v3.0 — standardized dataset format for interoperability
  • Bota Systems SensONE — 6-axis F/T sensor (simulated with calibrated noise model)

🗺️ Roadmap

Building toward a Kontakt-Foundation Model for Assembly — multi-task, multi-robot, force-aware:

Version What's New Observation Actions Timeline
v0 Proof-of-Concept State + F/T Joint-Space (7D) ✅ Done
v0.1 Domain Randomization State + F/T Joint-Space (7D) 🔄 Training
v0.2 "See & Feel" — Point Cloud + F/T + Task-Space Point Cloud (1024×3) + F/T (EE-Frame) 9D EE Delta Pose Q2 2026
v0 Catalog 5 contact-rich tasks Point Cloud + F/T 9D Task-Space Q3 2026
v1 Multi-Robot (Franka + UR5e) Point Cloud + F/T 9D → per-robot IK Q4 2026
v2 Real-World validated Point Cloud + F/T (calibrated) 9D + Impedance 2027

📜 License

  • Code: Apache-2.0
  • Model Weights: CC-BY-NC-4.0 (commercial use requires separate license)
  • Datasets: CC-BY-NC-4.0

📬 Contact


EXOKERN — Bridging the Haptic Gap in Robotic Manipulation