Learning Embodied Control Policies from Human Behavior
January 2026 — Present
Learning robotic control policies from human demonstrations using imitation learning and vision-language models.
Funded by Loyola University Chicago through the Provost Fellowship, this work explores how embodied agents can learn from human demonstrations. I investigate imitation learning, vision-language models, and large-scale human-activity datasets to help robots understand actions, recover task structure, and improve behavior through hindsight reflection.
A major part of the project is 3D object tracking. I developed a perception pipeline that combines SAM3 and SAM3D-Objects to track objects, reconstruct 3D meshes, and estimate six-degree-of-freedom kinematic states for robotic manipulation environments that require reliable localization and continuous state updates.
I also developed an end-to-end synthetic-data pipeline that produces adversarial robot datasets, along with a camera-calibration pipeline using PnP, RANSAC, and other modern extrinsics-estimation methods.
Highlights
- Built a perception pipeline for object tracking, 3D mesh reconstruction, and 6D kinematic-state estimation.
- Created synthetic robot datasets and calibrated experimental cameras for adversarial evaluation conditions.
Gallery
