Projects

Today's self-driving systems excel at detecting objects but still struggle with ambiguous or unusual situations — a person behaving unpredictably, a scene that doesn't match training data. I built a system that inserts a Vision-Language Model as a semantic "observer" between perception and planning in a standard autopilot stack (Autoware + CARLA). The VLM continuously checks whether what the vehicle sees makes sense and triggers safe fallback behaviours when it doesn't. By quantizing the model, I demonstrated that VLMs can operate at a consistent rate alongside real-time control loops, achieving strong hazard precision — a promising step toward VLMs as practical semantic safety layers.

How do you teach a robot to walk in a way that's not just functional but physically meaningful? I trained the Unitree Go2 in Isaac Lab using PPO with a torque-level PD control setup, but the key contribution is the reward function. Rather than only rewarding forward velocity, I added terms for foot clearance, contact timing, symmetry across legs and gait phases, smooth actions, and postural stability. The goal: locomotion that is energy-conscious, balanced, and actually transferable to real hardware — bridging "the robot moves" and "the robot moves like it should."

Multiple spacecraft coordinating to contain space debris — without a central planner. I developed a velocity-based controller inspired by cyclic pursuit, where each satellite adjusts its trajectory relative to its neighbours to form a stable ring around a target. The system operates under full orbital mechanics, modelled via Clohessy–Wiltshire equations, and was validated across three physics frameworks (Kepler, Newton–Euler, MuJoCo). Looking forward, this connects to differential drag as a passive, fuel-efficient alternative to thruster-based formation control.

3 drones and 3 Clearpath Jackal UGVs moving as a coordinated group, with no central controller. Each agent follows local flocking rules — separation, cohesion, alignment — using only its own sensing. On top of this, a Graph Neural Network layer handles dynamic goal assignment, deciding which agent goes where based on the current swarm configuration. Classical control handles safety and stability; the GNN handles higher-level coordination. The result is a scalable multi-robot architecture for complex, mixed-terrain environments.

An open playground for humanoid robot learning — starting with whole-body controllers (gravity compensation, PPO-trained gaits) and progressively adding motion planners on top. The goal is a modular sandbox where grad students can drop in new controllers and see what happens, without reinventing the simulation infrastructure. Currently: gravity comp working, shaky PPO running. Next: cleaner RL-based gaits and simple motion planners.

Modelling a high-agility hexcopter in MuJoCo, with a focus on getting the dynamics right before training any control policy. A recurring challenge: MuJoCo's tree-based kinematic structure doesn't support closed kinematic loops natively, requiring careful use of equality constraints to simulate real mechanical linkages. Working through this constraint-by-constraint, with plans to evaluate Genesis as an alternative once the model is physically consistent.

Modern autopilots fail silently on semantic anomalies — cascade failures across tightly coupled perception, planning, and control modules make real-time debugging nearly impossible. This paper deploys Cosmos-Reason1-7B as a closed-loop semantic observer layer on top of Autoware running in CARLA, processing four time-synchronized RGB camera views jointly at 1–2 Hz. The observer classifies scenes into MRM tiers (Emergency Stop, Comfortable Stop, Pull Over) and routes decisions to Autoware's deterministic MRM handler — never directly commanding the controller, preserving ISO 26262-aligned safety guarantees. FP16 + FlashAttention 2.0 sustains sub-500 ms inference across all four views. The Zenoh pub/sub bridge offloads inference to a remote GPU instance, sidestepping VRAM contention with Autoware's TensorRT models. A custom Unreal Engine 4.26 digital twin of NYU Brooklyn Commons serves as the test environment. CARLA Leaderboard results: baseline Autoware records 7 collisions and 40% route completion; the augmented stack records 0 collisions and 100% route completion on Run 1.