Kai Lu

I am a Ph.D. Candidate (2020 - 2025, now in the process of completion) in Computer Science at the University of Oxford, supervised by Prof. Andrew Markham. Prior to this, I received my B.E. degree with honor in Automation from Tsinghua University in 2020, while I was a student research assistant at the Department of Computer Science from 2017 to 2019, advised by Prof. Huaping Liu. I was a visiting student at Duke and UIUC in 2019, advised by Prof. Kris Hauser.

My research focuses on Embodied AI, with two main aspects: first, generalizable/ transferable/ dynamic robot skill learning (thesis subject); and second, LLM/ VLM/ VLA for multi-modal learning (latest work).

Research Interests:
     ⋅   Robotics - Robotic Manipulation, Mobile Manipulators, Human-Robot Interaction
     ⋅   Machine Learning - Deep Learning, Reinforcement Learning
     ⋅   Multi-Modality - Robotic 3D Vision, Tactile Sensing, VLM/LLM for Embodied Agents

Collaboration / Research Intern:
     ⋅   Mitsubishi Electric Research Labs (MERL), Meta Research (FAIR, Habitat Team)
     ⋅   Oxford Robotics Institute (ORI), vLAR at Hong Kong PolyU (vLAR)

Sevice:
     ⋅    Associate Editor (AE): International Journal of Advanced Robotic Systems (IJARS)
     ⋅    Reviewer: ICRA, ICLR, NeurIPS


Formal Bio     Resume     Github     Google Scholar     LinkedIn    

Email: kai.lu AT cs.ox.ac.uk / lukaiwork01 AT gmail.com

KitchenVLA: Iterative Vision-Language Corrections for Robotic Execution of Human Tasks

Kai Lu, Chenyang Ma, Chiori Hori, Diego Romeres
Patent (in submission) & IEEE International Conference on Robotics and Automation (ICRA) 2025 SafeVLM

Learning Generalizable Manipulation Policy with Adapter-Based Parameter Fine-Tuning

Kai Lu, Kim Tien Ly, William Hebberd, Kaichen Zhou, Ioannis Havoutis, Andrew Markham
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2024
Webpage   •   PDF

SpatialPIN: Enhancing Spatial Reasoning Capabilities of Vision-Language Models through Prompting and Interacting 3D Priors

Chenyang Ma, Kai Lu, Ta-Ying Cheng, Niki Trigoni, Andrew Markham
Conference on Neural Information Processing Systems (NeurIPS) 2024
Webpage   •   PDF

InteLiPlan: Interactive Lightweight LLM-Based Planner for Domestic Robot Autonomy

Kim Tien Ly, Kai Lu, Ioannis Havoutis
Technical Report 2024
Webpage   •   PDF

Learning to Catch Reactive Objects with a Behavior Predictor

Kai Lu, Jia-Xing Zhong, Bo Yang, Bing Wang, Andrew Markham
IEEE International Conference on Robotics and Automation (ICRA) 2024
Webpage   •   PDF   •   Code

Decoupling Skill Learning from Robotic Control for Generalizable Object Manipulation

Kai Lu, Bo Yang, Bing Wang, Andrew Markham
IEEE International Conference on Robotics and Automation (ICRA) 2023
Webpage   •   PDF   •   Code

Weakly Supervised Descriptor Learning for Pixel-Level Feature Matching

Kai Lu, Andrew Markham
Term Report 2021

Semi-Empirical Simulation of Learned Force Response Models for Heterogeneous Elastic Objects

Yifan Zhu, Kai Lu, Kris Hauser
IEEE International Conference on Robotics and Automation (ICRA) 2020
Webpage   •   PDF   •   Code

Deep Reinforcement Learning for Robotic Pushing and Picking in Cluttered Environment

Yuhong Deng*, Xiangfeng Guo*, Yixuan Wei*, Kai Lu*, Bin Fang, Di Guo, Huaping Liu, Fuchun Sun
*: co-first author, equal contribution
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2019
Webpage   •   PDF   •   Code

RoboCup 2019 Humanoid League Contest

Visual Localization Group, Tsinghua Team
RoboCup 2019
Webpage   •   PDF   •   Code