Kai Lu

I am a Ph.D. Candidate (2020 - 2024, now in the process of completion) in Computer Science at the University of Oxford, supervised by Prof. Andrew Markham. Prior to this, I received my B.E. degree with honor in Automation from Tsinghua University in 2020, while I was a student research assistant at the Department of Computer Science from 2017 to 2019, advised by Prof. Huaping Liu. I was a visiting student at Duke and UIUC in 2019, advised by Prof. Kris Hauser.

My research focuses on Embodied AI, with two main aspects: first, generalizable/ transferable/ dynamic robot skill learning (thesis subject); and second, LLM/ VLM/ Agent AI for multi-modal task discovery (latest work).

Research Interests:
     ⋅   Robotics - Robotic Manipulation, Mobile Manipulators, Human-Robot Interaction
     ⋅   Machine Learning - Deep Learning, Reinforcement Learning
     ⋅   Multi-Modality - Robotic 3D Vision, Tactile Sensing, VLM/LLM for Embodied Agents

Collaboration / Research Intern:
     ⋅   Mitsubishi Electric Research Labs (MERL), Meta Research (FAIR, Habitat Team)
     ⋅   Oxford Robotics Institute (ORI), vLAR at Hong Kong PolyU (vLAR)

Sevice:
     ⋅    Associate Editor (AE): International Journal of Advanced Robotic Systems (IJARS)
     ⋅    Reviewer: ICRA, ICLR, NeurIPS


Formal Bio     Resume     Github     Google Scholar     LinkedIn    

Email: kai.lu AT cs.ox.ac.uk / lukaiwork01 AT gmail.com

Learning Generalizable Manipulation Policy with Adapter-Based Parameter Fine-Tuning

Kai Lu, Kim Tien Ly, William Hebberd, Kaichen Zhou, Ioannis Havoutis, Andrew Markham
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2024
Webpage   •   PDF

SpatialPIN: Enhancing Spatial Reasoning Capabilities of Vision-Language Models through Prompting and Interacting 3D Priors

Chenyang Ma, Kai Lu, Ta-Ying Cheng, Niki Trigoni, Andrew Markham
Conference on Neural Information Processing Systems (NeurIPS) 2024
Webpage   •   PDF

InteLiPlan: Interactive Lightweight LLM-Based Planner for Domestic Robot Autonomy

Kim Tien Ly, Kai Lu, Ioannis Havoutis
Technical Report 2024
Webpage   •   PDF

Learning to Catch Reactive Objects with a Behavior Predictor

Kai Lu, Jia-Xing Zhong, Bo Yang, Bing Wang, Andrew Markham
IEEE International Conference on Robotics and Automation (ICRA) 2024
Webpage   •   PDF   •   Code

Decoupling Skill Learning from Robotic Control for Generalizable Object Manipulation

Kai Lu, Bo Yang, Bing Wang, Andrew Markham
IEEE International Conference on Robotics and Automation (ICRA) 2023
Webpage   •   PDF   •   Code

Weakly Supervised Descriptor Learning for Pixel-Level Feature Matching

Kai Lu, Andrew Markham
Term Report 2021

Semi-Empirical Simulation of Learned Force Response Models for Heterogeneous Elastic Objects

Yifan Zhu, Kai Lu, Kris Hauser
IEEE International Conference on Robotics and Automation (ICRA) 2020
Webpage   •   PDF   •   Code

Deep Reinforcement Learning for Robotic Pushing and Picking in Cluttered Environment

Yuhong Deng*, Xiangfeng Guo*, Yixuan Wei*, Kai Lu*, Bin Fang, Di Guo, Huaping Liu, Fuchun Sun
*: co-first author, equal contribution
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2019
Webpage   •   PDF   •   Code

RoboCup 2019 Humanoid League Contest

Visual Localization Group, Tsinghua Team
RoboCup 2019
Webpage   •   PDF   •   Code

2020

Outstanding Graduate Honor, Department of Automation, Tsinghua University

2019

Technological Innovation Scholarship, Department of Automation,Tsinghua University

2019

Second Place, Adult-Size Technical Challenge, RoboCup 2019 Humanoid World Final

2019

First Prize (Top 1% of all students), The 37th Tsinghua Challenge Cup Competition

2018

Champion, Robotic Innovation Contest, The 20th Chinese Robotics and Artificial Intelligence Competition

2016

Tsinghua Leading Talent Program (Top 1% of all students), Tsinghua University

2015

First Prize (Top 10 in Province), Chinese Physics Olympiad (CPhO)

2014

First Prize (Top 10 in Province), Chinese Mathematical Olympiad (CMO)

2014

Bronze Medal, China Western Mathematical Olympiad (CWMO)

2024

Associate Editor, International Journal of Advanced Robotic Systems (IJARS)

2024

Reviewer, ICRA 2025, ICLR 2025

2023

Reviewer, NeurIPS 2023, ICLR 2024, ICRA 2024

2023

Student Member, IEEE

2021+

Board Member, UK Tsinghua Alumni Association (UKTA)

2021+

Student Ambassador, Department of Computer Science, University of Oxford

2019

Member, Student Science Society, Department of Automation, Tsinghua University

2018

Organizer, C Language Programming Competition, Department of Automation, Tsinghua University

2017

Volunteer School Teacher, Teaching Support Program in Underdeveloped Areas (Taiping Village, China)

2024

Submitted my thesis "Embodied Visual Control Policy Learning for Robotic Mobile Manipulation"!

2024

Our paper SpatialPIN got accepted into NeurIPS 2024!

2024

Our paper GenRob got accepted into IROS 2024!

2024

Our paper DynCatch got accepted into ICRA 2024!

2023

Successfully defended my Confirmation of Status viva (Examiners: Prof. Niki Trigoni, Prof. Ioannis Havoutis).

2023

Two papers Multi-Body SE(3) Equivariance and DynPoint got accepted into NeurIPS 2023!

2023

Our paper DecoupSkill got accepted into ICRA 2023!

2022

Started my internship at vLAR group, Hong Kong Polytechnique University (PolyU, mentor: Prof. Bo Yang).

2022

Successfully defended my Transfer of Status viva (Examiners: Prof. Niki Trigoni, Prof. Alessandro Abate).

2021

Completed three assessment work for Transfer (one hardest-level course, one survey, and one term paper).

2020

Started my D.Phil. journey at University of Oxford.

2020

Received my B.Eng. degree with honor from Tsinghua University.