Kai Lu
I am a Ph.D. Candidate (2020 - 2025, now in the process of completion) in Computer Science at the University of Oxford, supervised by
Prof. Andrew Markham.
Prior to this, I received my B.E. degree with honor in Automation from
Tsinghua University in 2020, while I was a student research assistant
at the Department of Computer Science from 2017 to 2019, advised by
Prof. Huaping Liu. I was a visiting student at
Duke and UIUC in 2019, advised by Prof. Kris Hauser.
My research focuses on Embodied AI, with two main aspects: first, generalizable/ transferable/ dynamic robot skill learning (thesis subject);
and second, LLM/ VLM/ VLA for multi-modal learning (latest work).
Research Interests:
⋅ Robotics - Robotic Manipulation, Mobile Manipulators, Human-Robot Interaction
⋅ Machine Learning - Deep Learning, Reinforcement Learning
⋅ Multi-Modality - Robotic 3D Vision, Tactile Sensing, VLM/LLM for Embodied Agents
Collaboration / Research Intern:
⋅ Mitsubishi Electric Research Labs (MERL), Meta Research (FAIR, Habitat Team)
⋅ Oxford Robotics Institute (ORI), vLAR at Hong Kong PolyU (vLAR)
Sevice:
⋅ Associate Editor (AE): International Journal of Advanced Robotic Systems (IJARS)
⋅ Reviewer: ICRA, ICLR, NeurIPS
Kai Lu is a D.Phil. student (2020 - 2024) in Computer Science at University of Oxford, supervised by
Prof. Andrew Markham, and within the theme
of Artificial Intelligence and Machine Learning.
His departmental advisor is Prof. Niki Trigoni.
Prior to this, he received his B.E. degree with outstanding graduate honor in Automation from
Tsinghua University in 2020. Before that, he was selected for the Tsinghua Leading Talent
Program in 2016, which can be found in
Tsinghua's Book (Vol. 3) and SOHO.com.
During his study, he was a student research assistant
at State Key Lab of Intelligent Systems and Technology
at the Department of Computer Science from 2017 to 2019, advised by
Prof. Huaping Liu.
He was a visiting scholar at Intelligent Motion Lab at Duke University
and University of Illinois Urbana-Champaign in 2019, advised by
Prof. Kris Hauser.
He completed his bachelor thesis at
Robot Locomotion Lab and he was a member of the winning team in Humanoid League in RoboCup 2019,
advised by Prof. Mingguo Zhao.
After that, he was a visiting
scholar at Visual Learning and Reasoning Group at Hong Kong PolyU in 2022,
advised by Prof. Bo Yang.
In Oxford, he works closely with Prof. Guangrun Wang.
Currently, he is working on robot learning, reinforcement learning, and embodied intelligence,
with applications mainly on generalizable mobile manipulation.
He has published several papers in ICRA, IROS, and NeurIPS. He also served as a reviewer in ICRA, NeurIPS and ICLR.
Feel free to reach out for collaboration and discussion of research ideas.
Formal Bio Resume Github Google Scholar LinkedIn
Email: kai.lu AT cs.ox.ac.uk / lukaiwork01 AT gmail.com

Learning Generalizable Manipulation Policy with Adapter-Based Parameter Fine-Tuning
Kai Lu, Kim Tien Ly, William Hebberd, Kaichen Zhou, Ioannis Havoutis, Andrew Markham
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2024
Webpage
•
PDF
SpatialPIN: Enhancing Spatial Reasoning Capabilities of Vision-Language Models through Prompting and Interacting 3D Priors
Chenyang Ma, Kai Lu, Ta-Ying Cheng, Niki Trigoni, Andrew Markham
Conference on Neural Information Processing Systems (NeurIPS) 2024
Webpage
•
PDF

InteLiPlan: Interactive Lightweight LLM-Based Planner for Domestic Robot Autonomy
Kim Tien Ly, Kai Lu, Ioannis Havoutis
Technical Report 2024
Webpage
•
PDF
Learning to Catch Reactive Objects with a Behavior Predictor
Kai Lu, Jia-Xing Zhong, Bo Yang, Bing Wang, Andrew Markham
IEEE International Conference on Robotics and Automation (ICRA) 2024
Webpage
•
PDF
•
Code
Decoupling Skill Learning from Robotic Control for Generalizable Object Manipulation
Kai Lu, Bo Yang, Bing Wang, Andrew Markham
IEEE International Conference on Robotics and Automation (ICRA) 2023
Webpage
•
PDF
•
Code

Weakly Supervised Descriptor Learning for Pixel-Level Feature Matching
Kai Lu, Andrew Markham
Term Report 2021

Semi-Empirical Simulation of Learned Force Response Models for Heterogeneous Elastic Objects
Yifan Zhu, Kai Lu, Kris Hauser
IEEE International Conference on Robotics and Automation (ICRA) 2020
Webpage
•
PDF
•
Code

Deep Reinforcement Learning for Robotic Pushing and Picking in Cluttered Environment
Yuhong Deng*, Xiangfeng Guo*, Yixuan Wei*, Kai Lu*, Bin Fang, Di Guo, Huaping Liu, Fuchun Sun
*: co-first author, equal contribution
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2019
Webpage
•
PDF
•
Code

QP-Based Whole Body Control (QP-WBC) for Humanoid Robot Motion Control
Kai Lu
Bachelor Thesis
Tsinghua University

RoboCup 2019 Humanoid League Contest
Visual Localization Group, Tsinghua Team
RoboCup 2019
Webpage
•
PDF
•
Code