Lei Yao | PhD Student

Ray Lei Yao

I am a PhD candidate with the Department of Electrical and Electronic Engineering at the Hong Kong Polytechnic University, supervised by Prof. Lap-Pui Chau. I also work closely with Dr. Yi Wang @ PolyU and Dr. Moyun Liu @ HUST.

I am currently a visiting student in the Department of Computer and Information Science at University of Pennsylvania, advised by Prof. Lingjie Liu within the GRASP Lab.

I recieved my BSc and MRes in the School of Mechanical Science and Engineering, Huazhong University of Science and Technology advised by Prof. Youping Chen in 2020 and 2023, respectively.

I served as a research intern at Tencent Robotics X Lab from May. 2022 to Oct. 2022, focusing on robot dexterous manipulation.

/ / / / /

Photo is generated by Nano Banana

News

[Feb. 2026] 🎉 LaSSM is accepted by IEEE TCSVT. All code and models are released.

[Jul. 2025] 🎉 GaussianCross is acceptted by ACM MM 2025. Pretrained weights and downstream code are released.

[May 2025] 🏆 LaSSM and SGIFormer obtain first and second places on CVPR 2025 ScanNet++ Challenge, respectively.

[May 2025] 🙏 Thanks for Xiaoyang adding SGIFormer into Pointcept, you can try it in this repo now.

[Jan. 2025] 🎉 Our survey paper about Embodied AI is acceptted by Machine Intelligence Research.

[Nov. 2024] 🎉 Our SGIFormer is acceptted by IEEE TCSVT. All code and models are released.

[Oct. 2023] 🎉 One paper about robot compliance control is acceptted by ROBIO 2023 (Oral).

[Aug. 2023] 👨‍🎓 I join JC STEM Lab of ML & CV at PolyU as a PhD student.

Research

My current research interests lie in Spatial Intelligence, with particular interests in 3D scene understanding and 4D world model learning. My research objective is to bulid an embodied agent which has the capability to efficiently perceive and interact with complex real-world environment. Previously, I have worked on robot compliance control and nonprehensile manipulation.

	GaussianCross: Cross-modal Self-supervised 3D Representation Learning via Gaussian Splatting Lei Yao, Yi Wang, Yi Zhang, Moyun Liu, Lap-Pui Chau [page] \| [arxiv] \| [code] ACM MM 2025 We present GaussianCross, a novel cross-modal self-supervised 3D representation learning architecture integrating feed-forward 3DGS techniques.
	LaSSM: Efficient Semantic-Spatial Query Decoding via Local Aggregation and State Space Models for 3D Instance Segmentation Lei Yao, Yi Wang, Yawen Cui, Moyun Liu, Lap-Pui Chau [arXiv] \| [code] IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2026 🏆 First place on CVPR 2025 ScanNet++ Challenge. We introduce LaSSM, prioritizing simplicity and efficiency while maintaining competitive performance.
	SGIFormer: Semantic-guided and Geometric-enhanced Interleaving Transformer for 3D Instance Segmentation Lei Yao, Yi Wang, Moyun Liu, Lap-Pui Chau [page] \| [paper] \| [arxiv] \| [demo] \| [code] IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2024 🏆 Second place on CVPR 2025 ScanNet++ Challenge. Reproduced by the organizers based on our code. We introduce SGIFormer for 3D instance segmentation, which combines semantic-guided query initialization and geometric-enhanced interleaving transformer.
	A Survey of Embodied Learning for Object-Centric Robotic Manipulation Ying Zheng, Lei Yao, Yuejiao Su, Yi Zhang, Yi Wang, Sicheng Zhao, Yiyi Zhang, Lap-Pui Chau [paper] \| [arxiv] \| [code] Machine Intelligence Research, 2025 We provide a comprehensive survey of embodied perceptual learning, embodied policy learning, and embodied task-oriented learning
	A Data-Driven Phantom Zeros Prediction Algorithm for Traction Force Sensor in Kinesthetic Demonstration Lei Yao, Bing Chen, Moyun Liu, Jingming Xie, Youping Chen, Lei He [paper] ROBIO, 2023 (Oral Presentation) We model and predict phantom zeros based on a GA-optimized NN during kinesthetic demonstration.
	An Integrated Compensation Method for the Force Disturbance of a Six-Axis Force Sensor in Complex Manufacturing Scenarios Lei Yao, Qingguang Gao, Dailin Zhang, Wanpeng Zhang, Youping Chen [paper] Sensor, 2021 We propose an integrated compensation method to eliminate the disturbances of zero drift, system error, and gravity of robot end-effector.

Other Publications

Interaction-aware Representation Modeling With Co-Occurrence Consistency for Egocentric Hand-Object Parsing
ICLR, 2026
Yuejiao Su, Yi Wang, Lei Yao, Lap-Pui Chau

MASS: Mesh-inellipse Aligned Deformable Surfel Splatting for Hand Reconstruction and Rendering from Egocentric Monocular Video
CVM, 2026
Haoyu Zhu, Yi Zhang, Lei Yao, Lap-Pui Chau, Yi Wang

GVSynergy-Det: Synergistic Gaussian-Voxel Representations for Multi-View 3D Object Detection
arXiv, 2025 | [paper]
Yi Zhang, Yi Wang, Lei Yao, Lap-Pui Chau

Dynamic Bottleneck with a Predictable Prior for Image-based Deep Reinforcement Learning
Expert Systems with Applications, 2024 | [code]
Bang You, Bing Chen, Lei Yao, Youping Chen, Jingming Xie

Evaluation of Range Sensing-based Place Recognition for Long-term Urban Localization
IEEE Transactions on Intelligent Vehicles (TIV), 2024 | [code]
Weixin Ma, Huan Yin, Lei Yao, Yuxiang Sun, Zhongqing Su

Few-shot Class-agnostic Counting with Occlusion Augmentation and Localization
ISCAS, 2024
Yuejiao Su, Yi Wang, Lei Yao, Lap-Pui Chau

Show previous publications

MAENet: Boost image-guided point cloud completion more accurate and even
Information Fusion, 2025 | [code] GitHub Repo stars
Moyun Liu, Ziheng Yang, Bing Chen, Youping Chen, Jingming Xie, Lei Yao, Lap-Pui Chau, Jiawei Du, Joey Tianyi Zhou

Towards Better Unguided Depth Completion via Cross-Modality Knowledge Distillation in the Frequency Domain
IEEE Transactions on Intelligent Vehicles (TIV), 2024
Moyun Liu, Bing Chen†, Youping Chen, Jingming Xie, Lei Yao†, Yang Zhang, Qin Hu, Jiawei Du, Joey Tianyi Zhou

MENet: Multi-Modal Mapping Enhancement Network for 3D Object Detection in Autonomous Driving
IEEE Transactions on Intelligent Transportation Systems, 2024
Moyun Liu, Youping Chen, Jingming Xie, Yijie Zhu, Yang Zhang, Lei Yao, Zhenshan Bing, Genghang Zhuang, Kai Huang, Joey Tianyi Zhou

A Concise but High-performing Network for Image Guided Depth Completion in Autonomous Driving
Knowledge-Based Systems, 2024 | [code] GitHub Repo stars
Moyun Liu, Bing Chen, Youping Chen, Jingming Xie, Lei Yao, Yang Zhang, Joey Tianyi Zhou

SemiT-SAM: Building a Visual Foundation Model for Tooth Instance Segmentation on Panoramic Radiographs
MICCAI Workshop, 2024 | [dataset] | [code]
Jing Hao, Moyun Liu, Lei He, Lei Yao, James Kit Hon Tsoi, Kuo Feng Hung

BF3D: Bi-directional Fusion 3D Detector With Semantic Sampling and Geometric Mapping
Image and Vision Computing, 2023 | [code]
Yijie Zhu, Jingming Xie, Moyun Liu, Lei Yao, Youping Chen

Services

Teaching Assistant:

EIE546 Video Technology in PolyU, 2023Fall, 2024Fall

EIE522 Pattern Recognition: Theory and Applications in PolyU, 2024Spring, 2025Spring

EIE4100 Computer Vision and Pattern Recognition in PolyU, 2024Fall

Reviewer:

ICLR'25, AAAI'26

TCSVT, TMM

Pattern Recognition

Expert Systems with Applications

The Visual Computer

Education

	Hong Kong Polytechnic University (PolyU) 2023.08 - Present PhD Student , Department of Electrical and Eletronic Engineering Supervisors: Prof. Lap-Pui Chau
	Huazhong University of Science and Technology (HUST) 2020.9 - 2023.6 Master Student, School of Mechanical Science and Engineering Supervisor: Prof. Youping Chen
	Huazhong University of Science and Technology (HUST) 2016.9 - 2020.6 Undergraduate Student, School of Mechanical Science and Engineering Supervisor: Prof. Youping Chen

Experience

	Tencent Robotics X Lab 2022.05 - 2022.10 Research Intern, Robot Dexterous Manipulation Leader: Dr. Yu Zheng
	Imperial College London 2019.1 - 2019.2 Winter School, Hamlyn Centre

Selected Awards and Honors

2023: Hong Kong PolyU Presidental PhD Fellowship (PPPFS)

2023 & 2020: Outstanding Graduate of HUST

2021: National Scholarship

2021: Merit Graduate Student of HUST

2021 & 2020: First-class Graduate School Fellowship

2019: GoerTek Scholarship (Sponsored by GoerTek Co., Ltd.)

2019: National Encouragement Scholarship