Ray Lei Yao

I am a PhD candidate with the Department of Electrical and Electronic Engineering at the Hong Kong Polytechnic University, supervised by Prof. Lap-Pui Chau. I also work closely with Dr. Yi Wang @ PolyU and Dr. Moyun Liu @ HUST.

I am currently a visiting student in the Department of Computer and Information Science at University of Pennsylvania, advised by Prof. Lingjie Liu within the GRASP Lab.

I recieved my BSc and MRes in the School of Mechanical Science and Engineering, Huazhong University of Science and Technology advised by Prof. Youping Chen in 2020 and 2023, respectively.

I served as a research intern at Tencent Robotics X Lab from May. 2022 to Oct. 2022, focusing on robot dexterous manipulation.

profile photo

Photo is generated by Nano Banana

News

  • [Feb. 2026] 🎉 LaSSM is accepted by IEEE TCSVT. All code and models are released.
  • [Jul. 2025] 🎉 GaussianCross is accepted by ACM MM 2025. Pretrained weights and downstream code are released.
  • [May 2025] 🏆 LaSSM and SGIFormer obtain first and second places on CVPR 2025 ScanNet++ Challenge, respectively.
  • [May 2025] 🙏 Thanks for Xiaoyang adding SGIFormer into Pointcept, you can try it in this repo now.
  • [Jan. 2025] 🎉 Our survey paper about Embodied AI is accepted by Machine Intelligence Research.
  • [Nov. 2024] 🎉 Our SGIFormer is accepted by IEEE TCSVT. All code and models are released.
  • [Oct. 2023] 🎉 One paper about robot compliance control is accepted by ROBIO 2023 (Oral).
  • [Aug. 2023] 👨‍🎓 I join JC STEM Lab of ML & CV at PolyU as a PhD student.

Research

My current research interests lie in Spatial Intelligence, with particular interests in 3D scene understanding and 4D world model learning. My research objective is to bulid an embodied agent which has the capability to efficiently perceive and interact with complex real-world environment. Previously, I have worked on robot compliance control and nonprehensile manipulation.


SGIFormer project image

SGIFormer: Semantic-guided and Geometric-enhanced Interleaving Transformer for 3D Instance Segmentation

Lei Yao, Yi Wang, Moyun Liu, Lap-Pui Chau

IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2024

🏆 Second place on CVPR 2025 ScanNet++ Challenge*.

* Reproduced by the organizers based on our code.

We introduce SGIFormer for 3D instance segmentation, which combines semantic-guided query initialization and geometric-enhanced interleaving transformer.

Other Publications

Dynamic Bottleneck with a Predictable Prior for Image-based Deep Reinforcement Learning

Expert Systems with Applications, 2024 | [code]
Bang You, Bing Chen, Lei Yao, Youping Chen, Jingming Xie

Evaluation of Range Sensing-based Place Recognition for Long-term Urban Localization

IEEE Transactions on Intelligent Vehicles (TIV), 2024 | [code]
Weixin Ma, Huan Yin, Lei Yao, Yuxiang Sun, Zhongqing Su

Few-shot Class-agnostic Counting with Occlusion Augmentation and Localization

ISCAS, 2024
Yuejiao Su, Yi Wang, Lei Yao, Lap-Pui Chau

Show previous publications

MAENet: Boost image-guided point cloud completion more accurate and even

Information Fusion, 2025 | [code] GitHub Repo stars
Moyun Liu, Ziheng Yang, Bing Chen, Youping Chen, Jingming Xie, Lei Yao, Lap-Pui Chau, Jiawei Du, Joey Tianyi Zhou

Towards Better Unguided Depth Completion via Cross-Modality Knowledge Distillation in the Frequency Domain

IEEE Transactions on Intelligent Vehicles (TIV), 2024
Moyun Liu, Bing Chen, Youping Chen, Jingming Xie, Lei Yao, Yang Zhang, Qin Hu, Jiawei Du, Joey Tianyi Zhou

MENet: Multi-Modal Mapping Enhancement Network for 3D Object Detection in Autonomous Driving

IEEE Transactions on Intelligent Transportation Systems, 2024
Moyun Liu, Youping Chen, Jingming Xie, Yijie Zhu, Yang Zhang, Lei Yao, Zhenshan Bing, Genghang Zhuang, Kai Huang, Joey Tianyi Zhou

A Concise but High-performing Network for Image Guided Depth Completion in Autonomous Driving

Knowledge-Based Systems, 2024 | [code] GitHub Repo stars
Moyun Liu, Bing Chen, Youping Chen, Jingming Xie, Lei Yao, Yang Zhang, Joey Tianyi Zhou

SemiT-SAM: Building a Visual Foundation Model for Tooth Instance Segmentation on Panoramic Radiographs

MICCAI Workshop, 2024 | [dataset] | [code]
Jing Hao, Moyun Liu, Lei He, Lei Yao, James Kit Hon Tsoi, Kuo Feng Hung

BF3D: Bi-directional Fusion 3D Detector With Semantic Sampling and Geometric Mapping

Image and Vision Computing, 2023 | [code]
Yijie Zhu, Jingming Xie, Moyun Liu, Lei Yao, Youping Chen

Services

Teaching Assistant:

Reviewer:

  • ICLR'25, AAAI'26
  • TCSVT, TMM
  • Pattern Recognition
  • Expert Systems with Applications
  • The Visual Computer

Education

PolyU Logo
HUST Logo
HUST Logo

Experience

Tencent Logo

Tencent Robotics X Lab

2022.05 - 2022.10

Research Intern, Robot Dexterous Manipulation

Leader: Dr. Yu Zheng

IC Logo

Imperial College London

2019.1 - 2019.2

Winter School, Hamlyn Centre

Selected Awards and Honors

  • 2023: Hong Kong PolyU Presidental PhD Fellowship (PPPFS)
  • 2023 & 2020: Outstanding Graduate of HUST
  • 2021: National Scholarship
  • 2021: Merit Graduate Student of HUST
  • 2021 & 2020: First-class Graduate School Fellowship
  • 2019: GoerTek Scholarship (Sponsored by GoerTek Co., Ltd.)
  • 2019: National Encouragement Scholarship