Ray Lei Yao

I am a PhD student in the Department of Electrical and Electronic Engineering at Hong Kong Polytechnic University, supervised by Prof. Lap-Pui Chau (IEEE Fellow) and Dr. Yi Wang.

I recieved my BSc and MRes in the School of Mechanical Science and Engineering, Huazhong University of Science and Technology advised by Prof. Youping Chen in 2020 and 2023, respectively. I served as a research intern in the Robotics X Lab at Tencent from May. to Oct. 2022.

 /   /   /   /   / 

profile photo

News

  • [Jan. 2025] 🎉 A survey paper about Embodied AI is acceptted by Machine Intelligence Research.
  • [Nov. 2024] 🎉 Our SGIFormer is acceptted by IEEE Transactions on Circuits and Systems for Video Technology.
  • [Oct. 2023] 🎉 One paper about robot compliance control is acceptted by ROBIO 2023.
  • [Aug. 2023] 🎉 I join JC STEM Lab of ML & CV at PolyU as a PhD student.

  • Research

    My current research interests lie in the intersection of 3D Vision and Generalizable Visual Representation for Embodied AI, with particular interests in 3D scene understanding, sample-efficient and generalizable manipulation policy learning. My research objective is to bulid an embodied agent which has the ability to efficiently perceive and interact with complex real-world environment. Previously, I have worked on robot Compliance control and Nonprehensile manipulation.



    SGIFormer: Semantic-guided and Geometric-enhanced Interleaving Transformer for 3D Instance Segmentation
    Lei Yao, Yi Wang, Moyun Liu, Lap-Pui Chau
    [page] | [arxiv] | [demo] | [code] GitHub Repo stars
    IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2024

    We introduce SGIFormer for 3D instance segmentation, which combines semantic-guided query initialization and geometric-enhanced interleaving transformer.



    A Survey of Embodied Learning for Object-Centric Robotic Manipulation
    Ying Zheng*, Lei Yao*, Yuejiao Su, Yi Zhang, Yi Wang, Sicheng Zhao, Yiyi Zhang, Lap-Pui Chau
    [arxiv] | [code] GitHub Repo stars
    Machine Intelligence Research, 2025

    We provide a comprehensive survey of embodied perceptual learning, embodied policy learning, and embodied task-oriented learning



    Dynamic Bottleneck with a Predictable Prior for Image-based Deep Reinforcement Learning
    Bang You, Bing Chen, Lei Yao, Youping Chen, Jingming Xie
    [paper] | [code]
    Expert Systems with Applications, 2024

    We propose an information-theoretic dynamic bottleneck objective with a predictable parametric prior for learning compressive and dynamic-relevant representations.



    A Data-Driven Phantom Zeros Prediction Algorithm for Traction Force Sensor in Kinesthetic Demonstration
    Lei Yao, Bing Chen, Moyun Liu, Jingming Xie, Youping Chen, Lei He
    [paper]
    ROBIO, 2023 (Oral Presentation)

    We model and predict phantom zeros based on a GA-optimized NN during kinesthetic demonstration.

    An Integrated Compensation Method for the Force Disturbance of a Six-Axis Force Sensor in Complex Manufacturing Scenarios
    Lei Yao, Qingguang Gao, Dailin Zhang, Wanpeng Zhang, Youping Chen
    [paper]
    Sensor, 2021

    We propose an integrated compensation method to eliminate the disturbances of zero drift, system error, and gravity of robot end-effector.


    Other Publications

    Evaluation of Range Sensing-based Place Recognition for Long-term Urban Localization
    IEEE Transactions on Intelligent Vehicles (TIV), 2024 | [code]
    Weixin Ma, Huan Yin, Lei Yao, Yuxiang Sun, Zhongqing Su

    Few-shot Class-agnostic Counting with Occlusion Augmentation and Localization
    ISCAS, 2024
    Yuejiao Su, Yi Wang, Lei Yao, Lap-Pui Chau

    Towards Better Unguided Depth Completion via Cross-Modality Knowledge Distillation in the Frequency Domain
    IEEE Transactions on Intelligent Vehicles (TIV), 2024
    Moyun Liu, Bing Chen, Youping Chen, Jingming Xie, Lei Yao, Yang Zhang, Qin Hu, Jiawei Du, Joey Tianyi Zhou

    MENet: Multi-Modal Mapping Enhancement Network for 3D Object Detection in Autonomous Driving
    IEEE Transactions on Intelligent Transportation Systems, 2024
    Moyun Liu, Youping Chen, Jingming Xie, Yijie Zhu, Yang Zhang, Lei Yao, Zhenshan Bing, Genghang Zhuang, Kai Huang, Joey Tianyi Zhou

    A Concise but High-performing Network for Image Guided Depth Completion in Autonomous Driving
    Knowledge-Based Systems, 2024 | [code] GitHub Repo stars
    Moyun Liu, Bing Chen, Youping Chen, Jingming Xie, Lei Yao, Yang Zhang, Joey Tianyi Zhou

    SemiT-SAM: Building a Visual Foundation Model for Tooth Instance Segmentation on Panoramic Radiographs
    MICCAI Workshop, 2024 | [dataset] | [code]
    Jing Hao, Moyun Liu, Lei He, Lei Yao, James Kit Hon Tsoi, Kuo Feng Hung

    BF3D: Bi-directional Fusion 3D Detector With Semantic Sampling and Geometric Mapping
    Image and Vision Computing, 2023 | [code]
    Yijie Zhu, Jingming Xie, Moyun Liu, Lei Yao, Youping Chen


    Services

    Teaching:

  • TA of EIE546 Video Technology in PolyU, 2023Fall, 2024Fall
  • TA of EIE522 Pattern Recognition: Theory and Applications in PolyU, 2024Spring, 2025Spring
  • TA of EIE4100 Computer Vision and Pattern Recognition in PolyU, 2024Fall
  • Reviewer:

  • IEEE Transactions on Multimedia
  • International Conference on Learning Representations (ICLR)
  • The Visual Computer

  • Education
    Hong Kong Polytechnic University (PolyU)
    2023.08 - Present
    PhD Student , Department of Electrical and Eletronic Engineering
    Supervisors: Prof. Lap-Pui Chau, Dr. Yi Wang
    Huazhong University of Science and Technology (HUST)
    2020.9 - 2023.6
    Master Student, School of Mechanical Science and Engineering
    Supervisor: Prof. Youping Chen
    Huazhong University of Science and Technology (HUST)
    2016.9 - 2020.6
    Undergraduate Student, School of Mechanical Science and Engineering
    Supervisor: Prof. Youping Chen

    Experience
    Tencent Robotics X Lab
    2022.05 - 2022.10
    Research Intern, Robot Dynamic Nonprehensile Manipulation
    Leader: Dr. Yu Zheng
    Imperial College London
    2019.1 - 2019.2
    Winter School, Hamlyn Centre

    Selected Awards and Honors

  • 2023: Hong Kong PolyU Presidental PhD Fellowship (PPPFS)
  • 2023 & 2020: Outstanding Graduate of HUST
  • 2021: National Scholarship
  • 2021: Merit Graduate Student of HUST
  • 2021 & 2020: First-class Graduate School Fellowship
  • 2019: GoerTek Scholarship (Sponsored by GoerTek Co., Ltd.)
  • 2019: National Encouragement Scholarship



  • This homepage is designed based on Jon Barron's website and deployed on Github Pages.
    Last updated: Jan. 17 2025
    © Copyright 2025 Lei Yao