Wentong Li 李文通

PhD

Zhejiang University
Email: liwentong@zju.edu.cn

About Me

I did my Ph.D at College of Computer Science and Technology, Zhejiang University, fortunately supervised by Prof. Jianke Zhu and Prof. Lei Zhang (PolyU, HK). My recent research interests are visual understanding and multimodal large language models, particularly in:
1. Enabling MLLMs with downstream vision tasks, including visual referring and grounding for image/video/3D scene.
2. Efficient and effective MLLMs , including token reduction, lightweight mllm, efficient high-resolution understanding.
Before, I mainly focus on the field of the techniques for object detection, image segmentaion and their weakly-supervised/label-efficient approaches. Besides, I am also interested in autonomous driving tasks (HD-Map, 3D-Occupancy, etc.) and 3D reconstruction tasks.

Preprints

TokenPacker: Efficient Visual Projector for Multimodal LLM

Wentong Li*, Yuqian Yuan*, Jian Liu, Dongqi Tang, Song Wang, Jie Qin, Jianke Zhu, Lei Zhang,

ArXiv:2407.02392, 2024

Paper ｜ Code ｜中文解读｜ Daily Papers

Selected Publications

Osprey: Pixel Understanding with Visual Instruction Tuning

Yuqian Yuan*, Wentong Li*, Jian Liu, Dongqi Tang, Xinjie Luo, Chi Qin, Lei Zhang, Jianke Zhu

CVPR, 2024

Paper ｜ Code ｜ Online Demo ｜ Video Demo ｜中文解读

Box2Mask: Box-supervised Instance Segmentation via Level-set Evolution

Wentong Li, Wenyu Liu, Jianke Zhu, Miaomiao Cui, Risheng Yu, Xiansheng Hua, Lei Zhang

T-PAMI, 2024

Paper | Code(BoxInstSeg) ｜ Code(MMDet)

Label-efficient Segmentation via Affinity Propagation

Wentong Li*, Yuqian Yuan*, Song Wang, Wenyu Liu, Dongqi Tang, Jian Liu, Jianke Zhu, Lei Zhang

NeurIPS, 2023

Paper ｜ Code ｜ Project Page | 中文解读

Point2Mask: Point-supervised Panoptic Segmentation via Optimal Transport

Wentong Li, Yuqian Yuan, Song Wang, Jianke Zhu, Jianshu Li, Jian Liu, Lei Zhang

ICCV, 2023

Paper | Code

Box-supervised Instance Segmentation with Level Set Evolution

Wentong Li, Wenyu Liu, Jianke Zhu, Miaomiao Cui, Xiansheng Hua, Lei Zhang

ECCV, 2022

Paper | Code

Oriented RepPoints for Aerial Object Detection

Wentong Li, Yijie Chen, Kaixuan Hu, Jianke Zhu

CVPR, 2022

Paper | Code(MMDet) | Code(MMRotate)

MGMap: Mask-Guided Learning for Online Vectorized HD Map Construction

Xiaolu Liu, Song Wang, Wentong Li, Ruizi Yang, Junbo Chen, Jianke Zhu

CVPR, 2024

Paper | Code

Not All Voxels Are Equal: Hardness-Aware Semantic Scene Completion with Self-Distillation

Song Wang, Jiawei Yu, Wentong Li, Wenyu Liu, Junbo Chen, Jianke Zhu

CVPR, 2024

Paper ｜ Code

Fine-Grained Multi-View Hand Reconstruction Using Inverse Rendering

Qiqun Gan, Wentong Li, Jinwei Ren, Jianke Zhu

AAAI, 2024

Paper｜ Code

H2RBox: Horizontal Box Annotation is All You Need for Oriented Object Detection

Xue Yang, Gefan Zhang, Wentong Li, Xuehui Wang, Yue Zhou, Junchi Yan

ICLR, 2023

Paper ｜ Code(MMRotate) ｜ Code(Jittor) ｜中文解读

LiDAR2Map: In Defense of LiDAR-Based Semantic Map Construction Using Online Camera Distillation

Song Wang, Wentong Li, Wenyu Liu, Xiaolu Liu, Jianke Zhu

CVPR, 2023

Paper ｜ Code

Honors

Outstanding Graduate of Zhejiang Province, 2024
Outstanding Graduate of Zhejiang University, 2024
Tencent Scholarship, 2023
Five-A Postgraduate Student, 2023
Outstanding Postgraduate Student, 2020-2023
Longhu Scholarship, 2022
First-class Academic Scholarship, 2018-2023
National Scholarship, 2016

Research Intern

AntGroup | HangZhou | Dec.2022 - Present
Topic: MLLM, Weakly-supervised/Label-efficient Segmentation
DAMO Academy | HangZhou | July.2020 - Oct.2020
Topic: Oriented Object Detection, Instance Segmentation
Institution of Automation, CAS | Beijing | July.2018 - June.2019
Topic: Object Detection, Remote Sensing Image

Academic Services

Conference Reviewer:
AAAI2025, ICLR2025
CVPR2024, ICLR2024, ICML2024, ECCV2024, ACM MM2024, NeurIPS2024
CVPR2023, ICCV2023, NeurIPS2023, ACM MM2023
Journal Reviewer:
Transactions on Pattern Analysis and Machine Intelligence (TPAMI)
International Journal of Computer Vision (IJCV)
Transactions on Circuits and Systems for Video Technology (TCSVT)
Transactions on Multimedia (TMM)
Transactions on Geoscience and Remote Sensing (TGRS)
Pattern Recognition (PR)
ACM Computing Surveys
ISPRS Journal of Photogrammetry and Remote Sensing (P&RS)
Neurcomputing

Tech. Talks

AI TIME-大模型嘉年华, Osprey:Pixel Understanding with Visual Instruction Tuning, Video, slides, 2024/01.

Teaching Assistant

Teaching Assistant, Police Brain of Zhejiang Province, Image Processing and Analysis, Fall 2022.
Teaching Assistant, FDS2021: Foundation of Data Structure, Fall 2021.