Lastest Update: 18th Jan 2024 中文版本 (Chinese Version)
Projects
3D Face Reconstruction
3D face reconstruction from a singe photo.

And I modeled my grandma’s face using this technique.

CurlingHunter
A multi-target multi-camera curling tracking system, termed CurlingHunter, is proposed, which can be applied in actual curling games in real time to assist athletes to compete, enhance the interest of the game, etc. Due to the regulations of curling game, no auxiliary equipment can be added to the curling stones, hence only non-contact measurement methods such as machine vision can be used in CurlingHunter. CurlingHunter has solved these problems:
- The problem of accurately capturing relatively small curling stones through long-sighted distance (> 20 m) in the super-large space environment with many occlusions;
- The problem of lens distortion correction in large scenes without interfering with the ice tracks;
- The problem of occlusions which would interfere with tracking and accuracy, while curling stone is easily blocked by athletes wiping ice, other peoples or objects during games;
- The problem of tracking and re-identifying multiple curling stones due to that all curling stones have identical appearance features;

As the first system to be applied to curling game, CurlingHunter demonstrated excellent performances in 2022 Beijing Winter Olympic Games of curling and 2022 Beijing Winter Paralympic Games of curling. Although we focus on curling, our system is readily transferable to other sports.
Face Stylization
This project converts the given image into a dozen of different styles while keeping the characteristics of the input person: Sketch, American Comic, Manga, Oil painting, CG style, Disney style, etc..


3D Face Cartoonization
This project converts the given image into a 3D cartoon avatar.

Avatar Head Pose Estimation
This project estimates the head pose of the person in the video and ensure the obtained head pose accurate and smooth.

Avatar Auto-Creation and Blendshape Generation
Avatar automatically pinches faces and automatically generates blendshape. Note that there is no requirement for the topology of the model.


Generated avatar is driven by video. Please note that the generated blendshape does not have any cross-mold phenomenon.
ToF Body Depth Completation
A real-time body depth completion for AR Effect

Audio Driven Digital Human
Given text or audio, we generate the visual-audio synchronized 2D avatar. All parts of the head look natural when speaking without any weird mouth, teeth and nose movements.
At the same time, as a core researcher, I independently completed (a) Outdoor digital humans (b) Video translation as HeyGen’s competitors (c) Dynamic local segment video fusion, etc. (d) Lightweight small model for mobile terminal
We used this technology to reproduce the talk show performance of Professor Tang, the founder of SenseTime, at the annual meeting. Video Link

VIMI: Controllable character video generation
Given a reference image, action video and expression video, generate the corresponding video of the reference image.

3D Vison and Robotics
This project is about object 6-DOF estimation and robotics manipulation.The project has been going on for too long, so I just selected some remaining drawings.
- Something about point segmentation and object pose estimation.



- Something about robotics and manipulation.

