Yanhong Zeng 曾艳红

Yanhong Zeng is currently a researcher working on Generative AI in Ant Group, working with Yujun Shen. Before that, she led a small effort in advancing image/video generation in Shanghai AI Lab, working with Kai Chen. She obtained her computer science Ph.D. degree in the joint doctoral program between Sun Yat-sen University and Microsoft Research Asia (MSRA) in 2022, working with Jianlong Fu, supervised by Prof. Hongyang Chao and Dr. Baining Guo.

💗 Her research interest is advancing high-quality and controllable generative models and systems across media including images, videos, and audio. Her passion lies in democratizing creativity by transforming ideas into compelling and shareable contents.

CV  /  Email  /  Google Scholar  /  Twitter  /  Github  /  Linkedin

News

Selected Publications

MangaZero is a new large-scale manga dataset containing 43K manga pages and 427K annotated panels. DiffSensei is the first model that can generate manga images with high-quality and controllable multiple characters with complex scenes.

Auto Cherry-Picker is designed to synthesize training samples for both perception and multi-modal reasoning tasks from a simple object list in natural language. It employs a nowly designed metric, CLIS, to ensure the quality of the synthetic data.

MotionBooth is designed for animating customized subjects with precise control over both object and camera movements.

HumanVid is the first large-scale high-quality dataset tailored for human image animation, which combines crafted real-world and synthetic data.

FoleyCrafter is a text-based video-to-audio generation framework which can generate high-quality audios that are semantically relevant and temporally synchronized with the input videos.

Live2Diff is the first attempt that enables uni-directional attention modeling to video diffusion models for live video steam processing, and achieves 16FPS on RTX 4090 GPU.

StyleShot is a style transfer model that excels in text and image-driven style transferring without test-time style-tuning.

PowerPaint is the first versatile inpainting model that achieves SOTA in text-guided and shape-guided object inpainting, object removal, outpainting, etc.

PIA can animate any images from personalized models by text while preserving high-fidelity details and unique styles.

We present Make-it-Vivid, the first attempt that can create plausible and consistent texture in UV space for 3D biped cartoon characters from text input within few seconds.

In AOT-GAN, we propose aggregated contextual transformations and a novel mask-guided GAN training strategy for high-resolution image inpaining.

We collect a large dataset which is the first high-resolution dataset including 371.5k hours of 720p videos and the most diversified dataset covering 15 popular YouTube categories.

We propose a token-based generator with Transformers for image synthesis. We present a new perspective by viewing this task as visual token generation, controlled by style tokens.

We propose STTN, the first transformer-based model for high-quality image inpainting, setting a new state-of-the-art performance.

We propose PEN-Net, the first work that is able to conduct both semantic and texture inpainting. To achieve this, we propose cross-layer attention transfer and pyramid filling strategy.

We design a 3D human body reshaping system. It can take as input user's anthropometric measurements (e.g., height and weight) and generate a 3D human shape for the user.

Working Experience

Researcher, 2025.04 ~ present

Researcher, 2022.07 ~ 2025.03

Research Intern, 2018.06 ~ 2021.12

Research Intern, 2016.06 ~ 2017.06

Projects

Project Owner, 2023.04 ~ 2024.09

Lead Core Maintainer, 2022.07 ~ 2023.08

Miscellanea