Skip to content

yule-li/Human-Video-Generation

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

61 Commits
 
 
 
 

Repository files navigation

Human Video Generation

Paper List

2018

  • Face2Face: "Real-time Face Capture and Reenactment of RGB Videos" "CVPR" (2016) [paper][project]
  • PSGAN: "Pose Guided Human Video Generation" "ECCV" (2018) [paper]
  • DVP: "Deep Video Portraits" "Siggraph"(2018) [paper][project]
  • Recycle-GAN: "Recycle-GAN: Unsupervised Video Retargeting" "ECCV"(2018) [paper][project][code]
  • X2Face: "X2Face: A network for controlling face generation by using images, audio, and pose codes" "ECCV"(2018) [paper][project][code]
  • EBDN: "Everybody Dance Now" "arXiv"(2018) [paper][project]
  • Vid2Vid: "Video-to-Video Synthesis" "NIPS"(2018) [paper][project][code]

2019

  • NAR: "Neural Animation and Reenactment of Human Actor Videos" "Siggraph"(2019) [paper][project]
  • TETH: "Text-based Editing of Talking-head Video" "Siggraph"(2019) [paper][project]
  • VPC: "Deep Video-Based Performance Cloning" "Eurographics"(2019) [paper]
  • FSTH: "Few-Shot Adversarial Learning of Realistic Neural Talking Head Models" "CVPR"(2019) [paper][code unofficial]
  • TNA: "Textured Neural Avatars" "CVPR"(2019) [paper][project]
  • VOCA: "Voice Operated Character Animation" "CVPR"(2019) [paper][project][code]
  • Audio2Face: "Audio2Face: Generating Speech/Face Animation from Single Audio with Attention-Based Bidirectional LSTM Networks" "arXiv"(2019) [paper
  • RSDA: "Realistic Speech-Driven Animation with GANs" "arXiv"(2019) [paper][project][code]
  • LISCG: "Learning Individual Styles of Conversational Gesture" "arXiv"(2019) [paper] [project][code]
  • AUDIO2FACE: "EAUDIO2FACE: GENERATING SPEECH/FACE ANIMATION FROM SINGLE AUDIO WITH ATTENTION-BASED BIDIRECTIONAL LSTM NETWORKS" "ICMI"(2019)
  • AvatarSim: "A High-Fidelity Open Embodied Avatar with Lip Syncing and Expression Capabilities" "ICMI"(2019) code
  • NVP: "Neural Voice Puppetry: Audio-driven Facial Reenactment" "arXiv"(2019) [paper]
  • CSGN: "Convolutional Sequence Generation for Skeleton-Based Action Synthesis" "ICCV"(2019) [paper]
  • Few shot VID2VID: "Few-shot Video-to-Video Synthesis" [paper] [project] [code]
  • FOM: "First Order Motion Model for Image Animation" "NIPS"(2019) [paper] [project] [code]

2020

  • TransMoMo: "TransMoMo: Invariance-Driven Unsupervised Video Motion Retargeting" "CVPR"(2020) [paper] [project] [code]
  • poseflow: "Deep Image Spatial Transformation for Person Image Generation" "CVPR"(2020) [paper] [project] [code]
  • PIFuHD: "PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization" "CVPR(Oral)"(2020) [paper] [project] [code]
  • Hifi3dface: "High-Fidelity 3D Digital Human Creation from RGB-D Selfies" "arXiv"(2020.10) [paper][project] [code]
  • face-vid2vid: "One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing" "arXiv"(2020.11) [paper] [project] [code]
  • HeadGan: "HeadGAN: Video-and-Audio-Driven Talking Head Synthesis" "arXiv"(2020.12) [paper]
  • "Audio-driven Talking Face Video Generation with Learning-based Personalized Head Pose" "arXiv"(2020) [paper][code]

2021

  • Talking-Face_PC-AVS: "Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation" "CVPR"(2021) [code][project][demo]
  • Pixel Codec Avatar "Pixel Codec Avatars" "arXiv"(2021.04) [paper]
  • MRAA "Motion Representations for Articulated Animation" "CVPR"(2021) [project]
  • NWT "Towards natural audio-to-video generation with representation learning" "arXiv"(2021)[paper][project]
  • LipSync3D Data-Efficient Learning of Personalized 3D Talking Faces from Video using Pose and Lighting Normalization "arXiv"(2021) [paper][demo]
  • AD-NeRF Audio Driven Neural Radiance Fields for Talking Head Synthesis "ICCV"(2021) [paper][code][demo][project]
  • LSP Live Speech Portraits: Real-Time Photorealistic Talking-Head Animation [paper][code][project][demo]
  • FaceFormer FaceFormer: Speech-Driven 3D Facial Animation with Transformers "arXiv"(2021.12) [paper]
  • HeadNeRF HeadNeRF: A Real-time NeRF-based Parametric Head Model "arXiv"(2021.12) [paper][project]
  • FACIAL FACIAL: Synthesizing Dynamic Talking Face with Implicit Attribute Learning "ICCV"(2021) [paper][code]

2022

  • NPFAP Video-driven Neural Physically-based Facial Asset for Production "arXiv"(2022.02)[paper]
  • PGMPI Real-Time Neural Character Rendering with Pose-Guided Multiplane Images "ECCV"(2022) [paper][code][project]
  • VideoReTalking Audio-based Lip Synchronization for Talking Head Video Editing In the Wild "arXiv"(2022.11) [paper][code][project]
  • One-Shot-Talking-Face One-shot Talking Face Generation from Single-speaker Audio-Visual Correlation Learning "AAAI"(2022) [paper][code][demo]
  • RAD-NeRF: Real-time Neural Talking Portrait Synthesis:"arXiv"(2022.12)[paper][code]

2023

  • SadTalker Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation "CVPR"(2023) [paper][code][project]
  • GeneFace: Generalized and High-Fidelity Audio-Driven 3D Talking Face Synthesis "ICLRv"(2023) [project][code][dockerfile]
  • Towards Realistic Generative 3D Face Models "arXiv"(2023.04) [paper][project][code]
  • Live 3D Portrait: Real-Time Radiance Fields for Single-Image Portrait View Synthesis "SIGGRAPH" (2023) [project][paper]
  • StyleAvatar: Real-time Photo-realistic Portrait Avatar from a Single Video "SIGGRAPH" (2023) [code][project][paper]
  • OTAvatar : One-shot Talking Face Avatar with Controllable Tri-plane Rendering [code] "arXiv"(2023) [paper]
  • DisCoHead: Audio-and-Video-Driven Talking Head Generation by Disentangled Control of Head Pose and Facial Expressions "arXiv"(2023) [project]
  • GeneFace++: Generalized and Stable Real-Time Audio-Driven 3D Talking Face Generation:[project]
  • HumanRF: High-Fidelity Neural Radiance Fields for Humans in Motion "SIGGRAPH" (2023) [project][code]
  • PointAvatar: Deformable Point-based Head Avatars from Videos "CVPR"(2023) [project][code][paper]
  • SyncTalk:SyncTalk: The Devil😈 is in the Synchronization for Talking Head Synthesis "arXiv(2023.11)[project][code]

2024

  • Real3D-Portait: Real3D-Portrait: One-shot Realistic 3D Talking Portrait Synthesis "ICLR(2024)" [project][code][paper]
  • EMO: Emote Portrait Alive - Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditions "arXiv"(2024.02) [project][paper][code]

Applications

Face Swap

  • ZAO: a hot app.

Video generated based on ZAO

AI Host:

Video generated by SouGou

Dataset

Researchers & Teams

  1. Graphics, Vision & Video at MPII
  2. REAL VIRTUAL HUMANS at MPII
  3. Visual Computing Group at TUM
  4. Perceiving Systems Department at MPII