LatentSync: Audio Conditioned Latent Diffusion Models for Lip Sync
Paper • 2412.09262 • Published • 1
import torch
from diffusers import DiffusionPipeline
# switch to "mps" for apple devices
pipe = DiffusionPipeline.from_pretrained("ByteDance/LatentSync", dtype=torch.bfloat16, device_map="cuda")
prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k"
image = pipe(prompt).images[0]The checkpoints of LatentSync
This repo not only stores the pretrained U-Net and SyncNet checkpoints of LatentSync, but also stores the whisper checkpoints, auxiliary checkpoints for detecting face, calculating syncnet confidence score and so on. They have covered all you need for both inference and training of LatentSync
- Downloads last month
- -
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support