Lip Sync

Let's learn how to add LipSync on Ready Player Me with React Three Fiber and Three.js

We'll discover the concept of MorphTargets and Visemes

Live demo
https://r3f-lipsync-tutorial.vercel.app/

Get the starter pack here 🔗
https://github.com/wass08/r3f-vite-starter

Final Code
https://github.com/wass08/r3f-lipsync-tutorial

Rhubarb LipSync
https://github.com/DanielSWolf/rhubarb-lip-sync

Ready Player Me
https://readyplayer.me/

And the documentation
https://docs.readyplayer.me/ready-player-me/api-reference/rest-api/avatars/get-3d-avatars#examples-7

Convert MP3 to OGG
https://convertio.co/mp3-ogg/

Eleven Labs (TTS)
https://elevenlabs.io/speech-synthesis

Mixamo Animations
https://www.mixamo.com/#/

LeonardoAI
https://leonardo.ai/

React Three Fiber documentation
https://docs.pmnd.rs/react-three-fiber/getting-started/introduction

R3F Drei
https://github.com/pmndrs/drei