Convert speech audio into lipsync animation data for 2D/3D characters.
npm install speech-to-lipsyncConvert speech audio into lipsync animation data for 2D/3D characters.
- 🎤 Convert audio to phoneme-based lipsync.(current support:jawOpen,oo-shape,ee-shape)
- 🎭 Output ready for facial rig blendshapes animation.
- 📦 Lightweight and easy to integrate
- 🎧 Uses Meyda for low-level audio analysis
``javascript`
npm install speech-to-lipsync
`javascript
import { SpeechToLipsync } from 'speech-to-lipsync';
const speechPlayer = new SpeechToLipsync({ intensity: 0.8 });
speechPlayer.onUpdate = (features) => {
this.head.morphTargetInfluences[34] = THREE.MathUtils.lerp(this.head.morphTargetInfluences[34], threshold(features.jaw, 0.01, 1), 0.15)
}
const audioLoader = new THREE.AudioLoader();
audioLoader.load('./assets/speech.mp3', (buffer) => {
speechPlayer.setSource(buffer)
speechPlayer.play()
// speechPlayer.stop()
// speechPlayer.replay()
});
``
Please see from github