Javascript/WebAssembly live coding environment for music and synthesis
npm install wasm-musicWasm-music
==========
Demonstrated at the WebAssembly Summit 2020:
https://youtu.be/C8j_ieOm4vE
Go to petersalomonsen.com for the hosted version. Or have a look at demo videos.
The synthesizer itself is written in AssemblyScript, and contains simple oscillators, filters, delay, reverb and more. It also contains a simple pattern sequencer with the same structures as in 4klang, and so you can use the same javascript API when programming music. The difference is though when it comes to defining instruments as you have the full flexibility of writing the code as can be seen in the examples here synth1/assembly/instruments. You'd also have to build a mix for your song as can be seen in the synth1/assembly/mixes folder, and also reference it from synth1/assembly/index.ts.
A synth WebAssembly module is small, and the current examples typically less than 16kb. By embedding the pattern data in the Wasm, you can create tiny webapps or executables playing music.
AudioWorklet is used for playing music on the web with low latency, and this is currently only supported for the Chrome web browser. I've written a polyfill for being able to play in other browsers, but you don't get the low latency.
Go to https://petersalomonsen.com for resources like articles, videos and examples. Also there are my gists. You can easily load music from a gist by referencing to it in the URL, e.g.:
https://petersalomonsen.com/webassemblymusic/livecodev2/?gist=a74d2d036b3ecaa01af4e0f6d03ae7c4
- JavaScript Sequence API Documentation - Complete reference for writing JavaScript music sequences
Build:
from the synth1 folder:
npm run createbrowsersourcebundle
this will create a json containing all the sources for the AssemblyScript compiler running in the browser.
If you want to build the webassembly synth directly:
npm run asbuild (for optimized build)
npm run fastbuild (for faster build and wat file)
Run:
npm run serve
Browse to http://localhost:5000
Export to wav:
from the synth1 folder.
the synth1/index.js script will output audio to stdout, so you may pipe it into e.g. SoX) like this:
node index.js | sox -S -t raw -b 32 -e float -r 44100 -c 2 - out.wav
In the browser there's a button in the upper right with download icon with tooltip Export Wasm.
Pressing this will generate and download a Wasm file.
You can run this Wasm file in the browser, NodeJS or with a WebAssembly runtime like Wasm3 or Wasmer.
See examples with Wasm3 here: https://github.com/wasm3/wasm3/tree/master/test/benchmark/wasmsynth
You can the Wasm file to export raw audio data and import into e.g. Audacity with 32-bit float, little endian, stereo.
wasmer song.wasm > song.raw
Play directly from Wasmer by piping to sox:
wasmer song.wasm | sox -S -t raw -b 32 -e float -r 44100 -c 2 - -d
Create wav directly from Wasmer by piping to sox:
wasmer song.wasm | sox -S -t raw -b 32 -e float -r 44100 -c 2 - song.wav
or use the exported Wasm binary from nodejs or the browser.
If you choose to export the Wasm as a png file you can also compress it further by using e.g. oxipng. In ase of compress it remember to disable gray scale reduction ( as this is not supported by Safari ):
oxipng --ng music.wasm.png
You can export video without sound from the web app. To mux it with the audio try using FFMPEG with this command:
ffmpeg -i video.webm -i exportedsong.wav -c:v copy -c:a libopus -b:a 256k output.webm
Often when creating videos like this you'd want them to be able to loop seamless. Unfortunately most players will have a very small, but noticable delay when looping. An alternative is to use ffmpeg to loop the output video:
ffmpeg -stream_loop 3 -i input.mp4 -c copy output_looped.mp4
Convert them for intra-frame:
ffmpeg -i movie.mp4 -c:v libx264 -intra movie2.mp4
Wasm should be pronounced like awesome starting with a W
https://www.youtube.com/watch?v=C8j_ieOm4vE&t=1650s