Magenta RealTime 2 was released with native support to JAX/MLX. The JAX code allows the code to run on TPU/GPUs, while the MLX version allows the code and the exciting apps Google released to run locally on Mac devices.
We are amplifying the surface one can interact with this amazing model with:
Depthformer (generates the audio tokens), SpectroStream neural codec (encodes/decodes audio tokens), and MusicCoCa style encoder (text & audio prompts → style tokens).trust_remote_code mechanism). magenta-community/magenta-realtime-2 - magenta-community/magenta-realtime-2-smalltransformers.js support — WebGPU browser-based demo.