Meta dropped AudioCraft, an open-source tool that lets developers make real audio and music from text. It’s built for people who want to try AI-generated sounds without starting from scratch. The structure handles everything from melody creation to making effects like wind or footsteps – no extra software needed.
The code runs all three functions in one place: composing music, generating sound effects, and compressing files. Anyone can use it to build custom tools or tweak models for things like video games or virtual assistants. A team at Meta tested it with a sample scene involving rain and a car driving through a forest.
“Imagine a professional musician being able to explore new compositions without having to play a single note on an instrument. Or a small business owner adding a soundtrack to their latest video ad on Instagram with ease. That’s the promise of AudioCraft – our latest AI tool that generates high-quality, realistic audio and music from text.”
Also Read: Nebius AI Cloud 3.5 Introduces Serverless AI for Developers
The main components of AudioCraft include three models that work in unison to provide the functionality of generating audio. MusicGen can be used to create new musical compositions from text, whereas AudioGen can be used to produce environmental sounds or effects from user prompts. Finally, there is EnCodec, which is a powerful neural codec for compressing sound, as well as for improving the quality of the produced sound with fewer artifacts.
Meta stated that the models were trained using a combination of licensed music and publicly available sound datasets, allowing them to generate high-quality sound with long-term consistency in the output sound. The concept of the system was to tackle the complexity of sound processing, including both short-term and long-term sound components, such as rhythm and melody.
The company pointed out that AudioCraft aims at increasing the availability of audio generation compared to the usual approaches that required complex pipelines and closed systems. By open-sourcing the models and giving pre-trained weights, Meta allows researchers and developers to experiment, fine-tune, and create custom audio generation systems based on the framework.
“Having a solid open-source foundation will foster innovation and complement the way we produce and listen to audio and music in the future.”
AudioCraft is described as a tool that is intended to assist musicians, sound designers, and developers in speeding up their workflow, developing ideas, and prototyping new audio-based concepts. The company explained that the framework could be utilized for “applications that span music composition/content creation, research in audio modeling/signal processing, and more.”
“We see the AudioCraft family of models as tools for musicians and sound designers to provide inspiration, help people quickly brainstorm and iterate on their compositions in new ways. We can’t wait to see what people create with Audiocraft.”
Meta stated that the release aligns with its broader commitment to open research and innovation in AI, providing the community with tools to advance the field of generative audio and explore new forms of creative expression.