→
← Click here!
This repository contains the implementation of the music generation model MG2, the first novel approach using melody to guide the music generation that, despite a pretty simple method and extremely limited resources, achieves excellent performance.
Anyone can use this model to generate personalized background music for their short videos on platforms like TikTok, YouTube Shorts, and Meta Reels. Additionally, it is very cost-effective to fine-tune the model with your own private music dataset.
You can watch the introduction video on
Now you can try music generation with your own prompt on our
Tips: To generate high-quality music using MG2, you would want to craft detailed and descriptive prompts that provide rich context and specific musical elements.
To get started with MG2, follow the steps below:
git clone https://github.com/shaopengw/Awesome-Music-Generation.git
cd Awesome-Music-Generation
# Create and activate the environment from the provided environment file
conda env create -f environment.yml
conda activate MMGen_quickstart
# Ensure that the checkpoints are stored in the following directory structure
Awesome-Music-Generation/
└── data/
└── checkpoints/
# Update the paths to reflect your local environment setup
# Replace:
export PYTHONPATH=/mnt/sda/quick_start_demonstration/Awesome-Music-Generation:$PYTHONPATH
export PYTHONPATH=/mnt/sda/quick_start_demonstration/Awesome-Music-Generation/data:$PYTHONPATH
# With:
export PYTHONPATH=/your/local/path/Awesome-Music-Generation:$PYTHONPATH
export PYTHONPATH=/your/local/path/Awesome-Music-Generation/data:$PYTHONPATH
chmod +x quick_start.sh
bash quick_start.sh
Allow the script to run for several minutes. Upon completion, the results will be available in the following directory:
Awesome-Music-Generation/log/latent_diffusion/quick_start/quick_start
- Demo website
- Huggingface checkpoints
- Quick start (Inference)
- Training Datasets
- Training/fine-tuning code
- Online free generation service
- Checkpoints on larger datasets
We sincerely acknowledge the developers of the following open-source code bases. These resources are invaluable sparks that ignite innovation and progress in the real world 🎆!
- https://github.com/CompVis/stable-diffusion
- https://github.com/haoheliu/AudioLDM-training-finetuning
- https://github.com/LAION-AI/CLAP
- https://github.com/jik876/hifi-gan
- https://github.com/facebookresearch/faiss
The research is supported by the Key Technologies Research and Development Program under Grant No. 2020YFC0832702, and National Natural Science Foundation of China under Grant Nos. 71910107002, 62376227, 61906159, 62302400, 62176014, and Sichuan Science and Technology Program under Grant No. 2023NSFSC0032, 2023NSFSC0114, and Guanghua Talent Project of Southwestern University of Finance and Economics.
@article{wei2024melodyneedmusicgeneration,
title={Melody Is All You Need For Music Generation},
author={Shaopeng Wei and Manzhen Wei and Haoyu Wang and Yu Zhao and Gang Kou},
year={2024},
eprint={2409.20196},
archivePrefix={arXiv},
primaryClass={cs.SD},
url={https://arxiv.org/abs/2409.20196},
}