A Practical Guide to Generative Music AI for Developers April 2025

Main Topics: AI Music Case Studies, Course Roadmap

An Introduction to AI Music (50')

Session 1 PDF handout

Open Discussion

Main Topics: Environment Setup

GitHub Repository

Setting up your environment - Cloning the class repository (10')

Hands On

Setting up your environment - Hands On 1.1: Loading, visualizing, playing audio (10')

Setting up your environment - Hands On 1.2: Extracting Audio Features, RMS and ZCR (13')

Setting up your environment - Hands On 1.2: Extracting Audio Features, Spectrograms (13')

Setting up your environment - Hands On 2: Manipulating MIDI Data (20')

Main Topics: Audio vs Symbolic Music, Basics of Generative AI, Data Acquisition and Ethics

Statistical Basics of Generative Modeling in Artificial Intelligence (10')

Variational Autoencoders (16')

Hands On

Hands On 1.1: Lakh MIDI Dataset (12')

Hands On 1.2: Free Music Archive (6')

Hands On 2: Using RAVE

Main Topics: Human-Computer Interaction, Iterative Design, Continuous Deployment

Human-Computer Interaction & User-Centered Design

Open Discussion with Jordan Rudess

Deep Dive into MIDI & Spectrograms

Comparing Musical Representations & Encodec Deep Dive (14')

Understanding RVQ in Encodec (12')

Hands On

Hands On: Encodec (29')

Main Topics: Autoregressive modeling, the Transformer architecture, HuggingFace Hub

The Transformer architecture (15')

Understanding Anticipatory Music Transformers (13')

Hands On

Hands On: Using AMT to generate MIDI data (Part 1) (18')

Hands On: Using AMT to generate MIDI data (Part 2) (17')

Main Topics: MusicGen & Audio Generation with Transformers

Understanding MusicGen (8')

Hands On

Hands On: Using MusicGen to generate audio (38')

Main Topics: Diffusion Models, Latent Diffusion Models

Intro to Diffusion Models Part 1 (11')

Intro to Diffusion Models Part 2 (14')

Conditioning & Classifier-Free Guidance (10')

The UNet Architecture (6')

Inference-Time Optimization: DITTO (6')

Hands On

Hands On: Using Stable Audio Part 1 (15')

Hands On: Using Stable Audio Part 2 (18')

Introduction to TorchScript (18')

8-bit Linear Quantization (9')

ONNX and Graph Optimizations (15')

Main Topics: Landscape of companies in AI and Music, Available Commercial Products

Demo

Main Topics: Setting up a project specification, timeline, and scope

Training Part 1 (8')

Training Part 2 (13')

Training Part 3 (16')

Peer Review & Feedback

Session 11

Lab Session: Guided Coding & Troubleshooting

Milestone Check-Ins

Final Presentations

Next Steps