How Accurately and Creatively Can Generative AI Produce Music Across Genres? A Mixed-Methods Evaluation of ACE-Step 1.5

Generative artificial intelligence music tools such as Suno, Udio, and ACE-Step can now create full songs with vocals, instrumentation, and production quality convincing enough to sound like human-made music. However, musical "quality" is not a single measurable property, and treating "AI music" as one category by itself risks missing details on how performance varies by genre. This study asks: how accurately and creatively can generative AI produce music across different genres? Two hundred tracks were generated using the open-source model ACE-Step 1.5 across ten genres (Pop, Hip-Hop & Rap, Classical, EDM, Rock, Latin, Country, R&B, Jazz, and Metal), using five human-written lyric sets to isolate genre as the main variable. Each track was evaluated through two complementary methods: a computational audio analysis using Essentia's Discogs-trained genre classifier, and a human evaluation using a five-criterion rubric assessing harmonic language, melodic development, rhythmic integrity, dynamics, and emotional fidelity.

Results suggest that AI-generated music performance is uneven and genre-dependent instead of uniform: genres that relied on standardized and rhythmic structures were classified and rated more consistently than genres that needed more sophisticated harmonies or improvisation and creativity. Genre-classification confidence and rubric-based creativity scores did not always align, which may mean the model can recreate surface-level genre traits without capturing the expressions and qualities associated with human musicianship. The study also brings a methodological finding to light: the genre classifier's own training taxonomy assigns subgenre labels unevenly across genres, which can distort genre-accuracy independent of the audio itself.