Make a face, hear a genre. Face the Music reads your expression through your webcam, places it on a map of valence (negative to positive) and arousal (calm to energetic), and plays music that matches: disco funk when you’re beaming, metal when you’re furious, a piano ballad when you’re down.
It started as a project for my Design with AI class at the University of Miami. I trained a facial-expression classifier in Apple’s CreateML on five emotions (anger, fear, happiness, neutral and sadness) and tested how the size of the training set affects accuracy. The CreateML models overfit badly: near-perfect in training, but only 24–36% on a held-out test set. In the write-up I suggested a fix: find the face first, then classify the facial features instead of the whole image. The original demo also depended on Google’s MusicFX DJ, so it couldn’t live on its own.
The new version takes my own advice. Google’s MediaPipe finds your face and measures 52 facial muscle movements, and a small model I retrained on the same dataset turns those into emotion probabilities. On the same experiment, test accuracy goes from 36% to 47%, and the full model reaches 54%. The music comes from a generative engine with 15 genre recipes, from dark ambient to drum & bass, that blends between neighbouring genres as your face moves. You can also drag the point around yourself.
Everything runs in your browser, so your camera feed never leaves your device. The full write-up, covering data prep, the original CreateML experiments and how the rebuild compares, is on the site under “The project”. Headphones recommended.
