Disney+, ShazamKit, and Why It Matters for You

In our last newsletter (Wait - you're not subscribed yet? 😱), I mentioned the Augmented Reality experience released by Disney+ for their latest short movie, "Remembering".

The concept is pretty simple: after downloading the app on your phone, you leave it open and start playing the movie on your TV. After a few minutes, your device will chime and prompt you to point it at the TV; a short countdown goes on, and the AR experience will start. Yay!

So indeed, as you might have read, it's the first AR experience that connects directly to content on Disney+.

But I think something matters even more: the technology used to trigger the experience!

Disney+ has leveraged an SDK called “ShazamKit”. You might have used it before: Shazam, the well-known app that everyone can use to recognize songs, has been out for a while now. Its first version for iOS was released in 2008, and Apple bought the company in 2018.

Whoops, sorry - not that one!

Better!

Ahem, anyway, it seems they’ve been busy! Indeed, ShazamKit is an exciting technology. Yes, it can recognize songs, but it can also be programmed to detect sounds! This is what is used in the Remembering app: it was programmed with key parts of the soundtrack. ShazamKit recognizes these parts and can trigger an event in the application.

On their website, ShazamKit lists some apps that leverage their technology. Apart from the Yoga one, all of the others are song-recognition apps.

Come on, creative people, you can do better than that!

A treasure hunt where users have to search for the sounds in an area?

A helping app for deaf people?

A digital marketing experience where you have to open the app and react as fast as possible once the ad starts on TV to get rewards?

An experience in an escape game where you must play the right sounds to the application?