The Street Orchestra is the first to be released of two really exciting iPhone applications I have been involved with the past few months. I really enjoyed working on this application since I felt that it really brought a whole new experience to mobile gaming with its synchronized music playback!

The game was created by HiQ and SCP for the Gothenburg Symphony Orchestra.

Check out the video below, then go download the application free from the App Store.

For the technically interested, here’s some background information on how we created the note data structure and the device synchronization algorithms for the application.

Note Playback

To generate the music notes for the game we created our own note editor in Microsoft .NET. The editor had a grid with four lines – three lines for note events and one line for BPM events (beats per minute). Using the editor you would import an audio file and it would appear as a wave form. You would then estimate the BPM of the song and insert any optional BPM changes events in the song file. To check that the BPM was set correctly we had a switchable metronome sound that played in sync with the song. Once the metronome was set up correctly the notes would snap to the grid in 1/16, 1/8, 1/4 or 1/2 notes, so placing notes was more or less a point-and-click process. Of course we had zooming and other UI features implemented as well. The editor supports long-notes and special notes, but this was not something we had time to implement in the game for version 1.0. The editor could load and save files in a custom XML format that was directly readable by the iPhone game application.

Synchronization

The synchronization works as follows: One device sends out an audible synchronization sound once every second for ten seconds. When other players hear that sound they try to tap their screens at the exact moment the synchronization sound is heard. Internally the application will disregard the first few taps since they are most often not correct. For the rest of the taps the application will move the taps so that the second tap is moved -1 second, the third tap is moved -2 seconds, and so on. The application will then calculate a median value in time based on all taps and will then remove the worst tap. It will then repeat this process to remove the three worst taps. After that has been done a new median time value is created and (n+5) seconds are added to this value to create a moment in time five seconds after where the device thinks the last beep was sent. This time is then used to start the music playback. The end result: perfect syncing on a huge number of mobile phones every time.

Here’s another movie of how the game works in practice: