How SPYSCAPE Added Magic Into Their AR Game Using Nyckel
SPYSCAPE is a destination spy museum and experience in New York City, with interactive exhibits about fictional and real spies. The museum’s permanent exhibits include interactive features and immersive games through which visitors can figure out what kind of spy they would be. Are you a handler or a technician? Special ops or cryptographer?
- SPYSCAPE uses Nyckel image classification to add “magic” to their AR games
- 10 minutes to train first classifier
- Function was “spooky accurate” right away
- Nyckel-powered magic camera a game highlight
The HQ also features a rolling exhibit. 007 x SPYSCAPE has recently moved out, and a physical component of a new Batman x SPYSCAPE mobile augmented reality game has moved in. Batman x Skyscape can be played anywhere in the world, but, if you happen to be in NYC, the game will also take you through the exhibition. It gives you the immersive experience of actually working with Batman and the characters from that universe.
What you do in the game moves the story forward. Although you’re in fact communicating with software, SPYSCAPE wants it to feel like you’re communicating with real people. But the question was how to do this at scale? The game already used a large branching script about the length of a typical novel. This allows the characters to react to the choices that players make in the game, which begins to establish the feeling of reality.
The physical exhibits in New York use RF wristbands and NFC to maintain a personalized sense of reality throughout the game, producing an immersive magic that resonates with customers. But another powerful element of the HQ’s immersiveness comes from its use of picture recognition.
At one point in the exhibit, visitors have to go into a laboratory and take pictures with the Batcam. SPYSCAPE needed an AI to classify the photographs that visitors were taking within the game in order for the game to react to them in a realistic way.
SPYSCAPE already had a lot of infrastructure in AWS, so that was their first thought. But there were lots of competing priorities within the company at the time, and it didn’t make sense to take on a whole raft of machine learning researchers just to solve this problem. Outsourcing made sense, and that’s when Tech Lead Gwyn saw an advertisement for Nyckel on Hacker News.
There was very little to do in terms of API integration: a dozen or so example images per class of training data, uploaded to the Nyckel web interface; and a server written in Node, deployed in AWS, to do a little bit of munging of the output.
Gwyn and his team are so pleased with the magic of the ML-powered Batcam that they are now looking for other use cases within Skyscape.