Back in 2002, [Takahiro Yonemura] and others built a system that used a single webcam-like device to track a user’s hand and generate inputs for a computer. Given that this was more than 20 years ago, the technological limitations were much greater than they are today. It wasn’t until nearly a decade later that we got something like the Microsoft Kinect, and even now many VR and gesture controls require something more than a simple camera. Although [Takahiro Yonemura]’s original paper (Japanese, Google Translate link) is now many decades old, he’s recently picked the project back up and used it to develop this game.
The original system worked by attempting to automatically identify the user’s fingers and head. The team had focused on user inputs for standard PCs, drawing characters in the air and allowing the software to interpret them and pass them to a computer. In the modern iteration the scope is slimmed down a bit. The system instead detects an open or closed hand, and can recognize when it leaves the camera’s view. For the game built around this concept, the player controls a dragon who whips his tail around a map. Closing and opening a hand generates a click in the game, and moving a hand off-screen pauses the game.
Although the game has relatively simple controls, these types of games can be quite fun and popular. From the video clip on the Steam page it looks like [Takahiro] is having plenty of fun playing it. Although we’d prefer it be open source so others can experiment with his gesture controls, we can also experiment with other camera-only gesture controls like this project which allows the user to move and interact with 3D objects in a virtual space.