Puzzling Places is a VR jigsaw puzzle game created with photogrammetry RealityCapture scans where players puzzle, piece by piece, hyper-realistic miniatures from around the world. The core concept and gameplay is quite simple which makes it familiar and extremely fun for people of all ages. Coming soon to Oculus Quest and PlayStation VR!
Variety of models in Puzzling Places by Alexandre González Rivas, Gerpho3D, & Azad B.
Azad Balabanian from Realities.io , creators of Puzzling Placesshares the story behind Puzzling Places, his knowledge and plans for the future.
The purpose of the project was to create a go to app when people want a calm and meditative VR game to play, an alternative to all the shooters and horror games out there. Everybody knows how to puzzle, which makes Puzzling Places a game that can be played by anyone. It’s also a fantastic way to see, hear, and learn about new places from around the world in a fun and engaging way, to playfully interact with scanned content in VR and AR.
The origin story of the game is a result of a total accident, a true "bug turned into a feature" moment! One of the steps of our Photogrammetry optimizations is to separate large Photogrammetry scans into equally-sized "chunks" to support rendering optimizations like frustum and occlusion culling.
Level of difficulty in Puzzling Places. Scan by poiformat
One day in 2018, while importing one of our processed Photogrammetry worlds into a game engine, the chunks of the scans appeared out of order due to an error in the importer, and were jumbled up across the scene. Shahriar Shahrabi, the Graphics Programmer/Game Designer on our team joked, "what if we ship it like this and have people just puzzle the pieces back together instead?". And so, the idea was born! Over the next week or so, Shahriar developed a SteamVR prototype to play and evaluate the idea. Lo and behold, it ended up being extremely fun!
Although the concept of the game is simple, technically, it is anything but!
Assuming most people reading the RealityCapture blog are aware of the basics of Photogrammetry, we won't reiterate the introductory details. Our Photogrammetry datasets are typically a combination of terrestrial DSLR's (Sony A7rii's) and aerial drones (DJI Mavic 2 Pros, DJI Inspire) processed with RealityCapture. Occasionally, we also deploy LIDAR for certain locations that Photogrammetry would not be suited for.
We often capture locations with a specific lighting in mind which is baked-in in the albedo texture, similar to Photography, which can often result in stunning photorealism that is stunning to see in VR. If the model's texture lacks certain contrast or is somewhat bland, however, we like to bake CG lighting into the albedo texture using Blender which can create very realistic results.
Tsuruga Castle by Fuji
Tsuruga Castle 3D model - top view
It's worth noting that many of the puzzles that we've created recently were in fact captured years ago with all the RAW data stored and archived. One of the wonderful things about raw imagery and Photogrammetry is the ability to re-process datasets with new tools and know-how to achieve better results! Some of the more recent updates from RealityCapture such as "color correction" have been great at producing more consistent texture blends in complicated Photogrammetry projects, both in color as well as exposure.
Apart from using our own scans, puzzles for Puzzling Places are also made with wonderful scans from some of our Photogrammetry partners, including Gerpho3D, who capture stunning locations around France from their plane. As most of the world is still inaccessible due to travel restrictions, we're focusing heavily on creating partnerships with Photogrammetry professionals who capture interesting locations we can turn into puzzles.
Tatev Monastery by Realities.io
Our plan with Puzzling Places is to launch the game in Fall and continue developing new puzzles and gameplay features for years to come.
Mont St. Michel 3D model by Gerpho3D
We'd love to talk with Photogrammetry teams that would be interested in contributing their scans to be turned into puzzles! We're interested in having people pitch us either singular scanned locations or a set of scans centered around a location or theme (such as a "Slovakian Puzzle Pack” featuring many of its fantastic locations).
Market square Gliwice by poiformat
High-quality mesh exports from RealityCapture projects are exactly what we need to turn a scan into a beautiful miniature to puzzle together in VR. If you have a scan you think might make for a good puzzle, please contact us at firstname.lastname@example.org. Thank you RealityCapture for creating your fantastic Photogrammetry processing software. We have been frequent users of RC from the very start and are excited to see your future with Epic and Unreal Engine to come!
Realities was established in 2016 with the goal of bringing the real world into VR using Photogrammetry. With the onset of consumer PS VR headsets like the Oculus Rift and the HTC Vive, we launched our first app - Realities - which showcased our Photogrammetry scanning, processing, and optimizing for VR. It also became a platform for us to experiment with non-linear storytelling and documentary style VR experiences that go beyond 360 videos (which was the most popular framework for VR films at the time). With it, we created interactive experiences like America Deserta, Cologne Cathedral, Terres d'Exploration, and glückauf!