Nowadays, virtual reality provides people with the ability to truly immerse themselves within a virtual world. With VR you can educate, entertain, create stunning presentations, give people a feeling of actually visiting places around the world, but also transform the reality and deceive the senses of the viewer. The possibilities are endless.

XRLab, a joint venture between Entropy Studio, an award-winning VFX company, and Istituto Europeo di Design, an international network of private educational institutions focused on the field of creativity, has recently participated in a project organized by Museo Nacional Thyssen - Bornemisza in Madrid, on the occasion of its 25th anniversary.

The museum had already been involved with many multimedia presentations in the past and their website offers even a 360-degree virtual tour of the museum. As part of their 25th anniversary, they asked XRLab to recreate in VR these three of the paintings in the museum:

• Van Gogh's "Les Vessenots en Auvers"
• Mondrian's "New York City, 3"
• Balthasar van der Ast's "Chinese Vase with Flowers, Shells and Insects"

Javier Garcia-Lajara, who is currently coordinating the XRLab, explains: “In the case of Van Gogh, we wanted to create an experience with 6DOF and, to improve this, we wanted to use a portal that would transport us between two worlds, one completely real and the other figurative.”

The 3D models have been created using only photogrammetry, for which RealityCapture had been chosen. In order to achieve the desired result, the processing of the whole experience has been divided into two parts.

Process

First of all, they have focused on creating a 3D model of the room itself. This was not an easy task, as the ceiling and walls did not have any distinctive features. Moreover, the lighting conditions were not favourable, as well. However, the room was successfully captured, reduced to the maximum in 3DS MAX, and imported back into RealityCapture for texture projection, in this order.

The second part were the paintings, which were the main part of the VR experience. Their frames have so many details that it would have been almost impossible to reconstruct them in such a detail without the use of photogrammetry. You can closely explore the details of the frames of the paintings which are very art pieces by themselves.

Once the room and paintings were optimized, they were brought into Unity to create the two worlds, one real and second of a fiction, and connected them together through the portal. They had also decided to use PBR so that it would respond to the user’s movement, such as reflection on the ground.

To make the feeling of entering the other world as realistic as possible, in the final application and room assembly they used an Arduino controlled sensor. It detects the moment when a person is passing through the frame, which activates a fan and creates a feeling of a breeze while the person finds himself/herself standing in the field surrounded by wildflowers. You can feel really immersed within the painting and see a flower field gently moving in the wind and clouds flying across the sky.

The whole project is a truly beautiful experience which shows the power of Virtual Reality and serves as a great example of how VR can enrich the experience of visiting galleries.