IMMERSIVE EXPERIENCE - 1 (IE1)
Ajanta Immersive Experience
One of the exhibits at the Virtual Experiential Museum on Ajanta currently live at the National Museum, New Delhi, India. The IE 1 lets you teleport through the caves of Ajanta with the help of a head mounted Virtual Reality Device and an application made with Unity 3D. These cave experiences will give you an understanding of the scale and grandeur of the cave architecture. When you step into Cave 10, you will surely notice the simple octagonal pillars, the apse like shape of the cave and the sacred stupa at the back. As a virtual tourist, walk the pradakshina route round the stupa as did Buddhists who made these caves about 2000 years.
How we did it?
Our Configuration:
Immersive Experience - 1 (IE1) consists of a VR headset :
HTC VIVE or Oculus Rift S,
powered by a computer,
CPU : Intel i9 9900K, GPU : NVIDIA RTX 2080 .
Creating the experience:
To give the users an experience of the caves at Ajanta was a challenge. Our team of artists and photographers visited the site, took thousands of photos of each cave, and used Photogrammetry to create the 3D models. These photogrammetry models were then taken into Unity and with the use of Steam VR, a basic cave viewing application was made.
The Experience:
Visitors interact with the exhibit with the head-mounted display and handheld controllers.
A 65-inch 4K display (ie1.display) mirrors the view from the head mounted display of the VR headset for onlooking visitors.
The IE1 consists of two modes through the user can experience the exhibit.
A. Explore Mode:
In this mode the user can select a Cave from the menu and then explore that cave by following these instructions:
> Tap once to activate teleport
> Move controller up/down to adjust teleport distance
> Release tap to teleport
> Press trigger to select cave
> Point and press touch pad to choose cave
> Move controller up/down to adjust teleport distance
> Release tap to teleport
> Press trigger to select cave
> Point and press touch pad to choose cave
B. Guided Tour:
In this mode, the user can sit back, relax and the Virtual Tour Guide will take him around the caves, the user interacts with Gaze Pointer to control the show.
CASE STUDY
CASE 1: Designing the application, working out the user interactions and improving the teleportation system:
After importing the photogrammetry 3D models into Unity, the major challenge was to create an app which lets the user explore the caves, see paintings and sculptures inside them and get information about each cave and the elements in it. We started of by creating a simple teleportation system where the user presses a button on the controller which shows up a parabolic curve and at the end of the curve is dummy platform which indicates the final position at which the user will get teleported.
Next we built a cave selection menu which the user can use to select a different cave.
We added ambient music, audio narrations and information labels which helps the user gain more information about the caves and the elements inside it.
Major concern was to teach the users how to experience the whole application for that, we implemented a simple step by step in application tutorial where in the user learns by performing each action and then implementing them to enhance their experience.
CASE 2: Fixing the issues with the Photogrammetry model, lighting and the floor:
The photogrammetry 3D models contained a lot of unwanted objects such as cables and lights used in the actual caves which were to be removed in blender, once this was done and the models were imported in Unity the lighting and textures were all messed up. In order to fix all this we simply imported the models with unlit textures and all the embedded lights were replaced.
Some of the caves in Ajanta have an uneven floor surface, this made the user experience bad while teleporting, for this we created surface like colliders which ensured the user landed on the right regions.
CASE 3: Adding Spatial Audio and enhancing the overall experience:
To give the users a good immersive experience, ambient music and sound effects were a must. Our team of sound designers went to the actual site, collected audio cues from various locations inside and outside the caves and created a beautiful Spatial Audio. The major task was to include the audio in the app at different locations and creating a very realistic surround sound experience. For this, we used audio mixers in Unity, troubleshoot issues with audio source, modified the sound curve etc and hence made the overall experience very immersive.
CASE 4: Implementing the Guided Tour:
After the initial user testing we found that there was a lot of information for the users to process and the time for each user is limited hence we planned of having a Guided tour where in a virtual guide with the help of audio and text narration would give users an all around experience of the cave in a limited time. The interaction works this way, the guide starts the tour and moves around the various major attractions inside the cave, with the help of a gaze pointer, the user can move close to the guide, listen and read the narration and take a walk through around the caves.
