Geolens: Virtual Reality for archeologists
I was selected to go as a student abroad to the Center for Digital Media in Vancouver during the fourth semester of my computer science studies at Dania Games. Working together with archeologist Dr. Kevin Fisher from UBC’s department of Archeology we created a working prototype showing how Virtual Reality can be used as tool in archeological studies. We used VR as a way to visualize different conjecture levels of the old structures based on what is left today at the Kalavasos-Ayios Dhimitrios site on Cyprus. At the highest level of conjecture the user can experience what it was like living in the old buildings with added elements such as soundscapes and items likely to be used back in the time. The prototype can also be used as research tool by loading in data maps and viewing them directly inside the environment.
During the project I gained experience and knowledge using Unity and Virtual Reality as a platform. I also expanded on my communication skills working closely together with a team to produce a professional prototype for our client.
Table of contents
On a plane to Vancouver
Back in late 2018 I got an e-mail from the Campus Director at Dania Games, where I was currently on the third semester of my Computer Science studies. The email stated that I had been selected for the opportunity to take my forth semester as part of the Industry Project program at The Center for Digital Media in Vancouver, Canada. We had to take up the offer within a week … On January the 8th 2019, I was leaving on a jet plane towards Vancouver with three fellow students.
Arriving at the Center for Digital Media, I was presented to the team, which I would spend the next 4 months working together with. There were four talented MDM students and a faculty mentor. Two 3D artists, a project manager and a project lead. I came into the team as the lead programmer. Together we worked in an agile and professional style during the development of Geolens together with our client.
The Geolens Team.
Project requirements
The project was first introduced as an expansion to the idea of an earlier prototype. It focused on recreating the site in virtual reality but without any intractability other than moving around. The old prototype was created in a specialized engine for VR worlds. The new prototype had to be build from scratch using the Unity Engine and integrating with Steam VR. New 3D models were also needed based on the newest finds at the site. Other requirements such as those for intractability would be developed with the client as the project progressed onwards.
Development happened in close contact with the client following Scrum principles. Our product lead kept track of the important features and communication with the the client via Basecamp and at client meetings. Our project manager made sure the team always stayed healthy and happy; whether it was solving conflicts or fetching an indoor heater because the room-heating had failed. All while the our faculty mentor kept an eye on the project and made sure the client understood what could be expected of us as students during the project.
Development and Pipeline
As the only programmer on the team my role was to select, plan and estimate tasks required for each feature. At the same time I had to work together with the 3D artists to make sure our Unity pipeline was flowing smoothly. Together we gradually improved the pipeline over time. Thereby increasing the turnaround time for new features and 3D models. This pipeline was also part of the project handover documentation, so that it could be easily picked up if the client wanted to continue expanding the project.
Below are some of the different systems i have worked on throughout the project.
Movement
My only VR experience before this project was a year ago on VR Bully working with Google cardboard where the user was always standing still or sitting down. One of the first things we had to figure out was how to do movement on the HTC Vive or similar. Together with the client we experimented on different systems such as teleportation and smooth locomotion based on head direction. We ended up going with the latter for the prototype even though it might cause slight nausea. The client felt like it was necessary to make the user get a more realistic experience of the site site. To help combat the disorientation my focus was on a high fps, no stutter and a sound system that makes user feel more immersed in the virtual world.
Sound system
The client focused heavily on the effect of sound in the world. The user had to feel like they were actually there. With ICTUS Audio producing sounds for the project my job was to implement them and create interactive soundscapes as the user walked around the site.
The sound of footsteps
The site had two general material sounds: and . The right sound had to play based on which floor material was being walked on. This couldn’t be detected directly as the whole floor consisted of just one material. Instead another texture was created based on the floor plan of the site.
Since the site was modelled directly from the floor plan, the player position could be translated to the texture. A simple lookup on a map of color values was used to determine which sound should be played. Looking at the rightmost image below, you can see that the blue color indicates plaster and the red color pebble.
Site floor plan, reverb-map and footstep-map.
Different maps used by the custom sound system.
Site floor plan, reverb-map and footstep-map.
Reverberation in different areas
To really make the user feel the significance of entering different areas we had four different reverberation profiles which had to be smoothly transitioned between:
- Outside (Inverse alpha channel)
- Large open area (Green channel)
- Small semi-open area (Red channel)
- Closed room (Blue channel)
To transition between the different profiles, I used a similar process as with the footsteps. A map was drawn based on the floor plan using red, green, blue and alpha channels to linearly interpolate between profiles. The map can be seen as the middle image on the figure above. Smooth transitions are made with a gradient between colors.
Soundscapes
The final element of the sound system was to introduce different soundscapes. Some of the soundscapes were simply playing all the time in an area based on the conjecture level. Others were based on a random timer, varying slightly in pitch as they played while showing an object animation ex. window creaking.
User Interface
Geolens was created both as a way to let the user experience the site but also as a useful research tool. Together with the client we had to find new ways of combining traditional archeological studies with interactivity through virtual reality. It also had to be easy for people not too familiar with VR as platform. My role in this was figuring out how to make interactive UI systems which work well in a 3D virtual environment using the HTC Vives wands for interactions.
The conjecture and layer controller; and the info-popup.
The two main UI-elements.
The conjecture and layer controller; and the info-popup.
Info popup
One of the first UI elements I had to create was a simple information popup tool. It is used to show quick information on certain objects in the scene including text, multiple images and video. It works by using the wand as a laser beam to point on objects. Interactive objects have a small outline that thickens when pointing at them. This also triggers the popup (which is normally hidden) to be shown. Pressing the trigger while pointing, causes the popup to stay visible until pressed again.
One final feature of the info popups was the ability to save each popup. By toggling the save button, related research will open in the users browser once the Geolens application is closed.
Layer system
A lot of research in archeological studies is based on different 2D maps of the sites such as: heat-maps of activity or room importance. Our goal was to integrate these into the world and make them interactive. Together with the client we created a system where maps could be overlaid on top of the world with varying levels of transparency and different heights above the floor. The user can use their wand to pickup layers and change their heights at runtime.
Conjecture systems
Another important feature was to experience the site at different conjecture levels. The lowest level includes only the walls and items visible at the site today; the highest includes conjectured furniture and soundscapes. As the levels progress from low to high, more and more items will be added in (such as pottery found near the site).
For this I had to create a custom system for tagging different objects in the scene. The objects and sounds are enabled/disabled as different conjecture levels are selected by the user.
Other tasks
During the project I also had a few miscellaneous tasks. I was in charge of creating and maintaining the unity part of our pipeline for the 3D artists to commit to. I was also in charge of building and optimizing the light setup in the world to be fast and realistic for the virtual reality experience.
As the project neared completion, one of my last tasks was to ensure that all plugins had easy to use UI inside Unity. This meant that I had to create a few custom UI editors for some of the objects such as the info popup. I also had to create video guides on how to use Unity with our additions, to make the client feel more comfortable messing with the systems and Unity in general.
Personal takeaway
Taking part of this project was a huge learning opportunity for me. I was able to expand my knowledge about Unity and Virtual Reality as a platform, while also gaining experience working for a client together in a team.
I gained experience working on a large prototype project in Unity. Together with the team, I setup and maintained a shared pipeline using Unity Colab. I also got to work with more advanced features such as custom inspectors and their lightning system.
My knowledge of Virtual Reality platforms expanded greatly. Together with the project lead and client, we had to investigate and try different methods to ensure an optimal result in the application. For one we had to start simple and figure out a movement system that fit the clients needs but, but also without making the user feel sick. We had to find a balance between quality and optimization. Next I had to work out how to interact with Unity’s user interface systems from the VR platform; and then how to make them usable in a 3D world.
Working closely with a client as part of a team gave me lots of valuable experience with communication and working as an agile team. I had to learn doing client pitches and reaching an agreement quickly to iterate fast. I took part in planning and problem solving together with the team. All of this together with the delivery of video documentation meant that I got lots of practice and a confidence boost in my english communication.
Team members
- Project Manager
- Danka Ha
- Project Lead
- David Sigrist
- 3D artist
- Zehra Khan
- Basel Alnashashibi
- Lead Faculty
- Larry Bafia
Client
Kambe project - Kalavasos site
CDM page
Thanks to
ICTUS Audio
Center for Digital Media - Project page
View my other projects.
Featured Education