John Sigmier, Ph.D. student at Pennsylvania University, gives us an insight into an ongoing project to display 3D models of Etruscan temples in the Penn University Museum! Though not quite the Ancient Near East, this is a really interesting project that I think offers practical applications of VR for any museum exhibit.
ML: Can you briefly describe your project?
JS: The project is still a work in progress, but right now it consists of two 3D digital models of Etruscan temples that live on an augmented reality headset called a Microsoft Hololens. Our goal for the project is to present archaeological information in a way that allows audiences to understand not only the information itself, but also the nature of the sources behind it, and we’re using Etruscan architectural terracottas that are on display in the Penn Museum as a case study. These terracottas are decorated plaques that adorned the rooves of temples and protected their wooden beams from rotting in the elements, and pieces like them represent most of what survives from the temples themselves. We’re using augmented reality to reconstruct complete buildings based on the fragments on display in the Museum.
When you put on the Hololens headset in the museum gallery, you see the digital models alongside the artifacts themselves. Each different model represents a different kind of information, but they all incorporate the terracottas on display to link the models to the archaeological evidence and provide context for the artifacts. Our ultimate goal is to have a whole series of digital models that allow museum visitors to explore different aspects of the temples to go along with the objects on display.
ML: What was the impetus for creating an augmented reality display for the Penn Museum?
JS: This project came out of a digital archaeology seminar in which a recurring theme was how difficult it can be to create good visualizations of buildings that no longer exist. The power of an image is such that once you see it, it stays in your memory as something “true,” even if you know that something about it is speculative. This is a big problem for archaeological reconstructions—it’s easy to represent what we think a building might have looked like, but difficult to explain to our audiences why we think it looked that way and what we are uncertain about. In writing, we accomplish this by citing sources, which allows readers to track down evidence and assess how well it supports the argument. For visualizations, though, things aren’t as easy—annotating an image with explanatory text generally doesn’t overcome the imaginative authority that the visualization has, and manipulating the image itself to convey uncertainty (for example, by making uncertain elements slightly transparent) can compromise the visualization’s effectiveness as a representation of the material past.
We had the idea that augmented reality could be a way to get around this problem. For one, we wanted to be able to link our visualizations to the evidence that informs them, and AR allows us to display our 3D models alongside actual artifacts. The technology lets us show users the artifacts’ contexts within the reconstruction, and also illustrates how we use the parts of the temple that survive to reconstruct the whole thing. In addition, with AR we can make many different models, which means that we can have a model that shows only what we know for certain, a model that is highly speculative, and anything in between. This way, we address the authoritative image problem not by watering down our image, but by showing our viewers many images and letting them grapple with uncertainty, just as archaeologists do. We thought the Penn Museum’s Etruscan terracottas would make a great test case for our idea because quite a bit is known about Etruscan temple rooves thanks to the terracotta elements that survive, but not much is certain about the rest of the temples’ superstructures because they were mostly wood. The way that the terracottas are displayed in the Museum’s galleries made it easy to build models around them, so they offered a good opportunity to experiment with AR technology.
ML: Did you have any prior training in computer modeling, or any other field that helped you in this project?
JS: Not really—I’d done a bit of modeling before, but this project also gave me a chance to research different modeling software packages and play around with what works well for the Hololens. I have plenty more to do on that front, both in getting better with the software and in finding the best way to model and display the reconstructions.
ML: What was the biggest problem you encountered?
JS: Learning how to use new software for every step in the process has been challenging. The 3D models I built aren’t very complicated, but they still required many hours of work and many do-overs since I was unfamiliar with the modeling software. When you make a 3D model, there’s no room for ambiguity, so I had to do a lot of research and make as informed decisions as I could when I was uncertain, and then I had to figure out how to represent those decisions. After that, we had to find a way to have the models appear in the Hololens, which is something that we’ll continue to work on. We were able to get to a point where the models are stable and can be shown to visitors to the gallery, but we’d like to improve the presentation to make it more user friendly. For example, users can play around with the models by moving them or rescaling them, but in the future, we also want the models to snap to the artifacts that are on display so that it is easier to see the relationship between what is real and what is digital.
ML: How successful do you think you were at providing visitors to the museum with a better understanding of the uncertainty surrounding the reconstruction of Etruscan temples?
JS: I think we have been moderately successful so far—some of our test users immediately understood what we were trying to convey about uncertainty, whereas others were more struck by other aspects of the simulation, like how it felt to be inside a temple looking up. Of course, AR is versatile technology with a lot of potential uses for archaeology, so it’s definitely not a bad thing that our users were taking different conclusions away from the simulation. Our next steps will involve making the simulation clearer and easier to use so that the ideas that we are trying to represent are more apparent to people who are learning about Etruscan architecture or trying AR for the first time.
ML: How do you envisage this technology being used in the future?
JS: That’s really hard to predict! This was my first time using an AR headset, and I was really impressed by the experience—I’m sure that as the technology improves, it will be put to use in many applications. In archaeology, I think it has a lot of promise for all kinds of reconstructions. Certainly, AR is well-suited to the kind of thing we’ve done with models that include actual artifacts, so I expect we’ll see AR used more and more in museums and at archaeological sites. I also think AR offers some advantages over 2D reconstructions and 3D reconstructions that are displayed in a browser—it can convey a better sense of scale since it isn’t confined to a page, and it lets you explore different views that aren’t necessarily available in other kinds of visualizations. I’m also excited by the possibility of modeling in AR. For this project, I had to build my models on my laptop first, and then port them to the Hololens so that they could be displayed. If I had been able to run the modeling software on the Hololens and build the models in AR, the process would have been more efficient (some features that looked good on my laptop looked less good in the Hololens, so I had to go back and change them, which took time), and I would have had an easier time making some of those difficult judgment calls. I think that a modeling process that lets us inspect every architectural element we add at scale from all angles will really help architectural historians understand buildings better, just as sketching plans and elevations does already.
ML: Do you have plans to expand this project, and is your virtual temple going to be made a more permanent feature of the Penn galleries?
JS: We do plan to keep working on the project—our next big goals are to add more models to represent more levels of uncertainty and to tinker with the user interface to make the simulation easier to understand. We’d like this to become a fixture in the galleries, but we’re not yet sure what form that will take. Some of what we do will be driven by improvements to the technology—the next version of the Hololens, for example, is supposed to offer a wider field of vision than the version we’ve been using does, which will really open up what we’re able to display. We’ve also been looking at smartphone-based AR as an alternative to the Hololens. The AR headset offers a cool immersive experience, but for most people it is new technology; smartphones, on the other hand, are familiar to most visitors to the Penn Museum, so having a simulation that works on a phone might remove a big technological barrier to entry for our audience. We’re looking forward to seeing what we can do!
Thank you very much to John for this fascinating interview on the importance of uncertainty, and for the fantastic images he provided. This is an incredibly exciting project, and I think make a very important point about how we present ancient artifacts to the public.
The Digital Orientalist 