This is a guest post by Anastasia Pineschi, International Dunhuang Programme (IDP) Project Manager. Find out more at the end of this piece.
As research has transitioned to online platforms, the need to adopt emerging technologies into older infrastructure has become a critical path for digital projects. The International Dunhuang Programme (IDP), formerly known as the International Dunhuang Project, had its official start in April 1994, and now in the 30th year of the Programme we continue our efforts to identify and incorporate useful digital tools onto our online environment. These efforts have culminated in the February 2024 launch of a newly redeveloped website, idp.bl.uk, which customises existing digital standards and tools to best showcase non-Western materials.
The homepage for idp.bl.uk.
Image of the IDP website’s object viewer and metadata page.
The core of the collections in the IDP database come from Dunhuang, a stop along the Eastern Silk Road. The Dunhuang materials have revealed themselves to be essential sources of information for life along the Silk Road between around 100 BCE to 1200 CE. In addition to a wealth of paintings, murals, artefacts, and coins, one of the primary draws of the Central Asian collection were the manuscripts, which showcase over twenty different languages and scripts. The materials, and indeed especially the manuscripts, occupy a variety of formats, from long horizontal and vertical scrolls to bound booklets and double-sided artefacts.
Following a period of rapid acquisitions in the early 20th century, these objects are now spread across the globe in different collections. Recognising that the geographic dispersal of these materials continued to be a barrier to research even 100 years on, the IDP launched its first website in 1998, which served images of the materials online. The website was redeveloped in 2005 and again in 2012.
The old IDP website, active from 2012 – 2024.
Knowing that over the following decade the website became in dire need of a refresh, the IDP began work on redeveloping its website in 2022. We identified four major areas of improvement:
- The Image viewer
- The search capabilities
- The contextual resources
- Architecture improvements
The Image Viewer
The IDP database contains manuscripts, fragments, artefacts, paintings, coins, photographs, and a variety of other item formats. These can range from traditional booklets and back-and front artefact shots, to incredibly long vertical and horizontal scrolls. Where standard book formats often have an aspect ratio of 2:3, the stitched image of the Diamond Sutra has an aspect ratio of 200:1. It soon became clear that a plug-and-play image viewer solution would have a difficult time properly showcasing these longer-format items.
The Diamond Sutra, Or.8210/P.2.
When considering how we wanted to redevelop the viewer, it became necessary to not only examine the different presentations of objects in the collections, but also to understand how they had been catalogued in the back-end database in order to determine which metadata elements could be helpful in customising the image display. We decided to prioritise the IIIF Mirador viewer, as it allowed for some excellent customisation to better show these objects and also implement new image standards that would increase access to across multiple technological environments.
Some highlighted improvements we have made to the viewer include:
- Vertical and horizontal viewer: The IDP viewer now responds to metadata in the database that tells it whether an object should be viewed vertically or horizontally.
An example of a vertical scroll, with the viewer orientation changed.
- Reading direction: The languages and scripts in the IDP database have been categorised based on which orientation they should be read. The viewer then adjusts the image order and displays helpful reading tips for each item based on this metadata.
- Image sets: IDP has added several image sets added to item records over time, from stitched images to verso and recto sets to scientific analyses of paper fibres. The images in these sets are now grouped and you can move between image sets easily.
The image sets oriented according to reading direction, while being clearly grouped at the bottom of the viewer.
- IIIF: The implementation of IIIF allows users to embrace established IIIF tools, such as deep zoom and accessibility of metadata through manifest downloads. We are discussing the implementation of further tools in the IIIF suite.
Custom Searching
An instance of ElasticSearch in the redeveloped IDP website, which was far more powerful and provided us some customisation options to better suit the specificity of not just the collection, but the way people searched for items. In order to deal with the discrepancies for the ways that scholars searched for items on the database, the IDP team worked to create a list of custom relationships between commonly used search terms.
To do this we established a 3-tier system for search term equivalencies:
- Basic diacritical equivalencies: This category involves stripping the diacritical marks to their foundational letter, such as ṇ being related to its base letter, n. Many of these equivalencies were addressed by the implementation of a pre-existing plug-in, ASCII Folding Filter.
- Advanced diacritical equivalencies: Here, the IDP team focused on identifying diacritical marks that require additional letters to be added in order to capture all correct formats. So even though ś could be reduced to just an s, it could also be reduced to an ‘sh’. The equivalencies here required some curatorial research, as often diacritical conventions in catalogues have shifted over time, resulting in historical uses of diacritics that also had to be captured.
- Multiple-language equivalencies: These equivalencies identify terms across the IDP collections that are directly related to each other. This could be the case for subjects, such as Avalokiteśvara = Guanyin = 觀音, or even terms that researchers have used to search for pressmarks, such as some academics using ‘p’ as a shorthand for ‘Pelliot’.
Creating custom search equivalencies that capture a variety of relationships, such as the alternative names for archaeological sites, is one of the drivers of this work.
The equivalencies list is obviously a work in progress and we continue to refine this moving forward. Usability, navigation, and accuracy have been enormous priorities throughout this redevelopment, and we have many future projects which we hope will help improve the search further.
Contextual Resources
Our team has also developed contextual resources of varying formats which provide better jumping-off points for visitors into the broad offerings of the IDP database. The first I would like to point out are the collection categories, which are new texts that provide background information to segments of the collections. Users can now use these as a point-of-entry for the collection search, with texts covering useful information for researchers like alternative names for archaeological sites, historical summaries, and location data, alongside featured items selected by our curatorial team.
An image of the collection history for Dandan Uiliq.
I would also like to highlight the learning resources, which take a selection of items across the collections to provide a more in-depth understanding of themes in the collections, such as Buddhism on the Silk Roads. You can navigate easily between subthemes, click into collection highlights which have popup information panels, and browse glossaries and definitions for common terms used throughout the resources and the collections more widely.
A selection of images from the IDP learning resources.
Architecture Improvements
The website now has a built in separation between front-end and back-end which will allow it to remain accessible should the database go down. This means we expect over a 99% uptime in line with standards for best-practice website hosting in the UK, allowing researchers more consistent access to these materials.
Because each of the founding partners in the IDP stewards its own instance of the database, it was also critical to ensure that proper synchronisation for metadata and images was captured by the new website. To facilitate a more robust back-end architecture, we are in discussions about moving the 4D database to the cloud. We are also working to develop an internationalised language version in Chinese in collaboration with our Chinese partners, so we hope to make the new version of this website even more accessible across the world.
In conclusion, the IDP has brought together partners from around the world that have significant collections from the East and Central Asian Silk Road sites, with the intention of creating an online database of these materials to unite them digitally. The Programme also promotes research into the collections through contextual resources and academic investigations, as well as providing a collaborative forum for establishing preservation and digitisation standards.
IDP Workshop in 2018
We anticipate that there will be several more projects to come as we continue to adapt our data to the modern technological landscape, from adding GIS data and site maps to establishing dynamic links between our database and our contributing partners through IIIF and other linked open data tools. The IDP is a multi-national project that is heavily reliant on digital technologies to bring together collections all around the world, and it is continually growing, evolving and adapting to contemporary situations in order to provide the best platform for these objects.
____
Anastasia Pineschi is the International Dunhuang Programme (IDP) Project Manager at the British Library. She has previously worked with institutions like LACMA, Tate Modern, the Italian-American Museum of Los Angeles, and the Getty to manage collections, developing exhibitions and organising activities.
The Digital Orientalist 