The Wellcome Player, the Universal Viewer and IIIF

Digirati have been working with the Wellcome Library, The British Library and the National Library of Wales to further develop the viewing technolgies introduced in the Wellcome Player.

Please see IIIF and the Universal Viewer before implementing the data model described on these pages.

The data model

The Player is a viewer for ordered sequences of assets, with accompanying metadata.
A digitised book is an ordered sequence of image assets.

The core Player in the public GitHub repository knows how to play sequences of one kind of asset – a deep zoom image. In this respect it looks like an image viewer. However, other kinds of asset can be played by developing new modules for them. The Wellcome Digital Library holds audio, video and born-digital material, so the Player has a domain model that can work with any format, it’s not just for images.

The most obvious example of something to view in the Player is a digitised (i.e., scanned) book, which is a single sequence of page images from start to finish. The Player could be a viewer for just one image (e.g., a painting, which would be a single sequence containing a single image) or it could be used to view a multi-part work, each part consisting of its own separate sequence of many images. An example of the latter is a multi-volume encyclopaedia:


These parts could themselves be arranged in an arbitrary structure. A journal might be arranged into annual volumes and weekly issues, with additional separate image sequences for supplements or index volumes. An encyclopaedia or multi-volume book would be a flat list of image sequences, one for each volume. There is also structure within each image sequence, because we can divide the images (pages of a book) into logical sections (e.g., chapters).

There is structure within the images of a single sequence, and there is structure within the collection of image sequences that make up a playable work

Most objects in a library are single books or archives, i.e., works with one image sequence.

Structure serves several purposes. It allows the user to navigate around a complex work (finding a particular issue of a journal, or a particular chapter of a book). It also provides places to attach metadata such as access control information (to mark sections or parts of the structure with different sets of permissions) or other references to further information.

The Player starts with a URI from which it obtains a description of the work to be played. This URI might correspond in some way to a catalogue record number in a library, however informally that is defined.

<div class="wellcomePlayer" data-uri="http://wellcomelibrary.org/package/b18031511" />

This starting identifier is crucial in understanding the Player’s boundary. In the discussion of structures above, why can’t the Player just play an individual issue of a journal at a time? Well, logically it does – it can only present one sequence of images to the user at any one time. But the Player does not load up that single issue in isolation from all the other issues that make up the journal. A catalogue identifier or bibliographic record number might correspond to a much richer structure than just a single image sequence. The logical “thing” being viewed might have multiple separate image sequences, therefore the Player needs to allow the user to view and navigate around within an image sequence, AND offer the facility to move between the different image sequences a work comprises. The Player loads the information for a package, and then shows a particular image from a particular image sequence, which might be the default (e.g., the first image from the first image sequence) or it might be a different image from a different sequence, dictated by additional parameters passed to the Player as a URI fragment. This allows us to send a link directly to a particular section or page, but preserving the “catalogue record as URI” principle.

The Player loads a package via an HTTP request. The package contains information about asset sequences. The package gives the Player the information it needs to make further requests. A package containing multiple asset sequences is potentially very large, so the package doesn’t have to be fully populated with all the information about every asset in every sequence. It can contain links to further data about each sequence.

In the simplest scenario the package could be a static text file containing JSON, even a hand-authored text file as in the “Digitising a book” example. In practice the package data will more likely be generated dynamically on the server. The Wellcome Library have a Digital Delivery System (DDS) that integrates existing library systems with the digitisation process. It acts as the server, the Player is the client.

The domain model and METS

The Player was developed for the Wellcome Library. The Library uses METS as the output of its digitisation workflow. Although not the same as METS, METS is the biggest influence on the domain model for the package because we want a relatively simple transformation from data held in METS files to packages. Besides, METS is designed to describe works in a digital library, so we should not deviate from METS just for the sake of it. Other libraries are also likely to use METS, so conceptual alignment with METS will make it easier for others to implement their own DDS layers.

We also need to resist the temptation of just exposing the METS in a different format. The package delivered to the Player serves a different purpose. Its job is to tell the Player how to play the digital object, which is not quite the same as a complete description of a digital object. METS is verbose and loosely structured, which is a strength in describing diverse digital objects but not in delivering package data to a browser-based application.

The package has a core domain model, that should be common to any implementation, and extensions – where the package delivered to the Player has extra information that is handled by plug in modules in the Player.

There are 5 key concepts in describing a thing to be played in the Player.

Package, Structure, AssetSequence, Section, Asset

  • Package
    • (Set of 1..n) AssetSequence
      • (Set of 1..n) Asset
      • Section
        • (Set of 1..n) pointer to Asset
    • Structure
      • (set of 1..n) Pointer to AssetSequence
It’s easiest to see this without the “structural” elements. Here’s a package for a three page book:

As JSON, this looks like this:

{
  "identifier": "fig1",
  "assetSequences": [
    {
      "packageIdentifier": "fig1",
      "index": 0,
      "assetType": "seadragon/dzi",
      "assetCount": 0,
      "supportsSearch": false,
      "assets": [
        {
          "identifier": "448d8d51-7207-4efb-be00-0b4b43ab44d8",
          "order": 0,
          "orderLabel": " - ",
          "width": 2000,
          "height": 3000,
          "dziUri": "/samples/fig1/img000.dzi",
          "thumbnailPath": "/thumbs/fig1/thumb000.jpg"
        },
        {
          "identifier": "e6fd6300-f2e1-4f3b-8d8c-e6559732b360",
          "order": 1,
          "orderLabel": " - ",
          "width": 2000,
          "height": 3000,
          "dziUri": "/samples/fig1/img001.dzi",
          "thumbnailPath": "/thumbs/fig1/thumb001.jpg"
        },
        {
          "identifier": "ee5bbaf0-6402-478d-a7cd-480f93b557e5",
          "order": 2,
          "orderLabel": "page 1",
          "width": 2000,
          "height": 3000,
          "dziUri": "/samples/fig1/img002.dzi",
          "thumbnailPath": "/thumbs/fig1/thumb002.jpg"
        }
      ]
    }
  ]
}
      

If we introduce sections to divide the book into chapters, we don’t alter the flat list of assets. We assign assets to different sections elsewhere, alongside the flat asset sequence. This model borrows a linking/reference pattern from METS files. Instead of trying to allocate the actual images and their metadata to sections, it maintains all the assets used in a single flat assetSequence (e.g., all the page images) as a simple array – thereby generating a unique index number for each asset within the assetSequence. The sections within the assetSequence, which can have arbitrary nested structure (a section can contain 0..n sections), then just refer to a list of index numbers rather than a list of the actual files. This allows a file to appear in more than one section without referencing problems.

In this very simple example we have added a root section to our asset sequence (which is mandatory) and allocated all three pages to it. We have further allocated the middle page to a child section, called “Table of Contents”.

{
  "identifier": "fig2",
  "assetSequences": [
    {
      "packageIdentifier": "fig2",
      "index": 0,
      "assetType": "seadragon/dzi",
      "assetCount": 0,
      "supportsSearch": false,
      "rootSection": {
        "title": "Great Expectations",
        "sectionType": "Archive",
        "sections": [
          {
            "title": "Table of Contents",
            "sectionType": "TableOfContents",
            "assets": [
              1
            ]
          }
        ],
        "assets": [
          0,
          1,
          2
        ]
      },
      "assets": [
        // as before...
      ]
    }
  ]
}
       

There is still a logical sequence of images (assets) in a single assetSequence, even if we allocate them to sections in complex ways. A book’s pages must have some natural order. A shoebox full of archive papers has some order, even if the order is arbitrary and simply the sequence they happened to be found and catalogued in.

Similarly, the AssetSequences in a package still require some natural order, even if it is arbitrary. By analogy, if the order of an AssetSequence corresponds to the bound pages in a book, or the order a set of papers happened to have been piled in an archive box, the order of AssetSequences within a Package is the order you might shelve the volumes in, volume 1 first.

The domain model

This diagram also introduces the “SeeAlso” entity, which is an optional element that can be associated with structures, asset sequences, sections and individual assets. This allows the Player to alert the user of further information available at various levels in the package:

http://wellcomelibrary.org/player/b1988414x

The package file for this item includes the following:

"seeAlso": {
  "tag": "moh",
  "uri": "/moh/report/b1988414x",
  "markup": "Visit the Medical Officer of Health reports website to download tables and text from this report.",
  "data": {
    "shortBNumber": 1988414,
    "bNumber": "b1988414x",
    "normalisedPlace": "Port of London",
    "displayPlace": "Port of London",
    "thumbnail": "/thumbs/b1988414x/0/21b63f37-1ff2-4ec4-a3ed-27f5102f1a67.jpg",
    "title": "[Report of the Medical Officer of Health for Port of London]",
    "year": 1890,
    "yearString": "1890",
    "author": "",
    "pageCount": 82,
    "tableCount": 24,
    "publicationYear": 1890
  }
        

This domain model solves the complex problem of addressing the deep structure in a clean and simple way. To deep link to page 79 in volume 5 of an encyclopedia, you don’t need to know anything about arbitrarily complex structure in volumes and sections within a volume. You just need the volume’s index within the flat list of asset sequences, and the page image’s index within the flat list of image assets.

http://wellcomelibrary.org/osplayer/b18031511#?asi=3&ai=205

Within an AssetSequence, an asset index number always refers to a particular asset, regardless of how deeply buried it is in sections (or even if it is duplicated). Likewise, within a Package, the AssetSequence index always refers to a particular manifestation, no matter how the work as a whole is arranged into volumes, issues, supplements and so on.

Assets

The IAsset interface in the class diagram above represents the simplest possible abstract asset, but there is not enough information here to model sequences of Deep Zoom images. Assets have a family of IAsset-derived interfaces:

JSON Reference Behaviour

One requirement of the player is that it has to be able to deal with a JSON reference for an AssetSequence.

The convention we have adopted for the WellcomeLibrary is that the first assetSequence is not a reference, so that the Player can start playing it straight away without a further HTTP request. For packages with more than one asset sequence, the other asset sequences appear as JSON references. The Wellcome DDS allows this behaviour to be overridden; compare the default package for the Crick letter on this site’s home page with the “noneAreReferences” version:

Similarly compare these three views of a six volume (i.e., six asset sequence) package: (this last one is very large)

Search within

Many books on the Wellcome Library site (including the above one) are searchable. This feature is described in a bit more detail here.