As the year 2023 has started, we are happy to look at the results achieved within the Share-VDE and Share Family initiatives over the past year and share with you the goals for the next period.
Among the most recent news, it’s very important to mention that on December 7th the Share-VDE Advisory Council has approved the new Executive Summary. Following the Share-VDE Statement published in 2021, the new Summary outlines the scope of Share-VDE in the context of Linked Open Data for Libraries. We owe much gratitude to the joint effort of the Advisory Council members in writing this important document.
SVDE 2.0 system infrastructure
As you know, we are putting much effort into the work needed towards the production environment.
The adoption of AWS - Amazon Web Services was a milestone to better support system scalability, make it more robust and meet high availability expectations. Right now, we are working on the pre-conditions for the system components to progressively migrate from staging to production. To this aim, we will inform you about planned downtimes of the system in the coming period (one downtime is in course right now, see the alert message a few days ago).
Also, once the back-end environments for all the Share Family tenants will be switched to production and the module of the system managing regular data updates will be in place, the URIs of the entities stored in the various tenants will be persistent.
So far, SVDE 2.0 system processed around 65 million of MARC bibliographic records and 13 million of authority records, to create approximately 150,2 million of BIBFRAME entities. These have been created from the catalogues of SVDE institutions that we have loaded to https://svde.org [1], and for all of them a skin portal is available (see more on skin portals below).
During 2023, we will conclude this round of data ingestion and publication to the SVDE portal.
Regarding the other tenants of the Share Family: the process towards the production environment described above will take place in the same way as we are doing for Share-VDE also for the National Bibliographies tenant (https://natbib-lod.org/), the British National Bibliography skin portal (preview of its beta site: https://bl.natbib-lod.org/), and the PCC data pool that is hosted by the Share-VDE infrastructure (https://pcc-lod.org).
Also, a dedicated task group formed by volunteers PCC members is testing the PCC data pool discovery portal, and the results will greatly improve our work.
New content and overview of technical developments
Among SVDE brand new content, you might want to look at the demo of the Share-VDE entity discovery portal available at https://bit.ly/SVDE-discovery-live-demo. This walkthrough includes an introduction on the Share-VDE user experience and user interface design process. Also, supporting slides with further materials are available at https://bit.ly/SVDE-discovery-demo-slides.
In this video you will see some features of the discovery portal that is worth summarising:
- the configuration of the simple search function that enables the option of Publication simple search default, where this is relevant (e.g. in the National Bibliographies tenant and British National Bibliography skin portal);
- the initial version of Subject entity, that aggregates all subjects pertaining to a resource, with their Provenance. This will be enhanced by Concepts, ie. the pieces that form a subject string and that are currently added to the Subject pages in a rough version;
- the tracking of Provenance: the system can be queried via API to return the bibliographic records of a given Provenance (= institution) connected to an Instance, while the discovery portal shows which institutions have contributed e.g. to subjects. Provenance is also key to support the editing of linked data entities with JCricket editor, to track updates and collaborative services;
- as mentioned above, skin portals are available for individual institutions. While the main entity discovery portal of a tenant shows the data of all the institutions feeding the tenant's Cluster Knowledge Base (e.g. svde.org), the skin portal gives the ability to filter only the data of the institution that the skin portal has been designed for. To this aim, an ad hoc function was added to filter publications by what is available at the current library. It is enabled on skin portals at Publication (= Instance) level in these cases:
- in advanced search, see e.g. NYU data pre-filtered here https://nyu.svde.org/advanced-search/publications?q=(title+does_not_contain+xyz)&heldAtLibrary=true (see the toggle on the right of the screen, you can turn it on / off);
- in the Original work entity page that lists Publications, see e.g. https://nyu.svde.org/suite-de-la-mancha-flute-cello-piano-unknown-author-o781654264663247/library-publications (see the toggle on the right of the screen, you can turn it on / off);
- a caveat: this function is visible and clickable on all skin portals, but currently works on institutions’ skins that have item data clustered and rendered to the portal. As soon as other institutions’ item data will be clustered and rendered to the portal, the toggle will work there as well;
- all tenants of the Share Family can now search for classification numbers in advanced search.
Moreover, we are working to complete the connection with local library services, to enable the integration with e.g. circulation services, lending system etc.
Regarding other technical developments, the core of SVDE system, ie. the clustering module that creates clusters of BIBFRAME entities, has been reviewed based on the input that users and SVDE libraries have provided (in particular through the dedicated Sapientia Entity Identification Working Group). This revision cycle is still ongoing, and some items are being addressed by this group (see below).
Connected to that, in order to ensure flexibility in gathering users’ feedback and in reviewing the clustering logics, we are preparing a new test environment, which will immediately highlight the changes to data elaboration carried out from time to time.
Major milestones during 2023
One of the most important work strands involves the linked data entity editor JCricket. The work on the back-end APIs has been completed, while the respective front-end features are being developed. As soon as a preview of the early functions will be available, we will provide more updates on this. To give evidence to the ongoing work, we are organizing a step-by-step release method that will allow us to share progresses with the community as we go along.
Much work is also ongoing behind the scenes to complete all the pieces of the Share-VDE system pipeline, that we aim to finalise during 2023:
- the new RDF conversion tool, that will support the export of RDF data dumps (including an updated triple store database) and the import of Sinopia data to close this loop of LD4P project;
- the regular updates module (AKA “delta” updates), to enable periodic imports of MARC records for continuous update of SVDE data;
- the creation of entity clusters of items, generated from libraries’ item data;
- the development of tools / protocols for third parties’ data consumption and harvesting, including APIs, OAI-PMH, Atom feeds and Activity stream;
- the Share Family index, to set-up interconnections among the linked data entities stored in the various tenants.
Community work
Share-VDE initiative is also committed to a continuous community work, through its regular working groups and the participation to the discussion within the broader BIBFRAME community.
In the Share-VDE Advisory Council, Nina Servizzi from New York University is the new Chair starting her service in 2023, following Beth P. Camden from the University of Pennsylvania who served until 2022. We are very grateful to Beth for her wise guidance and we will continue to rely on her as Past Chair. We are very glad to welcome Nina in this role.
The SVDE User Experience – User Interface Working Group has been reconvened recently, having as co-chairs Abigail Sparling from the University of Alberta and Kayt Ahnberg from the University of Pennsylvania. This group steered the design of the current SVDE web portal a few years ago, and it’s now called to review the discovery features currently implemented for potential optimisations. These and discussions around new features for future versions of the discovery will be carried on also in cooperation with the National Bibliographies working group, that will also address discovery features specific to the National Bibliographies tenant.
The work of the Sapientia Entity Identification Working Group has among the current tasks the formalization of a SVDE ontology, and to give recommendations on the use of bf:Hub in SVDE data.
In the broader community, we are consolidating the cooperation about BIBFRAME data exchange, launched by the PCC and carried on by the BIBFRAME Interoperability Group that SVDE participate in.
Also, the continuous discussions with the Library of Congress, the PCC, the LD4P – Linked Data for Production project, and OCLC will be of great importance to contribute to BIBFRAME interoperability among linked data nodes.
Events and conferences
Finally, numerous events and conferences have taken place in the latest months where the Share-VDE team and member libraries had the occasion to contribute, and we hope for more in the future.
On January 19th Jim Hahn from the University of Pennsylvania will present at a session of the ALA/Core Linked Data Interest Group, addressing topics about BIBFRAME interoperability that also touch the SVDE experience.
The Share-VDE wiki Resources page collects the materials presented at such events.
As usual, we would be happy to have your feedback on the initiative, or any other questions or comments. Don’t hesitate to look at the SVDE wiki https://wiki.svde.org for more details on the initiative, or to contact us at [[1]].
[1] Library catalogues loaded to svde.org and converted to BIBFRAME so far:
Duke University https://duke.svde.org
Library of Congress https://loc.svde.org/
National Library of Finland https://natlibfi.svde.org/
National Library of Norway https://nln.svde.org/
New York University https://nyu.svde.org
Smithsonian Institution https://smithsonian.svde.org/
Stanford University https://stanford.svde.org
University of Pennsylvania https://penn.svde.org
University of Alberta Library https://ualberta.svde.org
