Menu

Software updates

New feature enhancement: new sFTP for publishers and Airflow

Last week, the latest DeepGreen update was rolled out to the production system. This update implemented a new sFTP for the delivery of publisher data, as well as the Airflow process monitoring tool. Airflow will make routing in DeepGreen more efficiently controllable in the future.

Based on feedback we received at BiblioCon, the update adjusted the metadata mapping to METS-MODS so that the respective corresponding authors are recognizable. MODS does not offer a predefined XML field or attribute to pass on the information about the corresponding author, so this will now be entered in the roleTerm field.

Update to New Python Version Completed

An important update for the DeepGreen data hub was completed on 25.07.2025. The production system was moved to a new server infrastructure and in this context DeepGreen was updated to Python version 3.11. This update was essential for the maintainability and stability of the service. After the maintenance work was completed, the SWORD interfaces for users were gradually reactivated.

New functional enhancement: Exclusion matching for name variants

We are very pleased to introduce a long-awaited functional enhancement for DeepGreen. From now on it is possible to exclude name variants during matching via the affiliation file in the DeepGreen account of the repositories.

If there are repeated mix-ups with a particular institution or if, for example, university XY is repeatedly assigned articles from medical university XY, university XY can now specifically exclude the name variation “medical university XY” via the affiliation file.

You can exclude name variants in the affiliation file by entering name variants in the “Excluded Name Variants” column for which you do not want to obtain any articles.

Two new functional enhancements for DeepGreen: Institutional identifiers and date filter for the routing history

We are very pleased to announce two new and long-awaited functional enhancements to the DeepGreen data hub. It is now possible to add institutional identifiers to the affiliation file of the repository accounts and to filter the routing history by date.

Institutional Identifier

In addition to name variations, domains and funding abbreviations, DeepGreen now also takes RoR and Ringgold IDs into account when matching publications.

With this enhancement, we want to make the matching process even more accurate in the future. Currently, institutional identifiers are not stored in the publisher data across the board.

We will use this functional enhancement to promote more institutional identifiers in the publisher metadata to publishers.

You can find instructions on how to store institutional identifiers in the affiliation file in our latest screencast.

Routing history date filter

It is now also possible to filter the routing history in the repository accounts by date and thus view all articles in the routing history in stages. Until now, the display was limited to 10,000 articles, which posed challenges for repositories with many deliveries. Now, by specifying a start and end date, it is possible to determine the exact period for which the routing history notifications should be displayed. This period can then be downloaded as a CSV file.

The Routing History date filter can also be used via the Web API.

 

Technical enhancements: Display ISSNs in the repository account

There is now a new technical development in the DeepGreen router. Repositories can now display the ISSNs they receive from the individual publishers in their DeepGreen account.
Clicking on “View License” after one of the licenses in the filter opens an ISSN list with the information “Title”, “Publisher”, “Embargo”, “EZB-ID”, “Print ISSN” and “E ISSN”.

It is also possible to display the list as a json record.

Technical enhancements: Display ISSNs in the repository account
Technical enhancements: Display ISSNs in the repository account

Adaptations to the Storage Infrastructure

Since the start of pilot operations in 2021, the DeepGreen router has processed approximately 1.5 million documents and a data volume of approximately 7 TB. This data continues to be held for potential new participant repositories to provide them with direct data retrieval.
In order to continue to ensure searchability and perform error-free backups of this large amount of data, the storage infrastructure was reorganized and put into a tree structure in early September.

Rollout of Metadata Mapping and Other Technical Innovations

Earlier this week, several technical adjustments were made to DeepGreen Router. The metadata mapping mentioned in the last DeepGreen consultation meeting has now been integrated into the production system. This mapping fixes several errors in the transformation of JATS files into OPUS or METS metadata formats, such as missing author names or affiliation information. As part of this effort, a tool was also developed and integrated into the user interface that allows DeepGreen administrators to test the transformations using XML files.
Another important technical innovation in this rollout is a logging mechanism that makes it possible to monitor the processing of ZIP files provided by the publishers. This now enables simple and transparent troubleshooting, which was previously very complicated and time-consuming.

New Functional Enhancements for Administration

At the end of December 2022, two new functional enhancements were added to the DeepGreen production system. They will facilitate the work of the DeepGreen employees and are not visible to users.

The first function will simplify license management. From now on, it is possible for the administrators to enter and manage the license and participant lists in the graphical interface of the DeepGreen router. A history function makes it now possible to document the different versions of the license and subscriber lists and to track the activity of the license lists.

The second function is a search function for notifications, which has also been implemented in the graphical user interface of the DeepGreen router. The DeepGreen administrators can now trace processed articles and find out whether an article was indeed delivered by the publisher.

Distinction between green and golden content now also possible with DeepGreen

Earlier this week, the next DeepGreen feature update was added to the production system. We are very pleased that with the new update DeepGreen is now able to distinguish between gold and green content within a journal. Previously, only a distinction at journal level was possible. S. Karger will be the first publisher to use this new feature. In addition to the content of the national licence and the pure gold OA journals, the gold and green content of the transformation contract will be available from Karger.

In addition, another SWORD feature is included in the new update. From now on, articles can be resent via SWORD using a button in the Routing History. In the routing history there is now an additional column “Deposit details”. In addition to the “Request deposit” button, which can be used to resend items via SWORD, this column contains information on the dispatch status of the respective item, for example, how often the item has already been sent, when it was last sent and what the current status is. It is also possible to send several items at once: Via the button “Select all on this page”, all articles on one page of the routing history can be selected and sent again via the button “Deposit selected notifications”.

The deposit logs, which were added to the “Manage SWORD Status” area with the last DeepGreen update, have also been expanded with this update (accessible via the View logs from last deposit button). Here you can now view not only the logs from the last deposit, but also from all previous deposits. 

You can find a screencast on this topic on our website:
https://info.oa-deepgreen.de/hilfe/#Screencasts

SWORD feature implemented in repositories

Earlier this week, the next DeepGreen software update was rolled out to the production system. With this update, new SWORD status management options were integrated into the repository accounts. Repository operators can now view the status of their SWORD interface in their account and list which articles were successfully delivered to their repository with the last delivery or which articles had an error. For those articles that were not delivered, they can view the error message that their repository returned to DeepGreen. In addition, repository operators can now independently activate and deactivate their SWORD interface in DeepGreen.

You can now find a screencast on this topic on our website:
https://info.oa-deepgreen.de/hilfe/#Screencasts

SWORD logging has been improved and expanded overall, so that DeepGreen administration can now better support troubleshooting.

Furthermore, the delivery method from DeepGreen to SWORD has been made configurable. Repository operators can now set in their account whether the articles (metadata, full texts and possibly additional materials) should be sent as one zip file or as individual files. We recommend “Single ZIP File” as the default setting.

A whole range of other bug fixes and change requests have been taken into account in this update:

  • The embargo date is now displayed correctly and in a separate column in the routing history of the repository accounts.
  • The display of publisher information has been corrected (if no publisher is specified in the metadata, the publisher field in the routing history is now left blank)
  • Eprints repositories can be supplied via SWORD
  • A new column for institutional identifiers has been added to the affiliation file.
  • The routing history in the repository accounts can now be sorted by different criteria

Load More