biblio-glutton-dbs-202404

Biblio-glutton database and index (2024-04 Crossref dump)

This repository contains the Biblio-glutton (https://github.com/kermitt2/biblio-glutton) databases and indexes

Due to the limitation of the HF upload size to a maximum of 50 GB, we have compressed the files into chunks.

The repository contains the following files:

• index.zip: contains the Zipped Elasticsearch index and engine.
• db: contains the LMDB fast storage databases:
  • db/crossref.z*: the crossref dump (2024/04)
  • db/hal.zip: the HAL identifiers
  • db/pmid.zip: the PMID identifiers mapping
  • db/unpayWall.zip: the unpayWall OA links (this comes from an old dump, we are planning to replace it with OpenALEX)

Getting started

Assuming you are in /home/user/glutton, you will have two directories:

• biblio-glutton: obtained by running git clone https://github.com/kermitt/biblio-glutton and containing the biblio glutton application
• biblio-glutton-index: for the Elasticsearch index, obtained by running unzip -d biblio-glutton-index` index.zip

index
├── elastic
│   ├── elastico_singleNode
│   └── elastico_singleNode.sh
└── elasticsearch-8.15.0

1. Clone the repository

git lfs install
git clone https://huggingface.co/sciencialab/biblio-glutton-dbs

1. Unzip the Index
2. TBD