sync
sync
Registry generation from the Zenodo API (fwl-io sync).
For every dataset in a manifest, query the pinned Zenodo record and rewrite the committed registry file with the record's file names and checksums. This is the only place where hashes enter the repository, so registry changes are always visible in review.
Concept DOIs are rejected here: a concept DOI resolves to the newest deposit of a record (the API answers with a redirect to the latest version, so the returned record id differs from the requested one), which would let the data change underneath pinned code. Only version DOIs are accepted.
All datasets of a manifest are attempted; per-dataset failures are collected and raised together at the end, so one bad entry does not leave the rest of the registry set unwritten.
fetch_zenodo_record(doi, api_base=ZENODO_API)
Return the full Zenodo API record for a pinned version DOI.
A concept DOI is rejected: the API resolves it to the newest deposit (the returned record id differs from the requested one, or equals the concept record id), which would let the data change underneath pinned code.
Source code in src/fwl_io/sync.py
45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 | |
fetch_zenodo_registry(doi, api_base=ZENODO_API)
Return the name-to-checksum mapping of a pinned Zenodo record.
Source code in src/fwl_io/sync.py
66 67 68 69 70 71 72 | |
sync_dataset(dataset, api_base=ZENODO_API)
Regenerate the committed registry file for one dataset.
Source code in src/fwl_io/sync.py
75 76 77 78 79 80 81 82 83 | |
sync_manifest(manifest_path, api_base=ZENODO_API)
Regenerate the registries of every dataset in a manifest.
Every dataset is attempted; if any fail, the successful registries are still written and a single aggregated error is raised at the end.
Source code in src/fwl_io/sync.py
86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 | |
zenodo_record_id(doi)
Return the numeric record id of a Zenodo version DOI.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
doi
|
str
|
A Zenodo DOI of the form |
required |
Returns:
| Type | Description |
|---|---|
str
|
The trailing record-id digits. |
Raises:
| Type | Description |
|---|---|
ValueError
|
When the string is not a Zenodo DOI. |
Source code in src/fwl_io/doi.py
16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 | |