scverse_misc.datasets.parse_registry

Contents

scverse_misc.datasets.parse_registry#

scverse_misc.datasets.parse_registry(path)#

Parse a YAML registry into (base_url, {name: DatasetEntry}).

The YAML has a top-level base_url (or s3_base_url) and a datasets mapping of name -> {type, files: [{name, url?/s3_key?, sha256?, fallback_urls?}], ...}. Any keys other than type and files are collected into the entry’s metadata.

Return type:

tuple[str | None, dict[str, DatasetEntry]]

Examples

base_url: https://example.com/data

datasets:

  example1:
    type: anndata
    files:
      - name: example1.h5ad
        s3_key: ABCDEFGH.h5ad
        sha256: 86126c1a3c163ea20abb14c1a9711aaff34e6c492ef6dd86298bbaf18cc3f5f3
        fallback_urls:
          - https://example.org/fallbackdata/ABCDEFGH.h5ad
          - https://example.net/data/ABCDEFGH.h5ad

  example2:
    type: anndata
    files:
      - name: example2.h5ad
        url: https://example.com/otherdata/DEFGHI.h5ad
        sha256: e7b358fef2b6da115b7ee3ab4c7fc55fd80e210bba427902e4afc9118992747c
Contents