Skip to content

paths

paths

Resolution of the data root, the optional shared cache, and offline mode.

The data root is resolved in this order:

  1. An explicit path passed by the caller.
  2. The FWL_DATA environment variable.

There is deliberately no silent fallback: the FWL ecosystem treats FWL_DATA as required, and inventing a default location would hide misconfiguration (on clusters it would quietly download gigabytes into a per-user directory instead of the shared tree). An unset root raises :class:MissingDataRootError with instructions.

FWL_DATA_CACHE may point at a read-only, pre-populated copy of the data tree (for example a group-shared directory on a cluster). It is searched before any download is attempted and is never written to.

FWL_IO_OFFLINE disables all network access when set to a truthy value (1, true, yes, on). In offline mode a file fetch (Fetcher.fetch, Fetcher.fetch_all) raises OfflineDataError for a missing or corrupt data file instead of a download, a model fetch (fetch_for, fwl-io fetch <model>) reports each failed dataset in one RuntimeError, and a command that needs the network (sync, mirror, mirror-publish, mirror-sync, mirror-pin, check-mirrors, mirror-status, latest) raises :class:OfflineModeError before it connects.

MissingDataRootError

Bases: RuntimeError

No data root is configured: FWL_DATA is unset and no path was given.

OfflineModeError

Bases: RuntimeError

An operation needs the network while offline mode is active.

existing_data_root(explicit=None)

Return the data root, which must already exist; this never creates it.

Raises:

Type Description
FileNotFoundError

When the resolved root is not an existing directory, so a mistyped root fails clearly rather than reading as an empty tree.
Source code in src/fwl_io/paths.py 
80
81
82
83
84
85
86
87
88
89
90
91
92
def existing_data_root(explicit: str | Path | None = None) -> Path:
    """Return the data root, which must already exist; this never creates it.

    Raises
    ------
    FileNotFoundError
        When the resolved root is not an existing directory, so a mistyped root fails
        clearly rather than reading as an empty tree.
    """
    root = resolve_data_root(explicit, create=False)
    if not root.is_dir():
        raise FileNotFoundError(f'the data root {root} does not exist or is not a directory')
    return root

is_offline()

Return True when FWL_IO_OFFLINE requests fully offline operation.

Source code in src/fwl_io/paths.py 
110
111
112
def is_offline() -> bool:
    """Return True when ``FWL_IO_OFFLINE`` requests fully offline operation."""
    return os.environ.get('FWL_IO_OFFLINE', '').strip().lower() in _TRUTHY

require_online(what)

Raise :class:OfflineModeError naming what when offline mode is active.

Source code in src/fwl_io/paths.py 
115
116
117
118
119
120
def require_online(what: str) -> None:
    """Raise :class:`OfflineModeError` naming ``what`` when offline mode is active."""
    if is_offline():
        raise OfflineModeError(
            f'{what} needs the network and offline mode is active; unset FWL_IO_OFFLINE'
        )

resolve_cache_root()

Return the read-only shared cache root, or None if not configured.

The cache is configured through the FWL_DATA_CACHE environment variable and is only used when the directory actually exists.

Source code in src/fwl_io/paths.py 
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
def resolve_cache_root() -> Path | None:
    """Return the read-only shared cache root, or None if not configured.

    The cache is configured through the ``FWL_DATA_CACHE`` environment
    variable and is only used when the directory actually exists.
    """
    raw = os.environ.get('FWL_DATA_CACHE')
    if not raw:
        return None
    cache = Path(raw).expanduser().absolute()
    if not cache.is_dir():
        return None
    return cache

resolve_data_root(explicit=None, *, create=True)

Return the writable root of the FWL data tree, creating it if needed.

Parameters:

Name Type Description Default
explicit str | Path | None

Caller-supplied override. When None, FWL_DATA must be set.

 None
create bool

Create the directory when it does not exist. A caller that only reads or removes from an existing tree passes False and checks for itself.

 True

Returns:

Type Description
Path

Absolute path to the data root directory.

Raises:

Type Description
MissingDataRootError

When neither an explicit path nor FWL_DATA is available.
Source code in src/fwl_io/paths.py 
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
def resolve_data_root(explicit: str | Path | None = None, *, create: bool = True) -> Path:
    """Return the writable root of the FWL data tree, creating it if needed.

    Parameters
    ----------
    explicit : str | Path | None
        Caller-supplied override. When None, ``FWL_DATA`` must be set.
    create : bool
        Create the directory when it does not exist. A caller that only reads
        or removes from an existing tree passes False and checks for itself.

    Returns
    -------
    Path
        Absolute path to the data root directory.

    Raises
    ------
    MissingDataRootError
        When neither an explicit path nor ``FWL_DATA`` is available.
    """
    if explicit is not None:
        root = Path(explicit)
    elif os.environ.get('FWL_DATA'):
        root = Path(os.environ['FWL_DATA'])
    else:
        raise MissingDataRootError(
            'no data root configured: set the FWL_DATA environment variable '
            'or pass an explicit data_root path'
        )
    root = root.expanduser().absolute()
    if create:
        root.mkdir(parents=True, exist_ok=True)
    return root