tinker_cookbook.stores.Storage

class tinker_cookbook.stores.Storage(Protocol)

Sync byte-level file I/O.

All paths are relative strings (e.g., "runs/001/metrics.jsonl"). The backend resolves them against its root.

Implementations must be pickle-serializable (for Ray/multiprocessing) — store only config (bucket, prefix, credentials path), not connections.

url(path)

Return a human-readable URI for a path.

Parameters:

• path (str)

Returns: str

read(path)

Read entire file. Raises FileNotFoundError if missing.

Parameters:

• path (str)

Returns: bytes

write(path, data)

Write data, creating parent dirs. Overwrites if exists.

Parameters:

• path (str)
• data (bytes)

Returns: None

append(path, data)

Append data to a file, creating if needed.

All backends must support this. Callers should keep individual appends small (< 4KB for POSIX atomicity). Cloud backends may implement via read-modify-write, append blobs (Azure), or internal buffering — the choice is transparent to callers.

On crash, the last append may be lost. Callers must tolerate this.

Parameters:

• path (str)
• data (bytes)

Returns: None

exists(path)

Return True if the file exists.

Parameters:

• path (str)

Returns: bool

stat(path)

Get file size and mtime, or None if missing.

Parameters:

• path (str)

Returns: StorageStat | None

read_range(path, offset, length)

Read bytes from offset. If length is None, read to end.

Raises FileNotFoundError if missing. Maps to Range GET on cloud backends.

Parameters:

• path (str)
• offset (int)
• length (int | None)

Returns: bytes

list_dir(prefix)

List immediate children under prefix. Returns names only.

For flat key-spaces (S3/GCS), this lists keys sharing the prefix up to the next / delimiter.

Parameters:

• prefix (str)

Returns: list[str]

remove(path)

Delete a file. No error if missing.

Parameters:

• path (str)

Returns: None

remove_dir(path)

Remove an empty directory. No error if missing or non-empty.

Cloud backends (S3/GCS) can treat this as a no-op since they don't have real directories.

Parameters:

• path (str)

Returns: None

flush()

Flush any buffered data to the backend.

No-op for backends that write directly (e.g. LocalStorage). Cloud backends with local staging upload buffered data on flush. Called automatically by the context manager on exit.

Returns: None