tinker_cookbook.stores.IncrementalReader
class tinker_cookbook.stores.IncrementalReader()
Reads a JSONL file incrementally, tracking the byte offset.
Only new bytes since the last read() are parsed. Memory is bounded
to max_records — oldest records are dropped when the limit is hit.
Uses Storage.read_range() for efficient partial reads.
Thread-safe: a threading.Lock protects all mutations so concurrent
SSE connections sharing the same TrainingRunStore don't race.
Created lazily by stores to preserve pickle-serializability of the parent store object.
property records
All buffered records (copy). Bounded by max_records.
Returns: list[dict[str, Any]]
property total_read
Total number of records read since creation (not bounded by max_records).
Returns: int
property known_keys
All metric keys seen (excluding 'step').
Returns: set[str]
read()
Read new lines since last call. Returns only new records.
Returns: list[dict[str, Any]]
aread()
Async version of read().
Returns: list[dict[str, Any]]
has_data()
Return True if records exist in memory or on disk.
Returns: bool