Dataset Fingerprinting and Artifact Versioning Strategies

What Dataset Fingerprinting Does

Dataset fingerprinting creates a cryptographic signature or version identifier for training data that proves two runs used exactly the same inputs. Without this, you might think you reproduced an experiment when in fact the underlying data changed silently due to schema evolution, backfill corrections, or deletion policies. Content addressing means computing strong hashes like SHA256 for datasets and using that hash as the artifact identifier. For a 100 GB dataset, you might compute per-shard hashes and store a manifest, enabling partial verification without reading everything.

Exact Snapshots vs Time Travel

The critical trade-off is exact data snapshots versus time travel reads. Copying entire datasets ensures immutability but becomes prohibitively expensive at 10 TB or larger scales. A full copy of a 5 TB dataset costs significant storage and takes hours to materialize. Time travel in data lakes and feature stores plus content hashing is cheaper, relying on underlying storage guarantees like Delta Lake or Iceberg to read data as it existed at a specific timestamp. This requires careful retention policies; if source systems delete data within your lookback window, reproducibility breaks.

Production Patterns

Uber Zipline versions feature definitions and materializations so training and serving read consistent time traveled snapshots. A marketplace model might reference a specific backfill version covering a 90 day window with explicit watermark boundaries. The pattern is to log source tables, snapshot or version IDs, query predicates, and the explicit list of sample IDs per split for perfect reproducibility.

Artifact Retention and Cost

Artifact retention quickly becomes a cost problem. Storing every checkpoint for every run leads to multi-TB growth monthly. A typical solution uses lifecycle policies: keep all artifacts for 30 days, then keep only top k performers per experiment, and deduplicate identical artifacts via content hashing. For a mid size org generating 85 GB per day, deduplication and top k retention can reduce storage by 3x to 10x.