Learn→Object Storage & Blob Storage→Multipart Uploads & Resumable Transfers→4 of 6

Object Storage & Blob Storage • Multipart Uploads & Resumable TransfersMedium⏱️ ~2 min

State Persistence and Cross-Session Resumability

What State to Persist
Resumability requires tracking upload state across client restarts. The minimum state: upload ID, file path, total file size, part size, and a list of completed parts with their ETags. The upload ID identifies the session on the server. Completed parts list tells the client what to skip on resume.
Store state to persistent storage before starting each part. If the client crashes mid part, that part is not recorded as complete and will retry. This is idempotent: uploading the same part twice produces the same result. The server overwrites the previous attempt.
State Storage Options
Local file: Write JSON to .upload-state/{uploadId}.json. Simple, works offline. Delete after successful completion. Risk: local storage fills up with orphaned state files from abandoned uploads.
Database: Store in a uploads table. Enables cross device resume: start on laptop, finish on phone. Requires network access to retrieve state. Adds dependency on database availability.
Server side: Some platforms provide list parts API. Client queries server for completed parts instead of tracking locally. Simplest client implementation but requires network round trip before resuming.
Handling File Changes
If the source file changes between sessions, uploaded parts become invalid. Detect this by storing file modification time and hash. On resume, verify the file unchanged. If changed, abort the old upload and start fresh. Uploading parts from different file versions corrupts the final object.
For streaming sources (data generated on the fly), checksums per part enable verification. Store the checksum with part state. On resume, regenerate the part data and verify checksum matches before skipping.
Upload Expiration
Servers expire incomplete uploads after a timeout, typically 7 days. Parts uploaded before expiration are deleted. The upload ID becomes invalid. Clients resuming after expiration must detect this (server returns 404 for the upload ID) and start fresh.
Store upload start time with state. Warn users approaching expiration. For long running uploads (days of intermittent progress), consider completing partial uploads and starting new ones before expiration.
💡 Key Insight: Persist state before each part upload, not after. This ensures crashes leave state consistent with server. On resume, query server for parts list to reconcile any state drift.

💡 Key Takeaways

✓Minimum state: upload ID, file path, total size, part size, completed parts with ETags

✓Persist state before starting each part, delete state after complete; crashes leave consistent state

✓Three storage options: local file (simple), database (cross device), server query (simplest client)

✓Detect file changes via modification time or hash; abort and restart if source file changed

✓Server expiration (typically 7 days) invalidates upload ID; client must detect 404 and restart

📌 Interview Tips

1Walk through the resume flow. On restart, read state file. Query server list parts API. Compare with local state. Upload any parts in local state but not on server (in flight when crashed). Upload remaining parts. Complete.

2Explain file change detection. Store {uploadId, filePath, fileSize, modTime, sha256, completedParts}. On resume, check modTime and optionally sha256. If different, the file changed, abort upload.

3Discuss the expiration edge case. User uploads 90% over 6 days. On day 8, server has deleted everything. Client gets 404, must restart. Solution: complete partial uploads before expiration, use object versioning to update later.

← Back to Multipart Uploads & Resumable Transfers Overview