Object Storage & Blob Storage • Presigned URLs & Access ControlMedium⏱️ ~3 min
Production Scale Patterns and Traffic Offloading Economics
At production scale, presigned URLs become a critical cost and performance optimization, but the implementation patterns differ significantly between upload and download use cases. Understanding the query per second (QPS) characteristics and bandwidth economics helps you architect the right solution.
For uploads at scale, consider a social platform with 5 million monthly active users, 20% daily active, each uploading 2 photos per day at 3 MB average. That is approximately 6 TB per day ingress. Proxying through application servers means handling those 6 TB inbound from clients, then 6 TB outbound to storage, doubling your bandwidth and Network Address Translation (NAT) processing costs. With presigned direct to storage, your app servers generate URLs (handling only kilobytes per request), while storage handles the actual bytes. At managed NAT charges around $0.045 per GB, shifting 6 TB daily off your servers saves roughly $270 per day in processing charges alone, plus reduced Central Processing Unit (CPU), memory, and autoscaling pressure.
The URL generation QPS profile matters for capacity planning. For simple single part uploads, 2 million uploads per day translates to approximately 23 QPS average with 10x to 20x peak bursts. However, multipart uploads change the calculus dramatically. A 5 GB video split into 16 MB parts requires 320 signatures. If 100 users simultaneously upload 5 GB files with 512 parts each, you face roughly 51,000 signatures generated over a few minutes, spiking to hundreds or thousands of QPS on your signing service. Amazon S3 and similar systems can generate signatures locally via HMAC (Hash based Message Authentication Code) with latency under 1 to 5 milliseconds p50 (50th percentile) on commodity instances, supporting tens of thousands of signatures per second per virtual CPU (vCPU), but you must architect for this load.
For downloads at consumer scale, the pattern shifts. Many large platforms front object storage with Content Delivery Networks (CDNs) using signed URLs or cookies at the CDN layer rather than generating per object presigned URLs. This approach shifts tens of gigabits per second from origin to edge, reduces object storage egress costs (typically approximately $0.09 per GB at low tier versus materially lower contracted CDN rates), and improves latency by 50 to 150 milliseconds p95. Direct object store presigned downloads work well for moderate scale or internal users, but at millions of requests per second, CDN integration with session based signing becomes essential for both cost and revocation capability.
💡 Key Takeaways
•Bandwidth cost arbitrage: Shifting 10 TB per month of upload traffic from application servers to direct storage saves hundreds to thousands of dollars monthly in NAT, egress, and compute costs. Every byte proxied by your app is paid twice: inbound from client and outbound to storage.
•QPS planning for multipart: Single part uploads generate predictable low QPS (approximately 20 to 50 QPS for millions of uploads per day). Multipart uploads with hundreds of parts per file can generate signature QPS spikes 100x to 1000x higher. Use larger part sizes (32 MB to 64 MB instead of 8 MB to 16 MB) or session token flows to reduce signing load.
•CDN integration threshold: Direct presigned downloads work well up to thousands of requests per second. Beyond that scale, CDN signed cookies or tokens become cost effective, reducing origin egress by 80% to 95% and improving p95 latency by 50 to 150 milliseconds through edge caching.
•Latency improvements from direct upload: Removing the application server hop saves 50 to 200 milliseconds Round Trip Time (RTT) for mobile users and eliminates head of line blocking under load. Users on congested servers or distant regions see the largest improvements, often 200 to 500 milliseconds faster time to upload completion.
📌 Examples
Instagram style photo sharing: Generate presigned PUT URLs for single 3 MB to 10 MB uploads. URL generation service deployed multi region (us-east, eu-west, ap-southeast) for <30 ms p95 latency. Signing service handles 500 QPS sustained, 5000 QPS peak with 4 vCPU instances horizontally scaled. Post upload, Lambda triggered by S3 event generates thumbnails and updates database.
YouTube style video platform: For 5 GB uploads, use multipart with 64 MB parts (approximately 80 parts per video). Generate all part URLs in single API call (batch signing). Client uploads parts in parallel (8 concurrent streams), then calls CompleteMultipartUpload. This reduces signing QPS by 4x versus 16 MB parts while maintaining upload parallelism.
Netflix style CDN delivery: Private video library fronted by CDN. App validates subscription, issues signed cookie valid for 4 hours at CDN edge. Cookie grants access to /user-789/* path prefix. CDN caches video segments, only forwarding misses to origin with short lived presigned URLs. Origin egress drops from 100 TB/month to 5 TB/month (95% offload), saving approximately $9,000 monthly in transfer costs.