ML Infrastructure & MLOps • Model Packaging (Docker, ONNX, SavedModel)Easy⏱️ ~2 min
What is Model Packaging and Why Does It Matter?
Model packaging transforms a trained machine learning model into a deployable artifact with a stable interface and runtime environment. It operates at two layers: the serialization format captures your computation graph, weights, and input/output signatures, while the execution environment bundles the runtime, libraries, and system dependencies needed to run predictions.
Without proper packaging, you cannot reliably move a model from training to production. A PyTorch model trained on a data scientist's laptop with CUDA 11.8 might fail completely when deployed to a serving cluster running different library versions. Packaging solves this by creating immutable artifacts with explicit contracts. At Netflix, a recommendation model packaged correctly includes not just the neural network weights, but also metadata about input shapes (for example, user features as 128 dimensional float32 vectors), expected preprocessing (tokenization rules, normalization constants), and the exact runtime versions.
The business impact is substantial. Poor packaging leads to deployment failures, silent accuracy degradation, and slow rollouts. A properly packaged model at Uber might achieve 3 second cold starts versus 45 seconds for an unoptimized container, directly affecting how quickly the platform can scale during demand spikes. Google's TFX pipelines treat model packaging as a first class concern, automatically validating that packaged models reproduce training metrics within 0.5% before promotion to production.
💡 Key Takeaways
•Serialization formats like Open Neural Network Exchange (ONNX) use protocol buffers to store computation graphs as directed acyclic graphs with typed operators, creating framework independent artifacts typically 50 to 500 MB in size
•SavedModel is TensorFlow's native format that bundles graphs, variables, and named signatures with tight integration into TensorFlow Serving, optimized for TensorFlow and Keras workflows
•Docker containers provide immutable execution environments, with lean inference images at 100 to 300 MB enabling 3 to 5 second cold starts compared to 30 to 60 seconds for bloated training containers over 1.5 GB
•Metadata such as input shapes, data types, model version, and preprocessing contracts enables safe upgrades, automated validation, and fleet wide observability across thousands of serving instances
•Production systems at Meta and Google treat packaging as a mandatory step with automated conformance tests that verify packaged models reproduce key metrics within 0.5 to 1% of training results before deployment
📌 Examples
Netflix recommendation model: packaged as ONNX with metadata specifying input of [batch, 128] float32 user embeddings, deployed to Kubernetes pods that achieve 150 millisecond p99 inference latency serving 10,000 requests per second
Uber's ETA prediction: SavedModel artifact with versioned signatures deployed via TensorFlow Serving, enabling atomic version switches and A/B testing with 1% traffic canaries before full rollout