Learn→ML Infrastructure & MLOps→Model Packaging (Docker, ONNX, SavedModel)→2 of 6

ML Infrastructure & MLOps • Model Packaging (Docker, ONNX, SavedModel)Medium⏱️ ~3 min

ONNX vs SavedModel: Choosing Your Serialization Format

Model Serialization Formats: Standardized ways to save trained models independent of the training code. SavedModel (TensorFlow native), ONNX (framework-agnostic), and TorchScript (PyTorch native) each make different trade-offs between portability, optimization potential, and ecosystem compatibility.
SavedModel Format
TensorFlow native format that captures the computational graph, weights, and signatures (input/output specifications). Advantages: first-class support in TensorFlow Serving with automatic batching and GPU management, direct integration with TF optimization tools (TF-Lite, TensorRT), and preservation of custom ops and training code. Limitations: TensorFlow-only ecosystem, larger file sizes than alternatives, requires TensorFlow runtime for inference. Best for: TensorFlow-native pipelines where you control the entire stack.
ONNX Format
Open Neural Network Exchange—a framework-agnostic format supported by PyTorch, TensorFlow, scikit-learn, and many others. Advantages: train in PyTorch, serve with ONNX Runtime; broad hardware support (CPU, GPU, edge devices); optimization tools work across frameworks. Limitations: conversion can lose custom operations, dynamic shapes require careful handling, not all ops have ONNX equivalents. Conversion validation is essential: compare outputs between original model and ONNX version on test inputs.
TorchScript Format
PyTorch serialization that captures the model as a graph rather than Python code. Two modes: tracing (records operations during example forward pass) and scripting (compiles Python code to TorchScript). Tracing is simpler but misses control flow; scripting handles conditionals but requires code compatibility. Advantages: stays within PyTorch ecosystem, enables C++ inference without Python, supports dynamic shapes better than ONNX. Limitations: PyTorch-only, some Python features cannot be scripted.
Decision Guide: Use SavedModel for TensorFlow pipelines. Use ONNX when you need framework portability or edge deployment. Use TorchScript for PyTorch models staying in PyTorch ecosystem. Always validate converted models against originals.

💡 Key Takeaways

✓SavedModel: TensorFlow native with best TF Serving integration

✓ONNX: framework-agnostic but requires careful conversion validation

✓TorchScript: PyTorch native with tracing (simple) vs scripting (handles control flow)

📌 Interview Tips

1Train in PyTorch, convert to ONNX, serve with ONNX Runtime

2Compare original and converted model outputs on test inputs to validate conversion

← Back to Model Packaging (Docker, ONNX, SavedModel) Overview