Experiment tracking is the habit of recording what you tried, what data and code you used, what metrics you got, and which artifact was produced. MLOps expands that habit into the full production lifecycle: registry, deployment, monitoring, rollback, and CI/CD.
Why it matters
A model that works once in a notebook is not a product. Teams need to reproduce results, compare runs fairly, know which model is live, detect drift, recover from bad releases, and explain decisions to stakeholders.
MLOps lifecycle
- Experiments: log parameters, datasets, code commits, metrics, plots, and artifacts for every serious run.
- Model registry: store approved model versions with stages such as candidate, staging, production, archived.
- Deployment: serve the model through an API, batch job, edge runtime, or embedded application.
- Monitoring: track latency, errors, prediction distributions, business metrics, and data quality.
- Data drift: detect when live input data changes compared with training or validation data.
- Rollback: keep a previous stable model and deployment path ready if the new release fails.
- CI/CD: automate tests, data checks, training jobs, model validation, packaging, and staged rollout.
Key terms
- Run: one tracked experiment execution.
- Artifact: saved output such as a model file, tokenizer, config, plot, or report.
- Lineage: the connection between data, code, run, model, and deployment.
- Champion/challenger: comparing the live model against a new candidate.
- Canary deployment: sending a small share of traffic to a new model before full release.
- Data contract: expected schema, ranges, freshness, and quality rules for input data.
Production checklist
- Define the target metric, guardrail metrics, latency budget, and rollback condition.
- Version data, code, configuration, and model artifacts.
- Use a model registry so the production model is not an unnamed file on a laptop.
- Run automated tests for code, data schema, feature generation, and model behavior.
- Deploy with health checks, logging, monitoring, and an owner who receives alerts.
- Monitor drift, data quality, prediction quality, cost, and user-facing failures.
- Document human review rules for high-risk decisions.
Visual explanation suggestion
Show experiment runs as cards flowing into a leaderboard, then into a model registry. From there, animate a staged deployment, production monitoring dashboard, drift warning, and rollback switch.
Common mistakes
- Only tracking final accuracy and losing the dataset, code, or configuration used to produce it.
- Promoting a model without slice metrics, latency checks, and data validation.
- Treating monitoring as optional after deployment.
- No rollback plan. If the new model fails, the team must improvise under pressure.
- Letting notebooks become production systems without tests or ownership.
Interview-style questions
- What should be logged for every ML experiment?
- How is a model registry different from a folder of model files?
- What is data drift, and how would you monitor it?
- How would you safely deploy and roll back an ML model?
- How does ML CI/CD differ from normal software CI/CD?
Related lessons
- ML Project Lifecycle
- Docker for ML
- REST API with FastAPI
- Cloud Deployment
- Model Monitoring & Drift
- CI/CD for ML
Related project/template CTA
Use the MLOps Starter Kit or FastAPI ML Deployment Template to practice tracking, registering, deploying, monitoring, and rolling back a model.