graviraja /MLOps-Basics

★ 8,923⑂ 1,766👁 121⑃ 24● 24▤ 0
Jupyter NotebookMITCreated 2021-05-02
+0
Daily
8,923
stars
1,766
forks
121
watchers
24
prs
24
issues
0
releases

README

MLOps-Basics

  There is nothing magic about magic. The magician merely understands something simple which doesn’t appear to be simple or natural to the untrained audience. Once you learn how to hold a card while making your hand look empty, you only need practice before you, too, can “do magic.” – Jeffrey Friedl in the book Mastering Regular Expressions

Note: Please raise an issue for any suggestions, corrections, and feedback.

The goal of the series is to understand the basics of MLOps like model building, monitoring, configurations, testing, packaging, deployment, cicd, etc.

!pl

 Week 0: Project Setup



Refer to the Blog Post here

The project I have implemented is a simple classification problem. The scope of this week is to understand the following topics:

- How to get the data?
- How to process the data?
- How to define dataloaders?
- How to declare the model?
- How to train the model?
- How to do the inference?

!pl

Following tech stack is used:

- Huggingface Datasets
- Huggingface Transformers
- Pytorch Lightning

 Week 1: Model monitoring - Weights and Biases



Refer to the Blog Post here

Tracking all the experiments like tweaking hyper-parameters, trying different models to test their performance and seeing the connection between model and the input data will help in developing a better model.

The scope of this week is to understand the following topics:

- How to configure basic logging with W&B?
- How to compute metrics and log them in W&B?
- How to add plots in W&B?
- How to add data samples to W&B?

!wannb

Following tech stack is used:

- Weights and Biases
- torchmetrics

References:

- Tutorial on Pytorch Lightning + Weights & Bias

- WandB Documentation

 Week 2: Configurations - Hydra



Refer to the Blog Post here

Configuration management is a necessary for managing complex software systems. Lack of configuration management can cause serious problems with reliability, uptime, and the ability to scale a system.

The scope of this week is to understand the following topics:

- Basics of Hydra
- Overridding configurations
- Splitting configuration across multiple files
- Variable Interpolation
- How to run model with different parameter combinations?

!hydra

Following tech stack is used:

- Hydra

References

- Hydra Documentation

- Simone Tutorial on Hydra


 Week 3: Data Version Control - DVC



Refer to the Blog Post here

Classical code version control systems are not designed to handle large files, which make cloning and storing the history impractical. Which are very common in Machine Learning.

The scope of this week is to understand the following topics:

- Basics of DVC
- Initialising DVC
- Configuring Remote Storage
- Saving Model to the Remote Storage
- Versioning the models

!dvc

Following tech stack is used:

- DVC

References

- DVC Documentation

- DVC Tutorial on Versioning data

 Week 4: Model Packaging - ONNX



Refer to the Blog Post here

Why do we need model packaging? Models can be built using

Top issues

Snapshot: 2026-11-16T09:23:14Z · Data license Apache-2.0 · JSON