Forem: Harshaja Agnihotri

Streamlining ML Workflows: Automating Hyperparameter Tuning with CI/CD and DVC

Harshaja Agnihotri — Wed, 10 Dec 2025 09:23:06 +0000

Streamlining ML Workflows: Automating Hyperparameter Tuning with CI/CD and DVC

Machine Learning development is complex. Unlike traditional software, it involves managing data dependencies, model versioning, and dynamic experimentation. Implementing a robust CI/CD (Continuous Integration/Continuous Delivery) pipeline is essential to automate these experiments and ensure high-quality delivery.

In this post, I’ll walk through how to combine Data Version Control (DVC) with GitHub Actions to track hyperparameter tuning automatically.

1. The Role of CI/CD in Machine Learning

CI/CD in ML goes beyond standard code testing. It enables us to automate model building, manage data versioning, and run experiments seamlessly. By integrating tools like DVC, we can treat ML experiments (like changing a hyperparameter) just like code commits.

2. Tracking Metrics with DVC

To effectively tune hyperparameters, you need to track how they affect model performance. DVC allows you to configure your dvc.yaml file to specifically track metrics files (like metrics.json) alongside your plots.

For example, your dvc.yaml might look like this to cache false for metrics, ensuring they are always regenerated during reproduction:

stages:
  train:
    cmd: python train.py
    deps:
      - data/processed
      - train.py
    outs:
      - confusion_matrix.png
    metrics:
      - metrics.json:
          cache: false

3. Automating the Comparison (The "Diff")

The real power comes when you tweak a hyperparameter (e.g., changing max_depth or n_estimators) and rerun your pipeline. Instead of manually checking results, DVC provides a command to compare experiments directly.

dvc metrics diff

4. CI/CD Integration with GitHub Actions

You can automate this entire process using GitHub Actions. By using the iterative/setup-dvc action, you can replace manual script execution with a standardized DVC pipeline.

A typical workflow involves:

Setting up DVC in the runner.
Running the pipeline with dvc repro .
Generating a report using dvc metrics diff --md and posting it as a comment on your Pull Request using CML (Continuous Machine Learning).

Conclusion

By leveraging DVC and CI/CD, you move from "guessing" which hyperparameter worked best to having a concrete, automated report attached to every code change. This ensures that every improvement to your model is tracked, verifiable, and reproducible.

Understanding MLOps: The Bridge Between Machine Learning and Real-World Impact

Harshaja Agnihotri — Sat, 11 Oct 2025 07:42:17 +0000

🚀 Understanding MLOps: The Bridge Between Machine Learning and Real-World Impact

If you’ve ever built a machine learning model that worked perfectly on your laptop but failed miserably when others tried to use it — you’ve already met the reason MLOps exists.

In simple terms, MLOps (Machine Learning Operations) is about making machine learning actually work in the real world — not just in Jupyter notebooks.

Just like how DevOps helped software engineers ship code faster and more reliably, MLOps helps data scientists and engineers move machine learning models from experimentation to production — efficiently, repeatedly, and safely.

💡 Why MLOps Matters

Let’s be honest — training a model is often the easiest part.

The real challenge begins when you ask:

How do I deploy this model so users can access it?
How do I update it when new data arrives?
How can I monitor if the model’s performance is dropping over time?
How do I ensure the process is reliable and reproducible?

That’s what MLOps is all about.

It’s not just a set of tools — it’s a culture, mindset, and process that helps teams:

Collaborate better between data scientists and engineers
Automate the boring (and error-prone) stuff
Build trust in ML systems that evolve over time

🌍 A Real-World Example: Credit Card Fraud Detection

Imagine you’re working at a fintech company like PayPal or Stripe.

Your team builds an ML model to detect fraudulent credit card transactions. It performs great in the lab — 98% accuracy! 🎉

But here’s what happens in production:

The data pattern changes — people start using cards in new ways after a major holiday.
The model starts giving false positives, flagging real customers as fraud.
The support team gets flooded with complaints.

Now you need to:

Retrain the model on new data
Deploy the updated version
Track how performance improves or declines over time

Doing all this manually, every week, is a nightmare.

That’s where MLOps saves the day. It sets up pipelines that automatically:

Collect fresh data
Retrain and validate the model
Deploy the new version
Monitor performance metrics continuously

Essentially, it helps your ML system adapt to the real world — without constant human babysitting.

🧠 MLOps in Simple Terms

If you’re from a software background, think of it this way:

Traditional Software	Machine Learning
You write logic manually	The logic is learned from data
Code rarely changes	Models must evolve with data
Testing is fixed	Model accuracy changes over time

MLOps brings structure to this constantly changing environment by introducing:

Versioning for data and models
Automation for training and deployment
Monitoring for live model performance

It’s the difference between “We have a model” and “We have a system that keeps learning and improving.”

🏢 How Companies Use MLOps

Here are some real-world stories:

Netflix uses MLOps to continuously improve its recommendation engine based on new user behaviour.
Uber uses it for predicting estimated arrival times (ETAs) — their models update as traffic patterns change.
Amazon relies on MLOps for inventory forecasting — models are retrained daily on new sales data.
Spotify uses it to manage multiple ML models for music recommendation and user personalization.

Without MLOps, all these systems would quickly become outdated or inconsistent.

🧩 The Lifecycle of MLOps (In Plain Words)

Think of an ML project like running a bakery 🍞.

You gather ingredients (data) – the fresher, the better.
You follow a recipe (training the model) – experiment to get the perfect result.
You bake and serve (deployment) – let people actually taste it.
You gather feedback (monitoring) – did customers like it? Too sweet? Too salty?
You tweak the recipe (retraining) – keep improving based on feedback.

MLOps simply makes sure all of this happens smoothly, consistently, and automatically.

🧭 Why Developers Should Care

Even if you’re a frontend or backend developer, MLOps is increasingly becoming a part of modern systems.

Here’s why it’s worth understanding:

Collaboration → You’ll work closely with data scientists and need to understand how ML models are integrated.
Automation → The DevOps mindset extends naturally to ML — pipelines, CI/CD, and monitoring.
Career Growth → Roles like ML Engineer, Data Engineer, and MLOps Engineer are growing fast.
Real-World Impact → MLOps turns “research projects” into actual products that reach users.

🌱 How to Start Learning MLOps

If you’re new, start by understanding the concepts, not the tools.

Here’s a simple path:

Understand the ML lifecycle → data collection, training, deployment, monitoring.
Learn DevOps basics → CI/CD, containers, version control.
Study real-world case studies → see how companies apply MLOps in production.
Gradually learn tools like MLflow, DVC, Docker, and FastAPI (only after understanding the “why”).

Remember: MLOps is not just about automation — it’s about building trust in ML systems that adapt and evolve.

🎯 Final Thoughts

MLOps is the natural next step for modern machine learning — where experimentation meets engineering.

It’s what turns a great idea into a reliable, scalable product.

And in today’s data-driven world, that bridge between science and systems is exactly where the magic happens.

If you’re a developer exploring AI or a data scientist stepping into engineering, understanding MLOps will make you the glue between innovation and real-world impact.

Understanding Request Waterfalls: A Key to Optimizing Web Performance

Harshaja Agnihotri — Tue, 23 Jul 2024 07:53:54 +0000

A request waterfall, visualized as a "waterfall chart," is an essential tool in web development and performance analysis. It illustrates the sequence and timing of resource loading within a web page, helping developers diagnose performance issues and optimize loading times. Let’s dive into the components of a request waterfall and how to read and use it effectively.

Key Components of a Request Waterfall

Resource Requests:

Each row in the waterfall chart represents a resource requested by the web page (e.g., HTML, CSS, JavaScript, images, fonts).
Timeline:

The horizontal axis represents time, usually in milliseconds. The timeline shows when each request starts and ends relative to the initial page load.
Request Phases:
- DNS Lookup: Time taken to resolve the domain name to an IP address.
- TCP Connection: Time taken to establish a TCP connection between the client and the server.
- TLS Handshake: If the request is made over HTTPS, the time taken to complete the TLS handshake.
- Request Sent: Time taken to send the HTTP request to the server.
- Waiting (TTFB - Time to First Byte): Time spent waiting for the server to respond after the request is sent.
- Content Download: Time taken to download the resource after receiving the first byte.
Resource Type and Size:

Information about the type of resource (e.g., script, stylesheet, image) and its size in bytes.

How to Read a Request Waterfall

Identify Bottlenecks:

Look for long bars that indicate slow-loading resources. These can highlight performance bottlenecks. Resources that block the rendering of the page, such as CSS and JavaScript files, are particularly important to optimize.
Parallel vs. Sequential Loading:

Resources that load in parallel can improve overall load times. However, some resources may load sequentially due to dependencies (e.g., a JavaScript file that depends on another script).
Critical Path:

The critical path consists of resources that must be loaded and processed before the page can be rendered. Optimizing these resources can significantly speed up page load times.

Common Issues Revealed by a Request Waterfall

DNS Delays:

Long DNS lookup times can slow down the initial request. Solutions include using faster DNS providers or caching DNS queries.
Slow Server Response:

Long TTFB can indicate server performance issues. Optimizing server configurations, improving backend performance, and using content delivery networks (CDNs) can help.
Large Resources:

Large images, scripts, or other resources can delay page rendering. Optimizing resource sizes and using techniques like lazy loading can improve performance.
Blocking Scripts:

JavaScript files that block rendering can cause delays. Deferring or asynchronously loading scripts can mitigate this issue.

Tools to Generate Request Waterfall Charts

WebPageTest: A free online tool that provides detailed waterfall charts along with other performance metrics.
Chrome DevTools: Built into the Google Chrome browser, it includes a "Network" tab that visualizes request waterfalls.
Firefox Developer Tools: Similar to Chrome DevTools, it includes a "Network" panel for analyzing request waterfalls.
GTmetrix: Another online tool that provides comprehensive performance reports, including waterfall charts.

Understanding and analyzing a request waterfall is crucial for web developers and performance analysts aiming to optimize the loading speed and user experience of their web pages. By identifying and addressing bottlenecks, developers can significantly improve web performance and ensure a smoother, faster experience for users.