Forem: Metta Surendhar

Tired of Writing the Same Tests Again and Again? Meet Keploy

Metta Surendhar — Tue, 09 Sep 2025 21:26:27 +0000

When I first started learning about APIs and CRUD operations, I tested everything manually. I would run the application, perform an action in the UI or call an API, and then check if it behaved correctly. At that point, I didn’t even know what unit testing or or API tests was.

Later, I realized that writing test cases is crucial. They help catch bugs early, reduce the risk of regressions, and give confidence when making changes. However, there was one big problem: writing tests felt repetitive and time-consuming. Instead of focusing on building features, I was spending hours writing boilerplate code for tests.

That’s when I came across Keploy, a tool that automatically records API calls and generates test cases and data mocks. This means you can test your application without writing traditional test scripts.

Why Automated Testing Matters

Before diving into code, let’s briefly revisit why automated tests — whether unit tests or API tests — are important:

Catch bugs early – Problems are detected before deployment.
Prevent regressions – Ensures that new changes don’t break existing features.
Improve confidence – Developers can refactor code without fear.
Save time in the long run – Manual testing becomes unnecessary for routine checks.

The only challenge? Writing and maintaining tests manually is tedious. This is where Keploy steps in.

What is Keploy?

Keploy is an open-source testing toolkit designed primarily for API testing. It helps you automate validation without writing traditional test scripts:

Records API calls and responses while you use your application.
Automatically generates test cases in YAML format.
Creates mocks for external dependencies (databases, third-party APIs).
Provides a simple test mode to replay requests and validate responses.

Think of Keploy as a record-and-replay testing tool. You interact with your app once, and Keploy creates reusable API-level tests for you.

Step 1: Build the To-Do List App

Before we dive into Keploy, let’s first build a simple To-Do List application. This app will serve as the foundation on which we’ll later generate automated test cases.

We’ll use:

Flask → A lightweight Python web framework.
SQLite → A simple database to store tasks.
Flask-SQLAlchemy → ORM (Object Relational Mapper) for managing database operations.
Jinja2 Templates → To render HTML pages.

The app is a classic CRUD (Create, Read, Update, Delete) example, which makes it a perfect candidate for learning automated testing.

Features

Our To-Do List app will support the following:

Add tasks: Enter a task and save it into the database.
View tasks: See all tasks with their creation time.
Update tasks: Edit an existing task’s content.
Delete tasks: Remove tasks you no longer need.
API endpoints: Access the same functionality programmatically with REST APIs.

This way, we’ll have both a web UI and a set of APIs for interaction.

Project Structure

D:.
├─ static
│  └─ css
├─ templates
├─ tests
└─ app.py

app.py – Main Flask application containing:
- Database model (Todo)
- UI routes (for web pages)
- API routes (for REST APIs)
templates/ – HTML templates used for rendering pages:
- base.html → Shared layout for all pages.
- index.html → Main page showing the task list.
- update.html → Page for updating an existing task.
static/css/ – Custom CSS for styling the UI.
tests/ – Placeholder directory where automated test cases will be generated later using Keploy.

Database Model

Inside app.py, we’ll define a simple Todo model using SQLAlchemy.

id: Unique identifier for each task.
content: The actual task description.
date_created: Timestamp of when the task was added.

Task List UI

Once you start the app and add some tasks, the home page (index.html) will display them in a neat table:

Column 1 → Task description.
Column 2 → Date created.
Column 3 → Update/Delete actions.

At the bottom, there’s a form input to quickly add new tasks.

✅ At this point, we’ve built a complete Flask To-Do List app with both UI and APIs. This will now serve as the base project for integrating Keploy and generating test cases automatically.

Step 2: Run the Application

You can clone the project from [To-Do-List-Flask] and follow the README to initialize and run the app.

Install virtualenv:

$ pip install virtualenv

Open a terminal in the project root directory and run:

$ python -m venv .venv

Then run the command:

$ .venv\Scripts\activate

Then install the dependencies:

$ (.venv) pip install -r requirements.txt

Finally start the web server:

$ (env) python app.py

This server will start on http://127.0.0.1:8080/ by default

Step 3: Install Keploy

If you’re on Windows, you’ll need WSL (Windows Subsystem for Linux).

Initialize wsl:

C:\Users\Dell> wsl
unix@DESKTOP:/mnt/c/Users/Dell$

Install command:

$ curl --silent -O -L https://keploy.io/install.sh && source install.sh

Check Installation:

$ keploy -v

👉 For Linux/macOS, follow the Keploy installation guide

Step 4: Record with Keploy:

Now comes the fun part. Keploy has a record mode that listens to API calls while you use your application.

Run:

keploy record -c "python app.py"

Now, perform some actions through both the UI and APIs.

UI Actions Captured

Create a task

View tasks

Update a task

Delete a task

Terminal Output

API Actions Captured

Create a task

curl -X POST http://localhost:8080/api/tasks \
  -H "Content-Type: application/json" \
  -d '{"content":"blog demo task"}'

Update a task

curl -X PUT http://localhost:8080/api/tasks/1 \
  -H "Content-Type: application/json" \
  -d '{"content":"updated task"}'

View tasks

curl -X GET http://localhost:8080/api/task

Delete a task

curl -X DELETE http://localhost:8080/api/tasks/1

Terminal Output

Each of these interactions gets stored as a test case in YAML format inside the keploy-tests/ folder.

Step 5: Run Tests with Keploy

Once tests are recorded, run them anytime with:

$ keploy test -c "python app.py"

Keploy will:

Replay all recorded requests.
Compare the actual responses with recorded ones.
Generate a detailed test report.

This ensures that your API behaves consistently over time.

Generated Test Cases:

Generated Reports:

Benefits of Using Keploy

Adopting Keploy in your development workflow brings several advantages:

Automatic Test Generation
- No need to manually write repetitive test cases.
- Keploy records your real API traffic and generates test cases in YAML format.
Keeps Tests Up-to-Date
- Whenever your API changes, just record again.
- Test cases evolve with your application, reducing maintenance overhead.
Data Mocks for External Services
- Keploy automatically generates mocks for databases, third-party APIs, or external dependencies.
- This allows tests to run reliably without depending on live external systems.
Language & Framework Agnostic
- Works with popular backend frameworks like Flask, Django, FastAPI, Spring Boot, Express.js, and more.
- Flexible enough to integrate into diverse tech stacks.
End-to-End Coverage
- Captures both UI-driven actions and API requests.
- Provides comprehensive testing without extra setup.
CI/CD Integration
- Generated tests can be run in pipelines.
- Ensures every deployment is validated with the same rigor as your local environment.

Conclusion

Automated testing is critical for building robust, bug-free applications, but writing and maintaining test cases can often feel repetitive and time-consuming. This is where Keploy changes the game.

By automatically recording real-world API interactions and generating tests with data mocks, Keploy ensures:

Your APIs behave consistently.
Tests evolve naturally with your application.
Development cycles become faster and more reliable.

In this blog, we built a Flask To-Do List app and saw how easily Keploy can record interactions, generate API test cases, and validate our endpoints. The key takeaway is:

👉 With Keploy, you record once and test forever.

If you’re working with Flask, Django, FastAPI, or any modern backend, I highly encourage you to give Keploy a try. It’s a huge productivity booster and ensures your applications remain reliable as they grow.

Final Year, No Guarantee - What It's Really Like Looking for a Job

Metta Surendhar — Fri, 05 Sep 2025 18:08:44 +0000

How tough is getting a job?

How stressful is getting a job? How much preparation is needed? How much time should I spend? How much better should I become? Am I ready? Will I be suitable for this job? Should I apply or not?

These are the questions running through my brain often. And honestly, I escape from them by procrastinating—scrolling through social media or distracting myself with other work instead of doing what’s needed.

Yes, I know this isn’t the right way. But the truth is, I don’t really know what I should do or how I should handle it.

The irony? I actually have good stories and experiences to share. I’m capable of competing. I’ve done the work needed for job seeking and building a career. All I need to do is put everything I’ve done over the past years together and prepare. But right now, it feels like a burden—or sometimes even useless.

Why? Because I’m not getting opportunities in on-campus placements. Off-campus companies reject me right at the resume shortlisting round. And the job market itself isn’t great right now.

Most companies seem to prefer experienced people. And even when there are fresher openings, many of them are purely for BE and BTech graduates.

Where I Stand ?

I’m not an Engineer, but I study in one of the top engineering colleges in India: College of Engineering, Guindy (CEG), Anna University (AU). It’s a pioneer institution, built during the British era, with a long history and many achievements.

Technically, I’m not called an “engineer” because of my course. But the reality is—I follow almost the same syllabus, build the same skills, and work on similar projects as BE CS and BTech IT students. The only difference is the title.

I’m not even a scientist, though my degree is M.Sc. Information Technology, an integrated course at CEG. Professionally, I don’t fit into the “scientist” label either, since our course doesn’t involve research. Instead, it’s packed with the technical learning, tools, and hands-on practice that an engineer or IT professional would need.

The tough part? This course is managed by the Mathematics Department. So we don’t get CS scholars or experienced professors—only teaching fellows, most of whom aren’t strong enough to guide us properly.

Because of this, our course doesn’t get proper on-campus placement opportunities. Most companies reject us immediately after seeing the course name. From the placement cell’s side, there’s also little to no support, even though we’ve been requesting it for years.

The frustrating part is that our students are equally skilled, and some are even better than BE CS and BTech IT students in terms of projects and internships. Yet, we don’t get proper recognition or placements.

With all this happening, I honestly don’t know how to face the situation.

Have you ever felt like no matter how much effort you put in, the system just doesn’t see it?

My Journey So Far

From my 2nd year onwards, I started taking small steps for the sake of my career and professional profile.

I created my LinkedIn account, began following students, alumni, professors, and tech pages.
I started learning web development—first simple frontend projects, then functional pages. I built my portfolio and worked with a college team as a frontend developer.

During the transition from 2nd to 3rd year, I learned version control, handling packages, and frameworks. I moved into backend development, did a full-stack project, and took on more backend roles in different projects.

By 3rd year, I was deeply interested in backend development. I worked on a college project as a backend developer for 3 months. At the same time, I also took up responsibilities:

I served as the General Secretary of my college, co-organizing events and fests, and voicing student concerns to the management.
I led a team of 5 on an alumni platform project, managing the entire backend myself.

I barely had personal time, but I didn’t mind. I loved what I was doing and stayed true to my responsibilities. Isn’t it funny how when you truly enjoy something, you forget about time?

Between 3rd and 4th year, I had to get an internship as part of my curriculum. During the internship drive, I attended only one company’s process—and I got selected.

In 4th year, I worked as a Platform Engineering Intern at Invisibl Cloud Solutions. Some of my alumni were there, so I got the chance to connect with them. I learned a lot—new tools, frameworks, and technologies—throughout those six months. Around this time, I also started attending tech meetups, made new connections, and gained great experiences.

After the internship, I continued with my course. Around then, I took responsibility

For sponsorship and logistics for my department symposium Mathrix, as the Head of Industry Relations.
I worked hard, brought in sponsorships, managed logistics, and actually enjoyed the whole process.

Where I’m Stuck Now

Now, I’m in my final year, preparing for placements. Alongside that, I’m handling two responsibilities:

I’m the Placement Representative, responsible for handling placements and bringing in companies.
I’m also one of the Heads of Marketing and External Relations at Guindy Times, our college media club.

With all this going on, I still want to learn ML—but right now, I feel stuck and don’t know how to move forward.

I know some of you reading this might be in the same state as me. And some of you may have already been through this and moved forward. If so, please share your experiences—it might help me, and others like me. After all, sometimes hearing someone else’s journey is all the push we need.

This Event Cleared My Doubts About MLOps, DevOps & Platform Engineering - OpsFusion 2024 Recap

Metta Surendhar — Sat, 12 Jul 2025 20:32:21 +0000

Recently, I had the opportunity to attend OpsFusion: Where Dev Meets ML—a technical meetup that brought together practitioners and enthusiasts across DevOps, MLOps, and Platform Engineering. The event was an excellent blend of hands-on sessions, real-world experiences, and emerging trends across these intersecting domains.

In this blog, I’ve shared a structured summary of each session, along with key takeaways that resonated with me.

MLOps in Vertex AI – by Navaneethan Gopal

This session focused on building end-to-end machine learning pipelines using Vertex AI, with a specific emphasis on automating the ML lifecycle beyond model development.

Key Highlights

The demonstration used a multi-class classification problem (Dry Beans dataset) developed in Google Colab using Gemini for code assistance.
It was emphasized that less than 1% of MLOps involves actual ML code. The remaining majority lies in operations—such as infrastructure, orchestration, testing, and monitoring.

Core Components of MLOps

Data collection and validation
Model training and testing
Debugging and analysis
Model monitoring post-deployment
Cross-functional collaboration

MLOps Lifecycle Phases

Discovery – problem and data exploration
Development – feature engineering, dataset versioning, and integration with feature stores
Deployment – serving the model through automated pipelines

Maturity Levels in MLOps

Level 0: Manual build and deploy
Level 1: Automated training workflows
Level 2: Fully automated and reproducible pipelines across environments

Vertex AI Pipeline Overview

The speaker provided a walkthrough of how to build and deploy a Vertex AI pipeline triggered from Bitbucket or a cronjob. The steps included:

Creating a GCS (Google Cloud Storage) bucket
Defining dataset and training components using XGBoost
Initializing and deploying the pipeline via SDK integration

Emerging Operations in ML

FMOps (Foundation Model Operations): Managing LLMs, latency, token usage, and cost
LLMOps: Operations tailored to Retrieval-Augmented Generation (RAG) and large language models
PromptOps: Monitoring and optimizing prompt performance and hallucination tracking

Kubeflow

Introduction to Kubeflow as a Kubernetes-native platform for ML workflows
Creating custom components and reusable pipelines

This session bridged the gap between foundational ML and scalable production pipelines, highlighting the growing need for robust, reproducible ML systems.

Trunk-Based Development with Terraform – by Harini Muralidharan

This session covered the developer-driven DevOps model, focusing on enabling application developers to define and manage infrastructure using Infrastructure as Code (IaC).

Context: Challenges in Traditional DevOps

Frequent inconsistencies between dev and production environments
Developer reliance on operations teams for even minor infrastructure changes
Lack of visibility and traceability in changes made to the system

Principles of Developer-Driven DevOps

Developers define and version infrastructure alongside application code
Early detection and mitigation of issues via automation
Promotes ownership without expecting developers to become operations experts

Introduction to Terraform

The session provided a deep dive into Terraform, its ecosystem, and how it enables scalable infrastructure on GCP.

Why Terraform?

Open-source and cloud-agnostic
Declarative syntax (HCL)
Native support for GCP
Strong community adoption and extensibility

Core Components

Providers: Connect Terraform with cloud services
Resources: Define infrastructure components
Variables & Outputs: Parameterization and visibility
State Management: Track infrastructure state across teams

Common Workflow

terraform init → terraform plan → terraform apply → terraform destroy

Integrating Terraform with CI/CD

Using CI/CD pipelines (YAML) to automate Terraform commands
Promotes consistent, reliable infrastructure changes with version control

Best Practices

Store code in Git with proper version control
Use remote state storage (e.g., GCS or Terraform Cloud)
Follow the principle of least privilege
Modularize Terraform codebases for reusability
Perform automated testing on infra modules
Monitor for configuration drift and enforce corrective actions
This talk emphasized the benefits of empowering developers while maintaining operational integrity, security, and scalability.

This session bridged the gap between foundational ML and scalable production pipelines, highlighting the growing need for robust, reproducible ML systems.

Platform Engineering vs DevOps: Evolution or Revolution? – by Crystal Darling

This session helped clarify the difference between DevOps, SRE, and the growing field of Platform Engineering.

Challenges in Traditional DevOps

Operations teams are often blocked by development timelines
Developers submit tickets for operational support, resulting in slow turnaround
Limited autonomy in environments, infrastructure, and tool usage

What Is Platform Engineering?

The practice of building and maintaining Internal Developer Platforms (IDPs)
Platform engineers build self-service tools and abstractions for developers
Treat developers as clients, providing them with consistent and secure environments

Key Platform Engineering Skills

Kubernetes orchestration
IaC tools like Terraform and Helm
CI/CD systems
CNCF tooling for observability, deployment, and monitoring

Core Message

Platform Engineering is not a rebranding of DevOps. It is a cultural and architectural evolution focused on developer experience, autonomy, and scalability.

Discussions on ML Research and Networking

The event concluded with group discussions on recent research papers from Microsoft and Google—specifically those related to Copilot, RAG, and the inner workings of generative systems.

It was a highly engaging session where I got to connect with fellow learners, exchange ideas, and hear how others are applying these concepts in real-world environments.

Closing Thoughts

Attending OpsFusion gave me a broader and more integrated view of how software systems are evolving—whether it’s about scaling ML models through MLOps, automating infrastructure with Terraform, or building robust internal platforms that make developer lives easier.

If you're someone who is navigating the intersection of ML, infrastructure, and deployment—or wants to bridge the gap between development and operations—events like these are immensely valuable.

Building Real-World Infrastructure as a Fresher: My Story with Logs, AI & Observability

Metta Surendhar — Sun, 29 Jun 2025 11:30:57 +0000

I'm thrilled to share that I’ve successfully completed my six-month internship(June 2024 – December 2024) as a Platform Engineer at Invisibl Cloud Solutions!

What started as an exploration of unfamiliar tools and domains quickly turned into one of the most fulfilling technical journeys I’ve had so far.

From building a log observability infrastructure to developing an AI-powered research agent, this internship helped me grow technically, professionally, and personally.

Internship Experience: Learning, Growth & Gratitude

The internship was hybrid in nature—split between working from home and at the office workspace. What made the experience even more special was that many of the senior staff were alumni from our college, making the workspace incredibly friendly and collaborative.

On the days I went to the office in person, I would ride along with Dinesh Kumar—it took us nearly an hour to reach the workspace, and we had some of the best conversations along the way.
Sometimes, we’d reminisce about college days, exams, and projects. Other times, we’d discuss our ongoing work, explore technologies, and talk about careers, placements, and what the future holds. Those morning rides were truly special—casual, thoughtful, and always enriching.

I personally loved going to the office because many of our seniors—alumni from our very own course—would be there. Since we shared that common ground, we had so much to talk about. Whether it was clearing doubts, learning about the industry, or just general chit-chat, they always made time for us.
During lunch, we’d all sit together, gossip, joke around, and just have fun. Looking back, those were some of my favorite memories—I genuinely miss those days.

Beyond office hours, I was deeply focused on learning and growing. Over the six months, I attended technical meetups, joined bootcamps, started blogging, and participated in hackathons. These experiences helped me not only sharpen my skills but also connect with the broader tech community.
All of this complemented what I was learning at Invisibl Cloud, helping me grow both in depth and in direction.

During my first three months, I worked on an Observability Infrastructure Project, where I dove deep into system logs, tools like Cribl and Grafana, and built a full-stack monitoring setup. Then, I transitioned to a Generative AI-based project centered around intelligent research paper discovery using RAG.

I’m proud to share that the demos for both projects received positive feedback from the client, which was deeply satisfying, especially considering both domains were completely new to me when I started.

Gratitude

I owe a huge thank you to the entire Invisibl Cloud Solutions team for this enriching opportunity.
A heartfelt thank you to Harish Ganesan, CEO of Invisibl Cloud Solutions, for not only trusting me with impactful work but also giving me the opportunity to work on the Gen AI project. His involvement and encouragement were truly motivating.
A big thanks to VijayRam Harinathan for his support and mentorship in the observability project—his feedback and belief in my work made a huge difference.
Special appreciation to Farhana S, whose consistent mentorship helped me navigate the observability space for the very first time.
I'm equally grateful to Suryaa Azhakhiamanavalan for his guidance on the Generative AI project. His mentorship turned this challenge into a rewarding experience.
And of course, Harshita Miranda, my project partner from day one. Working with her on both projects was a joy—we shared ideas, solved challenges together, and supported each other throughout.
Lastly, shoutout to my amazing friends who interned alongside me—Dinesh Kumar, Sree Varshan M, and Harini S. You all made the workspace vibrant and the learning process fun!

Project 1: Building Observability Infrastructure for System Logs

Objective: To extend the existing metrics-based monitoring stack by incorporating log observability across Windows and Linux systems.
Tech Stack: Grafana, Loki, Cribl Edge, Cribl Stream, rsyslog, Prometheus
Key Concepts: Log collection, log routing, centralized logging, visualization

As someone new to observability, I began with research into best practices and tools. The organization already had metrics monitoring, and I was tasked with building the logs monitoring infrastructure from scratch.

Windows Log Monitoring

Leveraged native Event Logs (Application, System, Security).
Collected logs using Cribl Agent.
Processed and routed them through Cribl Edge and Cribl Stream.
Stored in Grafana Loki.
Visualized using Grafana dashboards, with alerting and filtering options.

Linux Log Monitoring

Linux was more challenging due to the absence of structured default logs.
Created a Ubuntu virtual machine.
Researched and implemented rsyslog to generate logs in custom templates.
Integrated the logs into the same Cribl → Loki → Grafana pipeline.

Outcome: Successfully built and delivered a cross-platform proof of concept for full-stack log observability, integrated seamlessly into the existing infrastructure.

Project 2: Generative AI Research Agent

Objective: To build an intelligent AI agent capable of retrieving and summarizing research papers based on user queries.
Tech Stack: Haystack, FastAPI, Streamlit, Python, Arxiv API, Gemini, OpenSearch
Key Concepts: Agent pipelines, Retrieval-Augmented Generation (RAG), API development, LLM integration

In the second half of my internship, I worked on this exciting project with one other teammate. The goal was to help researchers find academic papers faster and more efficiently using Generative AI.

My Contributions

🌟Integrated the Arxiv API to fetch relevant research papers.
🌟Designed the agent pipeline using Haystack and Gemini, implementing RAG to combine retrieval with generation.
🌟Stored extracted data in OpenSearch for quick and context-aware access.
🌟Built a Streamlit-based POC to demo the functionality.
🌟Later developed a FastAPI version for production-level usage.

The first month of development was incredibly intense—we often worked for over 10 hours a day to shape the prototype. With consistent support and motivation from Suryaa Azhakhiamanavalan and Harish Ganesan, and after multiple review meetings and revisions, we (myself and Harshita Miranda) were able to complete the proof of concept within the first month.
Even though the pace felt heavy at the time, it turned out to be one of the most rewarding learning experiences of my internship.

Outcome: Delivered a fully functional prototype in under a month, and then enhanced it into an API-ready microservice with scalable architecture.

Key Takeaways

This internship gave me a crash course in:
✔️Observability tools and infrastructure, from system logs to dashboard visualization.
✔️Generative AI workflows, agent chaining, and RAG pipelines.
✔️Real-world problem solving across two very different but equally challenging domains.
✔️Working in a collaborative team, presenting demos to clients, and adapting to fast-paced learning curves.
✔️Most importantly, it showed me the importance of taking initiative, asking the right questions, and owning the full cycle of a product — from idea to implementation.

Final Thoughts

Looking back, I’m proud of how much I was able to learn and build in just six months. The trust, guidance, and opportunities I received from Invisibl Cloud Solutions shaped this internship into something I’ll always remember.

From configuring log protocols on Linux to chaining LLM agents for intelligent research—this journey has been transformative. I’m grateful for every challenge, every lesson, and every teammate who made it all worthwhile.

Hackathon Realities: What It's Like to Build, Code & Ship in a Weekend

Metta Surendhar — Thu, 16 Jan 2025 14:00:00 +0000

I had the honor of leading my team in Hackz 2024, an intense 24-hour hackathon that stretched from 10 AM on November 23rd to 10 AM on November 24th. The event was buzzing with energy, creativity, and an overwhelming sense of purpose. With over 1000 teams registered, 500+ submitted their ideas, and only 20 were selected for the final round, our journey to the top felt nothing short of incredible.

What made it even more special? We were the only team from the College of Engineering, Guindy (CEG) to make it this far—a proud moment for all of us.

Despite the hackathon being hosted at our college, many teams from CEG had submitted ideas, but we were the sole representatives from our campus in the finals. Representing CEG among teams from diverse colleges and states was both a responsibility and a privilege.

The Challenge: Building Financial Inclusion for Seniors

Our problem statement was as compelling as it was challenging:

Develop an AI-powered financial inclusion platform designed for elderly individuals to simplify digital banking and financial planning. The platform should support voice commands, provide timely alerts for financial milestones, and offer tailored scam protection. By addressing digital literacy challenges, this solution aims to enhance independence and promote safe, accessible engagement with financial services for seniors.

When we read this, we knew this was more than just a technical challenge; it was an opportunity to create a meaningful impact. With the rise in digital banking and scams targeting vulnerable populations, creating a tool to empower the elderly felt deeply significant.

Our Solution: A Fintech Platform for Empowering the Elderly

After a whirlwind of brainstorming, designing, and coding, we presented our prototype, packed with features designed specifically for seniors:

✔️ Voice Commands: For intuitive, hands-free interactions—perfect for users unfamiliar with complex interfaces.
✔️ AI Chat Assistant: A personalized guide to help with financial queries and planning.
✔️ Scam Protection Education: To safeguard users against fraud and teach them to spot red flags.
✔️ Expense, Savings, and Investment Insights: Tailored recommendations to support better financial management.
✔️ Ease of Use: Every feature was designed with accessibility and simplicity in mind to bridge the digital literacy gap.

My Role as Team Lead: A Learning Curve

As the team lead, I had the responsibility of steering our efforts. Coordinating a team in such a high-pressure environment was both thrilling and demanding. I learned the importance of quick decision-making, fostering collaboration, and staying calm under pressure.

I’m incredibly grateful to my teammates—Harini S., Sundar Balamoorthy, and Adhithya—for their dedication and hard work. Sundar and Adhithya were new to hackathons and development, but they stepped up brilliantly, proving that a willingness to learn and contribute matters more than experience. I hope this journey encourages them to take part in more hackathons.

Hackz 2024 was no ordinary hackathon. It brought together diverse minds from across states, creating a vibrant melting pot of ideas and innovation. The energy in the room as we worked through the night, fueled by adrenaline (and coffee!), was unlike anything else.

We worked tirelessly for 24 hours, juggling ideas, implementing features, and debugging issues, but it was all worth it. By the end, we had a working prototype—a tangible result of our collaboration and effort.

While we didn’t win, being among the top 20 teams out of 500+ submissions was a milestone we cherished.

I want to express my heartfelt gratitude to:

Navaneethan, our mentor, for his unwavering support and guidance. His encouragement kept us motivated till the very end. Navaneethan, I’m sorry we couldn’t bring home a trophy, but I’ve learned so much from you, and your insights will stay with me as I take on future projects.

CSEA for organizing such a well-structured and supportive event. From managing logistics to ensuring participants were cared for, they truly went above and beyond.

Temenos for sponsoring Hackz 2024 and fostering innovation among young developers.

Takeaways and Reflections

Hackz 2024 wasn’t just about creating a product; it was a journey of growth and discovery. Here’s what I’m taking away:

🌟 Leadership Lessons: Leading a team under a tight deadline taught me how to manage people, tasks, and time effectively.

🌟 Problem-Solving Skills: Tackling real-world challenges pushed me to think creatively and practically.

🌟 Understanding Expectations: Getting feedback from judges and mentors helped me understand what it takes to impress industry experts.

I also realized that mistakes are stepping stones to growth. Each bug we fixed, each feature we struggled to implement, and every moment of doubt taught me something valuable.

Hackz 2024 has left me inspired and more determined than ever to create impactful tech solutions. The journey doesn’t end here; it’s just the beginning. I plan to build on these experiences, improve my skills, and continue pushing boundaries.

To anyone considering participating in a hackathon: go for it! You’ll leave with more than just technical skills—you’ll gain memories, friendships, and a sense of achievement that’s hard to match.

Once again, thank you to everyone who made this journey unforgettable—my team, mentor, organizers, and sponsors. Here’s to many more hackathons and challenges ahead! 💪

What IBM's SRE Expert Wants You to Know About Observability - A Beginner's Guide

Metta Surendhar — Thu, 09 Jan 2025 14:00:00 +0000

During the Grafana and Friends Meetup in Chennai, I had the opportunity to attend an insightful session by Manojkumar, an SRE professional from IBM. His talk centered around observability and how IBM tackles real-world challenges using Grafana and AI.

Out of all the sessions that day, this one stood out as my personal favorite, and I couldn’t wait to share some key takeaways here!The talk covered four critical components in modern observability systems: logs, metrics, traces, and profiling.

These make up the foundation for any robust observability setup, and he explained how each one plays a role in monitoring and troubleshooting large-scale infrastructures.

The Observability Stack - Logs, Metrics, Traces, and Profiling

1. Logs:

Logs are often the first step in diagnosing issues.
They provide a granular record of everything happening within the system, from user activities to errors.
At IBM, logs are used to trace the precise sequence of events that can lead to potential failures or performance degradation.

2. Metrics:

Metrics come in when you need to track the overall health of your system.
By monitoring things like CPU usage, memory consumption, and response times, metrics give a top-level view of how different components are performing.
While logs help you understand the "what" and "when," metrics help you catch patterns before they escalate into critical issues.

3. Traces:

Traces become vital in distributed systems where a single request might travel through multiple services.
IBM uses traces to monitor each step of the request path, allowing them to pinpoint bottlenecks and understand complex interactions between microservices.

4. Profiling:

Profiling takes observability to the next level by digging into the code execution itself.
It’s useful for spotting inefficiencies in resource usage (like CPU or memory) at a granular level, making it easier to optimize and fine-tune system performance.
Profiling provides the precision needed to identify which parts of the code need optimization, especially in performance-critical applications.

Real-World Challenges & Solutions in Observability

Manojkumar didn’t just talk theory — also shared practical challenges he faced and the solutions implemented using Grafana and AI. Three problems, in particular, stood out:

Problem 1: Missing Logs in the Centralized Logging System

One of the biggest challenges they encountered was missing logs in their centralized logging system. Relying on CloudWatch metrics alone led to gaps in visibility, which made it hard to troubleshoot incidents.

To close the gaps, they decided to incorporate ElasticSearch metrics alongside CloudWatch data. This approach gave them a more comprehensive view and reduced the chance of missed log entries, ensuring no critical data was lost in the process.

Problem 2: Where to Start Diagnostics?

With data pouring in from multiple sources—Prometheus, MySQL, Oracle, AWS, Azure—it can be overwhelming to know where to begin diagnosing a system issue.

The team built a collective dashboard that aggregates data from all these different sources. This unified view streamlined their diagnostics process, allowing them to get a clearer picture faster. Instead of hunting for data in different places, everything was available in one interface, which reduced the mean time to recovery (MTTR).

Problem 3: Multiple Alerts for a Single Issue

Receiving multiple alerts for the same underlying issue was a common problem, leading to alert fatigue. This made it difficult to focus on the real issue amidst the flood of notifications.

By utilizing LLMs (Large Language Models) and KNN (K-Nearest Neighbors) algorithms, they were able to intelligently group alerts. The system consolidated related alerts into one primary notification using AI-driven operations through ClickHouse , drastically cutting down on unnecessary noise. This way, the team could focus on solving the root cause without getting overwhelmed by redundant alerts.

Why This Talk Stood Out for Me

As someone deeply interested in observability and system health, Manojkumar’s talk felt incredibly relevant and timely. I’ve worked with observability tools like Grafana and Cribl, but seeing how IBM integrates AI to enhance monitoring was eye-opening. Their ability to handle large-scale infrastructure challenges using observability and AI offered a glimpse into the future of system monitoring.

The solutions they’ve implemented—whether it's creating multi-source dashboards or using AI for alert grouping—demonstrate how powerful modern observability tools have become. It also reinforced the idea that observability is not just about collecting data; it’s about making sense of it efficiently to keep systems running smoothly.

His talk has inspired me to dive even deeper into observability. In the coming weeks, I’ll be exploring more advanced Grafana features and tools like LGTM Stack (Loki, Grafana, Tempo, Mimir) and Cribl for smarter log management.

Stay tuned as I continue this journey into understanding how we can use observability to improve system reliability and performance.

Cribl & Grafana: Build a Full Observability Pipeline From Scratch

Metta Surendhar — Thu, 02 Jan 2025 14:00:00 +0000

Data is the pulse of any system, and effectively managing it can bring significant value to your business. In this blog, we'll guide you step-by-step through setting up Cribl Edge for data collection, Cribl Stream for processing, and Grafana for visualizing your metrics. Whether you're new to Cribl or looking for a refresher, this guide will have you up and running in no time.

Here’s what we'll cover:

Setting Up Cribl Agent for Data Collection
Configuring Cribl Edge to Send Data to Cribl Stream
Processing Data with Cribl Stream
Utilizing Data in Grafana

Let’s dive in!

Step 1: Setting Up Cribl Agent for Data Collection

Efficient data collection is the first step towards real-time system monitoring. Cribl Edge helps you capture system metrics and logs from multiple sources and send them to Cribl Stream for processing.

Follow these instructions to install and configure Cribl Edge on Linux and Windows systems.

1.1 Create an Account in Cribl Cloud

Before we begin, we need to set up an account in Cribl Cloud:

Sign-Up Process: Go to Cribl Cloud and create an account.
Login: After signing up, log into Cribl Cloud with your credentials. The Cribl Cloud will be your primary interface for managing Edge nodes and data pipelines.

⚙️ Note: For learning purposes, we will use Cribl Cloud to manage our data collection agents.

1.2 Access the Edge Fleet

Navigate to Edge : After logging in, select the “Manage” button in the Cribl Edge section.
Fleet Overview: This will redirect you to the Edge page, where you can see a list of fleets and analytics
Navigate to Default Fleet: In Cribl Cloud, only one fleet (default_fleet) will be available by default. Click on default_fleet to view the monitoring data for Edge nodes, sources, and destinations.

1.3 Add an Edge Node

Edge Node Overview: Edge nodes are responsible for collecting and sending data from your system to Cribl Stream
Edge Node Installation:
- In the default_fleet page, click the "Add/Update Edge Node" button in the upper right corner
- Choose an environment (Linux or Windows) where you want to install the Cribl Edge agent.
- Linux Edge Node:
  - Hover over the Linux tab, click "Add", and copy the installation script.
  - Run the Script: Open a terminal and execute the script as the root user.
  - Start and Verify: After installation, ensure the agent is running with the command: systemctl status cribl.
- Windows Edge Node:
  - Hover over "Windows" and click "Add" to view the command prompt and PowerShell scripts.
  - Modify the Script: Edit the script by changing "/qn" to "/q" to ensure the installation runs in the foreground.
  - Run as Admin: Run the script with administrator privileges to install the agent

1.4 Check the Data Flow

Verify Agent Installation: Once the Edge node is installed, monitor its status in the Cribl Cloud by navigating to the Edge Node Monitoring page.
Real-Time Data Monitoring: Under Edge Fleet → default_fleet → Overview → Monitor, you can view metrics such as events in and bytes in to verify that the Edge node is collecting data.
List View for Health Status: Use the “List View” to check the health and status of each Edge node

Step 2: Configuring Cribl Edge to Send Data to Cribl Stream

Once the Cribl Edge agent is installed and collecting data, you need to configure it to send the collected data to Cribl Stream for further processing

2.1 Configure Source in Cribl Edge

Sources Overview: Data sources represent the type of data being collected (e.g., system metrics, logs).
Navigate to default_fleet → More → Sources to add a new data source. Depending on your environment, configure one of the following:
- Windows Metrics:
  - Enable the in_windows_metrics source and configure it by setting host metrics to "All."
  - Set processing settings with Fields to a field name and value like observ_data = 'edge_win_metrics' , Preprocessing Pipeline to passthru and Connect Destination set to Send to Routes.
  - Commit and deploy the changes.
- System Metrics:
  - Enable the in_system_metrics source and configure processing settings.
  - Set Fields to a field name and value like observ_data = 'edge_lin_metrics' , Preprocessing Pipeline to passthru.
  - Ensure Connect Destination set to Send to Routes and commit/deploy changes

⚠️ Note:

Can’t enable both Windows and Linux sources in the same fleet simultaneously.

The destination can also be connected via interface using quick connect for more details check the docs.

2.2 Configure Destination in Cribl Edge

Go to default_fleet → More → Destinations to add a new destination.
Use Cribl TCP as the destination for both Windows and Linux sources:
- Set a unique output ID (e.g., cribl_system).
- Enter the IP address from Cribl Cloud’s Access Details (can be get from your cribl cloud → Access details → Ingress IPs).
- Enter the port number (e.g., 10300).
- Commit and deploy the changes.

2.3 Verify Source and Destination Configuration

Verify that both source and destination are enabled (indicated by a check mark).
If there’s an issue (indicated by a cross mark), check the logs to resolve configuration errors.

2.4 Create the Data Route

Route Overview: The data route links the source (e.g., Windows or Linux metrics) to the destination (Cribl Stream).
Route Configuration:
- In Cribl Stream, go to default_fleet → More → Data Routes.
- Create a new route that links the source and destination:
  - Name the route and set filter expressions (observ_data == 'edge_win_metrics' for Windows and observ_data == 'edge_lin_metrics' for Linux) to ensure only Windows/Linux metrics are sent through this route.
  - Set the pipeline to passthru (default pipeline that doesn't modify data) and output to the Cribl TCP destination created earlier (cribl_tcp:cribl_system).
  - Save the changes and Commit and deploy it to activate the route.

2.5 Capture and Verify Data Flow

Status Check: Use the source and destination status and chart pages to view live data
Capture Events: Monitor live data capture in source, destination and the data route.
Verify Routing: Ensure that data flows seamlessly from source to destination by capturing data in the data route as well.
Troubleshoot: If data doesn’t flow as expected, check the logs in Cribl Edge for potential configuration errors.

Step 3: Processing Data with Cribl Stream

Now that Cribl Edge is sending data to Cribl Stream, the next step is to configure Cribl Stream to receive, process, and route this data.

3.1 Configure Source in Cribl Stream

Setting Up the TCP Source: Cribl Stream needs to listen for incoming data from Cribl Edge via a TCP connection.
- Navigate to Cribl Stream → Default → Data → Sources and add a source.
- Select Cribl TCP Source to match the configuration of the Cribl Edge TCP destination.
- Add a new source with a unique input ID, set the IP to bind to the edge which will be in default 0.0.0.0, and configure it with the same port used in Cribl Edge (e.g., 10300).
- Commit and deploy the changes.

3.2 Configure Destination in Cribl Stream

Destination Configuration: The processed data will be sent to Grafana using Prometheus Remote Write.
- Go to Cribl Stream → Default → Data → Destinations and select Prometheus destination.
- Create a new destination with a unique input ID like prometheus-output.
- Set the remote write URL , get the Prometheus Remote Write URL from your Grafana Cloud account (found under Prometheus → Send Metrics → Write URL )
- Commit and deploy the changes.

3.3 Create a Processing Pack

Processing Packs: A processing pack in Cribl Stream allows you to create modular pipelines to filter, enrich, or modify data before it reaches its destination.
- Go to Cribl Stream → Default → Processing → Packs and add a pack.
- Create a new pack (e.g., Cribl-Windows-Metrics).
- Use functions and routes within the pack to process data via adding a pipelines.
- For more details on packs and pipelines, refer to Cribl documentation

3.4 Configure the Data Route

Create a Data Route: Similar to Cribl Edge, create a data route that links the TCP source to the Prometheus destination.
- In Cribl Stream, go to Default → Routing → Data Routes and add a route.
- Set filter expressions based on the source tags (observ_data=='edge_win_metrics').
- Link the pack (Cribl-Windows-Metrics) to the source and set the output to Prometheus (prometheus:prometheus-output).
- Commit and deploy the changes.

3.5 Verify Data Flow

Monitor Event Flow: Use the data capture and status pages in Cribl Stream to verify that events are flowing correctly from the sources to the destinations.
Capture Data: Monitor live data for around 50 minutes and ensure the data is being processed and sent to Grafana.
Troubleshoot: If data doesn’t flow or doesn’t processed as expected , check the logs for potential configuration errors.

Step 4: Utilizing Data in Grafana

Once the data has been processed by Cribl Stream, you can visualize it in Grafana.

4.1 Create a Dashboard in Grafana

Log in to Grafana Cloud: If you don’t have an account, sign up at Grafana Cloud.
Create a Dashboard: After logging in, go to Create Dashboard and add a new panel.
- Data Source: Set the data source to Prometheus.
- Query Configuration: Use PromQL queries to retrieve data from Prometheus. For example, windows_cpu_percent_active to visualize CPU usage.
- Customize the Panel: Give the panel a meaningful name (e.g., Windows CPU Metrics).

4.2 Fine-Tuning Visualization

Panel Customization: Adjust time ranges, choose chart types (line, bar, etc.), and set thresholds for key metrics.
Multiple Panels: Add panels for different metrics (memory, disk usage, network I/O).
Deploy Dashboard: Save and deploy the dashboard for real-time monitoring.

4.3 Monitoring and Analyzing Data

Real-Time Data: Grafana will now display real-time metrics based on the data collected, processed, and routed from Cribl Edge and Stream.
Alerts and Notifications: Set up alerts in Grafana based on threshold values (e.g., high CPU usage).

Conclusion

And there you have it! By following these steps, you can successfully set up Cribl Edge for data collection, Cribl Stream for processing, and Grafana for visualizing the data. This guide provides a foundation for customization of your data pipelines, allowing you to monitor, process, and visualize large-scale metrics effectively.

In the next post, we will dive deeper into the detailed steps for creating dashboards, panels, and alerts in Grafana. Stay tuned!

New to Monitoring? Start With Grafana - Easy Steps to Visualize & Alert

Metta Surendhar — Thu, 26 Dec 2024 14:00:00 +0000

Let’s dive into one of the powerful tools in the observability stack—Grafana. Whether you’re monitoring system health, exploring logs, or tracing requests, Grafana acts as your all-in-one observability superhero. It helps you visualize, analyze, and act on your system’s data to gain deep insights, identify issues, and optimize performance.

Why Grafana?

Grafana is much more than just a dashboard—it’s a tool that empowers you to understand your system in real-time. Here’s why Grafana deserves a spot in your observability toolkit:

💠Visual Storytelling at its Best

Grafana transforms raw data into meaningful visualizations.
From graphs to dashboards, you can reveal patterns, trends, and potential bottlenecks that would otherwise stay hidden.
It brings your data to life, giving you the insights you need to make informed decisions.

💠Data from (Literally) Anywhere

Grafana integrates with a wide array of data sources—such as Prometheus, InfluxDB, Elasticsearch, and more.
No matter where your data comes from, Grafana brings it together in a unified view, allowing you to leverage existing data without switching tools or infrastructure.

💠Serious Analytics Power

Beyond visualizations, Grafana allows you to run complex queries, set custom alerts, and build tailored dashboards.
Whether you’re tracking metrics or responding to incidents, Grafana equips you with the analytic power to uncover actionable insights.

💠Open-Source Love ❤️and Community-Driven

As an open-source project, Grafana benefits from a vibrant, active community.
This means continuous improvements, access to hundreds of plugins, and endless community-driven resources to help you get the most out of Grafana.

Setting Alerts in Grafana 🔔

One of the most powerful features Grafana offers is alerting. It enables you to proactively respond to issues by setting up alerts that notify you when key metrics reach certain thresholds. Imagine being alerted to high CPU usage or memory exhaustion before it impacts your application. With Grafana, you can:

Create alert rules based on metric thresholds.
Choose alert notifications — via email, Slack, Webhook, or other platforms—so you never miss a critical event.
Define conditions that trigger alerts for the exact scenarios you want to monitor.

Setting alerts helps you catch potential problems early, ensuring your system is always performing at its best.

Silence When You Need It 🤫

Alerts are essential, but sometimes you need a little quiet—especially during planned maintenance or when you’re troubleshooting known issues. That’s where Grafana’s silencing feature comes in.

With silences, you can:

Temporarily mute alerts for specific services, applications, or instances.
Avoid unnecessary notifications during deployments or maintenance windows.
Ensure critical alerts still get through by selectively silencing less important ones.

Silencing ensures that you’re only notified of issues when it truly matters—no more alert fatigue!

My Journey with Grafana and Observability

As someone who recently stepped into the world of observability, Grafana quickly became my go-to tool. While initially focused on metrics, I soon realized the true power of Grafana lies in its ability to combine logs, traces, and metrics into a single, cohesive observability platform.

Here’s how I’ve leveraged Grafana in my own projects:

I connected Prometheus to Grafana to track key system metrics like CPU usage, memory consumption, and latency from Windows systems.
I explored logs with Loki, which allowed me to visualize log data, trace errors, and pinpoint issues from Linux and Windows systems.
Using Grafana Alerts and silences, I set up proactive monitoring that helped me identify and address issues before they affected users.

Grafana has transformed the way to approach observability, giving me the tools to see the big picture and act on the small details. It’s not just about visualization—it’s about control, insight, and continuous improvement.

Start Your Grafana Journey Today!

Ready to embrace the power of Grafana? Here are a few steps to help you get started on your observability journey:

💠 Download and Install

Grafana is easy and straightforward to install. Download the appropriate package for your operating system and follow the setup instructions.

💠 Connect Data Sources

Configure Grafana to connect to your data sources, whether it’s Prometheus, Loki,Tempo or others, and start building custom dashboards. Check out the Grafana documentation for detailed setup guides.

💠 Explore Features

Dive into Grafana’s powerful features, from visualizations to alerting and beyond. The Grafana community is full of plugins and tutorials to help you master the tool.

💠 Set Alerts and Silences

Set up your alerts and don’t forget to use silences when necessary, so you can stay ahead of any issues without being overwhelmed by notifications.

Grafana isn’t just a tool—it’s your partner in observability. It empowers you to visualize, act, and optimize your systems, all in one powerful platform. So, why wait? Start your Grafana journey today, and unlock the full potential of your data!

Stay Tuned!

In upcoming blogs, we'll delve deeper into creating effective Grafana dashboards, setting up alerts, and exploring more advanced features to enhance your observability capabilities. Don’t miss out!

Why Your System Needs Observability - The Beginner Guide Every Dev Should Read

Metta Surendhar — Thu, 19 Dec 2024 14:00:00 +0000

Ever wondered how tech giants keep their systems running smoothly even when handling millions of users? Or maybe you're curious about how you can ensure your own projects are rock solid? The answer lies in a little magic called Observability—and today, we’re going to dive right into it!

What’s the Buzz About Observability?

Imagine you’re debugging your code without any tools—no console logs, no debuggers, nothing but the code itself. Frustrating, right? Now, scale that up to managing an entire application or a complex system. That’s where observability comes in—it’s like having a comprehensive debugger for your entire system.

Observability allows you to understand what's happening inside your applications by analyzing the data they generate. With observability, you can identify and resolve issues before they escalate, optimize performance, and ensure everything runs smoothly.

Observability vs. Monitoring: What’s the Difference?

You might think, “Isn’t observability just a fancy term for monitoring?” Not quite. While both are critical for system reliability, they serve different purposes:

Monitoring is like setting up health checks on your system. It watches specific metrics or logs and alerts you when something goes wrong, like high CPU usage or a failed API request.
Observability goes beyond that—it’s about understanding why things are happening. Think of it like having the ability to step through the running code of your system in real-time, understanding each decision and interaction. It’s not just knowing something went wrong but also how and why it happened.

In essence, monitoring tells you when there's an issue, while observability helps you understand the root cause.

The Three Pillars of Observability

Observability works by collecting and analyzing the three types of telemetry data—logs, metrics, and traces. To fully understand observability, it’s essential to grasp the three main types of telemetry data:

Logs :

Logs are the detailed records of what’s happening inside your system. They capture events, errors, and other critical information.
For developers, logs are like the print statements in your code—they help you trace the flow of execution and understand what happened when an issue occurred.
It help you understand what actions were taken at specific times. They’re invaluable for troubleshooting specific issues, like why a server crashed or why a user experienced an error.

Metrics :

Metrics are numerical data that represent the performance and health of your system. They include things like CPU usage, memory consumption, request latency, and error rates.
It give you a quick snapshot of your system’s overall state. Think of them as the performance stats of your application, similar to how you’d monitor frame rates in a video game to ensure smooth performance.
They’re crucial for setting up alerts that notify you when something goes wrong, like a sudden spike in latency or a drop in request rates.

Traces :

Traces follow the path of a request as it moves through various services in your system. They help you visualize how different parts of your application interact and where bottlenecks or errors might occur.
They are like following a breadcrumb trail through your code, seeing exactly where each function call leads and how it impacts the system.
This is especially important in microservices architectures, where understanding the interaction between services is key to diagnosing performance issues.

By combining these data types, observability tools can offer a holistic view of your system’s health and behavior, allowing you to identify and fix problems faster.

Why Should You Care About Observability?

So, why all the fuss about observability? Here’s why it matters:

Proactive Problem-Solving:

Observability lets you catch issues before your users do.
Instead of waiting for an error report, you can detect and resolve problems early, ensuring a smoother user experience and less downtime.

Optimized Performance:

By keeping an eye on metrics and traces, you can identify inefficiencies and optimize your system to run faster and more efficiently.
This is crucial whether you're running a small application or a large-scale distributed system.

Enhanced Collaboration:

Observability data acts as a common language for your team.
Developers, DevOps engineers, and SREs can all work from the same data, making it easier to collaborate on solving problems and improving the system.

Getting Started with Observability

Ready to bring observability into your projects? Here’s how to get started:

Choose Your Tools:

Tools like Grafana, Prometheus, and Loki are great for getting your observability stack up and running.
Each specializes in different aspects—metrics, logs, and traces—so you can tailor your setup to your needs.

Set Up Monitoring:

Start small by setting up monitoring for your most critical systems.
Track basic metrics like CPU, memory, and disk usage to understand your system’s normal behavior.

Implement Alerts:

Alerts are your early warning system.
Set up thresholds for your metrics so you’ll be notified the moment something goes off the rails.

Explore and Experiment:

Observability is a vast field, and there’s always more to learn.
Experiment with different tools and techniques to find what works best for your systems.

My Journey into Observability

In my work, I had the opportunity to explore and implement observability using various tools. I extracted metrics from Windows and Linux logs through the Cribl TCP source, processed them in Cribl Stream, and then used Prometheus to store and visualize the data on Grafana dashboard panels.

I also set up alerts for key metrics like CPU, disk, and memory using Grafana Alertmanager and Mimir, ensuring that any critical issues were immediately flagged. Additionally, I utilized silences in Grafana to manage and suppress alerts during maintenance windows or non-critical periods.

Why You Should Start Today

Whether you’re managing a large-scale system or just starting out with a small project, observability is key to ensuring reliability and performance. It’s like having superpowers for your code—powers that let you see inside your systems and make sure everything’s running just the way it should.

Stay Tuned for More!

I’m excited to share more about observability in my upcoming posts, where we’ll dive deeper into specific tools and techniques. Whether you’re a beginner or a seasoned pro, there’s always more to learn, so stay tuned!

Why Running Containers as Root Is Risky - Use Rootless Containers Instead

Metta Surendhar — Thu, 12 Dec 2024 14:00:00 +0000

Running containers with root privileges has long been recognized as a security risk. When a container operates with root access, it potentially exposes the host system to severe vulnerabilities. If that container is compromised, an attacker could gain root-level control over the entire host, which is why the concept of "rootless" containers is so important.

Rootless Containers: What They Are and Why They Matter

Rootless containers are designed to run without requiring root privileges on the host system. This means that even if a container is breached, the attacker wouldn't gain root access to the host. Rootless containers enhance security by significantly reducing the potential damage that could be done by a compromised container.

Here’s how they work:

User Namespaces: Rootless containers leverage user namespaces, a feature of the Linux kernel that maps user and group IDs within the container to different, non-root IDs on the host. So, even if a process runs as "root" inside the container, it’s actually operating as a non-root user on the host, ensuring the host remains protected.
Control Groups (cgroups): These manage and limit resource usage like CPU, memory, and disk I/O for containerized processes, preventing any single container from consuming too many resources on the host.
Seccomp (Secure Computing Mode): This filters system calls made by containerized applications, restricting what actions they can perform on the host, thereby reducing the attack surface.
SELinux and AppArmor: These are security modules that enforce access controls on containerized processes, further isolating them from the host system.

Why You Should Care About Rootless Operation

Running containers as root is risky business. Any exploit within a container running as root could allow an attacker to break out of the container and gain root access to your host. This could spell disaster for the entire system. Rootless containers, on the other hand, are designed to prevent this scenario. Even if an attacker manages to breach the container, they’ll find themselves with limited access and unable to escalate privileges on the host.

How to Check Container Privileges in Docker

If you’re using Docker and want to check if a container is running with root privileges, you can use this command:

codedocker run —user root -it my-root-container

Running this command will explicitly start the container as the root user. If the container starts without issues, it’s running with root privileges—something you typically want to avoid in production due to the security risks involved.

To run a Docker container in a more secure, non-root mode, use the --user flag to specify a non-root user:

codedocker run --user 1000:1000 -it my-rootless-container

In this example, 1000:1000 refers to a non-root user and group ID. This ensures your container operates with limited privileges, enhancing overall security.

Why Podman Makes Rootless the Default

While Docker supports rootless containers, it wasn’t designed with this as the default setting. Podman, on the other hand, was built from the ground up with rootless operation as the standard. This makes Podman inherently more secure, especially for those environments where security is a top priority.

The Benefits of Running Rootless

Reduced Attack Surface: Rootless containers minimize the chances of a successful privilege escalation attack.
Compliance with Security Policies: If your organization mandates that applications must not run as root, rootless containers help you stay compliant.
Improved Security Posture: By running containers with the least amount of privilege necessary, you’re actively reducing your risk exposure.

Wrapping Up

Ensuring that your containers are running without root privileges is a critical step toward securing your containerized environments. Whether you’re using Docker, Podman, or another container engine, adopting rootless containers represents a significant leap forward in security. By limiting the privileges of your containerized processes, you’re safeguarding your infrastructure against potential exploits and attacks.

Why Modern Apps Run in Containers - A Journey from FreeBSD Jail to Today

Metta Surendhar — Thu, 05 Dec 2024 14:00:00 +0000

Introduction

The world of software development has changed a lot in the past decade, mainly because of container technology. What started as a small solution for isolating processes is now a key part of how we develop and deploy apps. From the early days of FreeBSD Jail to the big impact of Docker, containers have changed how we build, deploy, and scale applications. In this blog, we’ll look at the power of containers, the important role of Docker, the history of container technology, and the ongoing efforts to standardize this growing field.

The Power of Containers

Containers are essential in modern software development, especially for scalable, cloud-native apps. Their power comes from providing lightweight, portable, and consistent environments, with key features being isolation and image immutability.

Isolation:

One of the most significant advantages of containers is their ability to encapsulate applications and their dependencies in isolated environments.
Each container runs independently, with its own filesystem, network interface, and process space.
This isolation ensures that applications operate uniformly regardless of the underlying infrastructure, preventing conflicts between applications and enabling consistent performance across different environments.
Isolation also enhances security by limiting the scope of access that an application has, reducing the risk of unauthorized interactions with the host system or other containers.

Image Immutability:

A container image is an immutable, pre-packaged environment that includes:
- Application code
- Runtime
- Libraries
- Configuration files necessary to run the application
Once an image is created, it remains unchanged across different stages of deployment—from development to production.
This immutability ensures:
- Consistency and reliability
- Elimination of the "it works on my machine" problem
- Simplified rollbacks in case of deployment issues
The ability to create a consistent environment that can be replicated across multiple platforms is a key reason why containers are vital in modern development practices.

Why Docker Became a Game-Changer

Docker, introduced in 2013 by the company then known as DotCloud, revolutionized the way developers build, ship, and run applications. Before Docker, deploying applications across different environments was fraught with challenges due to discrepancies in operating systems, software versions, and configurations. Docker addressed these issues by providing a simple, consistent way to package applications and their dependencies into a single, portable container image.

Simplified Packaging and Deployment:

Docker enabled developers to adopt a "build once, run anywhere" approach.
By bundling an application and its dependencies into a single container image, Docker eliminated inconsistencies between development, testing, and production environments.
This significantly reduced the time and effort required to deploy applications.
It made it easier for developers to focus on building features rather than troubleshooting deployment issues.

Developer-Friendly Tools:

Docker provided a suite of tools that made containerization accessible to a broader audience.
The Docker CLI allowed developers to easily create, manage, and share container images.
Docker Hub, a centralized repository for container images, offered a vast library of pre-built images that could be used to jumpstart development.
This ease of use, combined with the availability of a wide range of ready-to-use images, made Docker an attractive option for developers looking to streamline their workflows.

Ecosystem and Community Support:

Docker’s open-source nature and the vibrant community around it led to the rapid development of complementary tools and frameworks.
Docker Compose simplified the management of multi-container applications.
Docker Swarm provided native container orchestration capabilities.
The integration of these tools into existing workflows further solidified Docker's position as the go-to solution for containerization.
Docker set the standard for the industry and inspired the development of many other container-related technologies and standards.

The Origins of Container Technology

The concept of containers is rooted in the idea of process isolation, a principle that dates back to the early days of Unix. However, the evolution of containers as we know them today can be traced through several key developments, each building upon the last to create the sophisticated container environments we use today.

FreeBSD Jail (2000):

The FreeBSD Jail was one of the first implementations of process isolation at the operating system level.
Introduced in 2000, it allowed administrators to partition a FreeBSD system into multiple independent "jails," each with its own filesystem, network interfaces, and users.
This early form of containerization provided a way to securely isolate applications while sharing the same OS kernel.
It laid the groundwork for future container technologies.

Linux Containers (LXC) (2008):

Linux Containers (LXC) brought the concept of containers to the Linux ecosystem.
Enabled the creation of lightweight, isolated environments using kernel features like cgroups (control groups) and namespaces.
LXC was the first comprehensive solution that allowed multiple Linux containers to run on a single host with a high degree of isolation.
Marked a significant step forward in container-based virtualization.
Provided a more flexible and scalable alternative to traditional virtual machines.

Docker (2013):

Docker took the concepts introduced by LXC and built upon them, offering a more user-friendly and developer-focused approach.
By abstracting the complexities of LXC and adding features like Docker Hub and Docker Compose, Docker made containerization accessible to a wider audience.
Docker’s innovation was not just in the technology itself but in how it packaged and presented that technology, making it easy to use and integrate into existing workflows.
The introduction of Docker marked the beginning of widespread container adoption, transforming containers from a niche technology into a mainstream solution for modern application development.

The progression from FreeBSD Jail to Docker illustrates the evolution of container technology from a specialized solution for server partitioning to a fundamental component of modern application development and deployment.

Standardizing Containers: The Open Container Initiative

As containers gained popularity, the need for standardization became increasingly apparent. The rapid adoption of containers by the industry led to the creation of various container formats and runtimes, which, without standardization, could lead to fragmentation and compatibility issues. To address these concerns, the Open Container Initiative (OCI) was formed in 2015.

Image Specification:

The OCI Image Specification defines the format and structure of a container image.
This standardization ensures that container images are portable.
Container images can be reliably used across different platforms and tools.
This fosters a more open and interoperable ecosystem.

Runtime Specification:

The OCI Runtime Specification defines how to run a container.
It includes how to create, manage, and destroy container instances.
By standardizing the runtime environment, the OCI ensures that containers behave consistently across different environments.
This reduces the risk of incompatibilities.
It simplifies the deployment process.

Distribution Specification:

The OCI Distribution Specification covers how to distribute and share container images, including:
- Transport
- Storage
- Retrieval
This specification standardizes the way container images are pushed to and pulled from registries.
Ensures that containers can be easily shared and deployed across various platforms and environments.

These standards have played a crucial role in the continued growth and adoption of container technology, ensuring that containers remain a reliable and versatile tool for developers and organizations alike.

Wrapping Up

The journey from the early days of process isolation to the modern era of containerization has been marked by significant advancements in technology, driven by the need for more efficient, scalable, and secure solutions. Containers have become an essential part of the software development lifecycle, offering unparalleled benefits in terms of isolation, consistency, and portability.

Docker's introduction revolutionized the way developers approached containerization, setting the stage for widespread adoption and the development of a thriving ecosystem. As the industry continues to evolve, initiatives like the Open Container Initiative are ensuring that containers remain standardized, interoperable, and accessible to all.

As developers and organizations look to the future, embracing the power of containers will be key to building and deploying applications that are not only robust and scalable but also secure and consistent across all environments. Whether you’re just starting with containers or looking to optimize your existing workflows, understanding the history, power, and potential of containers is essential for staying ahead in the rapidly changing world of software development.

Tired of Docker's Limitations? Here's Why Podman Should Be Your Next Container Engine

Metta Surendhar — Thu, 28 Nov 2024 14:00:00 +0000

Attending the CNCF Chennai Meetup was an enlightening experience, particularly with the insightful talk by Ram, Chief Evangelist at the Cloud Foundry Foundation. His presentation, titled "Cloud Native Containers: Myth, Truth, or Marketing?," provided a comprehensive overview of the evolution of containers, the rise of Docker, and the emergence of modern, efficient, open-source alternatives.

This blog summarizes and expands on Ram's talk, delving into the world of container technology and exploring the tools developers can use today to build and manage containers more securely and effectively.

Podman: The Leading Open Source Docker Alternative

As Docker gained popularity, the demand for more flexible, secure, and open alternatives also grew. Enter Podman—an open-source container engine developed by Red Hat that has quickly become a preferred choice for many developers and organizations. Podman offers many of Docker’s features but introduces significant enhancements tailored to meet the needs of today’s security-conscious and compliance-driven environments.

Exploring Podman

Podman provides a comprehensive, daemonless container management solution that is fully compatible with the Open Container Initiative (OCI) standards. This ensures that it can interoperate seamlessly with Docker images and other OCI-compliant tools.

Key features of Podman include:

Daemonless Architecture:
Unlike Docker, which relies on a central daemon to manage containers, Podman operates without a long-running background process. Each container is managed as an individual process by the user who initiated it. This approach not only improves resource efficiency but also enhances security by reducing the attack surface—there’s no single point of failure or a privileged process that can be exploited.
Rootless Operation: Security is at the heart of Podman’s design. By default, Podman runs containers as non-root users, minimizing the risk of privilege escalation attacks. This is a crucial advantage over Docker, where the daemon typically requires root privileges, posing potential security risks. While Docker can be configured to run in a rootless mode, Podman makes rootless operation the standard, ensuring a safer environment by default.
CLI Compatibility with Docker: Podman’s command-line interface (CLI) is designed to be nearly identical to Docker’s, making it easy for developers to switch from Docker to Podman without having to learn new commands. This compatibility even extends to Docker Compose, thanks to the podman-compose tool, which replicates Docker Compose functionality using Podman.

These features make Podman a compelling alternative for developers and organizations looking to enhance security, reduce dependency on privileged processes, and maintain compatibility with existing container workflows.

Top Open Source Tools for Container Builds

Beyond Podman, several other open-source tools offer unique features and capabilities for container builds. These tools are integral to modern DevOps practices, providing flexibility and efficiency in container creation and management.

1. Unleashing Buildah

Buildah is the backbone of Podman for building container images.
It allows users to create images from scratch or customize existing ones.
Offers the flexibility to build images without needing a Dockerfile.
Compliance with OCI standards ensures compatibility with any OCI-compliant container runtime.
Makes Buildah a powerful and versatile tool for developers looking to streamline their image-building process.

2. Kaniko: Building in Kubernetes

Kaniko is a build tool designed to run within Kubernetes clusters.
It enables developers to build container images inside containers or Kubernetes pods without requiring privileged access to the host.
This is particularly useful in CI/CD pipelines, where security is paramount, and running builds without elevated privileges is a best practice.
Kaniko’s integration with Kubernetes makes it a go-to choice for teams leveraging Kubernetes as their primary platform.

3. The Efficiency of BuildKit

BuildKit is a highly efficient and modern toolkit for building container images, originally developed as part of the Moby project.
It’s now widely used within the Docker ecosystem and beyond.
BuildKit supports advanced features like parallel builds, build caching, and multi-stage builds, significantly speeding up the image-building process.
Its flexibility and performance enhancements make it a robust option for developers looking to optimize their container build workflows.

4. NixOS: A Unique Approach

NixOS is an innovative Linux distribution that takes a declarative approach to system configuration.
While not a traditional container tool, NixOS’s ability to perform VM work through containers offers an unparalleled level of reproducibility and isolation.
This makes NixOS ideal for developers who need a highly controlled and consistent environment for their applications.
It is especially useful in scenarios where precise environment replication is critical.

5. Ko: Tailored for Go Developers

Ko is a specialized tool designed for Go developers.
It simplifies the process of building and deploying Go applications as container images.
Ko eliminates the need for a Dockerfile.
It automates the creation of optimized, small container images directly from Go source code.
This tool is particularly valuable for microservices architecture.
Efficient and rapid deployment of Go applications is essential with Ko.

6. Apko: Minimal and Secure

Apko, developed by Chainguard, focuses on building minimal and secure container images.
It emphasizes simplicity and ease of auditing, making it an excellent choice for security-conscious organizations.
Apko’s approach to container image creation helps minimize the attack surface, which is crucial in environments where security is a top priority.

7. Buildpacks: Simplifying Container Creation

Buildpacks offer a higher-level abstraction for building container images.
Originally developed by Heroku and now part of the Cloud Native Buildpacks project.
Developers can create optimized container images without writing Dockerfiles.
Buildpacks automatically detect the language and dependencies of an application.
They generate a container image tailored to the application's needs.
Support rebasing, allowing for base image updates without rebuilding the entire application.
Save time and resources in the deployment process.

Building Containers: Reproducibility, Isolation, and Security

As container usage grows, the importance of reproducible, isolated, and secure builds becomes increasingly clear. Here are some critical considerations for achieving these goals:

Reproducibility: A reproducible container build process ensures that the same container image can be generated consistently, with identical content, regardless of when or where it is built. This is vital for debugging, auditing, and maintaining compliance with regulations.
Isolation: Containers should be isolated not only during runtime but also throughout the build process. This prevents potential conflicts and ensures that the build environment does not influence the final container image, leading to more reliable and predictable deployments.
Parameter Less Builds: To enhance build reliability, containers should be built without relying on external parameters or environment variables, which can introduce variability and unpredictability into the build process.
Provenance: Understanding the origin of every component within a container is essential for security and compliance. Provenance tools help track and verify the source of software components, ensuring that your containers are built from trusted and secure sources.

Embracing the Future of Container Technology

The container technology landscape has evolved significantly since the early days of Docker. Developers now have access to a wide array of open-source tools, each offering unique features that cater to different aspects of containerization. Whether you’re seeking a secure, rootless container engine like Podman or specialized build tools like Buildah, Kaniko, or Buildpacks, the open-source community has created robust alternatives that can meet the diverse needs of modern software development.

As you explore these tools, it’s crucial to align them with your specific requirements—be it security, performance, ease of use, or integration with your existing CI/CD pipeline. By selecting the right tool for the job, you can ensure that your containerized applications are built and deployed efficiently, securely, and with minimal overhead.