Forem: Itay Eylath

From Metrics to Action: Turning Embedding Analysis into Sprint Tickets

Itay Eylath — Mon, 23 Feb 2026 09:04:38 +0000

In an agile Computer Vision startup, global accuracy is a vanity metric.

It tells you the model is working.

It doesn’t tell you what to fix next.

To move fast, you need actionable signals not dashboards.

Here’s how we transform raw embeddings into sprint ready decisions.

1. Stop Calculating Accuracy. Start Finding Confusion.

A single accuracy score hides local failures.

Instead of asking “How accurate is the model?”, ask:

Where exactly is the model failing?

We use 1-Nearest Neighbor (1-NN) evaluation to expose structural confusion between visually similar items.

import numpy as np
from sklearn.metrics import pairwise_distances

# Compute pairwise distances between embeddings
D = pairwise_distances(embeddings, metric="euclidean")

# Ignore self comparisons
np.fill_diagonal(D, np.inf)

# Identify closest neighbor
closest_idx = np.argmin(D, axis=1)
is_correct = labels == labels[closest_idx]

Instead of celebrating 88% accuracy, we extract:

Which classes are confused
How often confusion happens
Whether errors are isolated or concentrated

This immediately narrows the problem space.

2. Margin: Measuring Model Uncertainty

Accuracy is binary.

Confidence is continuous.

We define Margin as the gap between the closest and second closest neighbors:

Margin = Dist_{2nd} - Dist_{1st}

If the margin is near zero, the model is effectively guessing between two nearly identical items.

# Sort distances per sample
sorted_dists = np.sort(D, axis=1)

# Compute margin
margin = sorted_dists[:, 1] - sorted_dists[:, 0]

# Flag unstable samples
low_confidence = margin < 0.05

Low margin samples are early warning signals.

They tell you where instability lives before accuracy visibly drops.

3. Visualization for Intuition (Not Decisions)

Metrics drive action.

Visualization builds intuition.

We use PCA or t-SNE to project embeddings into 2D space:

from sklearn.manifold import TSNE

tsne_results = TSNE(
    n_components=2,
    perplexity=30,
    random_state=42
).fit_transform(embeddings)

Overlapping clusters often explain:

Why two classes are confused
Whether boundaries are weak
Whether the representation itself lacks separation

Visualization doesn’t replace metrics it explains them.

4. The Operational Matrix: Converting Signals to Ownership

Analysis only matters if someone owns the fix.

We translate technical signals into clear action items:

Signal	Meaning	Action Item	Owner
High Intra-class Distance	Same class samples are visually inconsistent	Clean dataset (remove low-quality samples)	Data Ops
Strong Confusion Pair	Two distinct classes overlap heavily	Add discriminative features or review labeling	CV / Product
Low Margin Cluster	Category boundary is unstable	Targeted data collection or fine-tuning	ML Team

This is where evaluation becomes execution.

5. The Agile Loop

Every iteration follows the same structure:

Run embedding evaluation
Extract top confusion pairs
Rank unstable classes by margin
Convert top issues into sprint tickets
Re-run metrics after fixes

Metrics → Ownership → Action → Re-measure

No guesswork.

No intuition battles.

No blind retraining.

What We Learned

Most persistent errors were structural, not architectural.
Margin was a better early warning signal than accuracy.
Concentrated confusion mattered more than global percentage.
Ownership accelerated improvement more than model complexity.

The biggest shift wasn’t technical.

It was operational.

The Bottom Line

In real world Computer Vision systems, the difference between stable and unstable performance isn’t a new architecture.

It’s a tighter feedback loop.

If a metric doesn’t lead to a sprint ticket, it’s just noise.

The best models aren’t the ones with the highest accuracy.

They’re the ones whose failures are measurable, owned, and shrinking every iteration.

This project is open source feel free to explore the code on GitHub and try it yourself:
https://github.com/itayeylath/metrics-to-actions

A Simple Docker Cleanup Script to Save Your Day

Itay Eylath — Mon, 15 Jul 2024 11:56:28 +0000

Are you tired of dealing with cluttered Docker containers? 🐳
Or any issues as a Linux user? 🛠️
Or maybe, you are just a lazy FullStack developer? 🥱

If you find your development environment getting messy with containers that you no longer need, I have a nifty little script that will solve your Docker woes in no time :-)

The Magic Script: 🪄

Let me introduce you to docker-cleanup.sh - a simple yet powerful script that will give your Docker environment a fresh start

Here's what the script looks like:

# Check if there are any containers
if docker ps -a &> /dev/null; then
    echo "Removing existing containers..."
    # Remove all containers
    docker rm -f $(docker ps -a -q)
    # Restart Docker
    echo "Restarting Docker..."
    sudo systemctl restart docker
    # Print Docker status
    echo "Docker status:"
    sudo systemctl status docker | grep "Active:"
else
    echo "No containers to remove."
fi

How It Works 🔍

Checks for Containers: The script starts by checking if there are any existing Docker containers.
Removes Containers: If containers are found, it removes all of them.
Restarts Docker: After cleaning up, the script restarts Docker to ensure everything is running smoothly.
Displays Status: Finally, it shows the status of Docker to confirm that everything is active and working.

How to Use It 💡

Using this script is super easy. Just follow these simple steps:

Save the script as docker-cleanup.sh (recommend to save it on home dir)
Open your terminal.
Execute the script bash docker-cleanup.sh

A Personal Touch 🤗

I face Docker environment issues regularly. This script has become my go-to solution, and I find myself using it at least once a day before I start working. It's simple, effective, and saves me a ton of time and hassle. I hope it works just as well for you! 🤞

GitHub Repo

Happy coding!

Google Analytics With React Typescript

Itay Eylath — Thu, 04 Jul 2024 11:55:02 +0000

What is GA ⁉️ 😯

Google Analytics (GA) is a powerful tool for tracking and analyzing website traffic and user behavior.

Here's how it operates:

GA collects data from your website using a JavaScript tracking code that you embed in your site’s pages. This code gathers information about your visitors and sends it to Google's servers, where it is processed and made available in various reports.

How's the implementation Work?

useGoogleAnalytics.ts as custom hook is the primary file for integrating with GA.
We're using an object to handle GA integration for flexibility and future-proofing.

GoogleAnalyticsProvider.tsx as a provider component to wrap the Routes

App.tsx modify your file by add GoogleAnalyticsProvider component

Why This Approach? ♻️

Support multiple: functionalities, easily manage page views, event tracking, and post-initialization configurations.
Future proofing: If we switch from ReactGA to another solution (e.g., a future GA version), we only need to update this object, not every instance in our codebase.
React Way: This method aligns with React’s best practices by using a component to handle side effects.

Comments In the code

Search it on the App // Recommend: and // Remark: to understand logic and getting the implantation better in your App!

GitHub Repo: 'google-analytics-react-ts'

useGoogleAnalytics.ts.tsx

import { useEffect } from 'react';
import ReactGA from 'react-ga4';

// Recommend: implement it with env variables to keep it as a secret 
export const trackingId = "GA_ID"
const appVersion = "APP_VERSION"
// Remark: use user ID in your app to make the analyze better
// Recommend: implement it with Redux 
const id = "user-id"

const useGoogleAnalytics = () => {

  useEffect(() => {
    if (trackingId) {
      try {
        ReactGA.initialize([
          {
            trackingId,
            gaOptions: {
              anonymizeIp: true,
              clientId: id
            }
          }
        ]);
        ReactGA.set({ app_version: appVersion });
      } catch (error) {
        // Recommend: reporting this error to an error tracking service
        console.log("Error initializing Google Analytics", { Error: error });
      }
    }
  }, [id]);

  const setOption = (key: string, value: unknown) => {
    ReactGA.set({ [key]: value });
  };

  const setUserId = (userId: string | number) => {
    setOption("userId", userId);
  };

  const sendData = (type: string, data: Object) => {
    ReactGA.send({ hitType: type, ...data });
  };

  const trackPageView = (pagePath?: string) => {
    if (!pagePath) {
      pagePath = location.href;
    }

    setOption('app_version', appVersion);
    sendData("pageview", { page: pagePath });
  };

  const trackEvent = (category: string, action: string, label?: string, value?: number) => {
    setOption('app_version', appVersion);
    ReactGA.event({ category, action, label, value });
  };

  return {
    setOption,
    setUserId,
    trackPageView,
    trackEvent,
  };
};

export default useGoogleAnalytics;

GoogleAnalyticsProvider.tsx

import React, { useEffect, PropsWithChildren } from 'react';
import { useLocation } from 'react-router-dom';
import useGoogleAnalytics, { trackingId } from '../hooks/useGoogleAnalytics';

const GoogleAnalyticsProvider: React.FC<PropsWithChildren<{}>> = ({ children }) => {
    const { trackPageView } = useGoogleAnalytics();
    const location = useLocation();

    useEffect(() => {
        if (trackingId) {
            try {
                trackPageView(location.pathname + location.search);
            } catch (error) {
                // Recommend: reporting this error to an error tracking service
                console.log("Error executing trackPageView Google Analytics", { Error: error });
            }
        }
    }, [location, trackPageView]);
    // Remark: this allows GoogleAnalyticsProvider to wrap other components without affecting the UI
    return <>{children}</>;
}

export default GoogleAnalyticsProvider;

App.tsx

import { BrowserRouter as Router, Route, Routes, useLocation } from 'react-router-dom';
import './App.css';
import GoogleAnalyticsProvider from './providers/GoogleAnalyticsProvider';

const App: React.FC = () => {

  return (
    // Remark: check in your App, the optimal placement for GoogleAnalyticsProvider-
    // to ensures early initialization to track all route changes but the errors still be captured (if you use an error tracking service)
    <GoogleAnalyticsProvider>
      <Routes>
        <Route
          path="/"
          element={
            <div>
              Google Analytics React TypeScript
            </div>
          }
        />
      </Routes>
    </GoogleAnalyticsProvider >
  );
}

export default App;

2 Ways Of Implementation

1 . General Tracking (the current repo) 😎

💪 Pros:

Simple Initialization: Your setup ensures that Google Analytics is initialized only once when the GAInitializer component mounts.
Page View Tracking: By providing a method to track page views, you can manually ensure that each page visit is logged.
Custom Events: The trackEventBuilder method offers flexibility to track various user interactions.

👎 Cons:

Manual Page View Tracking: Since you're only wrapping the App component and not each route change, you must manually call trackPageView on each route change, which is prone to human error if forgotten.
Lack of Route Change Tracking: Without automatic tracking of route changes, you might miss logging some page views if trackPageView is not called explicitly in each component.
Limited User Context: Your implementation does not track detailed user interactions or demographic data beyond what GA collects by default.

2. Add Event Tracking Function To EVERY Event We Want To Track 😴

💪 Pros:

Granular Control: you have precise control over exactly which events are tracked and how they are labeled, allowing for very detailed and specific analytics data.
Customization: each event can be customized with specific category, action, label, and value, providing rich insights into user behavior and interactions.

👎 Cons:

Maintenance Overhead: as your application grows, managing numerous individual tracking calls can become time-consuming and difficult to maintain consistently.
Code Duplication: you might end up repeating similar tracking code across multiple components or functions, leading to less DRY (Don't Repeat Yourself) code.

Content Security Policy (CSP) 🚔

To implement GA, include the GA script in your web application and set up tracking in your code. Configure your CSP to allow connections to GA servers by adding 'https://www.google-analytics.com' to your script-src and connect-src directives. If using Google Tag Manager, also include 'https://www.googletagmanager.com' in your script-src directive, and always test your CSP configuration thoroughly.

References ©️

Medium: 'Implementing Google Analytics to React with TypeScript'

YouTube: 'Add Google Analytics in React JS'