Forem: Santhosh

I Scraped 120 Years of Olympic History — and You Can Too

Santhosh — Wed, 13 Aug 2025 13:17:45 +0000

I’ve always been fascinated by the Olympics.

The stories, the records, the triumphs… but when I went looking for a clean dataset of every athlete in history, I hit a wall.

Sure, there’s Olympedia.org — an incredible resource — but no “Download” button.

So I decided:

If the dataset doesn’t exist, I’ll build it myself.

The result? A Python scraper that can pull every athlete profile from 1896 to today — perfect for data analysis and visualization projects.

📌 What This Script Does

With one command, you get:
✅ Name, gender, height, weight
✅ Birth & death info (date, city, country)
✅ National Olympic Committee (NOC)
✅ Last Olympic Games and sport
✅ Medal counts (gold, silver, bronze)

Saved neatly in a CSV ready for Pandas or Excel.

📊 What You Can Do With It

This isn’t just about scraping.

Once you have the data, you can:

Visualize medal trends over decades
Explore which sports certain countries dominate
Study athlete physique trends (height/weight) over time
Map birthplaces of medalists with GeoPandas

⚡ How Fast Is It?

With 10 threads and a 0.4 second delay per request,
you can scrape thousands of athletes in under an hour — without hammering the site.

🚀 Quick Start

1️⃣ Clone the repo

git clone https://github.com/Wydoinn/Olympedia-Athlete-Scraper.git
cd Olympedia-Athlete-Scraper
pip install -r requirements.txt

2️⃣ Run the scraper

# Start fresh
python scraper.py --start 1 --concurrency 10 --delay 0.4 --csv olympedia.csv

# Or resume where you left off
python scraper.py --resume

3️⃣ Open olympedia.csv and start exploring.

📂 The Data Format

Example row:

athlete_id,name,sex,height_cm,weight_kg,born_date,died_date,
born_city,born_region,born_country,died_city,died_region,died_country,
noc,games,year,sport,gold_medal,silver_medal,bronze_medal
19,Maurice Germot,M,178,68,1882-11-15,1958-01-06,
Vichy,Allier,FRA,Vichy,Allier,FRA,FRA,
Stockholm 1912,1912,Tennis,0,2,0

🧠 How It Works (in 20 Seconds)

Multi-threaded with ThreadPoolExecutor
Resumable with a progress.json checkpoint
Auto-stops after 1000 consecutive missing IDs
Parses HTML using BeautifulSoup
Writes CSV as it runs (so you can peek mid-scrape)

🧹 A Note on Responsible Scraping

Please be respectful:

Keep a delay between requests
Don’t flood the server
Always credit the source (Olympedia)

💬 What would you want to analyze first?
Drop a comment and let’s brainstorm!

YOLOv10 on Custom Dataset

Santhosh — Sun, 09 Jun 2024 12:33:41 +0000

What is YOLO?

You Only Look Once (YOLO) is a state-of-the-art, real-time object detection algorithm .

What makes YOLO popular?

Speed
Detection accuracy
Good generalization
Open-source

Google Colab is an excellent platform for running deep learning models due to its free access to GPUs and ease of use. This guide will walk you through the process of running the latest version, YOLOv10, on Google Colab.

Before You Start

To make sure that you have access to GPU. You can use nvidia-smi command to do that. In case of any problems navigate to Edit -> Notebook settings -> Hardware accelerator, set it to GPU, and then click Save.

!nvidia-smi

Install Required Packages

Clone the GitHub repository.

!git clone https://github.com/THU-MIG/yolov10.git

cd yolov10

!pip install .

Upload Data To Colab

Step 1: Mount Google Drive

from google.colab import drive
drive.mount('/content/drive')

Step 2: Upload Files Directly

from google.colab import files
uploaded = files.upload()

Step 3: Organize Data for YOLOv10

Images:

The images directory contains subdirectories for train and val (validation) sets.
Each subdirectory contains the corresponding images for training and validation.

Labels:

The labels directory mirrors the images directory structure.
Each text file in the labels/train and labels/val subdirectories contains the annotations for the corresponding images.

Annotations Format:

/my_dataset
  /images
    /train
      image1.jpg
      image2.jpg
      ...
    /val
      image1.jpg
      image2.jpg
      ...
  /labels
    /train
      image1.txt
      image2.txt
      ...
    /val
      image1.txt
      image2.txt
      ...
  data.yaml

Data Configuration File (data.yaml):

train: /content/my_dataset/images/train
val: /content/my_dataset/images/val

nc: N     # N for number of classes
names: ['class1', 'class2', ..., 'classN']

Download Pre-trained Weights

import os
import urllib.request

# Create a directory for the weights in the current working directory
weights_dir = os.path.join(os.getcwd(), "weights")
os.makedirs(weights_dir, exist_ok=True)

# URLs of the weight files
urls = [
    "https://github.com/jameslahm/yolov10/releases/download/v1.0/yolov10n.pt",
    "https://github.com/jameslahm/yolov10/releases/download/v1.0/yolov10s.pt",
    "https://github.com/jameslahm/yolov10/releases/download/v1.0/yolov10m.pt",
    "https://github.com/jameslahm/yolov10/releases/download/v1.0/yolov10b.pt",
    "https://github.com/jameslahm/yolov10/releases/download/v1.0/yolov10x.pt",
    "https://github.com/jameslahm/yolov10/releases/download/v1.0/yolov10l.pt"
]

# Download each file
for url in urls:
    file_name = os.path.join(weights_dir, os.path.basename(url))
    urllib.request.urlretrieve(url, file_name)
    print(f"Downloaded {file_name}")

Train Custom Model

!yolo task=detect mode=train epochs=100 batch=4 plots=True model=weights/yolov10n.pt data=data.yaml

Inference on Image

!yolo task=detect mode=predict conf=0.25 save=True model=runs/detect/train/weights/best.pt source=img.jpg

Inference on Video

!yolo task=detect mode=predict conf=0.25 save=True model=runs/detect/train/weights/best.pt source=video.mp4

Summary

This guide covers running YOLOv10 on Google Colab by setting up the environment, installing necessary libraries, and running inference with pre-trained weights. It also explains how to upload and organize data in Colab for YOLOv8, including the required directory structure and configuration files. These steps enable efficient training and inference for object detection models using Colab's resources.

Run Local LLMs Using LM Studio

Santhosh — Sun, 07 Apr 2024 12:40:43 +0000

In this article, I'll guide you through the process of running open-source large language models on a computer using LM Studio. LM Studio is compatible with macOS, Linux, and Windows.

What you can find in this article?

What is LM Studio?
What are the minimum hardware / software requirements?
Installing LM Studio on Windows
Running LM Studio
Chat with your model
Summary

1. What is LM Studio?

Software for running large language models (LLMs) locally. This is the most relevant meaning in the context of AI and machine learning. LM Studio is a software application that allows you to download, install, and run powerful LLMs on your own computer. This gives you more control and privacy compared to using cloud based LLMs like ChatGPT.

Here are some key features of LM Studio for running LLMs:

Discover and download various LLMs
Run models on your local machine with a compatible GPU
Integrate with AnythingLLM for a chatbot interface

If you're interested in learning more about LM Studio for LLMs, you can refer to their official website or their GitHub repository.

2. What are the minimum hardware / software requirements?

Apple Silicon Mac (M1/M2/M3) with macOS 13.6 or newer
Windows / Linux PC with a processor that supports AVX2 (typically newer PCs)
16GB+ of RAM is recommended. For PCs, 6GB+ of VRAM is recommended
NVIDIA/AMD GPUs supported

3. Installing LM Studio on Windows

LM Studio works flawlessly with Windows, Mac, and Linux. These quick instructional leads you through the installation processes, particularly for Windows PC.

➡️ Go to LM Studio page and download the file: Download

Download the file
Open the file (.exe)
It will automatically install on (C:) drive

4. Running LM Studio

Once LM Studio has been set up, you may open your application and download various models locally.

You can browse for any model available on Hugging Face. And in the page's upper right corner displays the estimated RAM and VRAM capacities. You can access the model card on the Hugging Face website in your browser and also use the README.md file to learn about the model.

The available files will be provided together with the compatibility of hardware, whether it can run or not. It shows the following compatibility guesses:

Full GPU offload possible
Partial GPU offload possible
Some GPU offload Possible
⚠️ Likely to large for this machine

After selecting the appropriate model for your computer, you may download and execute the model.

5. Chat with your model

Go to the chat page, load the model you have downloaded, and provide the prompt, it will respond with the proper answer. You can receive the response in plaintext, markdown, and monospace format. You may export the chat as JSON, simple text, formatted prompt, or as a snapshot.

You can configure the hardware, inference parameter, prompt format, and model initialization under the advanced configuration.

You can also load many models at once to receive diverse responses to the prompts that have been given.

LM Studio's local server lets you use LLMs on your machine mimicking OpenAI's API for privacy, customization, and easier integration with existing code.

6. Summary

LM Studio is a software application that allows you to download, install, and run powerful large language models (LLMs) on your own computer. This gives you more control and privacy compared to using open source LLMs. The guide outlines the requirements to run LM Studio and provides step-by-step instructions on how to install it on Windows, download models, and chat with them. LM Studio also offers features like advanced configuration, loading multiple models, and a local server for easier integration with existing code.