Forem: Karan Bhardwaj

How to setup the Nvidia TAO Toolkit on Kaggle Notebook

Karan Bhardwaj — Thu, 17 Oct 2024 04:30:37 +0000

Introduction

Action recognition plays a crucial role in enabling applications like video surveillance, sports analytics, and gesture recognition. Leveraging pre-trained models with NVIDIA’s TAO Toolkit makes it easier to train high-performance action recognition models efficiently.

TAO Toolkit can be set up using docker or NGC CLI. Since we will be working on the Kaggle Notebook, we will use the NGC CLI, as the Kaggle Notebook environment does not support docker.

Note: Kaggle Notebooks don't support Docker due to security concerns, resource management, and the provision of pre-configured environments for simplified workflows.

Installation Steps:

1. Install dependencies

First, install nvidia-pyindex, a repository manager for NVIDIA’s Python-based tools that simplifies the installation process for the TAO Toolkit and related components.

!pip install nvidia-pyindex

2. Install the Nvidia TAO Toolkit and NGC-CLI

The Nvidia TAO Toolkit contains a collection of pre-trained models for various tasks such as object detection, classification, segmentation and action recognition.

!pip install nvidia-tao

Next, install the NGC-CLI (NVIDIA GPU Cloud Command Line Interface), which interacts with NVIDIA's NGC catalog to manage pre-trained models.

!wget -O ngccli_linux.zip https://ngc.nvidia.com/downloads/ngccli_linux.zip
!unzip ngccli_linux.zip

3. Create an NGC account

Register for an account on the Nvidia NGC catalog to access the TAO toolkit models. Once registered, you can authenticate via the NGC CLI using your API key to download the desired models.

First, go to https://catalog.ngc.nvidia.com/ and sign up for a free account from the right menu.

Once signed in, go to the Setup section from the right drop-down menu and click on Generate Personal Key.

4. Configure the NGC CLI

Set up your environment to authenticate with NGC using the following commands. Keep your API key secure.

!chmod u+x ngc-cli/ngc

import os

# Declaring the input arguments as environment variables. 
# This way we can directly pass the arguments during cell runtime to any command request in the Kaggle notebook.

os.environ['API_KEY'] = 'your_api_key'
os.environ['TYPE'] = 'ascii'
os.environ['ORG'] = '0514167173176982'
os.environ['TEAM'] = 'no-team'
os.environ['ACE'] = 'no-ace'

# Passing the input arguments to the config command
!echo -e "$API_KEY\n$TYPE\n$ORG\n$TEAM\n$ACE" | ngc-cli/ngc config set

If you see the output below, your setup is complete. Hurray!!🥳🥳

Now that the NGC CLI is configured, you can list the available models:

!ngc-cli/ngc registry model list

If you want to download any specific model, you can do so by running the following command

!ngc-cli/ngc registry model download-version "nvidia/tao/actionrecognitionnet:deployable_onnx_v2.0"

Here I have downloaded the ActionRecognitionNet model. The model will be downloaded in the .onnx format.

By following the steps above, you’ve set up the TAO Toolkit on Kaggle Notebook. Now you can start exploring the world of high-performance computer vision with ease.

Happy Coding!🤗🤗

Passing Input Arguments in Kaggle Notebook Using Environment Variables

Karan Bhardwaj — Sun, 13 Oct 2024 18:41:10 +0000

Kaggle Notebook doesn't support interactive user input since it runs in a cloud environment where code cells are executed in sequence without waiting for user interaction.

So, in cases where we have to pass input arguments, we can bring the environment variable to our rescue.

Assuming the case that there is a command named some_command when executed asks for input argument, let's say an API key. So the steps to pass the API key will be as follows:

1. Declare an environment variable

We use the os library to declare an environment variable.

import os

# Instantiate the API key as an environment variable
os.environ['API_KEY'] = "whatever_is_the_key"

2. Passing the environment variable as a user input

Here, we will use the echo shell command to pass the API key as a user input argument to command some_command.

# run the shell command
!echo $API_KEY | some_command

What happened above is that "echo $API_KEY" generated the output (in this case, the API key "whatever_is_the_key"), and "|" sent this output as an input argument to some_command.

This way, you can pass input arguments to the commands you need to execute.

In case you have to pass multiple input arguments, you can modify echo shell command as,

# Assume you have environment variables as I, ME, and YOU
!echo "$I" "$ME" "$YOU" | some_command

This approach can be beneficial when automating tasks that require external inputs or when working with APIs in non-interactive environments like Kaggle

Happy Coding!🤗🤗

How to use Detectron2 Instance Segmentation on Videos

Karan Bhardwaj — Wed, 06 Mar 2024 18:47:48 +0000

Instance segmentation, a challenging task in computer vision that involves detecting and delineating individual objects within an image or video, has seen significant advancements in recent years. One such advancement is Detectron2, a flexible and efficient framework developed by Facebook AI Research. In this guide, we'll explore how to leverage the power of Detectron2 within the Google Colab environment to perform instance segmentation on videos.

Step 1: Check GPU availability

Check whether you have connected to GPU by changing the runtime from the Runtime tab in the dropdown menu.

After that check whether the GPU is accessible or not by running the command:

!nvidia-smi

If you see something like this, you are all set to go.

Step 2: Install detectron2

Run this single command to directly install detectron2.

!python -m pip install 'git+https://github.com/facebookresearch/detectron2.git'

Step 3: Import libraries

Import the required libraries.

# COMMON LIBRARIES
import os
import cv2

from google.colab.patches import cv2_imshow

# VISUALIZATION
from detectron2.utils.visualizer import Visualizer
from detectron2.utils.visualizer import ColorMode

# CONFIGURATION
from detectron2 import model_zoo
from detectron2.config import get_cfg

# EVALUATION
from detectron2.engine import DefaultPredictor

Step 4: Initialize the predictor

Choose a model as per your requirement from the model zoo. You can see the list of available models here.

cfg = get_cfg()
cfg.merge_from_file("detectron2/configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml")
cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.5
cfg.MODEL.WEIGHTS = "detectron2://COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x/137849600/model_final_f10217.pkl"
predictor = DefaultPredictor(cfg)

Step 5: Inference on Video

Set the path to your video in the following code, and execute the code. The output will be a video with segmentation applied.

import imageio
import numpy

# Load video
video_path = "path_to_your_video.mp4"
cap = cv2.VideoCapture(video_path)

# Initialize video writer
fps = cap.get(cv2.CAP_PROP_FPS)
output_path = '/content/output.mp4'
writer = imageio.get_writer(output_path, fps=fps)

# Perform instance segmentation on each frame
while cap.isOpened():
    ret, frame = cap.read()
    if not ret:
        break
    outputs = predictor(frame)

    # Find the classes (Optional)
    pred_classes = instances.pred_classes.cpu().numpy()

    # Find the segment points (Optional)
    pred_masks = instances.pred_masks.cpu().numpy()

    v = Visualizer(frame[:, :, ::-1], metadata=MetadataCatalog.get(cfg.DATASETS.TRAIN[0]), scale=0.8)
    frame = v.draw_instance_predictions(outputs["instances"].to("cpu")).get_image()[:, :, ::-1]
    # Write processed frame to output video
    writer.append_data(frame)

# Release video resources
cap.release()
writer.close()

How to Install Detectron2 on Windows

Karan Bhardwaj — Tue, 05 Mar 2024 19:32:51 +0000

Detectron2 is a powerful open-source object detection and segmentation framework built by Facebook AI Research. It's widely used for research and development in computer vision applications. However, installing Detectron2 on Windows 11 can be a bit tricky due to various dependencies. In this guide, I will take you through the step-by-step process to set up Detectron2 on your Windows 11 system.

Step 1: Set up a Conda Environment

First, let's create a new conda environment to isolate the installation:

conda create --name detectron2_env python=3.11.7
conda activate detectron2_env

Step 2: Install CUDA

If you have an NVIDIA GPU, you'll need to install the CUDA toolkit. Ensure you have the correct version compatible with your GPU:

conda install cuda=12.1 -c nvidia

Step 3: Install PyTorch

Install PyTorch with the required CUDA version:

conda install pytorch==1.9.1 torchvision==0.10.1 torchaudio==0.9.1 cudatoolkit=12.1 -c pytorch -c nvidia

For other versions of CUDA, and the required PyTorch version, you can refer to https://pytorch.org/get-started/locally/.

In case you are on CPU, use

conda install pytorch torchvision torchaudio cpuonly -c pytorch

Step 4: Update Visual C++ Redistributable

Ensure your system has the latest Visual C++ Redistributable installed. You can download and install it from the official Microsoft website.

Step 5: Install Cython and COCO Tools

Cython is a prerequisite for building Detectron2, while COCO tools required for evaluation:

pip install cython
conda install conda-forge::pycocotools

Step 6: Clone Detectron2 Repository

Clone the Detectron2 repository from GitHub:

git clone https://github.com/facebookresearch/detectron2.git

Step 7: Install Detectron2

python -m pip install -e detectron2

That's it! You've successfully installed Detectron2 on your Windows 11 system. You can now start using it for various computer vision tasks like object detection, instance segmentation, and more.

Remember to activate your conda environment whenever you want to work with Detectron2.

Happy coding!🤗🤗