Forem

Cover image for First Look at NVIDIA Jetson Orin Nano Super - The Most Affordable Generative AI Supercomputer

First Look at NVIDIA Jetson Orin Nano Super - The Most Affordable Generative AI Supercomputer

Ajeet Singh Raina on February 16, 2025

Table of Contents Comparing Jetson Nano Vs Jetson Orin Nano Super How powerful is NVIDIA Jetson Orin Super? Generative AI Capabilities ...
Read full post
Collapse
 
ajeetraina profile image
Ajeet Singh Raina • • Edited

Quick Update[17 feb 2025]::

I highly recommend using SSD instead of microSD card for AI workloads. I ordered "Crucial P3 500GB PCIe 3.0 3D NAND NVMe M.2 SSD, up to 3500MB/s - CT500P3SSD8" from Amazon. Will keep you all posted on the performance.

Expecting NVMe SSDs to reach atleast 3500 MB/s+ (35x faster than SD cards). Need a faster boot times, quicker model loading, and improved system responsiveness.

Collapse
 
ajeetraina profile image
Ajeet Singh Raina • • Edited

Quick Update: 22- Feb 2025:

Image description

My containers just got a power boost! ⚡ Upgraded my Jetson Orin Nano with a Crucial P3 NVMe SSD—more speed, more storage, and extra energy for AI workloads! 🚀 #JetsonOrin #EdgeAI #Containers #NVMePower

Collapse
 
ajeetraina profile image
Ajeet Singh Raina •

Here’s the latest blog on running Docker Desktop on the Jetson Orin Super board collabnix.com/running-docker-deskt...

Collapse
 
nathaniel_palmieri_9d5830 profile image
Nathaniel Palmieri •

Thank you for your insights.

Collapse
 
exaptedai777 profile image
Exapted Aint •

Will this run any GPU-requiring code or is it limited LLM installs only?

Collapse
 
ajeetraina profile image
Ajeet Singh Raina • • Edited

It’s not just restricted to LLM but run any GPU requiring code.

Collapse
 
click2install profile image
click2install •

Triton makes it super easy to spin up a model using various backends.

Collapse
 
ajeetraina profile image
Ajeet Singh Raina •

That's on my wishlist:

docker run --gpus all --rm -p 8000:8000 -p 8001:8001 -p 8002:8002 \
    -v /path/to/model/repository:/models \
    nvcr.io/nvidia/tritonserver:23.12-py3 \
    tritonserver --model-repository=/models
Enter fullscreen mode Exit fullscreen mode
Collapse
 
nkko profile image
Nikola Balic •

Great, would be even better to have some tests and results like tokens/sec for some models.

Collapse
 
darkpowerxo profile image
Sam Abtahi • • Edited

Anyone knows where to buy this in canada? Other than the the 999$ option on amazon instead of the 250 ?

Collapse
 
ajeetraina profile image
Ajeet Singh Raina • • Edited

Did you check https://www.amazon.ca/dp/B0BZJTQ5YP?ref=cm_sw_r_cso_cp_apin_dp_RBZDTNN21DGZNMPPJ3GW&ref_=cm_sw_r_cso_cp_apin_dp_RBZDTNN21DGZNMPPJ3GW&social_share=cm_sw_r_cso_cp_apin_dp_RBZDTNN21DGZNMPPJ3GW&starsLeft=1 Look like it’s out of stock.