Forem: Shivnath Tathe

Run a 397B AI Model for Free Using Claude Code (3 Commands)

Shivnath Tathe — Mon, 06 Apr 2026 12:18:10 +0000

Most people think you need expensive APIs or a powerful GPU to run large AI models.

You don't. Here's how I ran a 397B parameter model for free in under 5 minutes on Windows.

What You Actually Need

A Windows machine
An Ollama account (free)
Internet connection

That's it. No GPU. No API key. No Anthropic billing.

What's happening under the hood

Two things working together:

Claude Code CLI is just the terminal interface and agent shell. Nothing goes through Anthropic's servers.
Ollama hosts and runs Qwen3.5 397B on their own cloud infrastructure, completely free

Claude Code supports Ollama as a backend. So you get a familiar, powerful agent interface while Ollama handles 100% of the actual inference. Anthropic is not involved beyond providing the CLI shell.

Step 1: Install Claude Code

Open PowerShell and run:

irm https://claude.ai/install.ps1 | iex

Step 2: Install Ollama

irm https://ollama.com/install.ps1 | iex

Step 3: Launch Claude Code with the 397B model

ollama launch claude --model qwen3.5:397b-cloud

You'll be asked to verify with your Ollama account once. After that it just works.

What you get

Full Claude Code agent interface
397B parameter model (Qwen3.5)
~256K token context window
Cloud hosted, so no local GPU needed
Free as long as you have an Ollama account

Important notes

This routes through Ollama's cloud, so you need internet
Don't use it for sensitive or private data
Free access may change in the future, use it while it lasts

Why this matters

We're entering a phase where the agent layer and the model layer are completely separate.

Claude Code is just the interface. The model underneath can be swapped. Local or cloud, open or closed, free or paid.

This setup is a simple example of that shift. The barrier to running frontier-scale models is no longer hardware or money. It's just knowing how to connect the right tools.

Try it

Three commands. Less than 5 minutes. No credit card.

If you run into issues drop them in the comments.

I research LLM training and continual learning. Follow for more no-fluff AI content.

Your AI Is Not Thinking. It's Multiplying Numbers. Let Me Show You Exactly How.

Shivnath Tathe — Mon, 06 Apr 2026 06:25:24 +0000

Everyone's talking about AI like it's magic. I work with it daily. It's not. Here's what's actually happening inside.

I've fine-tuned LLMs. I've published research on them. I've built systems around them.

And the single most honest thing I can tell you about large language models is this:

At the bottom, it's matrix multiplication. That's it.

Not intelligence. Not reasoning. Not understanding. Matrices of floating point numbers being multiplied together, billions of times per second.

But here's the uncomfortable part. That doesn't mean nothing interesting is happening.

Let me break this down without the hype, without the doomsaying, and without the marketing.

What a "Model" Actually Is

Forget the word "model." It carries too much baggage.

What you're actually dealing with is a file. A very large file full of numbers, floats arranged in matrices. GPT-2 has 117 million of them. GPT-3 has 175 billion. These numbers are called weights.

That's the model. Numbers in a file.

When you send a message to an LLM, here's what happens mechanically:

Your text gets converted to tokens (integers from a vocabulary)
Each token gets looked up in an embedding table (a matrix) and becomes a vector
That vector passes through N identical blocks, each doing attention (matmul) and feedforward (matmul + nonlinearity)
Final layer produces logits over the vocabulary
Sample from that distribution and get the next token

Repeat until done.

No memory. No state. No "thinking." Pure function application.

What "Training" Actually Is

This is where people get philosophical, so let me be precise.

Training is not teaching. There's no curriculum, no explanation, no understanding being transferred.

Here's the actual process:

Show the model some text
It predicts the next token
Compare prediction to actual next token, compute loss (a single number)
Backpropagate gradients through every matrix
Nudge every weight by a tiny amount in the direction that reduces loss
Repeat roughly a trillion times

The only signal the model ever receives is: your probability distribution over the next token was wrong, adjust.

No grammar lessons. No semantic explanations. No world knowledge explicitly provided. Just: you were wrong, here's by how much, here's which direction to shift.

And yet grammar emerges. Semantics emerges. World knowledge emerges.

That's not magic. That's what happens when you apply a single optimization pressure billions of times across the entire written record of human thought.

Why Next-Token Prediction Even Works

This is the question nobody asks clearly enough.

The implicit assumption is that predicting the next word is a shallow task. A parlor trick.

It's not. Here's why.

Language is not random. Language is massively structured at every level simultaneously:

Surface statistics: "New York" appears together with near-deterministic frequency
Syntax: "The ___" expects a noun or adjective, not a verb. Every time.
Semantics: "eat" expects a food object. "drink" expects liquid. Violations sound wrong.
World knowledge: "Paris is the capital of ___" has essentially one answer across millions of documents
Discourse: A medical article doesn't randomly switch to cooking. Topics are coherent.
Causal structure: "He dropped the glass. It ___" and physics is implicitly encoded because text describing physics is consistent

To predict the next token accurately across all of these simultaneously, the model is forced to learn all of these regularities. Not because anyone labeled them. Because they're all load-bearing for loss reduction.

Shannon estimated English has roughly 1 to 1.5 bits of true entropy per character. Language is not a high-entropy signal. It's a highly compressible, deeply structured one.

Next-token prediction works because language itself is learnable. The model just exploits that ruthlessly.

What the Weights Actually Encode

Here's the honest answer: nobody fully knows.

What we do know from interpretability research is directional. Early layers tend to capture surface patterns, later layers tend to capture more abstract task-relevant signals. But it's distributed, messy, and not cleanly decomposable.

You cannot point to a weight and say "this one handles subject-verb agreement."

What the weights collectively store is a giant tangled function that behaves as if it knows grammar, semantics, world facts, and reasoning patterns. Because that behavior is what minimizes loss on human-generated text.

It's not a list of rules. It's a compressed statistical model of human language and thought, discovered purely by gradient pressure.

"Just Matrix Multiplication" Is Not the Same as "Trivial"

This is where I'll push back on the reductive take, including my own initial instinct.

Yes, it's matmul. But DNA is just chemical interactions. The brain is just electrical signals. Both produce systems of staggering complexity.

The Universal Approximation Theorem tells us that stacked layers with nonlinearities can approximate any function given enough capacity. The architecture isn't doing something magical. But the composition of many simple operations produces a function of extraordinary complexity.

"Just matmul" at the mechanistic level does not imply "nothing meaningful" at the behavioral level.

What it does imply is that the meaningful behavior is not designed, it's emergent. And that's a genuinely different and more honest framing than either "it's just statistics" or "it's basically thinking."

The Transformer Is Not The Only Answer

Here's something the hype cycle obscures: the transformer architecture is not special in principle. It's dominant in practice.

What you actually need to exploit language regularities:

A way to consume sequential input
A way to condition predictions on context
Enough capacity to store learned regularities
A next-token prediction objective on enough data

RNNs satisfied all four. So did CNNs on text. So do State Space Models like Mamba, which use no attention at all, run in O(n) instead of O(n squared), and are competitive with transformers on many benchmarks today.

The transformer won because attention handles long-range context without compression, and because it parallelizes perfectly on GPUs. The hardware fit mattered as much as the architecture itself.

Mamba is a serious contender. Hybrid architectures mixing SSM and attention layers are emerging. Five years from now, the dominant architecture is probably neither pure transformer nor pure SSM.

And honestly? It's probably something nobody is currently working on.

What We Still Don't Know

I want to be precise here because this matters.

We don't know why scaling works.

We know that scaling works. More parameters plus more data plus more compute equals better performance, with surprising consistency. But the mechanism by which scaling produces emergent capabilities, tasks the model couldn't do at smaller scale suddenly working at larger scale, is genuinely unresolved.

"Combinatorial pattern composition" is a label for the mystery, not an explanation of it.

We built something that behaves intelligently. We know the training procedure in full detail. We do not know why that procedure, at scale, produces the behavior it does.

That's not a gap that marketing will fill. That's an open research problem.

The Real Summary

Claim	Truth
It's just matrix multiplication	True, mechanistically
Nothing meaningful is happening	False
It understands language	False
It learns statistical structure	True
We fully understand why it works	False
The transformer is the final architecture	Almost certainly false

Why This Framing Matters

If you think LLMs are magic, you'll use them wrong. You'll trust outputs that are confident but wrong, anthropomorphize failure modes, expect capabilities that aren't there.

If you think LLMs are "just statistics" and therefore uninteresting, you'll also use them wrong. You'll dismiss genuine capabilities, fail to understand where they're reliable vs brittle.

The accurate framing is: a large function approximator trained via optimization that exhibits emergent structured behavior, whose full mechanism we don't yet understand.

Not magic. Not trivial. Something genuinely new that deserves precise thinking.

I research LLM training, continual learning, and quantization. If this sparked something, let's discuss in the comments.

Attention Is All You Need — Explained Like You’re Building It From Scratch

Shivnath Tathe — Thu, 02 Apr 2026 05:32:13 +0000

Everyone has seen this diagram.

And almost everyone says:

“Transformers use attention instead of recurrence.”

But that doesn’t actually explain anything.

So let’s rebuild this from scratch — the way you would understand it if you were designing it yourself.

🧠 The Real Problem Transformers Solve

Before transformers, models like RNNs and LSTMs processed text like this:

One word at a time → sequentially

This caused two big problems:

Slow training (no parallelism)
Long-range dependencies break

Example:

“The cat, which was sitting near the window for hours, suddenly jumped.”

To understand “jumped”, the model needs context from far back.

RNNs struggle here.

⚡ The Core Idea

Instead of processing words one by one:

What if every word could look at every other word at the same time?

That’s attention.

🔍 What “Attention” Actually Means

Forget formulas for a second.

Think like this:

Each word asks:

“Which other words are important for me?”

Example:

Sentence:

“The animal didn’t cross the street because it was tired.”

What does “it” refer to?

animal?
street?

Attention helps the model decide.

⚙️ How It Works (Intuition First)

Each word is converted into a vector.

From that vector, we create:

Query (Q) → what I’m looking for
Key (K) → what I offer
Value (V) → actual information

🧠 Matching Process

Every word does:

Compare its Query with all Keys

This gives:

similarity scores

Then:

normalize (softmax)
use scores to combine Values

💡 Translation:

“Take information from important words, ignore the rest”

🔥 Multi-Head Attention (Why multiple?)

One attention head might learn:

grammar

Another:

relationships

Another:

positional meaning

So instead of one view:

We use multiple perspectives in parallel

🧱 Transformer Block (Now the Diagram Makes Sense)

Each block has:

1. Attention

Words interact with each other

2. Add & Norm

Stabilizes training

3. Feed Forward

Processes each token independently

🔁 Why “Add & Norm”?

This is often ignored but critical.

It keeps gradients stable and prevents information loss

Without it:

deep transformers won’t train well

⚡ Encoder vs Decoder

From your diagram:

Left side → Encoder

Reads input
Builds representation

Right side → Decoder

Generates output
Uses:
- masked attention (can’t see future)
- encoder output

🔒 Masked Attention (Important for LLMs)

When generating:

Model should not see future words

So we mask them.

🚀 Why Transformers Changed Everything

Because they:

Allow parallel computation
Handle long-range dependencies
Scale extremely well

💥 The Real Insight

Transformers don’t “understand language”.

They do something simpler but powerful:

They learn relationships between tokens

🧠 Connecting to LLMs

When you do:

model.generate("Hello")

What happens?

Each token attends to previous tokens
Builds context dynamically
Predicts next token

🔥 Final Thought

The paper says:

“Attention Is All You Need”

But the deeper idea is:

You don’t need sequence —
You need relationships

Once you understand that…

Transformers stop looking complex
and start looking inevitable.

If you're building models or exploring low-bit training like I am, this perspective changes everything.

Because now the question becomes

How efficiently can we compute these relationships?