Forem: pickuma

Why Your Backtest Is Lying to You: Data Biases That Ruin Equity Research

pickuma — Tue, 19 May 2026 05:24:13 +0000

You built a backtest. It returns 18% annualized over the last
decade. The Sharpe is 1.4. The drawdown is acceptable. You're
already mentally apartment-shopping in the city you'll move to
after the strategy makes you wealthy.

It's probably wrong. Not because your code has bugs — backtest
code can be flawless and still produce numbers that have almost
nothing to do with what would have happened if you'd traded the
strategy live. The reasons are subtle and they have names.

Survivorship bias

The most famous one. If your historical universe is "S&P 500
today, backtested over ten years," you've quietly excluded every
company that was in the S&P 500 at some point and got booted out
— usually because it failed.

The companies that died don't show up in the dataset. Your
historical "S&P 500" is actually "the winning subset of the
S&P 500 visible from the future." Returns are biased upward,
sometimes by several percentage points a year for small-cap or
emerging-market strategies.

Fix: use a point-in-time index membership database, or build
one. CRSP and similar paid datasets handle this; free options
require more work and willingness to scrape historical index
constituent lists.

Look-ahead bias

You computed the trailing-twelve-month P/E using today's reported
financials. Trouble: those financials weren't available at the
time. Earnings get reported 30–90 days after a quarter ends, and
restatements happen after that.

If you're ranking companies on March 31, 2018, you should be
using financials that were actually filed and disclosed by
March 31, 2018 — not the latest revision visible today.

Fix: use point-in-time fundamentals. EDGAR preserves the
original filing dates, so it can be done, but it requires care.
Most price-data vendors hand you the latest-available value with
no timestamp — that's the bias source.

Universe selection bias

A close cousin of survivorship. If you backtest on "stocks that
have at least 10 years of clean fundamental data," you've
selected for stability — companies that didn't go bankrupt,
didn't merge, didn't get acquired, didn't change reporting
standards. Your universe is structurally biased toward survivors.

Fix: define your universe as a point-in-time set ("Russell
3000 as of the rebalance date") and accept that some constituents
will have noisy or incomplete data.

Slippage and transaction costs

You assumed you traded at the closing price with zero cost. In
reality, large orders move the market, bid-ask spreads exist,
and small-cap stocks can be impossible to fill at the quoted
price.

A strategy that backtests at 15% gross return after slippage and
costs is often 8–10% net. The smaller your average market cap
and the higher your turnover, the worse the gap.

Fix: model slippage explicitly. A reasonable starting point
is 20–50 basis points per round-trip for liquid stocks, 100+ bps
for small caps. Charge it to every trade.

Backtest overfitting

You ran 200 parameter combinations and the best one delivered
22%. Congratulations: you've discovered noise in your dataset,
not signal. By construction, the best of 200 trials on the same
data will look impressive even if the underlying strategy has no
edge.

Fix: walk-forward testing. Develop on one window, test on a
later out-of-sample window you never touched during parameter
selection. If the out-of-sample period underperforms
substantially, the rule was likely overfit. Bonferroni-style
corrections for multiple-testing apply here too.

Calendar and currency cleanup

Subtler than the rest, but important:

Time zones. "Closing price on date X" depends on which exchange and which time zone X refers to. Cross-listed names bite first.
Corporate actions. Stock splits and dividends have to be back-adjusted consistently. Yahoo Finance gives you adjusted prices; sometimes the adjustments are off for small caps or delisted names.
Currency. A non-USD strategy backtest needs FX adjustment at the same point in time as the prices, not at today's rate.

If your backtest dramatically outperforms the broad market over
a long window, treat that as a red flag, not a celebration.
Persistent strategy edges are uncommon and small. A 5%-per-year
alpha is exceptional. A 15%-per-year alpha is almost always a
data bias.

The honest closing

A backtest is a hypothesis, not a result. The version of you who
will trade the strategy live doesn't have access to next year's
returns — only to today's beliefs. The point of correcting for
biases isn't to make the backtest look worse; it's to make the
backtest tell you something true about what to expect.

If after correcting for all of the above, your strategy still
shows a clean edge: trade a small amount of real money for six
months and see if the live numbers match. If they don't, you've
learned the cheap version of the lesson everyone else learns the
expensive way.

Originally published at pickuma.com. Subscribe to the RSS or follow @pickuma.bsky.social for new reviews.

Index Funds, ETFs, Mutual Funds: What Actually Differs (and When It Matters)

pickuma — Tue, 19 May 2026 05:22:57 +0000

The marketing for retail investment products collapses three
distinct things into one fuzzy idea: "passive, low-cost, set it
and forget it." The labels — index fund, ETF, mutual fund — get
used interchangeably even though they describe different
structures with different mechanics.

For most people most of the time, the difference is small enough
to ignore. For some people some of the time, the difference is
big enough to matter. Here's where the lines actually are.

Quick definitions

Mutual fund: a pooled investment vehicle where shares are bought from and redeemed by the fund company at the day's NAV (net asset value). Settles end-of-day, not intraday.
ETF (exchange-traded fund): also a pooled vehicle, but shares trade on an exchange like a stock. Settles intraday. Uses an "authorized participant" creation/redemption mechanism that has tax implications.
Index fund: a strategy descriptor, not a structure. An index fund tracks a benchmark (S&P 500, total market, etc.). Index funds can be either mutual funds or ETFs.

So "mutual fund vs ETF" is a structural question. "Index fund vs
active fund" is a strategy question. They cut across each other.

Side-by-side

The tax line is the one most retail investors don't think about
and shouldn't ignore.

The tax structure difference, briefly

When investors redeem mutual fund shares, the fund may have to
sell underlying holdings to meet the redemption. Those sales
generate capital gains, which the fund distributes to all
remaining shareholders at year-end. You can owe tax on a fund
you've held for years even if you didn't sell a thing.

ETFs use a creation/redemption mechanism with authorized
participants where shares are exchanged for baskets of underlying
securities "in kind." That largely sidesteps the capital-gains
distribution problem. In taxable accounts, this is often the
single biggest practical difference.

In retirement accounts (401(k), IRA), the distinction is
irrelevant because the account itself shields the gains.

When the difference matters

Taxable brokerage account. ETF wins on tax efficiency, by enough to justify a 10–15 bps expense-ratio premium in many cases.
401(k) menu. You usually only have mutual fund options anyway. Pick the cheapest broad-market index option; the structural question is moot.
Fractional dollar investing. Some brokers let you buy fractional ETF shares; some don't. Mutual funds always settle in dollar amounts.
Active strategy access. Many active strategies are only available as mutual funds. If you specifically want a particular manager, you may not have a choice.

When it doesn't matter as much as people think

Returns of a broadly equivalent index fund vs ETF over a decade. Within 10–30 basis points for the same underlying index. The tax difference dominates the return difference for taxable accounts.
Expense ratios. The gap between mutual fund and ETF expense ratios on broad indexes has closed dramatically over the last fifteen years. A broad-market mutual fund and the corresponding ETF often charge nearly the same.

The most common mistake in this space isn't structural — it's
holding actively managed mutual funds in a taxable account.
Annual capital-gains distributions plus higher expense ratios
compound badly. If you're in that position, the question to
research first is "would an equivalent ETF lower my tax bill,"
not "is ETF or mutual fund better in general."

What people get wrong

"ETFs are riskier because they trade intraday." Not really. Intraday liquidity is a feature, not a leverage source. The underlying portfolio is what determines risk.
"Index funds always beat active funds." On average, over long horizons, after fees, yes — but "always" is too strong. The S&P-vs-active comparison varies by asset class, decade, and the particular cohort of active managers in the study.
"Mutual funds are obsolete." They aren't, especially in retirement accounts and for active strategies. They're a structure, not a relic.

The shortest useful summary

Pick the structure that fits where the money lives. Taxable
account → ETF, almost always. Retirement account → whichever
your plan offers, usually mutual fund. Active manager you
specifically want → mutual fund, accept the trade-offs. The
structure question matters less than picking the right index
and keeping costs low.

Originally published at pickuma.com. Subscribe to the RSS or follow @pickuma.bsky.social for new reviews.

What Diversification Actually Buys You (and What It Doesn't)

pickuma — Tue, 19 May 2026 05:21:41 +0000

"Diversification" is one of those finance words that does a lot
of marketing labor without much teaching labor. Every robo-
advisor mentions it. Every target-date fund's pitch implies it.
The definition that actually matters — what diversification does
and doesn't do — tends to get lost in the slogan.

The short version: diversification reduces the risk you don't
get paid for taking. It doesn't reduce all risk. The distinction
matters most exactly when people forget about it.

The mechanical idea

If you hold two assets whose returns are perfectly correlated,
the portfolio variance equals the weighted average of the
individual variances. You've diluted nothing.

If you hold two assets whose returns are less than perfectly
correlated, the portfolio variance is lower than the weighted
average. The amount of "free" risk reduction depends on how
uncorrelated the assets are.

That's diversification's claim: lower portfolio risk without
lower expected return. It's one of the few things in investing
that actually qualifies as a free lunch — but only within
limits.

Idiosyncratic vs systematic risk

The standard breakdown:

Idiosyncratic (specific) risk — the risk that one company fails for reasons specific to it. Accounting fraud, CEO scandal, a failed product launch.
Systematic (market) risk — the risk that everything drops together. Recession, rate shock, geopolitical event.

Diversification across enough individual stocks within an asset
class eliminates most idiosyncratic risk. Holding 30–50 stocks
covers most of it; holding 500 covers nearly all of it.

It cannot eliminate systematic risk. When the broad market
drops 30%, your diversified portfolio of US large-cap stocks
also drops roughly 30%. The fact that you held 500 names instead
of 1 doesn't help in that scenario.

Where the slogan breaks: correlation under stress

The number that diversification is built on — the correlation
between assets — isn't stable. In normal markets, the
correlation between US stocks and international stocks might
sit around 0.5. In a crisis (2008, March 2020, 2022), those
cross-equity correlations spike toward 1.

"Diversification fails when you need it most" is a real
phenomenon, not just a cliché. The assets that protected you in
calm years can all sell off together in panic months.

The implication: a portfolio diversified across "things that are
all equity-like" — US stocks, international stocks, REITs,
commodities — is less diversified in a crisis than the calm-
period correlations suggest.

What actually still diversifies in stress

A short list of asset relationships that have historically held
up reasonably well in past crises:

High-quality government bonds vs equities. Negative correlation during deflationary panics (notably 2008). Note: not in inflationary regimes — 2022 broke this.
Cash. Boring, but uncorrelated by construction.
Some long-volatility positions. Expensive to hold in calm periods.
Gold (sometimes). Inconsistent — works in some crises, doesn't in others.

The honest summary is that true crisis diversification is
expensive. It either costs you in calm periods (cash drag,
volatility premium decay) or it's unreliable (gold).

What people get wrong

"More holdings = more diversification." Adding the 200th S&P 500 name adds almost no diversification — you already captured the systematic risk by name 30. Adding a different asset class would help; adding more of the same kind doesn't.
"60/40 is a diversified portfolio." It's two assets. It's better than 100% stocks for most goals, but it's "lightly diversified," not "well-diversified" in the academic sense.
"Diversification means I won't lose money." It means you'll likely lose less than the unluckiest single bet in a normal market. It doesn't mean you'll lose nothing in a market-wide drop.
"International diversification doesn't work because correlations are high." Correlations are higher than they used to be, but not 1.0. The free lunch is smaller, not gone.

The useful framing

Diversification is a tool with a specific job: reduce the
variance of outcomes around the same expected return. It does
that job well within asset classes and partially across asset
classes. It does not do the job of eliminating market risk, and
nothing else does either — at least, not without paying for it.

The right question isn't "am I diversified" but "what risks am I
still exposed to, and am I being compensated for them?"
Systematic equity risk is compensated, in expectation, by the
equity risk premium. Single-name risk is not. Diversification is
the mechanism that turns the second kind into the first.

Closing

Diversification is more arithmetic than philosophy. Below a
certain level of holdings it adds a lot. Above that level it
adds little. Across asset classes it helps in calm periods and
partially in stress periods. The job it does is real and worth
doing — the job it doesn't do is the one the marketing implies.

Knowing the difference is most of what makes diversification
useful in practice.

Originally published at pickuma.com. Subscribe to the RSS or follow @pickuma.bsky.social for new reviews.

SEC EDGAR for Developers: The Free Fundamentals API Hiding in Plain Sight

pickuma — Tue, 19 May 2026 05:20:26 +0000

You finished the comparison shopping between Polygon and Alpha
Vantage, maybe even signed up for a free tier of each. Now you
want to pull fundamentals — revenue, earnings, free cash flow —
for your screener.

Don't reach for the paid tier just yet. The SEC publishes every
filing from every public US company through a free, well-
structured JSON API at data.sec.gov. It's unrated, public, and
the data is straight from the source.

What EDGAR actually exposes

Three endpoints do most of the work:

/submissions/CIK{cik}.json — every filing a company has made, with dates, accession numbers, and direct links to the documents.
/api/xbrl/companyfacts/CIK{cik}.json — every reported XBRL fact for that company, across all filings, indexed by concept (Revenues, NetIncomeLoss, Assets, etc.) and unit.
/api/xbrl/companyconcept/CIK{cik}/us-gaap/{concept}.json — just one concept across all reported periods.

The companyfacts endpoint is the workhorse. One JSON request
and you have every quarterly and annual fact the company has
filed with the SEC since they switched to XBRL in 2009.

The annoying part: CIK lookup

EDGAR keys companies by CIK (Central Index Key), not ticker.
You'll need to maintain or fetch a ticker → CIK map. The SEC
publishes one:

curl -A 'your-name you@example.com' \
  https://www.sec.gov/files/company_tickers.json

The User-Agent header is required — EDGAR rate-limits anonymous
requests and asks for an identifying string. They throttle at 10
requests/second across all clients; respect it.

A minimal example

Here's the smallest useful thing — pulling the last 4 quarterly
revenues for Apple (CIK 0000320193):

import requests

headers = {'User-Agent': 'side-project you@example.com'}
url = (
    'https://data.sec.gov/api/xbrl/companyconcept/'
    'CIK0000320193/us-gaap/Revenues.json'
)

r = requests.get(url, headers=headers)
data = r.json()

# units may include USD and USD/shares; pick USD
usd = data['units']['USD']
# 10-Q quarterly filings only
quarterly = [f for f in usd if f.get('form') == '10-Q']
quarterly.sort(key=lambda f: f['end'], reverse=True)
for f in quarterly[:4]:
    print(f"{f['end']}: ${f['val']:,}")

That's it. Free, structured, official.

When EDGAR wins

No rate-limit drama for any volume a side project can generate.
Authoritative — straight from filings. No vendor between you and the company's own numbers.
Historical depth since 2009 for most large filers, earlier in some cases.

Where EDGAR struggles

Concept fragmentation. Companies don't all use the same XBRL concept for the same thing. Apple uses Revenues; some others have used SalesRevenueNet or company-specific extensions. Real cleanup work.
Restated filings. When a company restates an earlier quarter, EDGAR contains both the original and the restated values. Your code has to decide which one is "the truth" for backtest purposes.
Calendar mismatch. Companies report on different fiscal calendars. You can't naively compare one issuer's Q1 ending in December to another's Q1 ending in September.
No price data. EDGAR is filings, not market data. You still need Polygon, Alpha Vantage, or similar for OHLC.

A reasonable production setup

For a magic-formula-style screener:

Maintain a local cik_map table (refresh weekly from SEC's company_tickers.json).
For each ticker in your universe, fetch companyfacts once and cache. Refresh on a quarterly cadence — fundamentals don't change daily.
Normalize concepts to your own internal names (revenues, net_income, total_assets, etc.) with a hand-curated mapping that tolerates fragmentation.
Get prices from your paid or free price-data API.
Run your ranking once a week — it's cheaper than the daily refresh cadence most tutorials suggest.

The hardest part of building a fundamentals pipeline is not the
API — it's the concept normalization. Budget more time for
"decide what counts as revenue across 500 issuers" than for
"wire up the HTTP request." Get it wrong and your screener
silently ranks the inconsistent reporters.

Closing note

The fundamentals data that paid APIs charge for is largely a
cleaned-up, normalized version of what EDGAR already gives you
for free. If your project is willing to do the cleanup, EDGAR is
the better foundation. If you'd rather pay a vendor to handle the
normalization, that's also a reasonable choice. Either way,
knowing the raw source exists changes how you think about the
cost of building.

Originally published at pickuma.com. Subscribe to the RSS or follow @pickuma.bsky.social for new reviews.

The Boring Math of Compound Returns (And Why Everyone Gets It Wrong)

pickuma — Tue, 19 May 2026 05:19:10 +0000

Compound interest is the kind of topic that gets called "the
eighth wonder of the world" in motivational graphics. The actual
math is mundane. The most common rules of thumb are subtly
wrong. And the most-shared retirement charts make assumptions
about return paths that real markets rarely deliver.

This isn't a takedown of compounding — it's a takedown of how
compounding gets explained. The reality is calmer and more
useful.

The basic formula isn't the whole story

The textbook compound formula is final = principal × (1 + r)^n.
That works perfectly if every year delivers exactly r.

Markets don't. Returns vary. The arithmetic average of yearly
returns and the actual compound return are not the same number.

Arithmetic vs geometric mean

Take an investment that returns +50% in year one and -50% in
year two.

Arithmetic mean: (50% + (-50%)) / 2 = 0%. "Average return was zero."
Geometric mean (actual outcome): start with $100, end at $150, then $75. The compound return is −25% over two years — about −13.4% per year.

The geometric mean is always ≤ the arithmetic mean, with
equality only when every return is identical. This gap is called
volatility drag or the variance penalty.

For a roughly stable broad-market portfolio, the gap is small
but not negligible — typically around 1–2% per year. For a
volatile asset (an individual small-cap, a leveraged ETF, a
single cryptocurrency), the gap can be much larger.

Why this matters in practice

Most "average return" figures you see in retail finance content
are arithmetic. The wealth you'd actually have is geometric.
Over decades, the difference compounds in the wrong direction.

A reasonable approximation: geometric ≈ arithmetic − (variance / 2).
That subtraction is the variance penalty in dollar form. Over
twenty or thirty years, it's the difference between a clean
projection and the path you'd actually live through.

The Rule of 72 (and where it fails)

"Years to double = 72 / interest rate." At 8%, money doubles in
about 9 years. The rule is a Taylor-series approximation that's
accurate around 6–10% and breaks down at higher rates.

At 12%: rule says 6 years, actual ~6.1. Fine.
At 20%: rule says 3.6 years, actual ~3.8. Off.
At 50%: rule says 1.44 years, actual ~1.7. Misleading.

For low rates, the rule slightly overestimates the doubling
time. The "Rule of 70" is closer for very low rates; the "Rule
of 72" is the practical compromise because 72 has more divisors
and gives nicer mental arithmetic.

Sequence of returns risk

Two retirees with identical average returns can end up with very
different outcomes if the returns arrive in different orders.

If you retire just before a bad decade for equities and are
drawing down a portfolio, the drawdowns compound against you in
a way that doesn't reverse just because returns recover later.
The withdrawals from a depressed portfolio book the losses
permanently.

This is why fixed-rate retirement projections ("assume 7% return
forever") overstate the safety of withdrawal rates. The same 7%
delivered with realistic year-to-year variance gives a wider
distribution of outcomes, with the bad tail materially worse
than the smooth path implies.

The fix isn't to abandon compounding — it's to use stochastic
projections (Monte Carlo) instead of single-path ones,
especially near and during the withdrawal phase.

What the "start early" charts get right and wrong

The right part: time in the market matters. Contributing
consistently from age 25 to 35 (then nothing) often ends up with
more than contributing the same amount per year from 35 to 65,
assuming the same return path.

The misleading part: that calculation assumes a flat compound
return for the entire period. Real return paths have decades-
long regimes — some decades have been great for US equities,
some have been bad. The "start early" math is correct in
expectation; the actual outcome for any specific cohort depends
heavily on which decades they were contributing during.

The single most useful rule of thumb for back-of-envelope
compounding is: doubling at 7% takes ~10 years (slightly under),
quadrupling takes ~20, an 8x takes ~30. Memorize that grid and
you can sketch realistic outcomes without a calculator.

What this means for decisions

Don't conflate the average return you read in marketing copy with the wealth path you'd actually experience. Volatility drag is small per year and large over decades.
Don't use Monte Carlo for everything, but don't use single-path 7%-forever models when the answer matters. The smoothness hides risk.
Time in the market is a real lever, but the magnitude depends on the regime you're contributing during, not just on the number of years.

Compounding is real, useful, and reliable. The mystification
around it is the problem.

Originally published at pickuma.com. Subscribe to the RSS or follow @pickuma.bsky.social for new reviews.

What 'Rules-Based Investing' Actually Means (And What It Doesn't)

pickuma — Tue, 19 May 2026 05:06:15 +0000

Open any investing subreddit and you'll find someone asking which
"AI-powered" trading service to subscribe to. The marketing makes
it sound like there's a secret algorithm sitting behind a paywall.
There usually isn't. Most of what gets sold as "AI investing" is
the same thing the old quant funds were doing in the early 2000s,
repackaged with newer buzzwords.

The actual category most of these strategies belong to is
rules-based investing — and it's worth understanding what that
means, because the term covers both very sensible discipline and
very silly product marketing.

The one-line definition

A rules-based investor decides what trigger means what action
before the event happens. The rule is written down. When the
trigger fires, the action is taken, regardless of how the investor
feels in the moment.

That's it. There's no claim of secret knowledge. There's no claim
about predicting markets. The whole point is to make the future
decision now, when emotions aren't loaded.

What it actually looks like

Examples that fit the definition:

Index rebalancing. "On the first trading day of January, sell whatever is now more than 5% above its target weight in the portfolio and buy whatever is more than 5% below." That's a rule.
Dollar-cost averaging. "On the 15th of every month, buy a fixed dollar amount of a broad-market ETF." Also a rule.
Stop-losses. "If a position drops 10% from the entry price, sell." A rule, applied without renegotiation.
The magic formula. "Once a year, rank stocks by earnings yield and return on capital, hold the top 30 for a year." Joel Greenblatt's version — also a rule.
Trend-following. "If the 50-day moving average crosses above the 200-day, increase equity allocation by X%." A more complicated trigger, same shape.

Notice that none of these claim to predict anything. They're all
about what to do in pre-specified situations.

What rules-based investing isn't

Three common misuses of the term:

It's not high-frequency or algorithmic trading. HFT is a different sport entirely — sub-millisecond latency arbitrage, market-making, statistical exploitation of order-book microstructure. You will not do that from your laptop.
It's not "AI investing." A neural network choosing stocks based on price patterns is also rules-based — just with an opaque rule. The label is marketing, not mechanism.
It's not a guarantee. A rule that has worked for a decade can fail in the eleventh year. Rules-based investing reduces certain kinds of mistakes; it does not eliminate all of them. Past performance is not predictive.

Why it actually works

The case for writing your rules down has nothing to do with
finding hidden alpha. It's behavioral.

When markets move violently, your brain wants to do something.
Selling at the bottom feels safe; buying the dip feels brave; both
feelings are misleading more often than they are useful. A
pre-committed rule short-circuits this. You execute the action
because the action was decided when you were calm.

The widely cited Fidelity Magellan observation — that the average
investor in that fund underperformed the fund itself, because they
bought after rallies and sold after drawdowns — is the canonical
illustration. The rule that says "don't react to last quarter's
returns" is unsexy and reliable.

Where rules-based strategies fail

Three failure modes are worth knowing before you commit to one:

Backtest overfitting. A rule that worked beautifully in historical data may have been fit to the noise of that data. The smaller and more recent the backtest window, the less you should trust the result. Walk-forward testing helps, but it doesn't fully cure the disease.
Regime change. Rules calibrated to one market environment can silently break when the environment shifts. The 60/40 stocks-and-bonds rule that ran smoothly from 1980 to 2020 came under real strain in 2022 when both legs sold off together.
Abandonment under stress. A rule only works if you actually follow it. A stop-loss you ignore the second time it fires is worse than no stop-loss at all.

A "rule" you keep changing is not a rule. If you find yourself
adjusting parameters because the recent results were uncomfortable,
you're back to discretionary investing — just with extra steps and
the illusion of discipline. Pick rules you can survive emotionally,
not just rules that look best in backtest.

How to start without overthinking it

If you want to put a rule in place this month:

Pick a frequency. Monthly is a common starting cadence. Weekly is too noisy for most people; quarterly is fine and reduces transaction costs.
Pick an asset universe. Broad index ETFs are sufficient for most people. You don't need to start with individual stocks.
Write the rule down. Two sentences, in a text file. Date it.
Execute and don't second-guess for at least four cycles. If after a year the rule isn't working, that's information. If after a month you're tempted to change it, that's you, not the rule.

That's the entire methodology. The rest is patience.

The honest closing

Rules-based investing isn't magic. It doesn't outperform the
market on average — by construction, broad rule-following can't.
What it does is make your behavior more predictable than your
emotions, which for most people is the single largest improvement
available without a finance degree.

That's a real edge. It just isn't the edge the product marketing
implies.

Originally published at pickuma.com. Subscribe to the RSS or follow @pickuma.bsky.social for new reviews.

Polygon vs Alpha Vantage: Which Stock Data API Fits a Side-Project Budget

pickuma — Tue, 19 May 2026 05:04:59 +0000

You want to build a stock screener. Maybe a magic-formula scan, maybe
a custom backtest, maybe just a dashboard that pings the close of your
watchlist every evening. You hit the data wall fast: free tiers from
the big-name APIs come with throttles or restrictions that turn a
weekend project into a debugging marathon.

The two most common entry points for developer-side stock data are
Polygon.io and Alpha Vantage. They occupy slightly different
positions — Polygon is the modern, well-funded data provider with
fast-moving infrastructure; Alpha Vantage is the older, broader-coverage
service that has been the default "free tier exists" answer since 2017.
Which one fits depends almost entirely on what you're building.

What you actually need from a stock data API

For most side projects, three things matter more than anything else:

What does the free tier let you do? A free tier that returns end-of-day OHLC for the full US universe is enough to run a magic-formula screen. A free tier rate-limited to 25 requests a day is not.
How wide is the coverage? Are you only looking at US stocks, or do you need options chains, forex, crypto, futures, or fundamentals?
Is the developer experience smooth? API key issuance, SDK quality, documentation freshness, response shape. The two providers are not equivalent here.

Latency, historical depth, and tick-level data also matter — but if
you're a side-project developer, you almost certainly don't need
microsecond-grade data. Daily or minute-level is plenty.

Pricing and free tiers at a glance

The shape below reflects each provider's published tiers at the time
of writing. Pricing pages change — verify before you commit.

The most important thing this table hides: Alpha Vantage's free-tier
limit dropped sharply over the last few years. If you're following an
older tutorial that assumes 500 requests a day, that's no longer the
deal. Polygon's free tier, by contrast, is limited per-minute rather
than per-day, which is more forgiving for development but less
forgiving for a hosted job.

Coverage breakdown

US equities (the common case)

Both providers have full US equity coverage on paid tiers. On free
tiers, both work — Polygon gives you per-minute throughput for fewer
endpoints; Alpha Vantage gives you broader endpoints with a stricter
daily cap.

Options

Polygon is the clearer pick on paid tiers. Full options chains,
greeks, and historical option prices are available on their higher
plans. Alpha Vantage exposes options data on its Premium plans, but
the shape is less ergonomic for chain-walking.

Forex and crypto

Alpha Vantage gives you forex and crypto on the free tier, within the
daily cap. Polygon requires the higher Stocks-plus-Currencies or
Advanced plan. If your hobby project is a multi-asset dashboard,
Alpha Vantage's free tier is a real advantage here.

Fundamentals

Both expose basic fundamentals (income statement, balance sheet, cash
flow). Polygon's fundamentals data is generally newer and better
structured. Alpha Vantage's was the de facto standard for free
fundamentals before SEC EDGAR's API matured — today, EDGAR is often
the better source for free fundamentals straight from filings.

Developer experience

Sign-up flow is similar: fill a form, get a key. The differences show
up in everyday use.

Polygon has typed SDKs in Python, Go, and JavaScript that map closely to the REST API. Docs render with example responses. WebSocket streaming is straightforward.
Alpha Vantage has a thinner SDK story (the community Python package is the most-used). Docs are denser and older-looking. The CSV vs JSON toggle is per-endpoint, which surprises people.

If you're prototyping and want to be reading clean JSON in your
editor within five minutes, Polygon's developer experience is
noticeably ahead in 2026. If you're already in a Python notebook with
pandas-datareader set up, Alpha Vantage feels familiar.

Rate limits in practice

A side-project screener doesn't need to hammer either API. But the
shape of the limit affects how you structure your code:

Polygon's per-minute rate limit means a sleep-and-retry loop is fine. You can scan several thousand tickers overnight on the free tier if you're patient.
Alpha Vantage's per-day limit makes the same scan impossible without a paid tier. Premium raises the cap by an order of magnitude or more.

For a magic-formula style scan that needs trailing financial data on
several thousand US tickers, the SEC EDGAR submissions API is free,
well-structured, and has no rate-limit drama. Use it for fundamentals;
reserve Polygon or Alpha Vantage for the price data.

When to pick which

Polygon if: you want fast iteration on US equities or options, expect to graduate to a paid plan within a few weeks, and care about modern developer experience.
Alpha Vantage if: you need forex or crypto on a free tier, you're scripting in Python notebooks, or you only need a handful of end-of-day quotes per day and can live with the 25-call cap.

If you can't decide, sign up for both — both keys are free, and
running them side by side for a weekend will surface the friction
points that actually matter to your project before you spend any
money.

Honest caveats

Stock data APIs change pricing, restructure tiers, and tighten
limits without much warning. The shape of this comparison should be
stable for a year or two; the specific numbers may not be. Before
you architect anything load-bearing on either provider, verify the
current free-tier and paid-tier limits on their pricing pages, and
budget for the price step-up if your project takes off.

And the usual reminder: better data does not become a trading edge
by itself. The same close prices everyone has access to are still
close prices everyone has access to.

Originally published at pickuma.com. Subscribe to the RSS or follow @pickuma.bsky.social for new reviews.

Why we're writing about investing on a developer tools blog

pickuma — Tue, 19 May 2026 04:53:31 +0000

Pickuma started as a place to write down what actually happened when we
ran a tool first-hand. That habit — try, document, disclose — turns out
to map cleanly onto investing too, so we're opening a small second lane:
Finance.

What's in scope

Three kinds of posts:

Developer × investor crossovers. Reviews of data APIs (Polygon, Alpha Vantage), screener tooling, and the occasional "build a stock screener in Postgres" walkthrough.
Investor guides. Hands-on takes on the boring infrastructure of investing — accounts, fees, what a piece of jargon actually means.
Market essays. Arguments about where the macro story might be wrong, grounded in numbers we can show, not vibes.

What's not in scope

We're not picking stocks. No target prices, no "this is the one"
essays, no guaranteed returns. If a post is making a directional
argument, it'll say what would prove it wrong.

We're not giving tax or legal advice. Rules vary by jurisdiction.
Ask a CPA, not a blog.

The editorial guardrails

Every finance article carries the Not investment advice notice at the footer (rendered automatically).
If we hold a position in something we're discussing, we say so inline.
Live prices get a date and source, or they don't show up.
AI assistance is disclosed the same way as on every other Pickuma post.

That's it. The first reviews and essays will follow shortly.

Originally published at pickuma.com. Subscribe to the RSS or follow @pickuma.bsky.social for new reviews.

Immich Review: Self-Hosted Google Photos Alternative for Developers

pickuma — Mon, 18 May 2026 14:33:03 +0000

What Immich Is

Immich is an open-source, self-hosted photo and video backup platform. The project ships a server, a web client, and native iOS and Android apps under AGPL-3.0. The mobile apps mirror the Google Photos flow: grant photo library access, and the app uploads new captures to a server you run. The web UI organizes the resulting library with albums, places, faces, and natural-language search.

The repo on GitHub ships a docker-compose.yml and a small set of services. Standing the stack up takes one command on any host with Docker installed. The team has been shipping the project publicly since 2022 and has crossed 60,000 GitHub stars, with a monthly release cadence.

Three audiences care about this project:

Households leaving Google Photos because of the 15 GB shared quota or the unease about cloud-side ML running on private photos.
Homelabbers who already run a NAS or a small server and want a "photos app" tier on top of their existing storage.
Developers who want an HTTP API and event hooks for their own automation — bulk imports, custom albums, family-facing share endpoints.

What makes Immich worth a review now rather than a year ago: stability landed, the mobile apps stopped requiring background-task workarounds on iOS for normal use, and the ML stack got faster CPU inference paths via ONNX runtime updates. If you bounced off Immich in 2023 because every other release broke your database schema, the picture is different today.

Architecture and Deployment

The reference deployment is six containers, all defined in the example compose file:

immich-server — the NestJS API plus microservices for jobs, ingest, and notifications.
immich-machine-learning — a Python service that runs ML models (face detection, face embeddings, CLIP image and text embeddings).
redis — job queue and short-term cache.
postgres — metadata, with the pgvecto.rs or pgvector extension for vector search.
A web client served from the server container.
An optional reverse proxy of your choice.

You point UPLOAD_LOCATION at a directory on the host, set a Postgres password, and run docker compose up -d. The web UI is reachable in under a minute on a small server. First-run setup creates an admin user and lets you invite additional users from the settings page.

Originals are stored on the filesystem and never re-encoded. JPEG, HEIC, most RAW formats via libraw, and the common video containers (MP4, MOV, ProRes) all import as-is. The server generates a thumbnail and a larger preview alongside each original, so a 50 GB phone roll lands closer to 65 GB on disk after processing.

Postgres holds metadata and the vector index. The machine learning container drives the first-run cost: bulk-importing an existing library puts every image through face detection, face embedding, and CLIP embedding. On a 4-core x86 box, an import of around 100,000 photos can take the better part of a day. After that, incremental ingest from phones is near-instant because the queue depth is just the new captures.

You can run the ML container on GPU via CUDA, OpenVINO, or Apple's CoreML on macOS hosts. The repo documents the configurations. On a homelab box with a discrete GPU, initial import drops from hours to minutes.

Immich is not a backup tool — it is the photos app. Your Immich server is one drive failure away from data loss. Set up restic, kopia, or rclone to push to S3, Backblaze B2, or a second disk before you migrate your phone library off Google Photos. The community's most common regret in support channels is skipping this step.

AI Features and What They Actually Do

Three AI features ship in the box.

Face clustering. Every photo goes through RetinaFace for detection and an ArcFace-style model for embeddings. Embeddings get clustered into "people" you can name. You name a cluster once and the label propagates to every other photo in that cluster. Accuracy on adults is good. Embeddings for young children drift as their features change, so expect to merge clusters by hand every few months until they grow up.

Smart search. CLIP-derived embeddings let you type red bicycle at night or kitchen with morning light and get a ranked result list. The search runs over precomputed embeddings stored in Postgres. Latency on libraries under 50,000 photos is sub-second on commodity hardware, and quality on natural-language queries holds up against the equivalent Google Photos search in our spot checks.

Duplicate detection. Perceptual hashing flags near-duplicates — rotation, compression, and edit variants — for manual review. You confirm or reject each batch from the UI rather than the server deleting anything on its own. Conservative defaults, which is the right call for an irreplaceable archive.

What the AI does not do today:

Text OCR on signs or handwritten notes lags Apple's on-device OCR.
Auto-generated "memories" reels exist but are skeletal next to Google Photos.
No voice tagging, AI captions, or generative editing.

For developers, the ML container is just an HTTP service. You can call it directly if you want to bolt your own pipelines onto it — for example, generating CLIP embeddings for photos already stored elsewhere and feeding them into the database.

The Tradeoffs of Self-Hosting

Mobile uploads carry platform constraints. iOS aggressively suspends background uploads, so a roll of 500 new photos may upload across several app opens rather than in one push. The Immich app uses every background API iOS exposes; the ceiling is Apple's, not Immich's. Android is more cooperative but still subject to manufacturer-specific power management.

The web UI is functional and clearly improving. Album sharing, public links, and collaborative albums all work. Comments and reactions exist but feel plain compared to Google Photos. If you share with non-technical family members, expect a short adjustment period — and consider keeping Google Photos on a free tier as the "share with grandma" path for the first six months.

Resource footprint is modest at rest and spiky during ML work. A household-scale instance (3–5 users, ~300,000 photos) runs steady-state on 4 GB RAM and 2 vCPUs. Initial import will saturate any CPU you give it; plan for a one-time spike or schedule it overnight.

Cost arithmetic versus Google One 2 TB ($9.99/month, $99.99/year on annual prepay): a capable Mini PC plus a 4 TB SSD is $300–500 one-time. Add Backblaze B2 offsite at $6/TB/month for proper backup and you spend roughly $144/year for 2 TB of cold copies. Plain hardware payback lands around year three. After that, you own the rails.

The right question is not whether Immich beats Google Photos on polish — it does not, and it does not need to. The right question is whether owning your photo archive is worth the operational tax of running one more service. For developers who already run a homelab, the answer is usually yes. For everyone else, evaluate honestly before you migrate.

Originally published at pickuma.com. Subscribe to the RSS or follow @pickuma.bsky.social for new reviews.

AI Research Slop: How to Filter Signal From the ArXiv Flood

pickuma — Mon, 18 May 2026 14:31:47 +0000

ArXiv's cs.LG and cs.CL categories now receive several thousand new submissions every month, and a non-trivial slice of that flood is what researchers on r/MachineLearning have started calling "slop": papers that recycle benchmarks, propose marginal deltas over published baselines, or in some cases were partly authored by the LLMs they claim to evaluate. The signal-to-noise ratio is degrading fast enough that people who used to read the daily arxiv mailing now report giving up. If you are trying to stay current as a developer or applied researcher, the question is not whether to filter — it is how to filter without missing the few papers that will reshape your stack six months from now.

What slop actually looks like in ML research

Slop is not a single failure mode. It is a cluster.

The first category is the marginal delta: a paper that tweaks a known architecture, runs it on the same three benchmarks everyone uses, and reports a 0.3-point improvement that may or may not survive a seed change. These papers were always common, but submission volume has multiplied them.

The second is benchmark reuse without acknowledgment. A model is fine-tuned on data that overlaps with the evaluation set, and the leaderboard win is reported as if it were honest. The 2024 wave of LLM contamination findings made this category much harder to ignore.

The third — and the one that surfaces most often in community threads — is LLM-assisted writing that crosses the line from polishing into authorship. Reviewers have flagged submissions where citations point to papers that do not exist, equations contain hallucinated symbols, and the related-work section reads like a summary generated from titles alone. Several venues, including ICLR and NeurIPS, have updated their author policies in response. Reviewer-side abuse exists too: there is meaningful evidence that some reviews are themselves LLM-generated, which sets up a feedback loop where slop reviews slop.

The honest framing is that the median paper has not gotten worse. The tail has gotten much longer, and the cost of finding the good papers inside that tail has gone up.

A triage workflow that scales past 20 papers a week

You cannot read everything. You probably should not even skim everything. The workflow below assumes you have roughly 60 to 90 minutes per week for paper intake and want most of that time spent on papers worth reading carefully.

Filter by source before content. Submissions to top venues that have already cleared first-round review are a different distribution than raw arxiv preprints. If a paper has not been posted by an author you recognize, an affiliation you trust, or accepted somewhere with real review, it goes to a "maybe later" pile. This is not gatekeeping — it is the only sane way to spend a finite reading budget.

Use the abstract as a hypothesis, not a summary. Read it once, then jump to the main results table or method figure. If the gap between the abstract's claim and the table's actual numbers is large, the paper is signaling slop. You will internalize this pattern within a few weeks.

Check whether code and weights ship. For empirical work, the absence of a public repository is no longer a neutral signal — it is a negative one. Papers with Code, Hugging Face, and GitHub links should be present and non-empty. Repositories with a README and no commits since submission are a known pattern.

Use citation graphs, not citation counts. Connected Papers and Semantic Scholar's "influential citations" view show you whether a paper is being built on or merely cited. A new paper with 200 citations that are all "we follow X" is not the same as one with 30 citations from groups extending its method.

Treat any paper whose related-work section cites work that does not exist as conclusively slop and stop reading. We have seen this in papers from named labs, not just anonymous submissions. A single hallucinated citation is enough — the rest of the document was not checked by a human carefully enough to trust.

Curation services and tools worth your subscription

The cheapest filter is to outsource part of it to someone whose taste you trust.

Sebastian Raschka's Ahead of AI publishes long-form roundups that triage papers around LLMs, training methods, and applied ML. The summaries are technical enough that you can decide whether to read the original from the summary alone.

Import AI by Jack Clark covers policy, capability progression, and a small number of papers per issue. Useful for spotting trends before they hit your feed.

The Batch from DeepLearning.AI is broader and shallower; helpful for orientation if you are not deep in ML day to day, less useful if you are.

alphaXiv layers comments on top of arxiv. Reading the discussion under a contested paper often saves you the trouble of reading the paper itself.

Semantic Scholar's Research Feeds lets you specify seed papers and surfaces new arxiv preprints ranked by relevance to that seed. Better than chronological for the same reason a recommendation engine beats a chronological feed.

Papers with Code is still the right starting point if you want to find the current state of the art on a specific benchmark and skip the marketing.

For deeper provenance work — checking whether a model's training data contaminates its evaluation — scite and Litmaps are worth knowing about, though both are paid.

Build your own filter and let it compound

The most underrated filter is the one you build yourself, paper by paper, over months. Every paper you read carefully should produce a one-paragraph note: what the claim is, what evidence supports it, and whether your prior moved. Six months in, you will notice that some authors and labs consistently move your prior and others do not. That ranking is more accurate than any third-party signal.

A simple Notion database with paper title, link, claim, evidence, and a 'would re-read' flag is enough. The 'would re-read' column is the highest-signal field — papers you actually return to are the ones that mattered.

The compound effect matters more than any single tool. A researcher who has read 200 papers carefully has a better filter than any service can provide, because their filter encodes their specific research interests. The tools above are scaffolding for getting to that point without burning out on slop first.

Originally published at pickuma.com. Subscribe to the RSS or follow @pickuma.bsky.social for new reviews.

Claude Code Agent View: Why Developers Aren't Sold on Anthropic's New CLI Dashboard

pickuma — Mon, 18 May 2026 14:30:31 +0000

What agent view actually ships

Anthropic added an "agent view" feature to Claude Code, its terminal-based coding agent. The pitch is straightforward: if you run multiple Claude Code sessions in parallel — one per branch, one per repo, one per task — the dashboard gives you a single pane to see what each session is doing and switch between them without juggling tmux panes or terminal tabs.

The feature targets a specific user: someone who already runs three or more Claude Code instances at once. That's a real workflow. Power users have been spawning agents per git worktree for months, often through custom shell scripts, terminal multiplexers, or wrapper tools like Conductor. Agent view tries to standardize that experience inside the CLI itself instead of leaving each user to roll their own.

What the dashboard shows is roughly:

The list of active Claude Code sessions on your machine
Each session's current state — idle, thinking, awaiting input, running a tool
A way to jump between sessions without leaving the dashboard
A consolidated view of what each agent is working on

The whole thing lives in your shell. It's a TUI, not a web app, and that's a deliberate choice consistent with how Claude Code positions itself: terminal-native, no separate desktop surface to keep open.

If you've never run more than one Claude Code session at a time, agent view solves a problem you don't have. The feature exists for the subset of users who already coordinate parallel agents and want less context-switching friction. Solo-session users can safely ignore it.

Why the developer response has been muted

The pushback we've seen isn't that the feature is bad. It's that the hard problems with multi-agent workflows live upstream of "which window am I looking at."

Here's what developers actually struggle with when running parallel Claude Code sessions, and where agent view lands on each:

Coordination, not just monitoring. Knowing three agents are working on three branches doesn't tell you whether they're stepping on each other's changes, duplicating work, or producing diffs that will conflict at merge time. Agent view surfaces state. It doesn't reconcile output.

Cost visibility. Running four agents in parallel multiplies your token spend by four. The dashboard doesn't surface per-session cost in real time, and that's the metric most teams care about before they let agents fan out across a codebase. You have to wait for the Anthropic usage page to update, then attribute spend back to sessions yourself.

Output quality across sessions. When you run multiple agents on related tasks — say, three different approaches to the same refactor — you want comparative review. Agent view shows what each is doing, not how to pick between them. The "diff A vs diff B, here's which I'd merge" view doesn't exist.

Existing tooling already covers the basics. tmux, screen, Zellij, kitty, WezTerm, even plain iTerm tabs handle session management. Developers who already built workflows around those tools see agent view as a reimplementation of capabilities they already have, with less flexibility than their custom setup.

The lukewarm response isn't "this is bad." It's "this isn't the bottleneck." The dashboard solves the easy problem (where are my sessions?) while leaving the hard ones (do these agents agree? what did this run cost? which output should I keep?) untouched.

There's a secondary critique that comes up too: a TUI dashboard in 2026 feels like a half-step. If you're already going to build session orchestration, why not a proper web UI that survives terminal crashes, supports remote sessions, and shows real cost graphs? Anthropic's answer appears to be that Claude Code's identity is CLI-first — but for power users juggling multiple machines or remote work, that constraint is the constraint.

What would actually move the needle

The more interesting question isn't whether agent view is useful — for some users it will be a small quality-of-life win — but what a real multi-agent control plane would look like. Based on what we've seen production teams ask for:

Per-session cost tracking with thresholds. "Stop this session if it crosses 50k tokens" beats a status indicator. "Show me which session burned 80% of today's spend" beats a list of session names.
Conflict detection at the diff level. If two parallel sessions both touch src/auth/login.ts, surface that before you waste compute on both. Even a static pass over each session's working tree would help.
Output comparison. A side-by-side view of multiple agents' diffs on the same task, with a pick-and-merge workflow. This is the feature that would make parallel agents genuinely productive instead of merely possible.
Persistent session state. Currently, killing your terminal effectively kills the session. A real control plane would let sessions survive terminal crashes, machine sleep, and laptop close — and ideally let you resume from another device.

None of those are in agent view today. Some are roadmap candidates, some are out of scope for a CLI tool entirely. The honest read: Anthropic shipped maybe 20% of the multi-agent UX problem, and the remaining 80% is still open — across every agent CLI on the market, not just this one.

For developers who want to keep using Claude Code, agent view is a no-cost upgrade. Turn it on, see if it helps your workflow, ignore it if it doesn't. The bigger question — how do you actually manage three or more AI agents on the same codebase without thrashing yourself or your token budget? — remains unsolved across the industry. Cursor, Cline, Aider, Continue, and Claude Code are each working on different slices of it.

Don't let agent view's existence trick you into running more parallel agents than you can review. Token spend scales linearly with sessions; review attention does not. The bottleneck is rarely how many agents you can spawn — it's how many outputs you can sanity-check before merging.

When agent view is worth turning on

Short answer: when you already run three or more Claude Code sessions at once and your bottleneck is genuinely "I can't keep track of which one is doing what." If that describes you, the dashboard will save you small amounts of friction many times per day, and that adds up.

If your bottleneck is anywhere else on the list above — cost, conflicts, review, persistence — agent view won't fix it, and you should keep building around it with the tools you already have. The dashboard isn't the answer to multi-agent coding. It's a session switcher, and judging it as anything more than that sets up the disappointment that's been showing up in the response so far.

Originally published at pickuma.com. Subscribe to the RSS or follow @pickuma.bsky.social for new reviews.

Anthropic Splits Agent SDK Billing: What Devs Need to Know About New Credit Pools

pickuma — Mon, 18 May 2026 14:29:15 +0000

Anthropic is changing how it bills Agent SDK traffic, and if you've been treating those costs as just another line on your standard Claude API invoice, that's about to stop being accurate. The company is separating programmatic Agent SDK usage — the traffic generated by third-party apps built on the SDK — from standard Claude billing, routing it instead into a new monthly credit pool. We pulled apart what's been announced to figure out which integrations need attention before this lands.

What actually changed

Until now, Agent SDK usage and direct Claude API calls drew from the same meter. You burned tokens, you paid per token, you watched a single number tick up on your dashboard. The new structure splits that meter in two.

Direct API calls — the ones you make from your own backend, with your own auth, hitting Claude endpoints directly — continue to behave the way they always have. Pay-as-you-go token pricing, the rate limits tied to your usage tier, the standard invoice line.

Agent SDK traffic, on the other hand, routes into a separate monthly credit pool. This is the usage generated when third-party apps wrap the SDK and your account is on the hook for the resulting calls. The credits are allocated per month, not per token, and that single change cascades into a different cost model, different rate limit semantics, and different forecasting math.

If your product imports the Agent SDK in production, this is the integration to audit first. Direct API consumers don't need to do anything yet, but anyone shipping on top of the SDK needs to revisit assumptions.

Why credit pools change your forecasting math

Pay-as-you-go billing is mostly linear: 2x the requests means 2x the bill. Credit pools are not linear. They're bounded by a monthly allocation, which changes both your ceiling and your burst behavior.

A few things shift in practice:

Burst days eat the pool faster. A single viral day or a noisy production bug can chew through credits that you assumed would last the month. Smoothing across the calendar is no longer something the billing system does for you — it's something you need to plan for in product behavior.

Forecasting becomes a question about pool fit, not just token spend. The right question stops being "how many tokens will we use next quarter" and becomes "which pool tier do we need, and how much headroom do we want for spikes." Tier upgrades become a discrete decision point rather than a continuous slope.

Exhaustion behavior matters now. When you run out of pool credits, what does your app do? Return a 429 to your users? Fail open to a fallback model? Queue requests until next month? That was a theoretical question under pay-as-you-go (you'd just pay more); under pool billing, it's a real one.

Internal attribution gets messier. If multiple features or teams share one Agent SDK pool, you now need to track usage at the feature level to know who's consuming the budget. Observability that just sums tokens won't tell you which integration is the cost driver.

If your Agent SDK calls sit in a latency-sensitive path, audit your fallback logic before the new billing structure activates. A drained credit pool behaves differently from a pay-as-you-go bill — your code needs to handle the "no credits left" case as a normal failure mode, not an unexpected one.

What to audit in your integration

Five things worth a pass before this lands:

1. Tag your Agent SDK call sites. Every code path that touches the SDK should be tagged in your observability so you can attribute spend to the right pool. If you're running both direct API calls and SDK calls from the same service, you want to know which one is which without grepping logs.

2. Re-read your rate limit handling. Pool credits and per-minute rate limits aren't the same thing, but they interact. Your retry logic should distinguish "rate limited, back off and retry" from "out of credits, fail fast." Lumping them together leads to retry storms when the pool is dry.

3. Forecast against the pool, not against tokens. Build a small dashboard that shows credits consumed against credits allocated for the month, with a projection line. If your current burn rate trends past 100% before month-end, you want to know on day 10, not day 28.

4. Revisit your unit economics. If you charge customers per request and the cost basis just changed shape, your margin assumptions may not hold. Per-pool pricing rewards predictable workloads and penalizes bursty ones — your pricing should reflect that if it doesn't already.

5. Update your runbook. What happens at 80% pool usage? At 95%? At exhaustion? The answer should not be "page someone and figure it out." Pre-decide the throttling, fallback, or upgrade path so the on-call engineer doesn't have to.

The pattern across all five: treat the credit pool as a finite resource with an SLA, not as a flexible meter that absorbs spikes for free.

What to check before this affects you

Three concrete steps for this week, regardless of how much SDK traffic you currently run:

Pull your last 90 days of usage and split it by call type. If you can't tell which calls would land in the new pool versus the standard API, that's the first observability gap to close.
Find the Anthropic billing documentation for the new pool structure and read it end to end before your finance team asks. The general shape of the change is public, but the specifics (exact tiers, credit-to-token conversion, overage behavior) live in the official docs.
Estimate which pool tier your current usage would land in, and price the next tier up as a planning input. You want to know the cost of headroom before you need it.

Most teams will not be dramatically affected by this in the first month. The teams that get burned are the ones who assume billing structure changes don't apply to them and discover the difference at quarter-end.

Originally published at pickuma.com. Subscribe to the RSS or follow @pickuma.bsky.social for new reviews.