Forem: Eitamos Ring

Three Rules for Designing a Go SDK Other People Will Actually Use

Eitamos Ring — Thu, 07 May 2026 18:00:00 +0000

I publish open-source Go libraries.
Not many people use most of them, and I've spent a fair amount of time trying to figure out why. Some of it is distribution. Some of it is the unsexy truth that nobody needed the thing I built. But a real chunk of it — bigger than I want to admit — is that the API was designed for me, the author, and not for the developer arriving cold from a Google search at 2am with a deadline.

This post is three rules I now apply when designing a Go SDK. They come from publishing postgresparser — a pure-Go PostgreSQL parser — and watching where new users got stuck. The examples are from that library, but the rules aren't about parsers. They're about what the surface of a Go package should look like if you want strangers to use it.

I'll also flag one place I broke my own rule, because the post would be dishonest without it.

Rule 1: Expose answers, not nodes

The single biggest mistake I see in Go SDKs (and that I've made myself) is shipping the internal data model as the public API. The author has built an AST, or a state machine, or a config tree, and they think: "great, I'll let the caller walk it." The caller does not want to walk it. The caller wants an answer to a specific question.

Here's what "expose the nodes" looks like in a SQL parsing context:

// What other Go SQL parsers tend to give you
tree, _ := parser.Parse(sql)
for _, stmt := range tree.Statements {
    if sel, ok := stmt.(*ast.SelectStmt); ok {
        for _, from := range sel.From {
            if rv, ok := from.(*ast.RangeVar); ok {
                tables = append(tables, rv.Relname)
            }
            // ...also handle JoinExpr, Subquery, RangeFunction,
            // RangeTableSample, RangeTableFunc, CTERef...
        }
    }
}

The user came to your library to find out which tables a query touches. You handed them a tree-walking exercise and a list of node types they have to learn. Every caller of your library now has to write — and maintain — the same boilerplate, with the same bugs, in slightly different ways.

Compare:

// What postgresparser gives you
result, _ := postgresparser.ParseSQL(sql)
fmt.Println(result.Tables)

That's it. Two lines. CTEs, subqueries, set operations, joins — all flattened into the same field, with aliases preserved. The IR (the actual AST-equivalent) still exists internally, but it's not what the caller binds to.

The principle: for every question your SDK answers, there should be a single field or function whose name is the question. "Which tables?" → Tables. "Which columns are filtered?" → ExtractWhereConditions. "How is each column used?" → ColumnUsage. If a user has to traverse three levels of struct to get an answer, the answer wasn't really exposed.

The objection I hear: but what if the caller wants something custom that we didn't anticipate? Fine — keep the IR public for the 5% case. But default to answering the 95% case in one line, and only fall back to the IR when the typed accessor doesn't cover the question.

Rule 2: Name the common case after the common case, and mark the variants

Most Go SDKs I see treat all of their entry points as peers. Parse, ParseStrict, ParseAll, ParseWithOptions, ParseFromReader — all listed in pkg.go.dev with the same visual weight, and the user has to read every one to figure out which they want.

This is the "tyranny of options" failure. The author thought of every variant; the user has to think about it too.

The fix is sequencing. Pick the version 80% of users want. Give that the short name. Make the other variants explicitly named after the thing that makes them different.

postgresparser's parsing entry points:

// 80% case — parses one statement, gives you a result.
result, _ := postgresparser.ParseSQL(sql)

// "I might pass multiple statements and want all of them."
batch, _ := postgresparser.ParseSQLAll(sql)

// "I want an error if more than one statement was passed."
result, _ := postgresparser.ParseSQLStrict(sql)

ParseSQL is the default. ParseSQLAll and ParseSQLStrict are explicitly named after the property that makes them different (handling all statements, strict-on-multi). A user reading the package docs sees ParseSQL first, tries it, and only goes looking for the variants if they hit a case it doesn't cover.

The wrong version of the same API:

// Don't do this
postgresparser.ParseSQL(sql, ParseOptions{Strict: true, AllStatements: false})
postgresparser.ParseSQL(sql, ParseOptions{Strict: false, AllStatements: true})

You've moved the decision from the function name (where it's documented and grep-able) to a config struct (where it's not). New users have to read the options struct just to call the function. Existing code has to be re-read every time someone wants to know what mode it's in.

The principle: the most common call should be the shortest call. Variants get names that describe how they differ. Config structs are for things that don't fit in a name, not for things that do.

Rule 3: Return structured data, not strings the caller has to re-parse

This one I see less often in writing about SDK design, but it's the one that bites users hardest in practice.

If your SDK has done work to extract structured information from unstructured input, don't throw the structure away on the way out. Returning []string when you could have returned []struct{...} is a tax you charge every caller forever.

postgresparser extracts WHERE conditions. The naive return type would be:

// Bad: caller has to re-parse what you already parsed
conditions, _ := analysis.ExtractWhereConditions(sql)
// returns: ["status = 'active'", "total > 100"]

// Now every caller writes a regex. They get it wrong.
// They handle = and != but forget IS NULL. They miss BETWEEN.
// They re-introduce the bug your library was built to solve.

What it actually returns:

type Condition struct {
    Column   string
    Operator string
    Value    interface{}
}

conditions, _ := analysis.ExtractWhereConditions(
    "SELECT * FROM orders WHERE status = 'active' AND total > 100",
)
for _, c := range conditions {
    fmt.Printf("%s %s %v\n", c.Column, c.Operator, c.Value)
}
// status = active
// total > 100

Now the caller can ask c.Column == "tenant_id" directly. They can switch on c.Operator. They can type-assert c.Value. None of them have to write a regex, and none of them re-introduce parsing bugs at the boundary of your library.

The principle: if the structure exists internally, expose the structure. Strings are for things that have no structure, or for things the user is going to print. Stringly-typed return values are how libraries become impossible to use correctly at scale.

The reverse also holds: if you find yourself writing a long regex inside a library you depend on, that library failed Rule 3.

Where I broke my own rule

In the spirit of not pretending I have all this figured out: postgresparser violates Rule 2 with ParseSQLWithOptions(sql, opts). It exists alongside ParseSQL(sql), takes a config struct with extraction flags like IncludeCreateTableFieldComments, and is exactly the "tyranny of options" pattern I just told you to avoid.

The honest reason it exists: comment extraction is expensive and most callers don't need it, but I didn't want to design a separate ParseSQLWithComments function because the option might evolve. So I shipped a WithOptions escape hatch and told myself it was fine. It's not fine — it's a slow leak that will get bigger as more options accrete. The right move would have been a separate named function for the one option that exists today, and a real opt-in API design when the second option arrives.

I'm flagging it so you can see what the wrong choice looks like even when the author knew the rule.

The point of including this isn't self-deprecation. It's that you will violate your own rules. The goal isn't a perfect API on day one; it's noticing the violation, naming it, and fixing it before the wrong shape hardens into a public contract you can't change.

TLDR;

If you can't remember three rules, remember the question they all answer: what does the user have to learn before they can use this library?

Rule 1 says: don't make them learn your AST.
Rule 2 says: don't make them learn your option matrix.
Rule 3 says: don't make them re-parse what you already parsed.

Every line of documentation a user has to read before their first successful call is friction. Some of it is unavoidable. A lot of it isn't, and that's where the design work is.

postgresparser is on GitHub at github.com/ValkDB/postgresparser if you want to see what these rules look like applied (and, per the section above, where they aren't yet). Issues and PRs welcome — particularly the kind that point out a rule I missed.

What I Learned Building a Pure Go PostgreSQL Parser

Eitamos Ring — Tue, 05 May 2026 06:19:05 +0000

Why I built it

I needed a PostgreSQL parser that could run inside Go tooling without CGO, external binaries, or runtime dependencies.

What made PostgreSQL parsing harder than expected

SQL is not one grammar
PostgreSQL has a lot of dialect-specific edge cases
AST shape matters more than “can it parse”
Error handling becomes a product feature
Real-world SQL is uglier than examples

Why pure Go mattered

No CGO, easy installation, works in CI, easy to embed in linters and developer tools.

What 200+ GitHub stars taught me

Developers care about boring installation
Parser APIs need to be simple
Good examples matter more than perfect docs
People want tooling, not academic grammar dumps

Where it’s going

This parser is becoming the foundation for Valk Guard, a local-first static analyzer for SQL and ORM usage. No LLM required. It works from ASTs and deterministic rules.

GitHub repo: GitHub repo: https://github.com/ValkDB/postgresparser

What every `?` in your SQL is hiding

Eitamos Ring — Mon, 04 May 2026 18:48:09 +0000

Take a query that comes out of pg_stat_statements:

SELECT date_trunc(?, o.created_at) AS week,
       count(*) AS total
FROM orders o
INNER JOIN customers c ON c.id = o.customer_id
WHERE o.created_at >= ?
  AND o.amount > ?
  AND c.plan = ?
GROUP BY ?
ORDER BY 2 DESC
LIMIT ?

Six question marks. Each one means something completely different.

The first, inside date_trunc, expects a string like 'week' — it's telling the function which time bucket to use. The second is a timestamp comparing against created_at. The third is a number comparing against amount. The fourth is a string joined through to the customers table — it has to match a plan value over there. The fifth, sitting bare inside GROUP BY, is a positional integer like 1, pointing back at the first column in the SELECT list. It's not a value, it's an index. The sixth, after LIMIT, is a page-size integer.

Six placeholders, four different value types, two completely different kinds of integer. There isn't a regex that gets all six right — not without re-implementing a SQL parser inside it.

postgresparser is the open-source Go/ANTLR PostgreSQL parser we maintain at ValkDB. Until this release, when you got back an AST, every ? was just a leaf node with positional information and nothing else. The parser knew exactly what each ? meant — it had to, in order to parse — but it never told you. So everyone downstream fell back to regular expressions, string scanning, and increasingly elaborate guesswork.

This week's release tells you what every ? actually is.

The new API

result, _ := analysis.AnalyzeSQL(querySQL)

for _, p := range result.Placeholders {
    fmt.Printf("placeholder %d: role=%s column=%s\n",
        p.Index, p.Role, p.ColumnRef)
}

placeholder 1: role=function_arg     column=         (date_trunc, arg 0)
placeholder 2: role=where_value      column=created_at
placeholder 3: role=where_value      column=amount
placeholder 4: role=where_value      column=plan
placeholder 5: role=group_by_ordinal column=
placeholder 6: role=limit            column=

Six placeholders, six correct classifications, no string scanning. Switch on the role, fill in the right value.

How the old way failed

Without role information, this is the pipeline most tools end up with:

┌─────────────────────┐
│  Normalized SQL     │
│  with ? placeholders│
└──────────┬──────────┘
           │
           ▼
┌─────────────────────────────────┐
│   Regex sweep for "?"           │
│                                 │
│   finds ? in string literals    │
│   finds ? in comments           │
│   can't see GROUP BY context    │
│   mis-IDs JSONB ? operator      │
│   picks same value twice for    │
│   same column on >= and <       │
└──────────┬──────────────────────┘
           │
           ▼
┌─────────────────────┐
│  Hand-written       │
│  per-position guess │
│  (fragile)          │
└──────────┬──────────┘
           │
           ▼
┌─────────────────────┐
│  Substituted SQL    │
│  often broken       │
└─────────────────────┘

The role-aware version skips all of that by walking the parse tree the parser already built. String literals are leaves of their own kind, so question marks inside them are never seen as placeholders. Comments are stripped before tree construction. The JSONB operator is parsed as an operator node, not a placeholder leaf, so it never enters the placeholder list. GROUP BY and ORDER BY ordinals carry their own dedicated role. And every placeholder's syntactic role — its actual position in the grammar — comes back attached.

The five footguns this release closes

1. The JSONB `?` operator is not a placeholder

PostgreSQL has three jsonb operators that look like placeholder tokens:

WHERE data ? 'key'                  -- "does jsonb contain top-level key?"
WHERE data ?| array['a','b']        -- "any of these keys?"
WHERE data ?& array['a','b']        -- "all of these keys?"

A regex sweep can't tell these apart from real placeholders. The new placeholder list excludes JSONB operator tokens by construction.

2. `INTERVAL ?` actually parses

Before this release, INTERVAL ? was rejected with a syntax error — a real problem if you consume pg_stat_statements, because every query that uses an interval literal gets normalized to that form. The grammar now accepts a parameter token in interval-operand position.

3. `?` inside string literals stays inside string literals

WHERE notes = 'has a ?'
WHERE notes = 'don''t mark me ?'

The collector walks the parse tree, never the raw SQL — so string-literal ? and comment ? simply don't appear in the placeholder list.

4. `GROUP BY ?` is an ordinal, not a value

pg_stat_statements rewrites GROUP BY 1, 2 to GROUP BY ?, ?. These placeholders need to be substituted with positional integers referring to SELECT-list slots — not with arbitrary values. A dedicated role makes this explicit.

5. Function-argument placeholders need to know their function

SELECT date_trunc(?, created_at), extract(? FROM created_at) FROM t

The first ? must be a string like 'week'. The second must be a string like 'year'. Both are function-args, but the function differs — so the right substitution differs. Each placeholder of this kind now carries its parent function name and argument index.

Who this is for

If you build an ORM or query builder and you've ever wanted to type-check a placeholder before binding to it, this is for you. If you build a SQL linter, a migration tool that rewrites queries between dialects, a monitoring agent that ingests pg_stat_statements, an AI-assisted SQL generator that emits parameterized queries — same. The common thread is that you have a normalized SQL string with ? placeholders in it, and you need to know what each one means before you can do anything useful.

If that sounds like work you've done, you've probably written a private placeholder classifier already. With this release, you don't have to.

Closing

The parser tells you what the SQL says; type inference belongs a layer up. The API stays narrow on purpose — roles, positions, and the structural context needed to make sense of them. Function-wrapper exposure on column usage is next on the roadmap; lateral-join and recursive-CTE refinements after that.

The parser knew. Now it tells you.

postgresparser — open-source PostgreSQL parser. Go, ANTLR-based. Contributions welcome.

A Protobuf for Database Schemas

Eitamos Ring — Wed, 18 Mar 2026 07:52:54 +0000

Every serious system has an interface definition for its wire format. gRPC has protobuf. REST has OpenAPI. GraphQL has its SDL. But databases -- the thing everything else is built on top of -- have nothing.

Your database schema is one of the most important artifacts in your system. It defines every table, column, type, constraint, relationship, and index. It encodes years of domain decisions. And yet there is no standard, portable, machine-readable format for it.

We built one. We call it ctxexport.json.

The problem is older than LLMs

Before you assume this is an AI-context story, consider how many times you have needed your schema outside the database itself:

Onboarding a new engineer who needs to understand the data model.
Diffing staging against production to catch drift before a deploy.
Running a linter in CI to enforce naming conventions or catch missing indexes.
Generating documentation that is not immediately stale.

Every time, you end up writing a bespoke script that queries information_schema or pg_catalog, parses the output, and feeds it into whatever tool you need. The script is Postgres-specific. It breaks when you add a second schema. Nobody maintains it.

pg_dump --schema-only exists, but it is a restore format, not a consumption format. It is Postgres-specific SQL with SET statements, ownership clauses, and an ordering designed for replay, not reading. Try parsing it reliably. Try feeding it to a linter. Try diffing two of them without drowning in noise.

MongoDB is worse. There is no mongodump --schema-only. Your schema lives in the shape of whatever documents happen to exist. Good luck extracting that into something a tool can reason about.

Extract once, use many ways

The core insight behind ctxexport.json is the same one behind protobuf: separate the definition from the consumption.

A protobuf .proto file is written once and compiled to Go structs, Python classes, TypeScript types, gRPC stubs, or REST gateways. The definition is the single source of truth. The consumers are many and varied.

ctxexport.json works the same way. You extract your schema once -- from Postgres, MongoDB, or whatever backend -- and produce a single canonical JSON file. That file contains entities (tables, views, collections), fields (columns with types, nullability, defaults), edges (foreign keys and inferred references), and access paths (indexes). Everything a tool needs to understand your data model, nothing it does not.

From that single artifact, you can:

Compile to a lighthouse map -- a compact table-and-relationship summary that fits in an LLM prompt.
Compile to full SQL DDL -- standard CREATE TABLE statements for any subset of tables.
Serve over MCP -- give an AI agent schema awareness without database credentials.
Diff across environments -- compare staging and production schemas as structured data, not text.
Lint offline -- check naming conventions, missing indexes, or orphaned foreign keys in CI.
Validate in CI -- catch schema regressions before they reach production.
Commit to git -- your schema becomes a versioned artifact with a real history.

None of these consumers need to know whether the source was Postgres or MongoDB. None of them need a live database connection. The extraction happened once, upstream, and everything downstream reads the same contract.

The sidecar pattern

Databases have never been good at carrying human knowledge alongside the schema. Your users.deleted_at column is a soft-delete flag, but the database only knows it is a timestamp with time zone. Your orders.payload column is JSONB with a specific structure, but the database sees an opaque blob.

A sidecar file (dbdense.yaml) layers descriptions and value annotations onto the extracted schema:

entities:
  payments:
    fields:
      status:
        values: ["pending", "authorized", "paid", "failed", "refunded"]
  users:
    fields:
      deleted_at:
        description: "Soft delete timestamp. NULL = active."

This merges at export time. The compiled DDL gets inline comments like -- Values: pending, authorized, paid, failed, refunded. Every downstream consumer -- linter, LLM, documentation generator -- picks it up automatically. Write it once in a YAML file committed to the repo.

Why JSON, not SQL

SQL DDL is human-readable but machine-hostile. Parsing CREATE TABLE statements reliably across dialects is a nightmare. Defaults are quoted differently. Constraints can be inline or out-of-band. Comments use different syntax. There is no standard way to represent a foreign key relationship as structured data.

JSON is boring and that is the point. It is a declarative state representation -- you look up a table by name, not by parsing DDL statement order. Every language has a JSON parser. The schema is simple: a version string, an array of entities, and an array of edges. You can validate it with a JSON Schema. You can diff it with jq. You can read it in any language without a SQL parser.

A minimal entity looks like this:

{
  "name": "payments",
  "type": "table",
  "fields": [
    {"name": "id", "type": "uuid", "is_pk": true},
    {"name": "status", "type": "text", "not_null": true, "values": ["pending", "paid", "failed"]}
  ]
}

Flat, predictable, zero ambiguity.

Stop treating your schema like a black box

The immediate use case is LLM context -- giving AI agents schema awareness without live database access. But the format is deliberately general. If your tool can read JSON, it can read a database schema. That was not true before.

The project is at github.com/valkdb/dbdense. The contract is documented in docs/ctxexport-contract.md. It supports Postgres and MongoDB today. The extractor interface is small enough that adding a new backend is a single file.

Your database schema is too important to be locked inside the database. Export it. Version it. Build on it.

Stop Sending 93K Tokens of Schema to Your LLM Agent!

Eitamos Ring — Wed, 18 Mar 2026 07:52:06 +0000

I've watched agents query information_schema over and over, spending 4-6 turns just to figure out which tables exist, what columns they have, and how they join. On a 500-table database, the full DDL is around 93,000 tokens. Most questions touch 3-5 tables. On a complex multi-table join, I measured a 64% token reduction by just giving the agent the schema upfront.

That's what dbdense does.

I built dbdense to fix this.

What it does

dbdense is a three-step offline pipeline: extract, compile, serve.

Extract connects to your database once and snapshots the schema into a portable JSON file (ctxexport.json). Tables, columns, types, primary keys, foreign keys, indexes -- everything an LLM needs to write correct queries.
Compile turns that snapshot into two artifacts:
- A lighthouse -- a compact table map (~4K tokens for 500 tables). It looks like this:
```
 T:users|J:orders,sessions
 T:orders|E:payload,shipping|J:payments,shipments,users
 T:payments|J:orders
```
Every table, its FK neighbors, and embedded docs. 23x smaller than full DDL. This stays in the agent's context so it always knows what's available.
- Full DDL -- standard CREATE TABLE statements with constraints, rendered on demand only for the specific tables the agent asks about.
Serve (optional) exposes the lighthouse as an MCP resource and the DDL via an MCP slice tool. The agent reads the map, picks the tables it needs, and gets back just those definitions.

After the extract, everything runs locally. The compiled artifacts are plain text you can commit to your repo. No database connection needed at runtime.

No credentials in the agent runtime

The export step is the only step that touches the database. After that, compile and serve work from the local snapshot. Your production database credentials never need to be in the agent's environment. The tool works offline and air-gapped.

The numbers

I ran an agentic benchmark: n=3, same 5 questions, same seeded Postgres database (20K+ rows, 8 tables), same model (Claude Sonnet 4). One arm had only a Postgres MCP tool. The other had the same tool plus dbdense schema context injected into the prompt.

Metric	Without schema context	With dbdense	Delta
Correct answers	13/15	13/15	equal
Avg turns	4.1	2.2	-46%
Tokens per run	285,922	187,603	-34%

Same accuracy. 34% fewer tokens. 46% fewer turns.

The savings scale with query complexity. On simple single-table filters, both arms performed about the same. On a complex multi-table join, the baseline agent spent 6+ turns querying information_schema to discover the schema. dbdense answered in 2 turns, using 64% fewer tokens for that query.

The two wrong answers (both on the same question, in both arms) returned identical incorrect results, pointing to question ambiguity rather than a schema context issue.

Sidecar enrichment

Databases lie by omission. A column named status with type text tells the LLM nothing about what values are valid. The agent either guesses or wastes a SELECT DISTINCT turn to find out.

dbdense supports a dbdense.yaml sidecar file where you annotate columns with descriptions and enum values:

entities:
  payments:
    fields:
      status:
        values: ["pending", "authorized", "paid", "failed", "refunded"]
  orders:
    fields:
      status:
        description: "Order lifecycle status."
        values: ["pending", "confirmed", "shipped", "delivered", "cancelled"]

These annotations merge into the compiled DDL as inline SQL comments. The LLM sees -- Values: pending, authorized, paid, failed, refunded right next to the column definition. No extra queries needed.

This also works for documenting JSONB structures, MongoDB embedded documents, or anything else the raw schema doesn't capture.

What it doesn't do

The snapshot is static. If your schema changes, re-run export. This is intentional -- schemas are stable; questions change.

The slice tool still depends on the LLM picking the right tables from the lighthouse. dbdense reduces the context problem; it doesn't solve table selection for the model.

It's not a pg_dump --schema-only replacement. The renderer covers columns, PKs, FKs, NOT NULL, defaults, unique constraints, and indexes, but skips triggers, RLS policies, and custom types.

Try it

go install github.com/valkdb/dbdense/cmd/dbdense@latest
dbdense export --driver postgres --db "postgres://user:pass@localhost:5432/mydb" --schemas public
dbdense compile --mode lighthouse --in ctxexport.json --out lighthouse.txt
dbdense compile --in ctxexport.json --out schema.sql

You now have two files: a lighthouse map and full DDL. Point your agent at them. If you use Claude Code, dbdense init-claude writes the MCP config for you.

The project is open source at github.com/valkdb/dbdense.

How does a linter know your column doesn't exist

Eitamos Ring — Mon, 09 Mar 2026 08:40:04 +0000

You write a query that SELECTs ghost_status from the orders table. Your code compiles. Your tests pass. But ghost_status was never created in any migration. In production, that query crashes.
Valk Guard catches this at PR time - with no database connection.
This post walks through exactly how. Not hand-waving. The actual code path, from source file to finding.
The setup
Here's a Go file using Goqu to build a query:
func ListBrokenUserOrderStatus(ctx context.Context) error {
_, _, err := goqu.From("users").
LeftJoin(
goqu.T("orders"),
goqu.On(goqu.I("orders.user_id").Eq(goqu.I("users.id"))),
).
Select("users.id", "users.email", "orders.ghost_status").
Where(goqu.I("orders.missing_flag").Eq("pending")).
ToSQL()
return err
}
And here's the migration that created the orders table:
CREATE TABLE orders (
id SERIAL PRIMARY KEY,
user_id INTEGER NOT NULL REFERENCES users(id),
total NUMERIC(10,2) NOT NULL,
status TEXT NOT NULL DEFAULT 'pending',
created_at TIMESTAMP DEFAULT now()
);
Notice: the query references orders.ghost_status. The migration never created that column. There is no ghost_status. Valk Guard reports:
VG105: projection column "ghost_status" not found in table "orders" schema; check SELECT list and schema/model mappings
How does it know?
Let's walk through each phase.
Phase 1: Query extraction
The Goqu scanner doesn't look for SQL strings. It walks the Go AST looking for method chains rooted in goqu.From().
When it finds one, it flattens the chain into a list of method calls: From("users") → LeftJoin(...) → Select(...) → Where(...). Each method gets parsed: From gives the base table, LeftJoin gives the join target, Select gives the projection columns, Where gives the predicates.
From these parts, the scanner synthesizes a SQL statement:
SELECT users.id, users.email, orders.ghost_status
FROM users LEFT JOIN orders ON orders.user_id = users.id
WHERE orders.missing_flag = 'pending'
This SQL never existed in your source code. Valk Guard constructed it from the AST of your Go code. That's the key difference from regex-based tools - regex can't walk a method chain and reconstruct what the query builder will produce.
Phase 2: Schema snapshot
Separately, Valk Guard finds all .sql files under your migration paths. Each file gets parsed through postgresparser, and every DDL statement gets applied to a Snapshot - an in-memory representation of your schema's current state.
The snapshot builder processes DDL actions in order:
CREATE TABLE orders (id, user_id, total, status, created_at) → registers the table with five columns
ALTER TABLE orders ADD COLUMN shipped_at TIMESTAMP → adds a sixth column
ALTER TABLE orders DROP COLUMN shipped_at → removes it

The end result is a map of table names to column definitions. For orders, that's: id, user_id, total, status, created_at. Five columns. No ghost_status.
This is the same principle as running all your migrations on an empty database - except it happens in memory, with no database, in microseconds.
Phase 3: Rule evaluation
Now VG105 runs. It takes the synthesized SQL (already parsed into a structured IR by postgresparser) and the schema snapshot, and does a straightforward lookup:
For each column in the SELECT list with usage type "projection", resolve which table it belongs to (using the alias or the single-table shortcut)
Look up that table in the snapshot
Check if the column exists in the table's column map
If not → finding

For ghost_status, the column usage says it belongs to orders (from the orders.ghost_status qualifier). The snapshot has an orders table. But orders.ghost_status is not in the column map. Finding.
The same logic powers VG106 (unknown filter column - catches WHERE orders.missing_flag = 'pending' from the same query) and VG107 (unknown table reference).
It also works with ORM models
The same snapshot system powers schema-drift rules (VG101–VG104). Instead of checking queries against migrations, these rules check ORM models against migrations.
Say you have a Go struct:
type Order struct {
ID int db:"id"
UserID int db:"user_id"
Total string db:"total"
Status string db:"status"
GhostStatus string db:"ghost_status"
}
Valk Guard's Go model extractor walks the AST, reads the db struct tags, and produces a ModelDef with columns: id, user_id, total, status, ghost_status.
VG101 then compares each model column against the migration snapshot. ghost_status isn't in the orders table → finding:
VG101: model "orders" references column "ghost_status" not found in table "orders" schema; check migration DDL or update model mapping
Two different rules, two different input paths (query vs. model), same schema snapshot, same answer.
What this means in practice
You don't need a running database. You don't need to run migrations. You don't need to connect to staging. Valk Guard reads your source code and your migration files, builds everything in memory, and cross-references them statically.
This runs in CI in seconds. It catches the kind of bug that usually shows up as a column "ghost_status" does not exist error in your logs at 2am - and moves it to a PR comment at 2pm instead.
go install github.com/valkdb/valk-guard/cmd/valk-guard@latest
valk-guard scan .
Repo: github.com/ValkDB/valk-guard

We didn't want an AI SQL reviewer. We wanted deterministic

Eitamos Ring — Sat, 07 Mar 2026 16:33:20 +0000

So we built Valk Guard.

Most SQL linters scan .sql files. The problem is, most SQL doesn't live in .sql files.

It lives in db.Query() calls. In Goqu builder chains. In SQLAlchemy ORM methods. In migration files mixed with application logic. By the time SQL reaches production, it's been assembled, concatenated, or synthesized by code that no .sql-only tool will ever see.

I built Valk Guard to solve that. It's a static analysis tool that walks your source code's AST, reconstructs the SQL your ORMs and query builders will generate, parses it through a real PostgreSQL grammar, and reports findings in CI-friendly formats. No database connection. No runtime. Just structure.

go install github.com/valkdb/valk-guard/cmd/valk-guard@latest
valk-guard scan .

19 rules enabled by default. Zero config. Takes seconds.

What it actually catches

Here's a Goqu chain in Go:

goqu.From("orders").Delete()

There's no raw SQL anywhere in that line. But Valk Guard walks the Go AST, recognizes the Goqu method chain, synthesizes DELETE FROM orders, feeds it through postgresparser, and fires VG003: DELETE without WHERE may affect all rows.

Same thing with SQLAlchemy:

session.query(User).delete()

No SQL string. Valk Guard's embedded Python AST extractor reconstructs it, and the same rule fires.

The full rule set covers three categories:

Query safety — UPDATE without WHERE (VG002), DELETE without WHERE (VG003), SELECT * (VG001), unbounded SELECT without LIMIT (VG004), leading wildcard LIKE '%...' (VG005), SELECT ... FOR UPDATE without WHERE (VG006).

Dangerous DDL — DROP TABLE / TRUNCATE in application code (VG007), CREATE INDEX without CONCURRENTLY (VG008).

Schema drift — ORM model references a column that migrations dropped (VG101). NOT NULL column missing from model (VG102). Type mismatch between model and DDL (VG103). Model table has no CREATE TABLE in migrations (VG104). Query SELECTs a column that doesn't exist in the schema (VG105). And several more cross-reference checks between your code and your migrations.

That last category is the one I haven't seen elsewhere. Valk Guard reads Go struct tags (db, gorm) and Python __tablename__ / Column(...) definitions, builds a schema snapshot from your migration DDL, and cross-references them. If your ORM model says email exists but your migration dropped it, that's VG101 at PR time — not a runtime panic in production.

Why AST, not AI

This was a deliberate choice, and it's worth explaining.

CI is not a brainstorming session. If a PR check comments on your code and changes its mind between runs, people stop trusting it. If it floods you with false positives, people add --skip-lint and move on. The tool is dead even if the idea was good.

I needed the opposite: same input, same output, every time. Testable. Explainable. Boring in the best way.

AI is useful for exploration and suggestions. But a blocking CI step needs determinism. Even structured-output approaches for LLMs improve schema conformance — they don't make a generative model behave like a static analyzer. The questions Valk Guard answers are structural: "does this statement have a WHERE clause?" "does this builder chain produce a bounded query?" "does this model match this schema?" Those are AST questions, not generation questions.

The same logic applies to regex. Regex is fine when the thing you're checking is a flat string. It falls apart when SQL is buried inside Go method chains or Python ORM calls. You can't regex your way through goqu.From("users").Where(goqu.C("id").Eq(42)).Select("name") and reliably reconstruct the query. You need to parse the source language's AST, understand the builder pattern, and synthesize the SQL. That's what Valk Guard does.

A small number of checks do use targeted regex after parsing — when a parser-extracted clause doesn't expose the exact field a rule needs. But that's regex as a surgical helper on already-parsed output, not regex as the foundation.

The pipeline

Source files go in. Findings come out. Here's what happens in between:

1. Extraction — Four scanners run concurrently. The raw SQL scanner handles .sql files with proper dollar-quoting and nested block comments. The Go scanner uses go/ast to extract SQL from db.Query, db.Exec, and db.QueryRow. The Goqu scanner walks builder chains and synthesizes SQL. The SQLAlchemy scanner invokes an embedded Python script (stdlib only — no pip dependencies) that parses ORM chains via Python's ast module.

2. Parsing — Every extracted statement goes through postgresparser, a pure-Go PostgreSQL parser I built on ANTLR. It produces a structured IR: tables, columns, joins, WHERE clauses, command type. No CGO, no database connection. Most queries parse in 70–350 µs.

3. Rule evaluation — Rules are dispatched by SQL command type for efficiency. Query rules (VG001–VG008) run against every parsed statement. Schema-drift rules (VG101+) cross-reference ORM model definitions against a migration-derived schema snapshot. Query-schema rules (VG105–VG108) validate that columns and tables referenced in queries actually exist.

4. Output — Findings are deduplicated, sorted by file and line, and formatted as terminal output, JSON, SARIF (for GitHub Code Scanning), or rdjsonl (for reviewdog PR comments).

The whole thing is ~8,100 lines of Go (plus ~700 lines of embedded Python), with nearly 1:1 test coverage. Three runtime dependencies: cobra, postgresparser, and yaml. That's it.

CI integration

Valk Guard was designed for pull request workflows. Exit code 0 means clean, 1 means findings, 2 means config/parser error. Hook it into reviewdog and you get inline PR comments:

- name: Run valk-guard
  run: |
    set +e
    valk-guard scan . --format rdjsonl > valk-guard.rdjsonl
    code=$?
    set -e
    if [ "$code" -gt 1 ]; then exit "$code"; fi

- name: Post review comments
  env:
    REVIEWDOG_GITHUB_API_TOKEN: ${{ secrets.GITHUB_TOKEN }}
  run: |
    reviewdog -f=rdjsonl -name="valk-guard" \
      -reporter=github-pr-review -filter-mode=added \
      < valk-guard.rdjsonl

Findings are non-blocking by default. Config errors fail the job. You can see real example PRs with live review comments in the valk-guard-example repo.

Where it fits

Valk Guard is not a runtime firewall, not a database advisor, and not a replacement for EXPLAIN ANALYZE. It's a guardrail for the most common and most expensive SQL mistakes — the ones that happen when someone pushes a DELETE FROM orders without a WHERE at 4pm on a Friday.

It's PostgreSQL-only. It doesn't auto-fix. It doesn't need a running database. It reads your source code, understands your ORMs, and tells you what's going to break — before it merges.

Less magic. More signal. Same answer every run.

Repo: github.com/ValkDB/valk-guard

Building a PostgreSQL Parser in Go: What Broke After We Open-Sourced It

Eitamos Ring — Wed, 18 Feb 2026 06:18:53 +0000

Building a PostgreSQL Parser in Go: What Broke After We Open-Sourced It

postgresparser is a pure-Go PostgreSQL SQL parser. It turns SQL text into structured metadata (tables, columns, joins, filters, DDL actions, parameters) without executing queries.

We thought it was solid. Open source proved we were wrong.

Here is what open source forced us to learn.

The biggest shift was not “more bug reports.” It was use-case expansion.

We built for our workflow. Users showed up with very different workloads.
In the first week after release, most feedback centered on deterministic batch parsing.

Our internal assumptions broke immediately

Inside a single team, ambiguous behavior survives because everyone “knows” the rules. Public users do not have that context.

The first pressure point was multi-statement SQL. We had ParseSQL (single statement) and figured batch parsing was “close enough.” It was not.

People were using the parser for:

CI linting pipelines
production tools
llm wrappers

People asked practical questions we could not answer cleanly:

Which exact statement failed?
Is this a warning or a hard failure?
Can I map diagnostics to the original SQL text reliably?

Those questions forced us to define strict contracts instead of relying on implied behavior.

If your tool consumes SQL in bulk, batch correlation is everything.

Broken behavior example

This input exposed the issue quickly:

SELECT 1;
SELECT FROM;
SELECT 2;

Early batch behavior made correlation awkward because results were compacted and diagnostics were not statement-first. If you’re building CI checks or migration tooling, “something in the batch failed” is not actionable.

Now each statement has deterministic correlation (Index, RawSQL, Query, Warnings), so downstream code can point to the exact source statement.

Before/after API diff

- type ParseBatchResult struct {
-   Queries          []*ParsedQuery
-   Warnings         []ParseWarning
-   TotalStatements  int
-   ParsedStatements int
- }
+ type StatementParseResult struct {
+   Index    int
+   RawSQL   string
+   Query    *ParsedQuery   // nil => IR conversion failure
+   Warnings []ParseWarning // statement-scoped warnings
+ }
+
+ type ParseBatchResult struct {
+   Statements       []StatementParseResult
+   TotalStatements  int
+   ParsedStatements int
+   HasFailures      bool
+ }

That shape is less convenient for quick demos, but much better for real integration.

Real SQL in the wild is much uglier than test fixtures

Open source usage also brought SQL shapes we did not have in internal tests:

trailing semicolons and odd whitespace
invalid syntax in the middle of an otherwise valid batch
mixed DDL + DML scripts
ONLY variants in DDL paths

The parser had to become resilient without becoming vague. That meant:

better statement-level warning attribution
explicit failure semantics (Query == nil)
tighter handling across DDL relation extraction paths

One concrete snippet (current behavior)

batch, err := postgresparser.ParseSQLAll(sql)
if err != nil {
    log.Fatal(err)
}

fmt.Printf("total=%d parsed=%d has_failures=%t\n",
    batch.TotalStatements, batch.ParsedStatements, batch.HasFailures)

for _, stmt := range batch.Statements {
    fmt.Printf("idx=%d failed=%t warnings=%d raw=%q\n",
        stmt.Index, stmt.Query == nil, len(stmt.Warnings), stmt.RawSQL)
}

That is the integration model people asked for: deterministic, inspectable, and boring in the best way.

Why this matters

Open source removed our ability to hand-wave edge cases.

The loop became:

implement
get challenged
simplify
lock behavior with tests
document the contract

That loop made postgresparser better than it would have been as an internal-only tool.
Internal tools can survive ambiguity. Public libraries cannot.

If you're building something on top of postgresparser, open an issue. Real-world SQL keeps improving the contract.

Building a Pure Go PostgreSQL SQL Parser (No CGO, No Server, No Runtime Dependencies)

Eitamos Ring — Mon, 09 Feb 2026 17:29:25 +0000

Why we built this

We needed PostgreSQL SQL parsing in environments where CGO was not an option:

Alpine containers
AWS Lambda
Distroless images
Scratch builds
ARM deployments
Anywhere CGO_ENABLED=0 is required

Most existing approaches either:

Depend on native Postgres parser bindings
Require CGO
Require running a Postgres server
Are too heavy for infrastructure tooling

So we built a pure Go PostgreSQL parser.

The goal

Not to replace Postgres parsing.

Not to be 100% server-compatible.

The goal was simple:

Give infrastructure and tooling systems structured query data safely and deterministically.

What it extracts

The parser outputs an intermediate representation (IR) with:

Tables (with aliases)
Columns
Joins
WHERE filters
GROUP BY
ORDER BY
CTEs
Subqueries

Example

result, err := postgresparser.ParseSQL(`
    SELECT u.name, COUNT(o.id) AS order_count
    FROM users u
    LEFT JOIN orders o ON o.user_id = u.id
    WHERE u.active = true
    GROUP BY u.name
    ORDER BY order_count DESC
`)

fmt.Println(result.Command)       // "SELECT"
fmt.Println(result.Tables)        // users, orders with aliases
fmt.Println(result.Columns)       // u.name, COUNT(o.id) AS order_count
fmt.Println(result.Where)         // ["u.active=true"]
fmt.Println(result.JoinConditions) // ["o.user_id=u.id"]
fmt.Println(result.GroupBy)       // ["u.name"]
fmt.Println(result.ColumnUsage)   // each column with its role: filter, join, projection, group, order

Now tooling can answer:

What tables does this query touch?
What joins exist?
What filters are applied?

Why ANTLR + Pure Go

We evaluated:

libpg_query bindings
WASM approaches
regex / string parsing
custom parsers

Tradeoffs we cared about

Requirement	Why
Pure Go	Simpler deploy, fewer runtime risks
No CGO	Works in restricted environments
Deterministic behavior	Important for tooling / analysis
Performance	Needed for production workloads

ANTLR gave us:

Mature grammar ecosystem
Strong parsing guarantees
Good performance with SLL mode

Performance

Most real-world queries parse in roughly:

~70–350 microseconds

(using SLL prediction mode)

Where this is useful

Typical use cases:

CI SQL validation
Query lineage hints
Migration safety checks
Static query analysis before deploy
“What tables does this service touch?” automation

Open Source

We’ve been using this internally for months and decided to open source it.

If you break it with weird SQL, please open issues — that’s how coverage improves.

👉 https://github.com/ValkDB/postgresparser

Why Database Indexes Keep Coming Up in My Performance Work

Eitamos Ring — Tue, 29 Jul 2025 06:44:05 +0000

I bounce between data pipelines, API fires, and new features all week, and there’s this one thing that keeps biting us. Slow pages. And 8 times out of 10 it’s the same root cause: we forgot the right index.

We had this analytics dashboard—we tuned the React, cached the API, CDN was spotless. Still slow. The query behind it? joining big tables and scanning like theres no tomorrow. No index on the join keys. Oops.

A quick demo to prove I’m not just ranting
I spun up a tiny test on my dev box: made an orders table, loaded 100k rows, timed a few queries. First with no indexes, then with some obvious ones.

CREATE TABLE orders ( id SERIAL PRIMARY KEY, customer_id INTEGER NOT NULL, order_date TIMESTAMP NOT NULL, total_amount DECIMAL(10,2) NOT NULL, status VARCHAR(50) NOT NULL, country VARCHAR(100) );
Before indexes (avg):

Customer lookup: 6.11 ms

Status filter: 8.47 ms

Date range: 6.73 ms

After indexes:

Customer lookup: 0.88 ms (~7x faster)

Status filter: 2.41 ms (~3.5x)

Date range: 1.48 ms (~4.5x)

Note: these are from my laptop which is also running Docker, two IDEs, Slack, and that Electron app we dont talk about… so, not a lab.

The classic slow page (you’ve seen this movie)
Admin page loads in 10 seconds, everyones pointing fingers. Frontend swears it’s fine, backend says “works on my machine”. The DB? doing full table scans through millions of rows because the where clause is on status and order_date and, yeah, neither is indexed.

Usual suspects

Foreign keys without matching indexes on the child table

Date columns everybody filters by (no index)

Status fields in every WHERE (also no index)

The tiny bit of code that tells the truth
Here’s how I timed the “customer orders” lookup in Go:

func testCustomerIDQuery(db *sql.DB, description string) { var total time.Duration for i := 0; i < numQueries; i++ { id := rand.Intn(100000) + 1 start := time.Now() rows, _ := db.Query(" SELECT id, customer_id, order_date, total_amount, status, country FROM orders WHERE customer_id = $1 LIMIT 10", id) if rows != nil { rows.Close() } total += time.Since(start) } fmt.Printf("%s avg: %v\n", description, total/time.Duration(numQueries)) }
And the “magic” is not magic, it’s just this:

CREATE INDEX idx_customer_id ON orders(customer_id);
CREATE INDEX idx_status ON orders(status);
CREATE INDEX idx_total_amount ON orders(total_amount);
CREATE INDEX idx_order_date ON orders(order_date);
CREATE INDEX idx_country ON orders(country);
CREATE INDEX idx_customer_date ON orders(customer_id, order_date);``
Add those, re‑run the exact same queries, and your 10‑second page quietly becomes sub‑second. It’s almost embarrassing how often that’s the fix.

Index size reality check
People ask “wont indexes be huge?”. From the same test (100k rows):

idx_customer_date: 3.1 MB

orders_pkey: 2.2 MB

idx_order_date: 2.2 MB

idx_total_amount: 2.2 MB

idx_customer_id: 1.9 MB

idx_country: 712 KB

idx_status: 688 KB

Call it ~12 MB total. For the speedup you get, thats cheap.

A couple gotchas (learned the hard way)
Composite order matters. (customer_id, order_date) helps WHERE customer_id = ? ORDER BY order_date DESC LIMIT 10. Flip it and you’ll be sad.

Check the plan. EXPLAIN (ANALYZE, BUFFERS)—you want Index Scan / Index Only Scan, not Seq Scan.

Not every column deserves an index. Super low selectivity (like a boolean) usually wont help with a plain b‑tree.

Writes pay the bill. Indexes speed reads, but inserts/updates get a bit slower—so pick the ones you actually use.

The boring checklist that works
Before you reach for sharding, a rewrite, or a shiny new DB:

Profile the slow endpoint.

EXPLAIN (ANALYZE, BUFFERS) the worst query.

If it’s scanning a big table, add the smallest useful index.

Re‑test. Ship. Sleep.

It’s not flashy. It won’t wow anyone at a meetup. But it’ll make your app feel fast, which is what users care about anyway.

Forem: Eitamos Ring

Three Rules for Designing a Go SDK Other People Will Actually Use

Rule 1: Expose answers, not nodes

Rule 2: Name the common case after the common case, and mark the variants

Rule 3: Return structured data, not strings the caller has to re-parse

Where I broke my own rule

TLDR;

What I Learned Building a Pure Go PostgreSQL Parser

Why I built it

What made PostgreSQL parsing harder than expected

Why pure Go mattered

What 200+ GitHub stars taught me

Where it’s going

What every `?` in your SQL is hiding

The new API

How the old way failed

The five footguns this release closes

1. The JSONB ? operator is not a placeholder

2. INTERVAL ? actually parses

3. ? inside string literals stays inside string literals

4. GROUP BY ? is an ordinal, not a value

5. Function-argument placeholders need to know their function

Who this is for

Closing

A Protobuf for Database Schemas

The problem is older than LLMs

Extract once, use many ways

The sidecar pattern

Why JSON, not SQL

Stop treating your schema like a black box

Stop Sending 93K Tokens of Schema to Your LLM Agent!

What it does

No credentials in the agent runtime

The numbers

Sidecar enrichment

What it doesn't do

Try it

How does a linter know your column doesn't exist

We didn't want an AI SQL reviewer. We wanted deterministic

What it actually catches

Why AST, not AI

The pipeline

CI integration

Where it fits

Building a PostgreSQL Parser in Go: What Broke After We Open-Sourced It

Building a PostgreSQL Parser in Go: What Broke After We Open-Sourced It

Our internal assumptions broke immediately

Broken behavior example

Before/after API diff

Real SQL in the wild is much uglier than test fixtures

One concrete snippet (current behavior)

Why this matters

Building a Pure Go PostgreSQL SQL Parser (No CGO, No Server, No Runtime Dependencies)

Why we built this

The goal

What it extracts

Example

Why ANTLR + Pure Go

Tradeoffs we cared about

Performance

Where this is useful

Open Source

Why Database Indexes Keep Coming Up in My Performance Work

1. The JSONB `?` operator is not a placeholder

2. `INTERVAL ?` actually parses

3. `?` inside string literals stays inside string literals

4. `GROUP BY ?` is an ordinal, not a value