Forem: kent-tokyo

Cheminformatics in Rust in 2025-2026: What Exists, What Doesn't, and Why

kent-tokyo — Mon, 25 May 2026 13:07:26 +0000

RDKit has been the dominant cheminformatics library since its open-source release in 2006. It is written in C++, wrapped in Python, and has accumulated nearly two decades of validated chemistry: SMILES and SMARTS parsing, multiple fingerprint types, 2D coordinate generation, 3D conformer generation, MMFF94 and UFF force fields, a PostgreSQL cartridge. Most cheminformatics pipelines assume it is present.

In mid-2026, Rust's answer is rdkit-sys — bindings to RDKit's C++ CFFI interface — and a collection of pure-Rust crates that stalled in 2020-2021.

What exists in 2025-2026

Crate	Type	Latest	Status
rdkit-sys	C++ FFI to RDKit	0.4.12 (Oct 2024)	Maintained
openbabel	C++ FFI to Open Babel	0.5.4 (Jan 2025)	Maintained
chemcore	Pure Rust	0.4.1 (Feb 2021)	Unmaintained
purr	Pure Rust (SMILES parser)	0.9.0 (Mar 2021)	Unmaintained
smiles-parser	Pure Rust (SMILES parser)	0.4.1 (Nov 2020)	Unmaintained
cosmolkit	Pure Rust (new attempt)	0.2.3 (May 2026)	New, unproven

The pattern in the pure-Rust column is consistent: implementations hit a wall around 2020-2021 and stopped. The active work is FFI bindings to existing C++ tools. A new attempt (cosmolkit) appeared recently with an ambitious scope — SMILES, SDF, conformers, molecular graphs — but with under 800 downloads it is too early to evaluate.

SMILES parsing is solved. The rest is not.

Parsing a SMILES string is a context-free grammar problem, and Rust handles those well. purr implements the full OpenSMILES specification. smiles-parser does the same. Both work. Neither has had a release since 2020-2021.

The problem starts after parsing.

A SMILES string like c1ccccc1 (benzene) uses lowercase atoms to indicate aromaticity. To do anything useful — calculate molecular weight, count implicit hydrogens, check valence — you need to convert it to a Kekulé structure: alternating single and double bonds. This is kekulization, and it is a constraint-satisfaction problem on the molecular graph.

chemcore, the most complete pure-Rust attempt, has supported kekulization since its initial release (v0.1.x, June 2020). A benchmark published alongside v0.3.1 in October 2020 showed it handling edge cases that RDKit cannot. But kekulization is one step. What chemcore does not have: fingerprints, 2D coordinate generation, SMARTS matching, or stereochemistry. The last release was February 2021. Getting past kekulization turned out not to be the finishing line.

Aromaticity: no agreed definition

Even with kekulization in place, aromaticity perception is harder than it looks — partly because aromaticity itself has no single agreed-upon definition in cheminformatics.

Hückel's rule — 4n+2 π electrons — works for monocyclic systems. For polycyclic aromatics and heteroaromatics, implementations diverge. Daylight's original SMILES aromatic model differs from RDKit's model, which differs from CDK's. An algorithm that kekulizes correctly under one model may fail under another.

Any pure-Rust toolkit that wants to produce output compatible with RDKit-generated SMILES needs to match RDKit's aromaticity behavior exactly, not implement some variant of Hückel. That requires reading RDKit's source code and testing against its outputs. It is months of work before any of it is visible to end users.

2D coordinate generation: not attempted

Every cheminformatics toolkit ships 2D depiction — you cannot work with molecules you cannot see. The layout problem is harder than it looks.

RDKit ships its own 2D depiction engine (rdDepictor) and also integrates Schrodinger's CoordGen library because rdDepictor alone produces clashing depictions for complex ring systems. Two tools are needed because neither is sufficient alone. CoordGen works by matching known ring scaffold templates and running iterative geometry optimization for everything else.

No pure-Rust crate has attempted 2D coordinate generation. Getting it right requires ring perception, a library of scaffold templates, and an optimization pass to resolve clashes. It is a multi-month project, and the output is still wrong until enough templates are added.

Substructure search: the graph is not the chemistry

petgraph (v0.8.3, 377M total downloads) provides VF2-based subgraph isomorphism and is actively maintained. VF2 is the standard algorithm for this — roughly an order of magnitude faster than Ullmann on typical molecule-sized graphs. The graph infrastructure exists in Rust.

SMARTS matching, which is how substructure search works in cheminformatics, requires more than graph isomorphism. A SMARTS pattern [#6;r6] means "a carbon atom in a 6-membered ring." Matching it requires: parsing SMARTS syntax, knowing which atoms belong to which rings, and matching node attributes with chemical semantics — atomic number, formal charge, aromaticity flag, implicit hydrogen count.

Connecting petgraph's isomorphism to a chemistry-aware molecular graph is exactly the glue code that no published Rust crate provides.

Why bindings are the rational choice

RDKit's changelog goes back to 2006. The codebase contains 200+ molecular descriptors, MMFF94 and UFF force fields with their respective validation papers, an ETKDG 3D conformer generator that uses torsion angle statistics from the Cambridge Structural Database, and a PostgreSQL cartridge for large-scale screening. The Python ecosystem wraps all of this: chembl_webresource_client for ChEMBL API access, PandasTools, scikit-learn integration for ML on fingerprints.

rdkit-sys exposes a fraction of this via RDKit's CFFI interface. Choosing bindings over a rewrite is not a concession. It is what you do when you look at how much chemistry is embedded in that C++ code and how long it took to get there.

What changed in 2024-2025, and what 2026 adds so far

2024-2025: rdkit-sys had three releases in 2024, the last in October, and moved into the rdkit-rs/rdkit monorepo. openbabel (Rust bindings) released 0.5.4 in January 2025 — it exposes Open Babel's OBSmartsPattern, which matters if you need substructure search without pulling in RDKit.

2026: The only 2026-specific addition is cosmolkit (v0.2.3, May 2026, 778 downloads). It claims an ambitious scope — SMILES, SDF, conformers, molecular graphs, "AI-ready workflows" — but it is too new to evaluate. Whether it addresses aromaticity perception and 2D layout, the parts that stopped every earlier attempt, is not clear from the current documentation.

As of this writing, nothing else has shipped in 2026. The structural gap between Rust and Python cheminformatics is the same as it was in 2025.

The actual hard part

The challenging problems in cheminformatics are not Rust-specific. Ownership and lifetimes will slow you down on day one; aromaticity will block you on month three. The chemistry fundamentals — aromaticity perception, 2D layout, stereochemistry, substructure matching — require domain knowledge that does not come from a Rust tutorial.

RDKit did not get where it is because C++ is better than Rust. It got there because a team of chemists and programmers spent two decades solving specific, hard chemistry problems. Whoever builds the Rust equivalent will need to solve the same problems.

I have been working around these gaps while building chem-wasm-lens, a pure-Rust molecular analysis library targeting the browser via WebAssembly. Restricting scope — no SMARTS, no full stereochemistry — made it possible to ship. But restricted scope is different from a general-purpose toolkit, and that distinction matters.

Rust in 2025-2026: From 'Most Loved Language' to Core Infrastructure

kent-tokyo — Sun, 24 May 2026 04:38:04 +0000

Rust has held the top spot in Stack Overflow's "most admired language" survey since 2016, nearly without interruption. But what happened in 2025 is no longer just about popularity polls. Rust is quietly but steadily becoming the language that underpins critical infrastructure.

"Experimental" Status Removed from the Linux Kernel

In December 2025, at the Kernel Maintainers Summit held in Tokyo, Rust's status in the Linux kernel was elevated from experimental to an officially recognized implementation language.

As LWN.net reported, the outcome was unambiguous: "The consensus among the assembled developers is that Rust in the kernel is no longer experimental — it is now a core part of the kernel and is here to stay." Maintainer Steven Rostedt noted there was zero pushback in the room. About five years after Linus first suggested the possibility in 2020, the matter was settled.

Rust code in the kernel currently stands at around 25,000 lines (compared to 34 million lines of C) — still a small share — but the subsystems adopting it are steadily expanding.

Rust Code Running in Production

Major components using Rust in the kernel today:

PHY drivers — network physical layer
null block driver — test block device
Android Binder driver — kernel-side implementation of Android's IPC mechanism
Apple AGX GPU driver — for Apple Silicon, via the Asahi Linux project
Nova GPU driver — for NVIDIA Turing-generation hardware (RTX 20 / GTX 16 series)

The Nova driver is architecturally interesting. It is split into two crates: nova-core (hardware initialization and communication) and nova-drm (Linux DRM API implementation), using an adapter pattern that maps different bus types — PCI, platform, USB — to the same types. Note that as of early 2026, full enablement is still in progress; development is ongoing.

Why Rust is Well-Suited for the Kernel

The majority of kernel driver bugs stem from memory safety issues: NULL pointer dereferences, use-after-free, buffer overflows, data races. In C, the only mitigation is "write carefully." Rust eliminates these classes of bugs at compile time, by construction.

Greg Kroah-Hartman has stated that Rust drivers are safer than their C counterparts, and this is exactly why. The 25,000-line figure is small, but it also means Rust is replacing the parts where "bugs would have been guaranteed if written in C."

The Technical Story Behind `async closures`

Among the stabilizations in 2025, async closures (Rust 1.85) were a deeper change than they appear.

What was wrong with the old workarounds?

Previously, writing an "async closure" looked like this:

// The closure couldn't borrow captured variables inside the Future.
// You had to either move ownership in, or wrap with Arc<Mutex<T>>.
let data = vec![1, 2, 3];
let f = |x: i32| {
    let data = data.clone(); // clone required
    async move { data[x as usize] }
};

The problem with |x| async move { ... } was that the returned Future could not borrow captures from the closure itself. Because the Future has a different lifetime from the closure, you had no way to pass references — you had to either move ownership or clone.

The `AsyncFn` Trait Hierarchy

The async closures introduced in Rust 1.85 come with a new trait hierarchy internally: AsyncFn / AsyncFnMut / AsyncFnOnce.

AsyncFnOnce
  └─ AsyncFnMut
       └─ AsyncFn

This mirrors the existing Fn* traits, but with a critical difference: the Future returned by an async closure can borrow from the closure itself (lending).

let data = vec![1, 2, 3];

// Rust 1.85+: the Future can borrow data directly
let f = async |x: usize| { data[x] };

// Usage at function boundaries
async fn apply<F: AsyncFn(usize) -> i32>(f: F, x: usize) -> i32 {
    f(x).await
}

This works because AsyncFnMut's CallRefFuture associated type is designed to propagate the &mut self lifetime into the Future. Just as FnMut returns &mut self to the caller, AsyncFnMut lets the Future hold that lifetime.

`let chains` — Quietly Important

Also stabilized in 2025, let chains (Rust 1.88, Rust 2024 edition only) look simple but are significant: you can now freely combine if let patterns and ordinary bool conditions with &&. Using this requires edition = "2024" in Cargo.toml.

// before: forced to nest or introduce temp variables
if let Some(user) = get_user(id) {
    if user.is_active() {
        if let Some(role) = user.role() {
            println!("{}", role);
        }
    }
}

// after: flatten conditions and pattern matches on one line
if let Some(user) = get_user(id)
    && user.is_active()
    && let Some(role) = user.role()
{
    println!("{}", role);
}

The same syntax works in while let and match guards, visibly reducing nesting depth throughout a codebase.

Safety Certification: A New Frontier

In December 2025, Ferrous Systems obtained IEC 61508 SIL 2 certification from TÜV SÜD for a subset of Rust's core library, under Ferrocene 25.11.0.

What is IEC 61508?

IEC 61508 is an international standard for functional safety of electrical, electronic, and programmable electronic safety-related systems. SIL (Safety Integrity Level) 2 corresponds to a "probability of dangerous failure of 10^-7 to 10^-6 per hour" — the level required in safety-critical domains such as aerospace, medical devices, industrial machinery, and automotive.

Historically, the de facto standard for safety-critical embedded systems has been C/C++ with MISRA. Rust achieving this level of certification means it has officially stepped into the domain where functional safety is required — not just systems programming for general use.

MISRA C 2025 Addendum 6

The same year, MISRA C 2025 Addendum 6 was published: an assessment of how MISRA C rules apply to Rust. The conclusion is that many existing C-specific rules are simply not applicable to Rust — the compiler enforces them by construction via the ownership model. This document also lays the groundwork for a future Rust-specific MISRA rule set.

Production Adoption Approaching 50%

According to JetBrains' State of Rust Ecosystem 2025, roughly half of surveyed organizations are now using Rust in production in non-trivial ways — a significant jump from around 38-39% in 2023.

Microsoft has started rewriting low-level Windows components in Rust, and Google, Amazon, and Meta have each introduced it into OS-level systems.

An Unexpected Fit for the LLM Era

In an era dominated by generative AI, Rust is being reassessed in an unexpected way.

LLM-generated code contains mistakes. Rust's compiler returns those mistakes immediately and concretely as build errors. The type system and borrow checker enumerate the mistakes an AI made — before the code ever runs.

error[E0502]: cannot borrow `data` as mutable because it is also borrowed as immutable
  --> src/main.rs:8:5
   |
6  |     let r = &data;
   |             ----- immutable borrow occurs here
7  |     println!("{}", r);
8  |     data.push(4);
   |     ^^^^^^^^^^^^ mutable borrow occurs here

The tight feedback loop — "if it compiles, memory safety is guaranteed" — is well-suited for pair programming with an LLM. Fixing Rust compile errors has higher reproducibility than debugging Python runtime errors.

The Biggest Concern: Adoption May Plateau

The State of Rust Survey 2025 found that the top concern is not a technical one.

"Not enough adoption in the tech industry" came in first at 42.1% (narrowly ahead of "the language is getting too complex" at 41.6%).

The learning curve remains steep and unresolved. Some feel the language itself is growing more complex over time. In early-stage startups where velocity is critical, Rust's upfront cost is real.

That said, this concern is also the voice of people who want to use Rust more but see adoption lagging. The fact that technical concerns did not dominate the top of the list suggests Rust has reached a meaningful level of maturity.

Summary

In one sentence: as of 2026, Rust is in the early stages of a shift from "the language people love" to "the language people need."

Linux kernel — Experimental status removed. Real drivers like Nova and Apple AGX are running.
async closures — The AsyncFn trait hierarchy lets Futures borrow from their enclosing closure.
let chains — Flatten if let + bool conditions, reducing nesting without temp variables.
Ferrocene / IEC 61508 — Rust now has a formal foothold in safety-critical domains.
LLM compatibility — The compiler becomes a clear feedback loop for AI-generated code.
Top concern is adoption speed — Technical concerns have receded; ecosystem breadth is now the focus.

Sources:

Open-source SDS tooling for Japanese MHLW compliance: the gap nobody filled

kent-tokyo — Sat, 23 May 2026 01:34:49 +0000

In March 2025, Japan's Ministry of Health, Labour and Welfare (MHLW) published a structured JSON schema for Safety Data Sheet data exchange. The schema covers roughly 200 deeply nested fields and is intended to standardize how SDS information moves between chemical management systems.

Most SDS tooling was not built for this.

What makes Japan's SDS requirements different

Japan's SDS requirements come from two laws: the Industrial Safety and Health Act (ISAH, 労働安全衛生法) and the Chemical Substances Control Law (化審法). Both mandate SDS for regulated chemicals, with format requirements governed by JIS Z 7253 — Japan's implementation of the UN Globally Harmonized System (GHS).

JIS Z 7253 follows the standard 16-section GHS structure. In principle, any GHS-compliant SDS satisfies the content requirements. What makes Japanese compliance distinct is a digital layer: the MHLW schema specifies how SDS content should be structured as machine-readable data, with field-level granularity that PDF documents cannot capture.

How GHS looks different by country

GHS uses a "building block" approach — each country adopts the elements it chooses. The result is that the same GHS-aligned document varies by jurisdiction:

Country/Region	Standard	GHS basis	Notable difference
Japan	JIS Z 7253:2019	GHS Rev. 6	MHLW digital schema; revised to GHS Rev. 9 in Dec 2025
United States	OSHA HazCom 2012	GHS Rev. 3	Updated to GHS Rev. 7 in 2024
European Union	CLP Regulation	GHS-aligned	Stricter on environmental hazards
China	GB 13690-2009	GHS Rev. 4 equivalent	Moving to GB 30000.1-2024 (GHS Rev. 8), mandatory from August 2025
Taiwan	CNS 15030	GHS-aligned	—

Japan-specific regulatory fields

The MHLW schema includes fields with no equivalent in EU REACH or US OSHA HazCom formats. These are the main reason international SDS tooling does not cover the schema out of the box:

Law	Example fields	What they capture
Chemical Substances Control Law (化審法)	`CaSCL.ClassificationStatus`, `CaSCL.RegistrationNumber`	Regulatory classification and registration numbers under this law
Industrial Safety and Health Act (安衛法)	`ISHAct.PublicationOfName`, `ISHAct.Notification`	Name disclosure and notification obligations
Poisonous and Deleterious Substances Control Law	`ControlledSubstancesAct.Applicability`	Whether the substance is classified as poison, deleterious, or specific poison
PRTR Law	—	Chemical release and transfer reporting obligations

Section 15 (Regulatory Information) is the most complex section in the schema — it contains separate subsections for each of these laws, each with its own field structure.

Why this matters now: the 2022 law revision

The MHLW published the schema in 2025, but the driver was a 2022 amendment to the Industrial Safety and Health Act. The amendment shifted Japan's chemical substance regulation from a prescriptive model (government designates specific hazardous substances) to an autonomous management model (companies assess and manage risk themselves).

The practical impact:

Enforcement date	Change
April 2023	Shift to autonomous management model — all substances with confirmed GHS hazard classifications brought progressively into scope
April 2024	SDS must now specify concentration ranges numerically (not just qualitatively)
April 2025	Protective equipment mandatory for substances with skin/eye hazards
April 2027	Risk assessment obligations expand to all regulated substances

With risk assessment coverage expanding significantly, companies need to process SDS data faster and more accurately. Manual PDF entry does not scale. The JSON schema is the infrastructure layer for automating this.

Where existing tools stop

Commercial SDS platforms

The major SDS authoring platforms — Sphera, EcoOnline, Chemwatch, Verisk 3E — have broad international coverage. Japanese is typically a supported output language. What they do not provide, as far as I have found, is export to the MHLW JSON schema. They produce Word or PDF output in the correct section structure, which satisfies the document requirement but not the structured data exchange requirement.

Japanese-market products like SDS Meister and SmartSDS support MHLW JSON output, but their PDF-to-JSON conversion coverage is limited — they are primarily SDS authoring tools, not bulk conversion tools for incoming supplier documents.

Open-source options

Tool	Language	MHLW JSON	PDF → JSON	Approach
sds_parser	Python	No	Yes	Regex, per-manufacturer rules
tungsten	Python	No	Yes	Rule-based, English-only
sds-converter	Rust	Yes	Yes	LLM-based extraction

sds_parser and tungsten solve a different problem: extracting SDS data in English, for specific known manufacturer formats. Neither targets the MHLW schema.

The format inconsistency problem

Even within JIS Z 7253-compliant documents, format varies by manufacturer:

Source of variation	Example
Section heading labels	"2. 危険有害性の要約" (JIS Z 7253) vs "2. Hazard(s) identification" (OSHA HazCom) vs "第2部分危险性概述" (GB/T 16483) — all mean the same thing
Section order	The 16 sections can appear in any order the manufacturer chooses
Concentration notation	"≥95%", "1〜5%", "約100%", "企業秘密" (trade secret) all need different handling
Language mixing	Japanese SDS documents regularly contain English chemical names and CAS numbers

A rule-based parser must enumerate every variant. In practice, manufacturer-specific headings add another layer of variation on top of the standard differences.

The schema itself

Two properties of the MHLW schema are worth knowing before implementing against it.

Section 3 (composition) is the hardest part

Section 3 stores component information as a repeating array. Each component object has nested fields for chemical identity, concentration range, and hazard classification. The same data appears differently depending on whether the source document covers a pure substance, a mixture, or a trade secret formulation.

{
  "Composition": {
    "CompositionAndConcentration": [
      {
        "ChemicalIdentity": {
          "CASNumber": "64-17-5",
          "ISHActNotificationNumber": "2-396"
        },
        "ConcentrationRange": {
          "ConcentrationRangeFrom": 95.0,
          "ConcentrationRangeTo": 100.0,
          "ConcentrationRangeUnit": "%"
        },
        "TradeSecretFlag": false
      }
    ]
  }
}

Typos locked into v1.0

The schema contains field name errors that are now part of the specification:

HumanExposureAndEmergencyMeasuress  ← trailing double-s
TestGuidline                        ← missing 'e' (not Guideline)
Desclaimer                          ← transposed letters (not Disclaimer)
gazetteNo                           ← lowercase first character

Correcting these would break all existing implementations, so they cannot be fixed in v1.0. An implementation that normalizes these to standard English spellings will fail schema validation.

sds-converter

I built sds-converter to address the MHLW schema gap. It handles both directions: PDF/DOCX/XLSX to MHLW JSON, and MHLW JSON to a JIS Z 7253-compliant Word document.

The core approach: rather than enumerating format variants with rules, the tool passes raw section text and the corresponding MHLW schema fields to an LLM and asks it to map values. The LLM handles heading label variation naturally. The output is validated against the schema before writing.

cargo install sds-converter

# PDF → MHLW JSON
sds-converter to-json --input input.pdf --output output.json

# MHLW JSON → JIS Z 7253 Word document
sds-converter to-docx --input output.json --output result.docx --lang ja

The LLM backend is pluggable — Claude, GPT, Gemini, Mistral, Groq, or local models via Ollama. A --quality flag adjusts cost versus accuracy for batch workloads.

Known limitations:

Issue	Status
Scanned PDFs without a text layer	Not supported — requires upstream OCR
Section 3 tables with merged cells	Extraction sometimes fails on complex DOCX layouts
Precision fields mixed with "not measured" entries	Occasional type errors in Section 9 output

These are open problems, not design decisions.

The open gap

The MHLW schema represents a real need for anyone handling chemical compliance in Japan at volume. Commercial tools cover the authoring side; the bulk conversion of incoming supplier PDFs to structured data has no open-source solution targeting this schema — other than sds-converter, which I developed and which is the only implementation I am aware of.

The repository is open. Contributions on the extraction side — particularly Section 3 table handling — are welcome. If you work in cheminformatics or chemical compliance and have approached the MHLW compliance problem differently, I would be interested to hear it.

sds-converter: Converting Safety Data Sheets to MHLW Standard JSON with Rust and LLMs

kent-tokyo — Fri, 22 May 2026 23:09:11 +0000

Background

Safety Data Sheets (SDS) are mandatory documents for every chemical product — solvents, adhesives, industrial gases, cleaning agents. Every manufacturer that supplies a hazardous chemical must provide one. In Japan, the governing standard is JIS Z 7253, which defines 16 sections covering chemical identity, hazard classification, first aid, storage, transport information, and more.

The Ministry of Health, Labour and Welfare (MHLW) published a standard JSON schema in March 2025 for electronic SDS data exchange between chemical management systems. The schema has roughly 200 deeply nested fields covering all 16 sections.

The problem is that real SDS documents don't arrive structured to this schema.

Why SDS documents are hard to parse

Even two documents both compliant with JIS Z 7253 will differ in ways that break rule-based parsers:

Section order — manufacturers arrange the 16 sections freely within the standard
Field labeling — the same data appears under different headings across JIS Z 7253, GHS/OSHA HazCom, GB/T 16483, CNS 15030, and company-specific layouts
Value representation — "≥99.5%", "99.5% or higher", "approximately 100%" all mean the same thing
Language mixing — Japanese SDS regularly embed English chemical names and CAS numbers mid-sentence
Implicit information — section 9 (physical/chemical properties) often has half its fields missing because manufacturers only fill in what's relevant to their product

The MHLW schema compounds this: it has intentional typos that must be reproduced exactly. HumanExposureAndEmergencyMeasuress ends in double-s. TestGuidline is missing an e. Desclaimer has transposed letters. These are in the official spec, and validation fails if you "fix" them.

To handle SDS from international manufacturers (GHS/OSHA format) or Chinese suppliers (GB/T 16483 format) in the same pipeline, you'd need separate parsers for each format. Writing and maintaining those is impractical. I built sds-converter to handle this with an LLM instead.

The 16 sections

#	Schema key	JIS Z 7253 section
1	`Identification`	Chemical identity and company information
2	`HazardIdentification`	Hazard identification
3	`Composition`	Composition / information on ingredients
4	`FirstAidMeasures`	First-aid measures
5	`FireFightingMeasures`	Fire-fighting measures
6	`AccidentalReleaseMeasures`	Accidental release measures
7	`HandlingAndStorage`	Handling and storage
8	`ExposureControlPersonalProtection`	Exposure controls / personal protection
9	`PhysicalChemicalProperties`	Physical and chemical properties
10	`StabilityReactivity`	Stability and reactivity
11	`ToxicologicalInformation`	Toxicological information
12	`EcologicalInformation`	Ecological information
13	`DisposalConsiderations`	Disposal considerations
14	`TransportInformation`	Transport information
15	`RegulatoryInformation`	Regulatory information
16	`OtherInformation`	Other information

Installation and quick start

cargo install sds-converter

# PDF → MHLW standard JSON
export ANTHROPIC_API_KEY=sk-ant-...
sds-converter to-json --input input.pdf --output output.json

# MHLW JSON → JIS Z 7253-compliant Word document
sds-converter to-docx --input output.json --output result.docx --lang ja

# Schema validation
sds-converter validate --input output.json

# Extract raw text (no LLM call — useful for debugging)
sds-converter extract-text --input input.pdf

Supported input: PDF, DOCX, XLSX, TXT.

How the conversion works

Step 1: Text extraction

Text is pulled from the PDF or DOCX file. Use extract-text to inspect exactly what gets sent to the LLM — useful when extraction quality is lower than expected.

Note: Encrypted PDFs and scan-only (image) PDFs are not supported — text extraction requires selectable text.

Step 2: Parallel LLM extraction

The 16 sections are split into two groups and extracted with two parallel LLM calls, halving per-file latency:

GROUP_A (sections 1–9): identification, hazard, composition, first aid, fire fighting, accidental release, handling, exposure, physical properties
GROUP_B (sections 10–16): stability, toxicology, ecological, disposal, transport, regulatory, other

Results from both calls are merged. Sections skipped in the first pass are automatically retried. HTTP rate-limit responses (429/529) trigger exponential backoff retries (2s → 4s → 8s, up to 3 attempts).

Step 3: JSON output

The merged result is written as MHLW SDS data exchange format v1.0 JSON.

LLM backend and quality settings

Choosing a provider

# OpenAI GPT (gpt-4o-mini by default)
sds-converter to-json --input input.pdf --output output.json \
  --provider openai --api-key $OPENAI_API_KEY

# Google Gemini (gemini-2.0-flash by default)
sds-converter to-json --input input.pdf --output output.json \
  --provider gemini --api-key $GEMINI_API_KEY

# Local LLM via Ollama (any OpenAI-compatible endpoint)
sds-converter to-json --input input.pdf --output output.json \
  --provider local --base-url http://localhost:11434/v1 \
  --model llama3.2 --api-key dummy

`--provider`	Default model	Environment variable
`anthropic`	`claude-haiku-4-5-20251001` (low/medium) · `claude-sonnet-4-6` (high)	`ANTHROPIC_API_KEY`
`openai`	`gpt-4o-mini`	`OPENAI_API_KEY`
`gemini`	`gemini-2.0-flash`	`GEMINI_API_KEY`
`mistral`	`mistral-small-latest`	`MISTRAL_API_KEY`
`groq`	`llama-3.3-70b-versatile`	`GROQ_API_KEY`
`cohere`	`command-r-plus`	`COHERE_API_KEY`
`local`	`llama3`	`LOCAL_LLM_API_KEY` (optional)

Quality preset

--quality controls both the model and how much text is sent to the LLM per call:

`--quality`	Model (Anthropic)	Max text fed to LLM	Use case
`low`	claude-haiku-4-5	15,000 chars	Speed/cost priority
`medium` (default)	claude-haiku-4-5	30,000 chars	Balanced
`high`	claude-sonnet-4-6	60,000 chars	Accuracy priority

At high, the full document text including the later sections (transport information, regulatory) is included. Use --quality high when complete 16-section coverage matters.

Batch mode

sds-converter to-json \
  --input-dir ./pdfs/ \
  --output-dir ./json/ \
  --lang ja \
  --concurrency 4

Validation

validate checks structural completeness of the extracted JSON and returns warnings without hard-failing — partial results are still usable.

sds-converter validate --input output.json

Examples of what it checks:

Section 1: no product name (TradeNameJP or TradeNameEN)
Section 1: SupplierInformation missing
Section 2: neither Classification nor HazardLabelling extracted
Section 3: CompositionAndConcentration list is empty

When using the library, convert_to_json returns a (SdsRoot, Vec<String>) tuple — the warnings are surfaced inline.

Output JSON structure

{
  "Datasheet": {
    "IssueDate": "2024-03-31",
    "SDS-SchemaVersionNo": "1.0"
  },
  "Identification": {
    "TradeProductIdentity": {
      "TradeNameJP": "Sample Product"
    },
    "SupplierInformation": {
      "CompanyName": "Sample Corp",
      "Phone": "03-0000-0000"
    }
  }
}

The full schema covers all 16 JIS Z 7253 sections with ~200 fields. The official spec and developer manual are on the MHLW website (Japanese).

Using as a library

[dependencies]
sds-converter-core = "0.1"

PDF → JSON

use sds_converter_core::{
    converter::{AnthropicBackend, LlmConfig},
    convert_to_json, ConvertConfig, Language,
};

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let backend = AnthropicBackend::new(
        std::env::var("ANTHROPIC_API_KEY")?,
        LlmConfig::default(),
    );
    let config = ConvertConfig {
        source_language: Some(Language::Japanese),
        output_language: Language::Japanese,
        ..Default::default()
    };
    let (sds, warnings) = convert_to_json(
        std::path::Path::new("input.pdf"), &backend, &config
    ).await?;
    for w in &warnings { eprintln!("WARN: {w}"); }
    std::fs::write("output.json", serde_json::to_string_pretty(&sds)?)?;
    Ok(())
}

JSON → Word document

use sds_converter_core::{convert_from_json, ConvertConfig, Language, SdsRoot};

fn main() -> anyhow::Result<()> {
    let sds: SdsRoot = serde_json::from_str(&std::fs::read_to_string("output.json")?)?;
    let config = ConvertConfig {
        output_language: Language::Japanese,
        ..Default::default()
    };
    convert_from_json(&sds, std::path::Path::new("result.docx"), &config)?;
    Ok(())
}

Custom LLM backend

use sds_converter_core::{LlmBackend, SdsError};

struct MyBackend;

impl LlmBackend for MyBackend {
    async fn complete(&self, system: &str, user: &str) -> Result<String, SdsError> {
        // Call your LLM API, return the raw JSON string response
        todo!()
    }
}

Language support

Language	`--lang`	Source standard	Output DOCX headings
Japanese	`ja`	JIS Z 7253	JIS Z 7253
English	`en`	GHS/OSHA HazCom	GHS Rev.10 / ISO 11014
Simplified Chinese	`zh-cn`	GB/T 16483-2012	GB/T 16483-2012
Traditional Chinese	`zh-tw`	CNS 15030	CNS 15030

Comparison with alternatives

Open-source

	sds-converter	sds_parser	tungsten
Language	Rust	Python	Python
AI/LLM	Yes (pluggable)	No (regex)	No (rule-based)
MHLW JSON	Yes	No	No
Bidirectional	Yes (↔ DOCX)	No	No
Multilingual	ja / en / zh-CN / zh-TW	Limited	English only

Commercial (Japan)

	sds-converter	SDS Meister	SmartSDS	Dr.EHS Chemical
AI	Yes (your API key)	No	Yes (translation)	AI-OCR
MHLW JSON	Yes	Yes	Yes	Yes
PDF → JSON	Yes	No (authoring only)	Partial (JP only)	Yes
Open-source	MIT/Apache-2.0	No	No	No

sds-converter is the only open-source tool that supports the MHLW schema, runs entirely locally, and handles the full round-trip.

Crate structure

sds-converter-core — library. LLM extraction, DOCX generation, MHLW schema types.
sds-converter — CLI binary. to-json, to-docx, validate, extract-text subcommands.

Feedback welcome, especially on section 3 component table extraction and non-Japanese document accuracy.

https://github.com/kent-tokyo/sds-converter

Why Font Rendering in Rust Is Harder Than It Looks

kent-tokyo — Fri, 22 May 2026 13:48:14 +0000

When you want to render text in a pure Rust project — a GUI app, a document generator, a terminal — you quickly find that "render some text" is actually four distinct problems stacked on top of each other. The C ecosystem has a library for each one. Rust has options too, but coverage is uneven, and the naming conventions obscure which crates actually depend on C.

The four layers

Layer	C library	What it does
Font discovery	fontconfig	Finds installed fonts by name, family, or script
Text shaping	HarfBuzz	Converts a Unicode string → glyph IDs with positions
Rasterization	FreeType	Turns outline vectors → pixel bitmaps
Layout	Pango	Wraps all three, handles BiDi, line breaking

On macOS/iOS the shaping and rasterization layers are CoreText. On Windows they're DirectWrite. In both cases, the system library is C or Objective-C, not Rust.

Each layer depends on the one below it: shaping needs the raw font bytes to read OpenType tables, rasterization needs the shaped glyph IDs, layout needs both. You can't swap in one layer without thinking about the others.

What text shaping actually does

Shaping is the step between "here is a Unicode string" and "here are the glyph IDs and positions to draw." Most developers don't know it exists until something renders wrong.

The string "fi" contains two Unicode codepoints (U+0066, U+0069). But most fonts substitute those two characters with a single ligature glyph — "ﬁ". In Arabic, the same letter takes a different shape depending on whether it appears at the start, middle, or end of a word. Even Latin fonts have kerning rules that adjust spacing between specific letter pairs.

All of this is encoded in the font file's OpenType tables: GSUB (glyph substitution) and GPOS (glyph positioning). A shaping engine reads those tables and converts your input string into a list of (glyph ID, x, y) tuples. HarfBuzz is the de facto standard for this. Without a shaper, you get incorrect rendering for anything beyond the simplest Latin text.

The naming trap

Several font crates look like pure Rust but are C wrappers:

harfbuzz-rs — a safe Rust wrapper around libharfbuzz. The API feels like Rust, but libharfbuzz is compiled from C++. Run cargo tree and you'll see harfbuzz-sys in the graph.

freetype-rs — same pattern. Safe API, C underneath. Look for freetype-sys.

fontconfig — bindings to the C fontconfig library.

If you're targeting wasm32-unknown-unknown or cross-compiling without a C toolchain, these fail at build time. The -sys suffix in cargo tree output is the reliable signal.

Pure Rust, layer by layer

Font discovery: `fontdb`

fontdb reads font directories directly, parses font metadata from binary headers, and supports querying by family name, weight, style, and script coverage. No C. Works on Linux, macOS, Windows, and WASM.

It doesn't use fontconfig under the hood — it scans directories directly. For most use cases this is fine. If you need fontconfig's alias resolution ("sans-serif" mapping to a specific installed font), fontdb covers part of this logic but not all of it.

Text shaping: `rustybuzz` or `harfrust`

rustybuzz is a complete port of HarfBuzz v10.1.0 to Rust. It passes 2,221 out of 2,252 HarfBuzz shaping tests. Performance is 1.5–2x slower than C HarfBuzz. Maintained under the harfbuzz org. Used in cosmic-text.

harfrust is a newer port, matching HarfBuzz v13.0.0. It started as a fork of rustybuzz, migrating the font parser from ttf-parser to read-fonts (Google's fontations project). Less than 25% slower than HarfBuzz on common fonts.

For new projects, harfrust tracks HarfBuzz more closely (3 major versions ahead of rustybuzz). rustybuzz has more production field time — it's what cosmic-text ships.

Known limitations of both: Arabic rendering works for common patterns, but unusual ligature constructions that require building lookup rules at runtime aren't supported. Some variable font interpolation features (avar2) are also not implemented.

Rasterization: `swash` or `fontdue`

swash handles both glyph outline extraction and rasterization. Supports ligatures and color emoji (CBDT, COLRv1). Used in cosmic-text.

fontdue focuses on rasterization only — no shaping, no color emoji. Simpler API and lighter weight if you're only rasterizing pre-shaped glyphs for basic Latin text.

Layout: `cosmic-text`

cosmic-text assembles the full stack: fontdb for discovery, rustybuzz for shaping, swash for rasterization, and its own layout engine for line breaking and BiDi text. Version 0.14.2 was released April 2025.

If you're building a GUI, a terminal emulator, or anything that needs correct multi-line Unicode text, this is the fastest path to a working stack. It's used in COSMIC (the Pop!_OS desktop environment), Iced, and Floem.

When the stack breaks

Complex scripts. rustybuzz handles most scripts well. Arabic works for common patterns but fails on ligature constructions that require building font lookup rules on the fly. Indic scripts (Devanagari, Bengali, Tamil) handle standard shaping but haven't had the production volume that C HarfBuzz has accumulated over 15+ years.

Font fallback. If a character isn't covered by your selected font, neither fontdb nor cosmic-text handles the fallback automatically. cosmic-text's FontSystem gives you font-matching utilities and codepoint coverage queries, but you write the chain logic.

Variable fonts. Basic variation instance selection works. Complex interpolation between instances (avar2) is not yet fully supported.

Practical summary

Goal	Crate(s)
Full text rendering pipeline	`cosmic-text`
Shaping only, modern (v13)	`harfrust`
Shaping only, battle-tested	`rustybuzz`
Rasterization with color emoji	`swash`
Lightweight rasterization (Latin only)	`fontdue`
Font metadata / discovery	`fontdb`

Checklist before shipping:

[ ] cargo tree | grep -- -sys — verify no C dependency slipped in
[ ] Test on all target platforms (font path scanning differs between Linux, macOS, Windows)
[ ] Check your target script against the known limitations for rustybuzz or harfrust
[ ] If targeting WASM, fontdb won't find system fonts — bundle font bytes directly

If you're doing font subsetting rather than rendering — extracting just the glyphs you need to embed in a PDF or WASM module — the relevant crates are different. But knowing that the shaping layer reads GSUB and GPOS tables directly informs which tables you need to keep when subsetting. I've been building that tooling in harumi.

Questions welcome in the comments.

Why Pure Rust WASM Is Harder Than It Looks

kent-tokyo — Thu, 21 May 2026 11:24:28 +0000

Most Rust developers expect the pitch to hold: compile to WebAssembly, ship to the browser, no C required. Add the wasm32-unknown-unknown target, run wasm-pack build, and you're done.

Then you try it on a real project. Four traps, none of them in the getting-started docs: a deprecated allocator with a memory leak, hidden C dependencies scattered through the ecosystem, a random number crate that silently misbehaves, and bundle sizes that balloon from Emscripten glue.

The Problem with "Just Use Rust"

The core appeal of Rust for WASM is real: no garbage collector, predictable memory, and — in theory — no need for the C/C++ toolchain that browser WASM has historically required.

But the Rust crate ecosystem has C dependencies scattered throughout it, often invisible until compile time. A project that builds fine on Linux will fail on wasm32-unknown-unknown because three levels down in the dependency tree, something links to libz, openssl, or a system RNG.

When I built chem-wasm-lens — a molecular analysis library for the browser — I needed to stay completely C-free. The alternative, RDKit (the standard C++ cheminformatics toolkit), compiles to a ~40MB WASM bundle via Emscripten. My target was under 200KB gzipped. Getting there meant systematically hunting down every C dependency.

The Allocator Story

Every WASM tutorial from a few years ago recommended wee_alloc: a tiny allocator that shrank bundle size and was the default in wasm-pack new.

wee_alloc was archived in August 2025. It has a known memory leak and is no longer maintained. If you have it in an existing project, remove it:

# Cargo.toml — remove this line
wee_alloc = "0.4"

// lib.rs — remove these lines
#[global_allocator]
static ALLOC: wee_alloc::WeeAlloc = wee_alloc::WeeAlloc::INIT;

The current state of WASM allocators:

Default (dlmalloc): Rust's wasm32-unknown-unknown target uses dlmalloc as the default global allocator. It's a pure Rust implementation — no C, no system calls. For most browser WASM projects, this is fine.
lol_alloc: Written as a wee_alloc replacement. Smaller than dlmalloc, but the author documents it as not production-ready.
talc: A newer allocator, benchmarked as smaller and faster than dlmalloc. Worth watching for size-critical projects.

For most projects: just use the default and move on. The allocator is no longer the interesting problem. The interesting problem is transitive C deps.

The Hidden C Dependency Tree

The most common wasm-pack build failure isn't your code — it's a crate three levels deep that silently links to C. The build error typically looks like:

error: failed to run custom build command for `openssl-sys v0.9.x`
  ...
  Could not find directory of OpenSSL installation.

Or for ring:

error: failed to run custom build command for `ring v0.17.x`
  ...
  the target `wasm32-unknown-unknown` is not supported

-sys crates are the signal. Any crate with a -sys suffix wraps a native library:

openssl-sys — OpenSSL
libz-sys — zlib
bzip2-sys — libbz2
libsqlite3-sys — SQLite
ring — cryptography (C code for performance-critical paths)

You probably didn't add any of these directly. reqwest pulls in openssl-sys by default. flate2 uses libz-sys by default. sqlx brings in libsqlite3-sys.

How to detect them:

cargo tree --target wasm32-unknown-unknown | grep "\-sys"

Example output from a project using reqwest and flate2:

├── flate2 v1.0.x
│   └── libz-sys v1.1.x (*)       ← C dependency
├── reqwest v0.12.x
│   └── openssl-sys v0.9.x (*)    ← C dependency

Any -sys crate in the output is a potential blocker.

Common replacements:

C-dependent crate	Pure Rust alternative
`openssl` / `openssl-sys`	`rustls`
`flate2` (default features)	`flate2` with `default-features = false, features = ["miniz"]`
`image` (old versions)	`image` 0.25+ (mostly pure Rust)
`reqwest` with system TLS	`reqwest` with `default-features = false, features = ["rustls-tls"]`

Here's what a typical fix looks like in Cargo.toml:

[dependencies]
# Before (pulls in openssl-sys):
# reqwest = "0.12"

# After (pure Rust TLS):
reqwest = { version = "0.12", default-features = false, features = ["rustls-tls", "json"] }

# Before (uses libz-sys by default):
# flate2 = "1.0"

# After (pure Rust miniz backend):
flate2 = { version = "1.0", default-features = false, features = ["miniz"] }

The ring situation deserves a specific note. ring powers most TLS stacks in the Rust ecosystem, and it contains C code for performance-critical paths. On wasm32-unknown-unknown, it fails to compile.

For browser WASM, this is usually not a problem — the browser's fetch API handles TLS transparently, so Rust code never touches it. For non-browser WASM runtimes (WASI, Wasmtime), look at RustCrypto crates, which provide pure Rust cryptographic primitives, or aws-lc-rs, which has better WASM support than ring.

The `getrandom` Footgun

Many crates need random numbers — UUID generation, HashMap initialization, cryptographic primitives. They all end up depending on getrandom.

On wasm32-unknown-unknown, getrandom doesn't know where it is. The target name alone says nothing about whether you're in a browser, a WASI runtime, or a bare-metal environment. Without explicit configuration, the build fails or panics at runtime, depending on the version.

First, find which version is in your tree:

cargo tree | grep getrandom

The fix depends on the version:

getrandom 0.2.x — add the js feature:

# Cargo.toml
[dependencies]
getrandom = { version = "0.2", features = ["js"] }

getrandom 0.3.x — feature flag plus a backend declaration:

In 0.3.x, getrandom introduced a "backend" model that separates feature flags from backend selection. Both are required:

# Cargo.toml
[dependencies]
getrandom = { version = "0.3", features = ["wasm_js"] }

# .cargo/config.toml
[target.wasm32-unknown-unknown]
rustflags = ["--cfg", "getrandom_backend=\"wasm_js\""]

The transitive dependency trap. You can't set feature flags for indirect dependencies directly. If uuid depends on getrandom and you don't use getrandom yourself, you still need to declare it explicitly so Cargo's feature unification propagates the flag:

[dependencies]
uuid = { version = "1", features = ["v4"] }
# Explicitly declare getrandom to force the js feature through the tree
getrandom = { version = "0.2", features = ["js"] }

The getrandom docs also recommend against enabling js/wasm_js in libraries — it breaks non-browser WASM builds. For a browser-only library like chem-wasm-lens, enabling it unconditionally is the right call, but the documentation buries this distinction.

Bundle Size

This is where C dependencies have their most visible cost.

C libraries compiled to WASM via Emscripten carry significant overhead: libc, libc++, a malloc implementation, and Emscripten runtime glue — regardless of how much of it you actually use. Tree shaking doesn't cross the FFI boundary. The result:

RDKit.js: ~40MB (Emscripten-compiled C++ cheminformatics)
OpenSSL compiled to WASM: ~1–2MB just for the crypto primitives

Pure Rust compiles lean. chem-wasm-lens ships at 411KB uncompressed, ~200KB gzipped — and that includes SMILES parsing, 2D coordinate generation, SVG rendering, ECFP4 fingerprint similarity, and PDB parsing. The size difference isn't magic; it's the absence of Emscripten glue, Rust's dead code elimination working cleanly across #[wasm_bindgen] exports, and wasm-opt running automatically on release builds.

Check your own binary size after building:

wasm-pack build --release
ls -lh pkg/*.wasm

If the .wasm is unexpectedly large, profile it with Twiggy to see what's taking up space:

cargo install twiggy
twiggy top pkg/your_crate_bg.wasm

The Current Ecosystem State

Where things stand, mid-2025:

Task	Pure Rust option	Status
Memory allocation	`dlmalloc` (default)	Solid
Compression	`flate2` with `default-features = false, features = ["miniz"]`	Solid
Serialization	`serde` + `serde-wasm-bindgen`	Solid
HTTP (browser)	`gloo-net` or JS `fetch` via `web-sys`	Solid
Random numbers	`getrandom` with `js`/`wasm_js` feature	Works, requires explicit config
Cryptographic primitives	RustCrypto crates	Solid for most algorithms
TLS (non-browser WASM)	—	Gap — `ring` doesn't build cleanly; `aws-lc-rs` has better WASM support
Image processing	`image` 0.25+	Mostly pure Rust; some format decoders still use C
Date/time	`web-time` or `js-sys::Date`	Solid

Practical Checklist

Before shipping a Rust WASM project:

Audit with cargo tree

   cargo tree --target wasm32-unknown-unknown | grep "\-sys"

Any -sys crate is a potential blocker.

Remove wee_alloc — archived in August 2025, has a memory leak. The default dlmalloc is fine.
Handle getrandom explicitly — run cargo tree | grep getrandom to find the version, then add it explicitly to [dependencies] with the right feature flag, even if you don't use it directly.
Replace system TLS — swap openssl/openssl-sys for rustls, or use reqwest with default-features = false, features = ["rustls-tls"].
Fix flate2 — use default-features = false, features = ["miniz"] to disable the libz-sys backend.
Build with --release — wasm-pack build --release runs wasm-opt automatically. Debug builds skip optimization and are typically 3–5x larger.

Closing Thought

The Rust-to-WASM path is genuinely good, and it's improving. The C ABI for wasm32-unknown-unknown is being standardized (Rust blog, April 2025), wee_alloc's archival simplified the allocator story, and wasm-pack keeps getting better.

Most popular Rust crates predate WASM as a real target, and they picked up C dependencies along the way. The audit is tedious but a one-time fix — once you know which -sys crates to watch for and what replaces them, it doesn't keep coming back.

I hit all four of these problems building chem-wasm-lens, a pure Rust + WASM molecular analysis library for the browser. The table above is what I found trying to go from a ~40MB Emscripten-compiled baseline down to ~200KB gzipped.

Why CJK Support in Rust Is Hard

kent-tokyo — Wed, 20 May 2026 11:30:23 +0000

Most Rust developers don't think about CJK until they need it. Then they discover that embedding Japanese text in a PDF, building a search index over Chinese content, or normalizing Korean input involves a stack of interlocking problems that Latin-script tooling simply never had to solve.

This post breaks down why CJK is genuinely hard — not just "different" — and where the Rust ecosystem still has gaps.

The Scale Problem

The first thing that surprises developers: a full CJK font file is enormous.

A Latin font like Inter Regular is around 300 KB. A full Japanese font — say, Noto Sans CJK JP — is over 15 MB. That's because Unicode's CJK Unified Ideographs block alone defines over 92,000 characters, and a production font needs to cover most of them.

For most use cases you don't need all 92,000 glyphs. If you're generating a PDF invoice with a customer name and address, you might use 50 distinct CJK characters. But a naive approach embeds the entire font, making a simple document balloon to 15 MB.

The solution is font subsetting: extract only the glyphs actually used, rebuild a minimal font binary, and embed that. It sounds straightforward. It isn't.

The Three Hard Problems

1. Font Subsetting for CJK

Subsetting a Latin font is well-understood. Subsetting a CJK font involves:

Glyph ID remapping. A font maps Unicode code points to internal Glyph IDs (GIDs). After subsetting, the GID space is compacted — the 50 glyphs you kept now have new GIDs from 0 to 49. Every reference to the old GIDs in the font binary and in your document needs to be updated.

CMap table reconstruction. The font's cmap table maps Unicode → GID. After subsetting, this table must be rebuilt to reflect the new GID assignments. Get this wrong and the font renders garbage or fails to load entirely.

Advance width recalculation. Fonts store per-glyph advance widths (how far the cursor moves after each character). After GID remapping, the width table must be reindexed. In PDF specifically, the /Widths array in the CIDFont object must match the new GIDs exactly — a mismatch causes text spacing to break in subtle, hard-to-debug ways.

Type0/CIDFont object graph. PDF represents CJK fonts as a two-level structure: a Type0 (composite) font wrapping a CIDFont. The CIDFont references the embedded font stream and the ToUnicode CMap. Building this object graph correctly requires understanding the PDF spec at a level most developers would rather avoid.

In pure Rust, the allsorts crate handles TTF subsetting. It works well for TrueType fonts. OpenType CFF fonts (.otf files with PostScript outlines) are more complex and allsorts coverage is incomplete — this is a known gap in the Rust ecosystem.

2. ToUnicode CMap Generation

PDF separates rendering (which glyph to draw) from semantics (what Unicode character that glyph represents). Rendering uses GIDs. Semantics are stored in a separate stream called the ToUnicode CMap.

Without a ToUnicode CMap:

Copy-pasting text from a PDF produces garbage characters or empty strings
Search within the PDF doesn't find CJK text
Screen readers can't read the document

The CMap is a PostScript-like stream that maps GID ranges to Unicode code points. For CJK fonts with thousands of glyphs, generating this stream correctly — with proper range compression for consecutive code points — requires care. A naive one-entry-per-glyph approach technically works but produces unnecessarily large streams.

3. Normalization and Variant Characters

CJK text has an encoding problem that Latin scripts largely don't: the same logical character can have multiple valid representations.

Unicode normalization forms (NFC, NFD, NFKC, NFKD) affect how composed characters are stored. Japanese text in particular mixes hiragana, katakana, kanji, and Latin characters, each with their own normalization quirks. Fullwidth ASCII (Ａ, Ｂ, Ｃ) and halfwidth katakana (ｱ, ｲ, ｳ) are canonically equivalent to their standard forms under NFKC but not NFC.

CJK Compatibility Ideographs (U+F900–U+FAFF) are compatibility mappings for characters that appear in legacy encodings. U+FA30 (㌍) is canonically equivalent to U+30AD U+30ED (キロ). Depending on whether you normalize before indexing, the same string might or might not match a query.

Variant selectors add another layer. CJK Unified Ideographs sometimes have multiple visual forms (simplified vs. traditional Chinese, Japanese vs. Korean glyph shapes). Unicode encodes this with Variation Selectors — invisible code points that follow a base character to select a specific glyph. 葛 followed by VS17 (U+E0100) selects a specific variant used in place names. A text search that isn't VS-aware will fail to match these strings.

For fuzzy matching over CJK content, you need to decide which of these equivalences to collapse before indexing. The right answer depends on the use case: a legal document system probably wants exact glyph matching; a general search index probably wants NFKC normalization.

The Legacy Encoding Problem

Modern CJK text is Unicode, but a significant amount of real-world content is still encoded in legacy formats:

Encoding	Used for
Shift-JIS	Legacy Japanese text, older Windows software
EUC-JP	Unix-era Japanese
GB2312 / GBK / GB18030	Simplified Chinese
Big5	Traditional Chinese (Taiwan/Hong Kong)
EUC-KR / CP949	Korean

Converting these to Unicode isn't just a lookup table — legacy CJK encodings have overlapping code spaces, vendor extensions, and edge cases that differ between Windows, macOS, and Linux implementations.

The encoding_rs crate (originally written for Firefox) is the authoritative pure Rust implementation of the WHATWG Encoding Standard and handles most of these correctly. This is one area where the Rust ecosystem is actually in good shape.

Why Everything Still Leans on C

The elephant in the room: most production CJK text processing still depends on C or C++ libraries.

HarfBuzz — text shaping (converting Unicode to positioned glyphs) — is C++. For CJK, shaping is relatively simple compared to Arabic or Indic scripts (no complex ligatures or bidirectional reordering), but HarfBuzz is still the de facto standard.

FreeType — font rasterization — is C. If you're rendering CJK text to a bitmap, you're almost certainly using FreeType bindings.

ICU (International Components for Unicode) — normalization, collation, locale-aware string comparison — is C++. The icu4x project is a ground-up Rust rewrite led by the Unicode Consortium, and it's making solid progress, but it's not yet a drop-in replacement for all ICU use cases.

The consequence for Rust developers: if you need CJK support and reach for crates that wrap these C libraries, you give up WASM compatibility, you complicate cross-compilation, and you add a build-time dependency on the system libraries or vendored C sources.

The Current State of Pure Rust CJK

Here's an honest assessment of the pure Rust ecosystem for CJK work:

Task	Pure Rust option	Status
Legacy encoding conversion	`encoding_rs`	Solid
Unicode normalization	`unicode-normalization`	Solid
TTF font subsetting	`allsorts`	Good for TrueType; OTF/CFF partial
PDF CJK text embedding	`harumi`	New, niche
Text shaping	None	Gap — no pure Rust HarfBuzz equivalent yet
Font rasterization	`fontdue`	Works but limited CJK testing
Full Unicode collation	`icu4x`	In progress

The gaps are real. Text shaping in particular is a hard open problem for pure Rust — for simple CJK rendering you can get away without a full shaper, but for mixed CJK/Latin text with proper kerning and ligatures, you eventually need something HarfBuzz-level.

What This Means in Practice

If you're building something that needs to handle CJK text in Rust:

For encoding conversion, use encoding_rs. Don't roll your own.
For normalization, use unicode-normalization and decide up front which form you want. For search, NFKC is usually the right default.
For PDF with CJK, the pure Rust path exists but requires understanding the subsetting pipeline. Wrapping pdfium or a C-based library is currently the easier path if WASM compatibility isn't a requirement.
For fuzzy search over CJK, normalization before indexing matters more than the search algorithm itself.
For text rendering, if you can accept a C dependency, HarfBuzz + FreeType is the proven path. Pure Rust rendering is possible for simple cases.

Closing Thought

CJK support isn't a single feature — it's a stack of problems that compound. The good news is that the Rust ecosystem is making real progress on each layer. The bad news is that each layer requires understanding the layer below it, which is why CJK support tends to be either "works perfectly" or "completely broken" with little middle ground.

If you're working on any of these problems — subsetting, normalization, collation, shaping — I'd love to compare notes.

I've run into most of these problems while building harumi, a pure Rust PDF library with CJK font subsetting. The gaps in the table above are the ones I've personally hit.

I Built a Next-Gen DNS Diagnostic CLI in Rust That Visualizes DNSSEC Trust Chains

kent-tokyo — Wed, 20 May 2026 09:32:59 +0000

If you've ever tried to debug a DNSSEC misconfiguration using dig, you know the pain. You're staring at a wall of raw text, manually cross-referencing DS records against DNSKEY records, tracing through TLD delegations one query at a time. It works — but it's exhausting.

I wanted something better. So I built shohei — a Rust-powered DNS diagnostic CLI that makes the invisible visible.

github.com/kent-tokyo/shohei

What Is shohei?

shohei is a next-generation DNS diagnostic tool that goes well beyond what dig or drill offer. It renders DNS resolution as color-coded terminal trees, walks you through DNSSEC trust chains step by step, and supports modern transports like DoH and DoT — all from a single command.

Think of it as dig if dig was designed for humans first.

The Three Hard Problems shohei Solves

Building a DNS diagnostic tool sounds straightforward until you try to make it actually useful. Here are the problems that motivated most of the design.

1. DNSSEC Trust Chains Are Invisible in Standard Tools

DNSSEC forms a cryptographic chain from the root zone down to individual records. The chain looks like this:

Root KSK (trust anchor, hardcoded in resolvers)
  └── Root ZSK (signs root zone records)
        └── DS record for .com  ← hash of .com's KSK
              └── .com KSK
                    └── .com ZSK (signs .com zone)
                          └── DS record for example.com
                                └── example.com KSK
                                      └── example.com ZSK
                                            └── RRSIG on A record ← what you care about

When something breaks in this chain — a DS mismatch, an expired RRSIG, a missing DNSKEY — dig gives you raw records and leaves you to piece together the failure yourself. shohei walks the whole chain and tells you exactly where validation fails and why.

. (Root)
└── com.  [DS 30909 → DNSKEY 30909 ✓]
    └── example.com.  [DS 12345 → DNSKEY 12345 ✓]
        └── www.example.com.  A 93.184.216.34
            └── RRSIG (ZSK tag 12345) expires 2026-06-01 ✓

A broken chain looks like:

. (Root)
└── com.  [DS 30909 → DNSKEY 30909 ✓]
    └── example.com.  [DS 99999 → DNSKEY ✗ NO MATCHING KEY FOUND]
        └── ⚠ Validation failed: DS/DNSKEY mismatch at example.com.

2. Iterative Resolution Is a Black Box

Standard resolvers are recursive — they do the work for you and return an answer. That's convenient, but useless when you're debugging a delegation problem, a glue record mismatch, or propagation that hasn't reached a specific nameserver yet.

shohei's --trace mode sends queries the way a resolver actually would — manually, hop by hop:

[1] Query root servers for example.com. A
    → a.root-servers.net. (198.41.0.4)  4ms
    ← REFERRAL: com. NS [a.gtld-servers.net., ...]

[2] Query TLD servers for example.com. A
    → a.gtld-servers.net. (192.5.6.30)  11ms
    ← REFERRAL: example.com. NS [ns1.example.com., ns2.example.com.]
       GLUE: ns1.example.com. A 205.251.196.1

[3] Query authoritative servers for example.com. A
    → ns1.example.com. (205.251.196.1)  8ms
    ← ANSWER: www.example.com. A 93.184.216.34  TTL 3600

Each hop shows which server was queried, its IP, round-trip time, and exactly what was returned. Delegation loops, missing glue records, and lame delegations all become immediately visible.

3. Resolver Disagreements Are Hard to Diagnose

Split-horizon DNS, stale caches, and regional differences mean two resolvers can return completely different answers for the same query. Figuring out why usually involves running dig @8.8.8.8 and dig @1.1.1.1 separately, then comparing the output by hand.

shohei's --compare mode queries both resolvers concurrently and shows a unified diff:

Comparing 8.8.8.8 vs 1.1.1.1 for example.com A

  ANSWER
  8.8.8.8:   93.184.216.34  TTL 3521
  1.1.1.1:   93.184.216.34  TTL 120

  AUTHORITY
+ 8.8.8.8:   example.com. NS ns1.example.com.  TTL 172800
- 1.1.1.1:   (none)

  FLAGS
  8.8.8.8:   QR AA TC RD RA AD  ← AD set (DNSSEC validated)
  1.1.1.1:   QR AA TC RD RA     ← AD not set

The TTL difference here tells you 1.1.1.1 has a nearly-fresh cache entry while 8.8.8.8 is serving a stale one. The missing AD flag on 1.1.1.1 is immediately visible.

Feature Overview

DNSSEC Chain Visualization

Full trust chain from root to record, color-coded by validation status. Green for valid, red for failures, yellow for warnings (e.g., signature expiring within 7 days).

Iterative Resolution Tracing (`--trace`)

Manual hop-by-hop resolution with timing, referral details, and glue record display. Shows exactly what a recursive resolver would do.

DoH and DoT Support

Modern DNS transports with no external dependencies:

shohei example.com --doh https://cloudflare-dns.com/dns-query
shohei example.com --dot 1.1.1.1

Useful for testing whether a domain resolves correctly over encrypted transports, or for bypassing a local resolver that might be interfering.

Resolver Comparison (`--compare`)

Side-by-side diff of two resolvers. Flags TTL differences, missing records, DNSSEC validation status mismatches, and flag differences:

shohei example.com --compare 8.8.8.8 1.1.1.1

Multi-Type Queries

Query multiple record types in a single command instead of running dig four times:

shohei example.com A AAAA MX TXT SOA

Output is grouped by type with consistent formatting.

Watch Mode (`--watch`)

Auto-refresh at a set interval. Useful for monitoring TTL countdown, propagation in real time, or catching flapping records:

shohei example.com --watch 5  # refresh every 5 seconds

The display updates in-place, so changes are immediately visible without scrolling.

Interactive TUI (`--tui`)

Built with ratatui, the TUI gives you three navigable panels in a single terminal window:

Records panel — all RRsets for the queried domain, sorted by type
DNSSEC panel — the full trust chain with validation status
Trace panel — iterative resolution hops with timing

Switch between panels with Tab, scroll with arrow keys, and exit with q. Useful when you need to explore a domain's DNS health interactively rather than in a single pass.

Reverse DNS (`--ptr`)

PTR lookups for IPv4 and IPv6, with automatic in-addr.arpa / ip6.arpa formatting:

shohei --ptr 8.8.8.8
shohei --ptr 2001:4860:4860::8888

Batch Mode & Script-Friendly Output

Pipe a list of domains for bulk querying, with JSON output for downstream processing:

cat domains.txt | shohei --batch --json | jq '.[] | select(.dnssec_valid == false)'

The --minimal flag strips color and tree formatting for plain-text pipelines.

Why Rust?

Three reasons:

Performance. DNS diagnostics involve parallel resolution paths — querying multiple nameservers concurrently, running resolver comparisons simultaneously, tracing iterative hops while fetching DNSSEC records in parallel. Rust's async ecosystem via tokio handles this efficiently without GC pauses that would skew timing measurements.

Correctness. DNSSEC validation is stateful and involves cryptographic operations across multiple record types. Getting it wrong silently is worse than getting it wrong loudly. Rust's type system lets you encode validation state as types — a validated chain is a different type than an unvalidated one, and the compiler enforces that distinction.

hickory-dns. The hickory-dns crate (formerly trust-dns) is a mature pure-Rust DNS implementation with first-class DNSSEC, DoH, and DoT support. It handles wire-format parsing, cryptographic validation, and transport negotiation — the parts that are easy to get subtly wrong — so I could focus on the UX layer.

Core Dependencies

Crate	Purpose
`hickory-dns`	DNS resolution, DNSSEC validation, DoH/DoT transport
`clap`	CLI argument parsing with derive macros
`ratatui`	Interactive TUI framework
`owo-colors`	Terminal colorization without ANSI escape string building
`comfy-table`	Table rendering with Unicode box-drawing
`tokio`	Async runtime for concurrent resolver queries
`serde_json`	JSON output serialization

Installation

cargo install shohei

Or build from source:

git clone https://github.com/kent-tokyo/shohei
cd shohei
cargo build --release
./target/release/shohei example.com

The TUI is included by default. If you want a smaller binary without ratatui, build with --no-default-features.

Quick Reference

# Basic query — records + DNSSEC chain
shohei example.com

# Multiple record types in one shot
shohei example.com A AAAA MX TXT SOA

# Iterative resolution trace (hop by hop)
shohei example.com --trace

# DNSSEC chain only
shohei example.com --dnssec

# Compare two resolvers
shohei example.com --compare 8.8.8.8 1.1.1.1

# Query over DoH
shohei example.com --doh https://cloudflare-dns.com/dns-query

# Query over DoT
shohei example.com --dot 1.1.1.1

# Watch mode (refresh every 5s)
shohei example.com --watch 5

# Reverse DNS
shohei --ptr 8.8.8.8

# Interactive TUI
shohei example.com --tui

# Batch mode with JSON output
cat domains.txt | shohei --batch --json

# Minimal output for scripting
shohei example.com --minimal

What's Next

DANE/TLSA validation — cross-reference TLS certificate fingerprints against DNS to verify DANE configurations
NSEC/NSEC3 zone walking — for security research and CTF scenarios
HTML report export — a shareable snapshot of a domain's full DNS health, including DNSSEC chain and trace output
CAA record checking — verify Certification Authority Authorization records alongside DNSSEC

Feedback Welcome

shohei is MIT licensed and open to contributions. If you work with DNS professionally — or just find yourself reaching for dig more than you'd like — I'd love to hear what features would be most useful.

github.com/kent-tokyo/shohei

Built with Rust, a healthy obsession with DNSSEC, and too many late nights staring at zone files.

I Built "harumi" — A Pure Rust PDF Editing Library with CJK Support

kent-tokyo — Mon, 18 May 2026 09:58:14 +0000

Overview

harumi is a Pure Rust library that lets you dynamically add CJK text (Japanese, Chinese, Korean) to existing PDFs. Unlike bindings-based solutions, it has zero C dependencies and handles font subsetting automatically.

crates.io: https://crates.io/crates/harumi
GitHub: https://github.com/kent-tokyo/harumi

Why Another Rust PDF Crate?

The existing Rust PDF ecosystem leaves a gap:

Crate	Limitation
`lopdf`	Low-level; no font subsetting or CMap generation
`printpdf`	Create-only; can't edit existing PDFs
`pdfium-render`	Requires linking against the C-based PDFium library

harumi fills that gap: append-only editing of existing PDFs, Pure Rust, with automatic CJK font subsetting and ToUnicode CMap generation built in.

The Three Hard Problems of CJK in PDF

Getting Japanese (and CJK in general) right inside a PDF isn't just about "embedding a font." There are three distinct challenges:

1. Font Subsetting

A full Japanese font file can easily exceed 10 MB. For practical file sizes you must extract only the glyphs actually used and rebuild the font binary — this is subsetting. harumi does this automatically at save time.

2. ToUnicode CMap Generation

PDFs separate rendering (Glyph IDs) from semantics (Unicode code points). Without a ToUnicode CMap, copy-paste and text search produce garbled output. harumi generates this mapping for every font it embeds.

3. Glyph Advance Width Recalculation

After subsetting, Glyph IDs are reassigned. The advance widths stored in the PDF must be recalculated to match — otherwise text spacing breaks. harumi handles this as part of the save pipeline.

Lazy Subsetting Pipeline

harumi uses a lazy subsetting design to handle all three problems in one pass:

embed_font() — store raw font bytes; no processing yet
Collect all text draw calls across all pages
Walk every page at save() time, gathering the complete set of used characters
Subset the font to only those glyphs
Reassign Glyph IDs
Build the ToUnicode CMap
Recalculate advance widths and write the final CIDFont object

This single-pass approach avoids redundant font processing and keeps the implementation straightforward.

Feature Overview

use harumi::Document;

let mut doc = Document::open("input.pdf")?;

// Append text (including invisible text for search layers)
doc.page(0).add_text("Hello, 世界！", font, 12.0, x, y)?;

// Draw shapes and embed images
doc.page(0).draw_rect(x, y, width, height, color)?;
doc.page(0).embed_image(image_bytes, x, y, width, height)?;

// Page operations
doc.rotate_page(1, 90)?;
doc.delete_page(2)?;
doc.reorder_pages(&[2, 0, 1])?;

// Merge and split
let other = Document::open("other.pdf")?;
doc.merge(other)?;
let parts = doc.split_at(&[3])?;

// Extract text
let text = doc.extract_text(0)?;

// Metadata
doc.set_title("My Document")?;

doc.save("output.pdf")?;

Current Status & Roadmap

harumi is published on crates.io and the source is available on GitHub.

Planned improvements:

Broader CJK font format support
Form field editing
Performance optimizations for large documents

Feedback, issues, and contributions are very welcome!

Forem: kent-tokyo

Cheminformatics in Rust in 2025-2026: What Exists, What Doesn't, and Why

What exists in 2025-2026

SMILES parsing is solved. The rest is not.

Aromaticity: no agreed definition

2D coordinate generation: not attempted

Substructure search: the graph is not the chemistry

Why bindings are the rational choice

What changed in 2024-2025, and what 2026 adds so far

The actual hard part

Rust in 2025-2026: From 'Most Loved Language' to Core Infrastructure

"Experimental" Status Removed from the Linux Kernel

Rust Code Running in Production

Why Rust is Well-Suited for the Kernel

The Technical Story Behind async closures

What was wrong with the old workarounds?

The AsyncFn Trait Hierarchy

let chains — Quietly Important

Safety Certification: A New Frontier

What is IEC 61508?

MISRA C 2025 Addendum 6

Production Adoption Approaching 50%

An Unexpected Fit for the LLM Era

The Biggest Concern: Adoption May Plateau

Summary

Open-source SDS tooling for Japanese MHLW compliance: the gap nobody filled

What makes Japan's SDS requirements different

How GHS looks different by country

Japan-specific regulatory fields

Why this matters now: the 2022 law revision

Where existing tools stop

Commercial SDS platforms

Open-source options

The format inconsistency problem

The schema itself

Section 3 (composition) is the hardest part

Typos locked into v1.0

sds-converter

The open gap

sds-converter: Converting Safety Data Sheets to MHLW Standard JSON with Rust and LLMs

Background

Why SDS documents are hard to parse

The 16 sections

Installation and quick start

How the conversion works

Step 1: Text extraction

Step 2: Parallel LLM extraction

Step 3: JSON output

LLM backend and quality settings

Choosing a provider

Quality preset

Batch mode

Validation

Output JSON structure

Using as a library

PDF → JSON

JSON → Word document

Custom LLM backend

Language support

Comparison with alternatives

Open-source

Commercial (Japan)

Crate structure

Why Font Rendering in Rust Is Harder Than It Looks

The four layers

What text shaping actually does

The naming trap

Pure Rust, layer by layer

Font discovery: fontdb

Text shaping: rustybuzz or harfrust

Rasterization: swash or fontdue

Layout: cosmic-text

When the stack breaks

Practical summary

Why Pure Rust WASM Is Harder Than It Looks

The Problem with "Just Use Rust"

The Allocator Story

The Hidden C Dependency Tree

The getrandom Footgun

Bundle Size

The Technical Story Behind `async closures`

The `AsyncFn` Trait Hierarchy

`let chains` — Quietly Important

Font discovery: `fontdb`

Text shaping: `rustybuzz` or `harfrust`

Rasterization: `swash` or `fontdue`

Layout: `cosmic-text`

The `getrandom` Footgun

Iterative Resolution Tracing (`--trace`)

Resolver Comparison (`--compare`)

Watch Mode (`--watch`)

Interactive TUI (`--tui`)

Reverse DNS (`--ptr`)