Forem: Na'aman Hirschfeld (Goldziher)

Kreuzberg v4.0.0-RC.8 is Available

Na'aman Hirschfeld (Goldziher) — Mon, 15 Dec 2025 13:06:14 +0000

Hi Peeps,

I'm excited to announce that Kreuzberg v4.0.0 is coming very soon. We will release v4.0.0 at the beginning of next year - in just a couple of weeks time. For now, v4.0.0-rc.8 has been released to all channels.

What is Kreuzberg?

Kreuzberg is a document intelligence toolkit for extracting text, metadata, tables, images, and structured data from 56+ file formats. It was originally written in Python (v1-v3), where it demonstrated strong performance characteristics compared to alternatives in the ecosystem.

What's new in V4?

A Complete Rust Rewrite with Polyglot Bindings

The new version of Kreuzberg represents a massive architectural evolution. Kreuzberg has been completely rewritten in Rust - leveraging Rust's memory safety, zero-cost abstractions, and native performance. The new architecture consists of a high-performance Rust core with native bindings to multiple languages. That's right - it's no longer just a Python library.

Kreuzberg v4 is now available for 7 languages across 8 runtime bindings:

Rust (native library)
Python (PyO3 native bindings)
TypeScript - Node.js (NAPI-RS native bindings) + Deno/Browser/Edge (WASM)
Ruby (Magnus FFI)
Java 25+ (Panama Foreign Function & Memory API)
C# (P/Invoke)
Go (cgo bindings)

Post v4.0.0 roadmap includes:

PHP
Elixir (via Rustler - with Erlang and Gleam interop)

Additionally, it's available as a CLI (installable via cargo or homebrew), HTTP REST API server, Model Context Protocol (MCP) server for Claude Desktop/Continue.dev, and as public Docker images.

Why the Rust Rewrite? Performance and Architecture

The Rust rewrite wasn't just about performance - though that's a major benefit. It was an opportunity to fundamentally rethink the architecture:

Architectural improvements:

Zero-copy operations via Rust's ownership model
True async concurrency with Tokio runtime (no GIL limitations)
Streaming parsers for constant memory usage on multi-GB files
SIMD-accelerated text processing for token reduction and string operations
Memory-safe FFI boundaries for all language bindings
Plugin system with trait-based extensibility

v3 vs v4: What Changed?

Aspect	v3 (Python)	v4 (Rust Core)
Core Language	Pure Python	Rust 2024 edition
File Formats	30-40+ (via Pandoc)	56+ (native parsers)
Language Support	Python only	7 languages (Rust/Python/TS/Ruby/Java/Go/C#)
Dependencies	Requires Pandoc (system binary)	Zero system dependencies (all native)
Embeddings	Not supported	✓ FastEmbed with ONNX (3 presets + custom)
Semantic Chunking	Via semantic-text-splitter library	✓ Built-in (text + markdown-aware)
Token Reduction	Built-in (TF-IDF based)	✓ Enhanced with 3 modes
Language Detection	Optional (fast-langdetect)	✓ Built-in (68 languages)
Keyword Extraction	Optional (KeyBERT)	✓ Built-in (YAKE + RAKE algorithms)
OCR Backends	Tesseract/EasyOCR/PaddleOCR	Same + better integration
Plugin System	Limited extractor registry	Full trait-based (4 plugin types)
Page Tracking	Character-based indices	Byte-based with O(1) lookup
Servers	REST API (Litestar)	HTTP (Axum) + MCP + MCP-SSE
Installation Size	~100MB base	16-31 MB complete
Memory Model	Python heap management	RAII with streaming
Concurrency	asyncio (GIL-limited)	Tokio work-stealing

Replacement of Pandoc - Native Performance

Kreuzberg v3 relied on Pandoc - an amazing tool, but one that had to be invoked via subprocess because of its GPL license. This had significant impacts:

v3 Pandoc limitations:

System dependency (installation required)
Subprocess overhead on every document
No streaming support
Limited metadata extraction
~500MB+ installation footprint

v4 native parsers:

Zero external dependencies - everything is native Rust
Direct parsing with full control over extraction
Substantially more metadata extracted (e.g., DOCX document properties, section structure, style information)
Streaming support for massive files (tested on multi-GB XML documents with stable memory)
Example: PPTX extractor is now a fully streaming parser capable of handling gigabyte-scale presentations with constant memory usage and high throughput

New File Format Support

v4 expanded format support from ~20 to 56+ file formats, including:

Added legacy format support:

.doc (Word 97-2003)
.ppt (PowerPoint 97-2003)
.xls (Excel 97-2003)
.eml (Email messages)
.msg (Outlook messages)

Added academic/technical formats:

LaTeX (.tex)
BibTeX (.bib)
Typst (.typ)
JATS XML (scientific articles)
DocBook XML
FictionBook (.fb2)
OPML (.opml)

Better Office support:

XLSB, XLSM (Excel binary/macro formats)
Better structured metadata extraction from DOCX/PPTX/XLSX
Full table extraction from presentations
Image extraction with deduplication

New Features: Full Document Intelligence Solution

The v4 rewrite was also an opportunity to close gaps with commercial alternatives and add features specifically designed for RAG applications and LLM workflows:

1. Embeddings (NEW)

FastEmbed integration with full ONNX Runtime acceleration
Three presets: "fast" (384d), "balanced" (512d), "quality" (768d/1024d)
Custom model support (bring your own ONNX model)
Local generation (no API calls, no rate limits)
Automatic model downloading and caching
Per-chunk embedding generation

from kreuzberg import ExtractionConfig, EmbeddingConfig, EmbeddingModelType

config = ExtractionConfig(
    embeddings=EmbeddingConfig(
        model=EmbeddingModelType.preset("balanced"),
        normalize=True
    )
)
result = kreuzberg.extract_bytes(pdf_bytes, config=config)
# result.embeddings contains vectors for each chunk

2. Semantic Text Chunking (NOW BUILT-IN)

Now integrated directly into the core (v3 used external semantic-text-splitter library):

Structure-aware chunking that respects document semantics
Two strategies:
- Generic text chunker (whitespace/punctuation-aware)
- Markdown chunker (preserves headings, lists, code blocks, tables)
Configurable chunk size and overlap
Unicode-safe (handles CJK, emojis correctly)
Automatic chunk-to-page mapping
Per-chunk metadata with byte offsets

3. Byte-Accurate Page Tracking (BREAKING CHANGE)

This is a critical improvement for LLM applications:

v3: Character-based indices (char_start/char_end) - incorrect for UTF-8 multi-byte characters
v4: Byte-based indices (byte_start/byte_end) - correct for all string operations

Additional page features:

O(1) lookup: "which page is byte offset X on?" → instant answer
Per-page content extraction
Page markers in combined text (e.g., --- Page 5 ---)
Automatic chunk-to-page mapping for citations

4. Enhanced Token Reduction for LLM Context

Enhanced from v3 with three configurable modes to save on LLM costs:

Light mode: ~15% reduction (preserve most detail)
Moderate mode: ~30% reduction (balanced)
Aggressive mode: ~50% reduction (key information only)

Uses TF-IDF sentence scoring with position-aware weighting and language-specific stopword filtering. SIMD-accelerated for improved performance over v3.

5. Language Detection (NOW BUILT-IN)

68 language support with confidence scoring
Multi-language detection (documents with mixed languages)
ISO 639-1 and ISO 639-3 code support
Configurable confidence thresholds

6. Keyword Extraction (NOW BUILT-IN)

Now built into core (previously optional KeyBERT in v3):

YAKE (Yet Another Keyword Extractor): Unsupervised, language-independent
RAKE (Rapid Automatic Keyword Extraction): Fast statistical method
Configurable n-grams (1-3 word phrases)
Relevance scoring with language-specific stopwords

7. Plugin System (NEW)

Four extensible plugin types for customization:

DocumentExtractor - Custom file format handlers
OcrBackend - Custom OCR engines (integrate your own Python models)
PostProcessor - Data transformation and enrichment
Validator - Pre-extraction validation

Plugins defined in Rust work across all language bindings. Python/TypeScript can define custom plugins with thread-safe callbacks into the Rust core.

8. Production-Ready Servers (NEW)

HTTP REST API: Production-grade Axum server with OpenAPI docs
MCP Server: Direct integration with Claude Desktop, Continue.dev, and other MCP clients
MCP-SSE Transport (RC.8): Server-Sent Events for cloud deployments without WebSocket support
All three modes support the same feature set: extraction, batch processing, caching

Performance: Benchmarked Against the Competition

We maintain continuous benchmarks comparing Kreuzberg against the leading OSS alternatives:

Benchmark Setup

Platform: Ubuntu 22.04 (GitHub Actions)
Test Suite: 30+ documents covering all formats
Metrics: Latency (p50, p95), throughput (MB/s), memory usage, success rate
Competitors: Apache Tika, Docling, Unstructured, MarkItDown

How Kreuzberg Compares

Installation Size (critical for containers/serverless):

Kreuzberg: 16-31 MB complete (CLI: 16 MB, Python wheel: 22 MB, Java JAR: 31 MB - all features included)
MarkItDown: ~251 MB installed (58.3 KB wheel, 25 dependencies)
Unstructured: ~146 MB minimal (open source base) - several GB with ML models
Docling: ~1 GB base, 9.74GB Docker image (includes PyTorch CUDA)
Apache Tika: ~55 MB (tika-app JAR) + dependencies
GROBID: 500MB (CRF-only) to 8GB (full deep learning)

Performance Characteristics:

Library	Speed	Accuracy	Formats	Installation	Use Case
Kreuzberg	⚡ Fast (Rust-native)	Excellent	56+	16-31 MB	General-purpose, production-ready
Docling	⚡ Fast (3.1s/pg x86, 1.27s/pg ARM)	Best	7+	1-9.74 GB	Complex documents, when accuracy > size
GROBID	⚡⚡ Very Fast (10.6 PDF/s)	Best	PDF only	0.5-8 GB	Academic/scientific papers only
Unstructured	⚡ Moderate	Good	25-65+	146 MB-several GB	Python-native LLM pipelines
MarkItDown	⚡ Fast (small files)	Good	11+	~251 MB	Lightweight Markdown conversion
Apache Tika	⚡ Moderate	Excellent	1000+	~55 MB	Enterprise, broadest format support

Kreuzberg's sweet spot:

Smallest full-featured installation: 16-31 MB complete (vs 146 MB-9.74 GB for competitors)
5-15x smaller than Unstructured/MarkItDown, 30-300x smaller than Docling/GROBID
Rust-native performance without ML model overhead
Broad format support (56+ formats) with native parsers
Multi-language support unique in the space (7 languages vs Python-only for most)
Production-ready with general-purpose design (vs specialized tools like GROBID)

Is Kreuzberg a SaaS Product?

No. Kreuzberg is and will remain MIT-licensed open source.

However, we are building Kreuzberg.cloud - a commercial SaaS and self-hosted document intelligence solution built on top of Kreuzberg. This follows the proven open-core model: the library stays free and open, while we offer a cloud service for teams that want managed infrastructure, APIs, and enterprise features.

Will Kreuzberg become commercially licensed? Absolutely not. There is no BSL (Business Source License) in Kreuzberg's future. The library was MIT-licensed and will remain MIT-licensed. We're building the commercial offering as a separate product around the core library, not by restricting the library itself.

Target Audience

Any developer or data scientist who needs:

Document text extraction (PDF, Office, images, email, archives, etc.)
OCR (Tesseract, EasyOCR, PaddleOCR)
Metadata extraction (authors, dates, properties, EXIF)
Table and image extraction
Document pre-processing for RAG pipelines
Text chunking with embeddings
Token reduction for LLM context windows
Multi-language document intelligence in production systems

Ideal for:

RAG application developers
Data engineers building document pipelines
ML engineers preprocessing training data
Enterprise developers handling document workflows
DevOps teams needing lightweight, performant extraction in containers/serverless

Comparison with Alternatives

Open Source Python Libraries

Unstructured.io

Strengths: Established, modular, broad format support (25+ open source, 65+ enterprise), LLM-focused, good Python ecosystem integration
Trade-offs: Python GIL performance constraints, 146 MB minimal installation (several GB with ML models)
License: Apache-2.0
When to choose: Python-only projects where ecosystem fit > performance

MarkItDown (Microsoft)

Strengths: Fast for small files, Markdown-optimized, simple API
Trade-offs: Limited format support (11 formats), less structured metadata, ~251 MB installed (despite small wheel), requires OpenAI API for images
License: MIT
When to choose: Markdown-only conversion, LLM consumption

Docling (IBM)

Strengths: Excellent accuracy on complex documents (97.9% cell-level accuracy on tested sustainability report tables), state-of-the-art AI models for technical documents
Trade-offs: Massive installation (1-9.74 GB), high memory usage, GPU-optimized (underutilized on CPU)
License: MIT
When to choose: Accuracy on complex documents > deployment size/speed, have GPU infrastructure

Open Source Java/Academic Tools

Apache Tika

Strengths: Mature, stable, broadest format support (1000+ types), proven at scale, Apache Foundation backing
Trade-offs: Java/JVM required, slower on large files, older architecture, complex dependency management
License: Apache-2.0
When to choose: Enterprise environments with JVM infrastructure, need for maximum format coverage

GROBID

Strengths: Best-in-class for academic papers (F1 0.87-0.90), extremely fast (10.6 PDF/sec sustained), proven at scale (34M+ documents at CORE)
Trade-offs: Academic papers only, large installation (500MB-8GB), complex Java+Python setup
License: Apache-2.0
When to choose: Scientific/academic document processing exclusively

Commercial APIs

There are numerous commercial options from startups (LlamaIndex, Unstructured.io paid tiers) to big cloud providers (AWS Textract, Azure Form Recognizer, Google Document AI). These are not OSS but offer managed infrastructure.

Kreuzberg's position: As an open-source library, Kreuzberg provides a self-hosted alternative with no per-document API costs, making it suitable for high-volume workloads where cost efficiency matters.

Community & Resources

GitHub: Star us at https://github.com/kreuzberg-dev/kreuzberg
Discord: Join our community server at discord.gg/pXxagNK2zN
Subreddit: Join the discussion at r/kreuzberg_dev
Documentation: kreuzberg.dev

We'd love to hear your feedback, use cases, and contributions!

TL;DR: Kreuzberg v4 is a complete Rust rewrite of a document intelligence library, offering native bindings for 7 languages (8 runtime targets), 56+ file formats, Rust-native performance, embeddings, semantic chunking, and production-ready servers - all in a 16-31 MB complete package (5-15x smaller than alternatives). Releasing January 2026. MIT licensed forever.

I benchmarked 4 Python text extraction libraries (2025)

Na'aman Hirschfeld (Goldziher) — Sun, 06 Jul 2025 10:25:38 +0000

TL;DR: Comprehensive benchmarks of Kreuzberg, Docling, MarkItDown, and Unstructured across 94 real-world documents. Results might surprise you.

📊 Live Results: https://goldziher.github.io/python-text-extraction-libs-benchmarks/

Context

As the author of Kreuzberg, I wanted to create an honest, comprehensive benchmark of Python text extraction libraries. No cherry-picking, no marketing fluff - just real performance data across 94 documents (~210MB) ranging from tiny text files to 59MB academic papers.

Full disclosure: I built Kreuzberg, but these benchmarks are automated, reproducible, and the methodology is completely open-source.

🔬 What I Tested

Libraries Benchmarked:

Kreuzberg (71MB, 20 deps) - My library
Docling (1,032MB, 88 deps) - IBM's ML-powered solution
MarkItDown (251MB, 25 deps) - Microsoft's Markdown converter
Unstructured (146MB, 54 deps) - Enterprise document processing

Test Coverage:

94 real documents: PDFs, Word docs, HTML, images, spreadsheets
5 size categories: Tiny (<100KB) to Huge (>50MB)
6 languages: English, Hebrew, German, Chinese, Japanese, Korean
CPU-only processing: No GPU acceleration for fair comparison
Multiple metrics: Speed, memory usage, success rates, installation sizes

🏆 Results Summary

Speed Champions 🚀

Kreuzberg: 35+ files/second, handles everything
Unstructured: Moderate speed, excellent reliability
MarkItDown: Good on simple docs, struggles with complex files
Docling: Often 60+ minutes per file (!!)

Installation Footprint 📦

Kreuzberg: 71MB, 20 dependencies ⚡
Unstructured: 146MB, 54 dependencies
MarkItDown: 251MB, 25 dependencies (includes ONNX)
Docling: 1,032MB, 88 dependencies 🐘

Reality Check ⚠️

Docling: Frequently fails/times out on medium files (>1MB)
MarkItDown: Struggles with large/complex documents (>10MB)
Kreuzberg: Consistent across all document types and sizes
Unstructured: Most reliable overall (88%+ success rate)

🎯 When to Use What

⚡ Kreuzberg (Disclaimer: I built this)

Best for: Production workloads, edge computing, AWS Lambda
Why: Smallest footprint (71MB), fastest speed, handles everything
Bonus: Both sync/async APIs with OCR support

🏢 Unstructured

Best for: Enterprise applications, mixed document types
Why: Most reliable overall, good enterprise features
Trade-off: Moderate speed, larger installation

📝 MarkItDown

Best for: Simple documents, LLM preprocessing
Why: Good for basic PDFs/Office docs, optimized for Markdown
Limitation: Fails on large/complex files

🔬 Docling

Best for: Research environments (if you have patience)
Why: Advanced ML document understanding
Reality: Extremely slow, frequent timeouts, 1GB+ install

📈 Key Insights

Installation size matters: Kreuzberg's 71MB vs Docling's 1GB+ makes a huge difference for deployment
Performance varies dramatically: 35 files/second vs 60+ minutes per file
Document complexity is crucial: Simple PDFs vs complex layouts show very different results
Reliability vs features: Sometimes the simplest solution works best

🔧 Methodology

Automated CI/CD: GitHub Actions run benchmarks on every release
Real documents: Academic papers, business docs, multilingual content
Multiple iterations: 3 runs per document, statistical analysis
Open source: Full code, test documents, and results available
Memory profiling: psutil-based resource monitoring
Timeout handling: 5-minute limit per extraction

🤔 Why I Built This

Working on Kreuzberg, I worked on performance and stability, and then wanted a tool to see how it measures against other frameworks - which I could also use to further develop and improve Kreuzberg itself. I therefore created this benchmark. Since it was fun, I invested some time to pimp it out:

Uses real-world documents, not synthetic tests
Tests installation overhead (often ignored)
Includes failure analysis (libraries fail more than you think)
Is completely reproducible and open
Updates automatically with new releases

📊 Data Deep Dive

The interactive dashboard shows some fascinating patterns:

Kreuzberg dominates on speed and resource usage across all categories
Unstructured excels at complex layouts and has the best reliability
MarkItDown is useful for simple docs shows in the data
Docling's ML models create massive overhead for most use cases making it a hard sell

🚀 Try It Yourself

git clone https://github.com/Goldziher/python-text-extraction-libs-benchmarks.git
cd python-text-extraction-libs-benchmarks
uv sync --all-extras
uv run python -m src.cli benchmark --framework kreuzberg_sync --category small

Or just check the live results: https://goldziher.github.io/python-text-extraction-libs-benchmarks/

🔗 Links

📊 Live Benchmark Results: https://goldziher.github.io/python-text-extraction-libs-benchmarks/
📁 Benchmark Repository: https://github.com/Goldziher/python-text-extraction-libs-benchmarks
⚡ Kreuzberg (my library): https://github.com/Goldziher/kreuzberg
🔬 Docling: https://github.com/DS4SD/docling
📝 MarkItDown: https://github.com/microsoft/markitdown
🏢 Unstructured: https://github.com/Unstructured-IO/unstructured

🤝 Discussion

What's your experience with these libraries? Any others I should benchmark? I tried benchmarking marker, but the setup required a GPU.

Some important points regarding how I used these benchmarks for Kreuzberg:

I fine tuned the default settings for Kreuzberg.
I updated our docs to give recommendations on different settings for different use cases. E.g. Kreuzberg can actually get to 75% reliability, with about 15% slow-down.
I made a best effort to configure the frameworks following the best practices of their docs and using their out of the box defaults. If you think something is off or needs adjustment, feel free to let me know here or open an issue in the repository.

Announcing Kreuzberg v2.0: A Lightweight, Modern Python Text Extraction library

Na'aman Hirschfeld (Goldziher) — Sat, 15 Feb 2025 18:44:30 +0000

🔍 What’s Kreuzberg?

Kreuzberg is a Python library that provides a unified async/sync interface for extracting text from PDFs, images, Office documents, and more.

Key Features

Async First: Optimized async using anyio and worker processes.
Minimal Dependencies: Much smaller footprint compared to alternatives.
Serverless-and-Docker Ready: Perfect for serverless functions and containerized deployments.
Local Processing: All processing is done locally, with no API calls or cloud services.
Modern Python: Built for Python 3.9+ with rigorous typing and extensive testing.
Versatile: Supports various formats, including PDFs, spreadsheets, Markdown, LaTeX, and more.

🚀 What’s New in Version 2.0?

Kreuzberg v2.0 brings significant enhancements to performance, usability, and feature set. Here’s what’s new:

Sync APIs: Kreuzberg supports synchronous extraction methods alongside async workflows.
Batch Processing: Efficiently process multiple files or byte streams in parallel.
Smart PDF Handling: Automatically fall back to OCR when direct text extraction fails.
Metadata Extraction: Retrieve metadata like document titles or creators using Pandoc.
Excel Multi-Sheet Support: Handle even the most complex spreadsheets.
Enhanced Performance: Worker processes for faster, resource-efficient extraction.

Check out the v2.0 changelog for more details.

🎯 Who’s It For?

Kreuzberg is ideal for developers building:

Retrieval-Augmented Generation (RAG) systems
LLM-powered applications
Document indexing, analysis, and automation tools

If you’re looking for a lightweight, efficient solution for text extraction, Kreuzberg is a great choice.

⚖️ How Kreuzberg Compares

Here’s how Kreuzberg stacks up against alternatives:

Python OSS Libraries
- Unstructured.io: Feature-rich but heavy, making it unsuitable for serverless or low-resource environments.
- Docling: Another strong alternative but larger and heavier—better suited for high-volume, GPU-based workloads.
Non-Python OSS Libraries
- Apache Tika: Requires a Java server running as a sidecar, with Python client libraries available.
- Grobid: Excellent for structured research text extraction but comes with a ~20GB Docker image.
Commercial APIs
Paid solutions like Azure Document Intelligence or AWS Textract offer best-in-class OCR and layout extraction. However, they come with pricing concerns and cloud dependencies, unlike Kreuzberg.

Staring ⭐ is Caring

If Kreuzberg sounds like the library you’ve been looking for, check it out on GitHub.

Please star the repo ⭐—it helps others discover the project and motivates me to keep improving it!

Functional Snippet: withSuppress

Na'aman Hirschfeld (Goldziher) — Sun, 01 Sep 2024 12:31:51 +0000

Sometimes, suppressing errors is handy.

E.g. you see this:

try {
  scrollableRef.update()
} catch {}

Let's create something to handle this in a cleaner fashion. We'll use a functional style that prefers wrap functions to extend functionality.

We "wrap" a function with error handling, returning a function with an identical signature:

import { isPromise } from '@tool-belt/type-predicates';

/**
 * Wraps a function with error suppression and optional logging.
 *
 * @param fn - The function to execute.
 * @param errorMessage - The error message to log if an exception is thrown.
 * @returns The wrapped function with error suppression.
 */
export function withSuppress<T extends (...args: any[]) => any>(
  fn: T,
  errorMessage?: string,
): (...args: Parameters<T>) => ReturnType<T> extends Promise<infer U>
  ? Promise<U | undefined>
  : ReturnType<T> | undefined {
  const log = errorMessage ? (msg: string) => console.error(msg) : null;

  return (...args: Parameters<T>) => {
    try {
      const result = fn(...args);

      if (isPromise(result)) {
        return result.catch((error) => {
          log?.(`${errorMessage}\n${error?.stack ?? error}`);
          return undefined;
        }) as ReturnType<T> extends Promise<infer U> ? Promise<U | undefined> : never;
      }

      return result;
    } catch (error) {
      log?.(`${errorMessage}\n${error?.stack ?? error}`);
      return undefined;
    }
  };
}

And now we can suppress errors elegantly.

Here are some example uses:

# generate content

Here is a Vitest test suite for the function:

import { withSuppress } from './functional';

describe('withSuppress', () => {
  it('should return the value from a synchronous function without errors', () => {
    const syncFn = () => 42;
    const wrappedFn = withSuppress(syncFn);
    expect(wrappedFn()).toBe(42);
  });

  it('should return undefined and log error when synchronous function throws', () => {
    const errorMessage = 'Sync error occurred';
    const syncFn = () => {
      throw new Error('Test Error');
    };
    const consoleErrorSpy = vi.spyOn(console, 'error').mockImplementation(() => {});

    const wrappedFn = withSuppress(syncFn, errorMessage);
    expect(wrappedFn()).toBeUndefined();
    expect(consoleErrorSpy).toHaveBeenCalledWith(
      expect.stringContaining(`${errorMessage}\nError: Test Error`),
    );

    consoleErrorSpy.mockRestore();
  });

  it('should return the resolved value from an asynchronous function without errors', async () => {
    const asyncFn = async () => await Promise.resolve(42);
    const wrappedFn = withSuppress(asyncFn);
    await expect(wrappedFn()).resolves.toBe(42);
  });

  it('should return undefined and log error when asynchronous function rejects', async () => {
    const errorMessage = 'Async error occurred';
    const asyncFn = async () => await Promise.reject(new Error('Test Error'));
    const consoleErrorSpy = vi.spyOn(console, 'error').mockImplementation(() => {});

    const wrappedFn = withSuppress(asyncFn, errorMessage);
    await expect(wrappedFn()).resolves.toBeUndefined();
    expect(consoleErrorSpy).toHaveBeenCalledWith(
      expect.stringContaining(`${errorMessage}\nError: Test Error`),
    );

    consoleErrorSpy.mockRestore();
  });

  it('should return undefined if no error message is provided and the function throws', () => {
    const syncFn = () => {
      throw new Error('Test Error');
    };
    const consoleErrorSpy = vi.spyOn(console, 'error').mockImplementation(() => {});

    const wrappedFn = withSuppress(syncFn);
    expect(wrappedFn()).toBeUndefined();
    expect(consoleErrorSpy).not.toHaveBeenCalled();

    consoleErrorSpy.mockRestore();
  });

  it('should return undefined if no error message is provided and the async function rejects', async () => {
    const asyncFn = async () => await Promise.reject(new Error('Test Error'));
    const consoleErrorSpy = vi.spyOn(console, 'error').mockImplementation(() => {});

    const wrappedFn = withSuppress(asyncFn);
    await expect(wrappedFn()).resolves.toBeUndefined();
    expect(consoleErrorSpy).not.toHaveBeenCalled();

    consoleErrorSpy.mockRestore();
  });

  it('should handle functions that return promises without errors', async () => {
    const asyncFn = async () => await Promise.resolve('async result');
    const wrappedFn = withSuppress(asyncFn);
    await expect(wrappedFn()).resolves.toBe('async result');
  });

  it('should handle synchronous functions returning undefined without errors', () => {
    const syncFn = () => undefined;
    const wrappedFn = withSuppress(syncFn);
    expect(wrappedFn()).toBeUndefined();
  });
});

Starlite December '22 Updates

Na'aman Hirschfeld (Goldziher) — Sun, 11 Dec 2022 13:31:27 +0000

Starlite December '22 Updates

Hi Pythonistas!

This post is an update on the development status of Starlite. Let me start, as usual, with a short intro about what is Starlite - for those of you unfamiliar with it.

Starlite is a Python ASGI API framework. ASGI is an async python API specification, originally from the good folk over at the Django project, and an ASGI API framework is a framework that follows this specification. You might be familiar with other ASGI frameworks, with the most famous one these days being FastAPI.

Starlite began as an alternative to FastAPI - it was originally built on the same foundation- Starlette, which offers a ready to use ASGI "tool-kit". The name of the framework - Starlite, was meant to highlight this relationship. Yet, over time Starlite grew in sophistication and complextiy, and we made the decision to drop Starlette as a dependency because this no longer made any sense.

Dropping Starlette and Benchmarks

Since version v1.39.0 Starlite (released on the 12.11.22) no longer has any dependency on Starlette. Between this version and v1.45.0 that was released today (11.12.22), we have invested significant effort into benchmarking and optimizing code. One of our maintainers, @provinzkraut (Janek Nouvertné), has done amazing work rewriting our benchmarking framework. You can read more about this in here and run the benchmarks on your own by cloning the benchmark repository. The results are pretty impressive if I may say so myself:

Other Important Changes

Contrib and Security Backends

Another important development is the inclusion of a starlite.contrib namespace, which we will be expanding on in the future.

This namespace includes an optional OpenTelemetry Integration, which was a long awaited feature.

We also added Security Backend support, which was discussed in some length here. The new security backend supports all of the different session backends Starlite supports, and there is also an optional JWT Security Backend as part of contrib.

Yield Based Dependencies

We made some updates to dependencies following a request on reddit - we now support yield based dependencies in the Starlite Dependency Injection framework. Additionally we made some optimizations to dependency injection which allows us to resolve dependencies in parallel and do some caching, the result is a significant boost in speed.

Call for Contributiors and Maintainers

The original imperative for creating Starlite was to create a community driven alternative to FastAPI. This was and remains a core pillar of Starlite- to have multiple maintainers and be as open, inviting and accessible for contributions as feasible. The project follows the all-contributors specification and we attribute all types of contribution - code, testing, refactoring, code reviews, documentation, design of the docs, writing and evangelizing etc.

We are a growing group of contributors and maintainers (5 maintainers at present), and we are always looking for more people to be involved. You're invited to join us on our discord server, or checkout our GitHub repository where you are welcome to write in both discussions and issues.