Forem: Xiao Ling

How to Extract Structured Data and Photos from Passports in JavaScript with Dynamsoft Capture Vision

Xiao Ling — Mon, 11 May 2026 02:36:58 +0000

Passport onboarding, hotel check-in, and KYC flows usually need more than plain OCR. You need MRZ parsing, document localization, and a reliable portrait crop from the same capture pipeline, ideally in the browser without shipping data to a server. This sample uses Dynamsoft Capture Vision on the web to read passport and ID card MRZ data, draw document boundaries, and extract the portrait photo from uploaded images or a captured camera frame.

What you'll build: A plain JavaScript passport and ID scanner that uses Dynamsoft Capture Vision 3.4.2001 to extract MRZ text, structured fields, document borders, and portrait photos from uploaded images or a frozen camera frame.

Live demo

https://yushulx.me/javascript-barcode-qr-code-scanner/examples/mrz_scanner/

Watch the JavaScript Passport Scanner Demo

Prerequisites

A modern browser with camera and clipboard support
A local web server such as Python's built-in HTTP server
Dynamsoft Capture Vision Bundle 3.4.2001
A valid Dynamsoft license key for MRZ features

Get a 30-day free trial license

Step 1: Load the Dynamsoft Bundle and Build the Page Shell

The page uses a plain HTML layout with one image view, one camera view, an overlay canvas, and a face crop canvas.

<!DOCTYPE html>
<html lang="en">

<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>AI-Powered MRZ & Passport Scanner</title>
    <link rel="stylesheet" href="style.css">
    <!-- Dynamsoft Capture Vision Bundle for MRZ -->
    <script src="https://cdn.jsdelivr.net/npm/dynamsoft-capture-vision-bundle@3.4.2001/dist/dcv.bundle.min.js"></script>
    <!-- Face/Border detection powered by Dynamsoft IdentityProcessor (ReadPassportAndId template) -->
</head>

<body>
    <div class="app-container">
        <!-- Header / License Config -->
        <header>
            <h1>🛂 MRZ & Passport Scanner</h1>
            <div class="license-bar">
                <input type="text" id="licenseKey"
                    placeholder="LICENSE-KEY">
                <button id="initBtn">Initialize MRZ</button>
                <a href="https://www.dynamsoft.com/customer/license/trialLicense/?product=dcv&package=cross-platform"
                    target="_blank" rel="noopener noreferrer" class="trial-link">Get MRZ License</a>
            </div>
        </header>


        <!-- Main Content -->
        <main>
            <!-- Left Panel: Image Source & Display -->
            <section class="panel left-panel">
                <div class="toolbar">
                    <button id="btnLoad" title="Load Image" disabled>📂 Load</button>
                    <button id="btnCamera" title="Start Camera" disabled>📷 Camera</button>
                    <button id="btnPaste" title="Paste from Clipboard" disabled>📋 Paste</button>
                    <span id="status">Ready</span>
                </div>

                <div class="viewer-container" id="dropZone">
                    <div id="cameraView" class="hidden">
                        <canvas id="cameraOverlay"></canvas>
                    </div>
                    <div id="imageView" class="hidden">
                        <img id="displayImage" src="" alt="Input">
                        <canvas id="overlayCanvas"></canvas>
                    </div>
                    <div class="placeholder-text" id="placeholderText">
                        Drag & Drop Image Here<br>or Select Source
                    </div>
                </div>
            </section>

Step 2: Initialize Capture Vision and Load the MRZ Template

The initialization step loads the license, WASM modules, MRZ specifications, and the custom findPrecisePortraitZone.json template used by the ReadPassportAndId workflow.

els.initBtn.addEventListener('click', async () => {
    let key = els.licenseKey.value.trim();
    if (!key) {
        key = "LICENSE-KEY";
    }

    try {
        els.status.textContent = "Initializing MRZ SDK...";
        els.initBtn.disabled = true;

        // Initialize License
        await Dynamsoft.License.LicenseManager.initLicense(key, true);

        // Load WASM modules
        els.status.textContent = "Loading WASM modules...";
        await Dynamsoft.Core.CoreModule.loadWasm(["DLR", "DDN"]);

        // Create Code Parser for MRZ parsing
        parser = await Dynamsoft.DCP.CodeParser.createInstance();

        // Load MRZ specs
        els.status.textContent = "Loading MRZ specs...";
        await Dynamsoft.DCP.CodeParserModule.loadSpec("MRTD_TD1_ID");
        await Dynamsoft.DCP.CodeParserModule.loadSpec("MRTD_TD2_FRENCH_ID");
        await Dynamsoft.DCP.CodeParserModule.loadSpec("MRTD_TD2_ID");
        await Dynamsoft.DCP.CodeParserModule.loadSpec("MRTD_TD2_VISA");
        await Dynamsoft.DCP.CodeParserModule.loadSpec("MRTD_TD3_PASSPORT");
        await Dynamsoft.DCP.CodeParserModule.loadSpec("MRTD_TD3_VISA");

        // Preload the custom MRZ recognition models referenced by the sample settings file.
        await Dynamsoft.CVR.CaptureVisionRouter.appendDLModelBuffer([
            "MRZCharRecognition",
            "MRZTextLineRecognition"
        ]);

        // Create Capture Vision Router
        cvr = await Dynamsoft.CVR.CaptureVisionRouter.createInstance();

        // Load MRZ template
        const settingsResult = await cvr.initSettings("./findPrecisePortraitZone.json?t=" + new Date().getTime());
        if (settingsResult?.errorCode && settingsResult.errorCode !== 0) {
            throw new Error(`Template load failed: ${settingsResult.errorString || settingsResult.errorCode}`);
        }

        const hasReadPassportAndId = await cvr.checkTemplateNameValidity("ReadPassportAndId");
        if (!hasReadPassportAndId) {
            const templateNames = await cvr.getTemplateNames();
            throw new Error(
                `Template ReadPassportAndId is not available after loading findPrecisePortraitZone.json. Available templates: ${templateNames.join(", ")}`
            );
        }

        isMRZReady = true;

        if (typeof faceProcessor !== 'undefined') {
            await faceProcessor.setCVR(cvr);
        }

The JSON template is where the sample ties MRZ reading to document detection and normalization.

{
  "CaptureVisionTemplates": [
    {
      "Name": "ReadPassportAndId",
      "ImageROIProcessingNameArray": [
        "roi_passport_and_id"
      ],
      "SemanticProcessingNameArray": [
        "sp_passport_and_id"
      ],
      "OutputOriginalImage": 1,
      "MaxParallelTasks": 0,
      "Timeout": 20000
    }
  ],
  "TargetROIDefOptions": [
    {
      "Name": "roi_passport_and_id",
      "TaskSettingNameArray": [
        "task_passport_and_id",
        "task-detect-and-normalize-document"
      ]
    }
  ]
}

Step 3: Process Uploaded Images and Parse MRZ Text

For still images, the app loads the image into the UI first, clears the previous intermediate results, runs cvr.capture(), and then parses the concatenated MRZ text.

function loadImage(base64Image, captureSource = null) {
    currentImageCaptureSource = captureSource ?? base64Image;
    els.displayImage.src = base64Image;
    els.placeholderText.classList.add('hidden');
    els.imageView.classList.remove('hidden');
    els.cameraView.classList.add('hidden');

    els.displayImage.onload = async () => {
        resizeCanvas();
        els.status.textContent = "Processing...";
        clearResults();

        // Show spinner
        els.loadingSpinner.classList.add('visible');

        const ctx = els.overlayCanvas.getContext('2d', { willReadFrequently: true });
        ctx.clearRect(0, 0, els.overlayCanvas.width, els.overlayCanvas.height);

        try {
            // Run MRZ detection (only if license is initialized)
            if (isMRZReady && cvr) {
                try {
                    // Clear previous intermediate results so IdentityProcessor
                    // only sees data from this capture call
                    if (typeof faceProcessor !== 'undefined') {
                        faceProcessor.clearIntermediateResults();
                    }

                    // Use ReadPassportAndId template: one capture returns MRZ text,
                    // document border (DetectedQuad), and intermediate units for
                    // IdentityProcessor.findPortraitZone()
                    const captureSource = currentImageCaptureSource ?? els.displayImage;
                    const result = await cvr.capture(captureSource, "ReadPassportAndId");
                    const items = result.items || [];

                    let mrzTexts = [];
                    for (const item of items) {
                        if (item.type === Dynamsoft.Core.EnumCapturedResultItemType.CRIT_TEXT_LINE) {
                            mrzTexts.push(item.text);

                            // Draw overlay and capture MRZ zone
                            const location = item.location;
                            if (location && location.points) {
                                drawOverlay(location);
                            }
                        }
                    }

                    if (mrzTexts.length > 0) {
                        // Display raw MRZ text with newlines for readability
                        els.mrzRawText.textContent = mrzTexts.join('\n');

                        // Parse MRZ - join all lines into single string (no separators)
                        // TD3 passport has 2 lines of 44 chars each = 88 chars total
                        const mrzForParsing = mrzTexts.map(t => t.trim()).join('');

                        const parseResults = await parser.parse(mrzForParsing);
                        displayParsedMrz(parseResults);
                    } else {
                        els.mrzRawText.textContent = "No MRZ detected.";
                        els.mrzResults.textContent = "No MRZ detected.";
                    }

                    // Process face/border from the same captured result
                    if (typeof faceProcessor !== 'undefined') {
                        await faceProcessor.processCapturedResult(
                            result, els.displayImage, els.faceCropCanvas, els.overlayCanvas
                        );
                    }

Step 4: Draw Document Borders and Portrait Crops from One Capture Result

The helper class uses the same CapturedResult to locate the document quadrilateral, call IdentityProcessor.findPortraitZone(), and draw the cropped portrait.

async processCapturedResult(capturedResult, imageElement, faceCanvas, canvasOverlay) {
    if (!this.isInitialized) {
        this._clearFaceCanvas(faceCanvas);
        return;
    }

    let detectedQuad = null;
    let portraitZone = null;

    try {
        // 1. Extract detected document border from captured result items
        if (capturedResult && capturedResult.items) {
            for (const item of capturedResult.items) {
                if (item.type === Dynamsoft.Core.EnumCapturedResultItemType.CRIT_DETECTED_QUAD) {
                    detectedQuad = item;
                    break;
                }
            }
        }

        // 2. Find portrait zone using the same unit group assembly as the Python reference.
        portraitZone = await this.findPortraitZoneForCapturedResult(capturedResult);

        // 3. Draw document border
        if (detectedQuad) {
            const { width: srcW, height: srcH } = this._getSourceDimensions(imageElement);
            this.drawDocumentBorder(detectedQuad, canvasOverlay, srcW, srcH);
        }

        // 4. Draw portrait zone and crop
        if (portraitZone) {
            this.drawAndCropPortrait(portraitZone, imageElement, faceCanvas, canvasOverlay);
        } else {
            this._clearFaceCanvas(faceCanvas);
        }

    } catch (e) {
        console.error('FaceProcessor error:', e);
        this._clearFaceCanvas(faceCanvas);
    }
}

drawAndCropPortrait(portraitZone, sourceImage, faceCanvas, canvasOverlay) {
    const pts = portraitZone.points;
    if (!pts || pts.length < 4) return;

    const { width: imgW, height: imgH } = this._getSourceDimensions(sourceImage);

    const clamped = this._clampPoints(pts, imgW, imgH);

    this._drawQuad(clamped, canvasOverlay, 'Portrait (Identity)', false, '#0070f3');
    if (!this._drawPortraitQuadCrop(sourceImage, clamped, faceCanvas)) {
        this._clearFaceCanvas(faceCanvas);
        return;
    }

Step 5: Freeze the Camera Frame Before Scanning

Instead of scanning every live frame, this sample freezes the current camera image first, shows it immediately, and then runs MRZ and portrait detection on that frozen canvas.

async function captureAndFreezeCamera() {
    const video = els.cameraView.querySelector('video');
    if (!video || video.readyState < 2 || !video.videoWidth || !video.videoHeight) {
        els.status.textContent = "Camera frame is not ready yet.";
        return;
    }

    const frameCanvas = document.createElement('canvas');
    frameCanvas.width = video.videoWidth;
    frameCanvas.height = video.videoHeight;
    const frameCtx = frameCanvas.getContext('2d', { willReadFrequently: true });
    frameCtx.drawImage(video, 0, 0, frameCanvas.width, frameCanvas.height);
    const frozenFrameDataUrl = frameCanvas.toDataURL('image/jpeg', 0.95);

    isCameraRunning = false;

    if (els.cameraOverlay) {
        const ctx = els.cameraOverlay.getContext('2d', { willReadFrequently: true });
        ctx.clearRect(0, 0, els.cameraOverlay.width, els.cameraOverlay.height);
    }

    if (videoStream) {
        videoStream.getTracks().forEach(track => track.stop());
        videoStream = null;
    }
    video.srcObject = null;

    document.getElementById('btnCamera').textContent = "📷 Camera";
    els.cameraView.classList.add('hidden');
    els.imageView.classList.remove('hidden');
    els.status.textContent = "Processing captured frame...";
    clearResults();
    els.loadingSpinner.classList.add('visible');

    lastCameraResult = {
        base64Image: frozenFrameDataUrl,
        mrzTexts: [],
        mrzLocations: [],
        detectedQuad: null,
        portraitZone: null
    };

    try {
        await showCapturedCameraFrame(frozenFrameDataUrl);

        if (isMRZReady && cvr) {
            try {
                if (typeof faceProcessor !== 'undefined') {
                    faceProcessor.clearIntermediateResults();
                }

                const result = await cvr.capture(frameCanvas, "ReadPassportAndId");

Source Code

https://github.com/yushulx/javascript-barcode-qr-code-scanner/tree/main/examples/mrz_scanner

How to Generate Custom Dynamsoft Barcode Reader Templates in Python for Hard Barcode Images

Xiao Ling — Wed, 06 May 2026 09:13:49 +0000

Default barcode presets are designed for broad coverage, not for every extreme image you will see in production. When a symbol is tiny, low-contrast, tightly cropped, or surrounded by distracting texture, the right fix is usually not to keep randomizing parameters. A better approach is to let AI follow a strict workflow: inspect the image, validate the current template, probe controlled variants, classify the failure mode, and then generate the smallest focused JSON template that matches the evidence.

What you'll build: A Python workflow that uses AI, OpenCV, and Dynamsoft Barcode Reader helper scripts to diagnose hard barcode images, generate focused JSON templates, and validate them before import.

Follow the Workflow from Diagnosis to Focused Template Generation

Why hard barcode images often need custom DBR templates instead of more blind retries.
How to turn barcode-tuning knowledge into an AI skill and prompt that produce consistent results.
How AI visual analysis and OpenCV-based validation work together in the same tuning loop.
How to validate and probe DBR templates with small Python helpers before changing JSON.
How to generate a focused single-barcode DataMatrix template for a real tiny-symbol image.

Why Focused DBR Templates Matter for Hard Barcode Images

Dynamsoft Barcode Reader is flexible because it lets you tune scope, localization, decode, and acceptance behavior independently. That flexibility is exactly why custom templates matter. A hard image might fail because the scene contains too much background, because localization is too conservative, because the decoder needs a different grayscale or binarization path, or because the acceptance rules reject a symbol that is otherwise readable.

The problem is not capability. The problem is undirected tuning. Without a disciplined workflow, it is easy to change DeblurModes, LocalizationModes, and MinQuietZoneWidth all at once and learn nothing from the result. In this project, AI is useful because it does not just generate JSON. It follows a falsifiable decision tree encoded in the skill and prompt. It also knows when to stop. For example, the same repo contains an Aztec case where raw plus basic variants still prove that missing context pixels are the real blocker, which means preprocessing is more justified than more template churn.

Prepare the Inputs for Repeatable Template Tuning

Use the default DBR preset as a baseline rather than a guarantee for extreme images.
Let AI classify whether scope, localization, decode, or acceptance settings deserve attention first.
Treat raw plus basic-variant failure as a stop signal for template-only tuning.
Keep focused JSON templates small so they are easier to validate and import into the desktop app.

Prerequisites

Python 3.9+
Get a 30-day free trial license for Dynamsoft Barcode Reader.
The dependencies in template-optimizer/requirements.txt if you only want the optimizer package, or requirements.txt if you also want the full GUI app
A difficult barcode image
A baseline DBR template file if you already have one

Install the standalone optimizer dependencies:

pip install -r template-optimizer/requirements.txt

Step 1: Encode the Tuning Rules into a Reusable AI Workflow

The first step is to stop asking AI for ad hoc parameter guesses. In this project, the canonical workflow now lives inside the self-contained template-optimizer/ directory: template-optimizer/SKILL.md captures the decision logic, template-optimizer/PROMPT.md provides a portable user-facing entrypoint, and template-optimizer/KNOWLEDGE.md stores the tuning reference. The .github/ files are still useful in VS Code Copilot, but only as thin wrappers around that portable package.

This is the copyable prompt contract from template-optimizer/PROMPT.md:

Read `template-optimizer/SKILL.md` and `template-optimizer/KNOWLEDGE.md` first. Follow the Python-only template-optimizer workflow and use the bundled helper scripts under `template-optimizer/tools/` for single-image triage before broad benchmarking.

Task: tune this barcode image.

- Image path: `<image-path>`
- Current template path: `<template-path-or-none>`
- Format hint: `<format-or-none>`

And this discipline now lives in the canonical template-optimizer/SKILL.md workflow, with .github/skills/template-optimizer/SKILL.md acting only as an optional Copilot wrapper:

## Procedure

1. Observe the image before touching the template.
Use the image tool on the provided barcode image and describe visible symptoms in concrete terms: quiet-zone loss, border clipping, blur, low contrast, warping, wrinkles, inversion, reflection, background texture, finder-pattern damage, or perspective skew.

2. Validate the current template directly through the simplest DBR path.
The validator must initialize the DBR license before creating `CaptureVisionRouter`; otherwise treat `NO_RESULT` as invalid evidence.
Run `python template-optimizer/tools/validate_dbr_template.py <image> --template-file <template>`.

3. Establish a broader baseline only after the direct validation result is known.
Run `python template-optimizer/tools/probe_dbr_templates.py <image>`.

That structure matters because AI now has a repeatable way to reason about DBR flexibility. It must classify the problem before it edits JSON. That is what turns AI from a template randomizer into a tuning assistant.

Step 2: Run the Portable Skill and Prompt in Practice

This part is critical. If an article shows the generated skill and prompt but never explains how to use them, many readers still will not know how to make the workflow run in a real tool.

The simplest packaging rule is this: treat template-optimizer/ as the product, and treat .github/ as optional integration glue for VS Code Copilot. In practice, a developer can copy just the template-optimizer/ directory into another workspace, install template-optimizer/requirements.txt, provide a barcode image plus any current template path, and ask the agent to read template-optimizer/PROMPT.md. That is enough to run the same workflow in Copilot, Claude, or another coding agent that can read files and execute local Python commands.

Inside VS Code Copilot, the flow becomes:

Open the workspace that contains template-optimizer/.
Optionally keep .github/prompts/ and .github/skills/ if you want native Copilot discovery.
Provide the barcode image, the current template path, and any known format constraints in Copilot Chat.
Use template-optimizer/PROMPT.md directly, or ask Copilot to use the template-optimizer skill if the optional wrapper is present.
Let the agent call the bundled Python scripts under template-optimizer/tools/, such as validate_dbr_template.py, probe_dbr_templates.py, and compare_dbr_template_profiles.py, to turn the visual diagnosis into evidence.

In other words, the portable prompt decides where the workflow starts, the canonical skill decides how it should reason, and the local scripts decide how the conclusions get validated. The .github/ files just make that package easier for Copilot to discover.

What about other tools? They can use the same method, but you need to separate the workflow itself from the file format that stores it.

Tools such as OpenCode and Claude Code can usually reuse the same methodology, but they may not natively understand GitHub Copilot-style files such as .github/prompts/*.prompt.md and .github/skills/*/SKILL.md. That is exactly why the prompt was moved into template-optimizer/PROMPT.md: the portable package no longer depends on .github/ to be usable. In those tools, you usually just tell the agent to read template-optimizer/PROMPT.md, template-optimizer/SKILL.md, and template-optimizer/KNOWLEDGE.md.

That does not mean the logic has to be rewritten. The portable parts are these three:

the workflow itself: inspect the image, validate, probe, classify, and then generate a focused template
the local script capability: template-optimizer/tools/validate_dbr_template.py, template-optimizer/tools/probe_dbr_templates.py, and JSON template editing
the image input capability: the AI must be able to actually inspect the image, not just read a text description of it

So if you want to use this approach in OpenCode or Claude Code, at least three conditions should be true:

The tool supports image input or multimodal visual analysis.
The tool can access the workspace files and run local terminal commands.
You are willing to translate the VS Code-style prompt and skill content into whatever instruction format that tool expects.

Once those conditions are met, the approach is not limited to VS Code. GPT-5.4 can be used, and other multimodal AI systems with visual capability can also be used. VS Code Copilot Chat can consume this file structure more directly, while OpenCode and Claude Code are better treated as environments where the same rules are adapted and reused. What needs to travel is not one specific file format, but the full loop of visual diagnosis, script-based validation, and focused template generation.

Step 3: Combine AI Vision with OpenCV-Based Validation

This workflow uses both AI and OpenCV, but they do different jobs.

In the first step, the model inspects the image itself and produces a visual diagnosis. That part is not driven by OpenCV code. It comes from the model's own multimodal reasoning: looking at the image and describing visible symptoms such as a tiny symbol, a noisy background, missing quiet zone, clipping, inversion, blur, or perspective distortion. That matters because template tuning is rarely just a brute-force search problem. Someone, or something, has to decide which parameter family deserves attention first.

GPT-5.4 is one option for that role, but it is not the only one. Other multimodal AI systems can also be used to generate focused templates, as long as they can actually inspect images and reason about visible symptoms before touching the JSON. The key requirement is not a specific vendor or model name. The key requirement is visual capability plus the ability to participate in a disciplined validation loop.

After that, OpenCV and DBR take over for the verification step. OpenCV loads the image, generates controlled variants when needed, and feeds those pixels into CaptureVisionRouter. DBR then gives the hard evidence: NO_RESULT, a successful decode, or a change in behavior after a specific template edit. In other words, AI proposes the diagnosis, while OpenCV and DBR test whether that diagnosis is actually true.

That combination is important because each side covers the other's weakness. AI alone is good at spotting visual patterns quickly, but it still needs a deterministic validation path. OpenCV alone is good at repeatable pixel operations, but it has no built-in understanding of which image symptom is most likely tied to scope, localization, decode, or acceptance settings. Put together, the workflow becomes much stronger: AI chooses a plausible next move, and OpenCV plus DBR turn that move into evidence.

Step 4: Validate the Current Template Before Editing JSON

The smallest trustworthy DBR check in this repo is template-optimizer/tools/validate_dbr_template.py. It loads one image, resolves one template name, initializes the license, and calls capture() directly. That makes it the right first step before you change anything.

The README already exposes the direct validation command:

python template-optimizer/tools/validate_dbr_template.py path/to/image.png --template-file candidate-template.json

The implementation is deliberately minimal:

def decode(image_path, template_file=None, template_name=None, fallback_profile="none"):
    ensure_dbr_license()
    router = CaptureVisionRouter()
    if template_file:
        err, msg = router.init_settings_from_file(str(template_file))
        if err != 0:
            print(f"[DBR] Template load failed ({err}): {msg}")

    cv_img = cv2.imread(str(image_path))
    if cv_img is None:
        raise RuntimeError(f"Failed to load image with OpenCV: {image_path}")

    result = router.capture(cv_img, template_name)
    return result.get_decoded_barcodes_result()

Example input image used throughout Steps 4-6: a tiny DataMatrix sits on a pink object within a large dark textured scene.

On the example image used in this project, that discipline immediately pays off. The raw image is dominated by dark textured background, while the DataMatrix occupies only a tiny region on a pink object. Running the default preset and the built-in templates on the original image produced only NO_RESULT, which is strong evidence that the first problem is not acceptance tuning. It is scene scope and localization.

Step 5: Probe Variants to Classify the Failure Before Editing JSON

Once the direct check fails, the next step is not to guess. It is to probe the image systematically. The helper script does exactly that by trying the preset, every bundled example template, and optional preprocessing variants.

The README exposes the broad evidence pass like this:

python template-optimizer/tools/probe_dbr_templates.py path/to/image.png --variant-set basic --report-json tuning-report.json

The variant generator is intentionally small and transparent:

def build_variants(image_path, variant_set):
    variants = [("original", image_path)]
    if variant_set == "none":
        return variants

    source = cv2.imread(str(image_path), cv2.IMREAD_COLOR)
    if source is None:
        return variants

    variant_dir = Path.cwd() / "_probe_variants" / image_path.stem
    variant_dir.mkdir(parents=True, exist_ok=True)

    def save_variant(name, image):
        path = variant_dir / f"{name}.png"
        cv2.imwrite(str(path), image)
        variants.append((name, path))

    padded = cv2.copyMakeBorder(
        source,
        40,
        40,
        40,
        40,
        cv2.BORDER_CONSTANT,
        value=(255, 255, 255),
    )
    save_variant("padded_40", padded)

    for scale in (2, 4):
        resized = cv2.resize(source, None, fx=scale, fy=scale, interpolation=cv2.INTER_NEAREST)
        save_variant(f"nearest_{scale}x", resized)

For the example image in this project, that probe gave the kind of evidence you want before generating a template:

The original image stayed NO_RESULT across the default preset and all built-in templates.
Padding alone did not fix the issue.
nearest_4x plus ReadBarcodes_ReadRateFirst decoded one DATAMATRIX result: DM-18Batch.
The variant results show that the main blocker is tiny effective symbol size in a large noisy scene, not an unrecoverable pixel failure.

That pattern is important. It means the pixels are recoverable, but the raw scene is too hostile for the broad default path. In other words, this is a good candidate for AI-generated focused tuning. In the final focused template, the key fix was to narrow the ROI to the centered symbol region so DBR spends its effort on the relevant area instead of the full textured background.

Step 6: Generate a Focused Single-Barcode DataMatrix Template

After the probe showed a single tiny DataMatrix and a scale-sensitive recovery path, AI can generate a much smaller template that matches the actual image. In this case, the key decisions are straightforward:

limit the format to BF_DATAMATRIX
limit the expected count to 1
add stronger localization modes for a tiny centered symbol
narrow the ROI to the center of the image instead of searching the whole frame
keep both normal and mirrored format specs available

AI can generate a focused template like the following when the image evidence clearly points to a single tiny DataMatrix:

{
    "BarcodeFormatSpecificationOptions" :
    [
        {
            "BarcodeFormatIds" :
            [
                "BF_DATAMATRIX"
            ],
            "DataMatrixSizeOptions" :
            [
                "DMS_DEFAULT"
            ],
            "ExpectedBarcodesCount" : 1,
            "FindUnevenModuleBarcode" : 1,
            "HasVerticalQuietZone" : 1,
            "MinQuietZoneWidth" : 4,
            "MinResultConfidence" : 30,
            "MirrorMode" : "MM_BOTH",
            "Name" : "bfs-single-tiny-datamatrix-both",
            "PartitionModes" :
            [
                "PM_WHOLE_BARCODE",
                "PM_ALIGNMENT_PARTITION"
            ],
            "ReturnPartialBarcodeValue" : 1
        }
    ],
    "TargetROIDefOptions" :
    [
        {
            "Name" : "roi-read-single-tiny-datamatrix",
            "Location" :
            {
                "Offset" :
                {
                    "FirstPoint" : [40, 30, 1, 1],
                    "SecondPoint" : [65, 30, 1, 1],
                    "ThirdPoint" : [65, 65, 1, 1],
                    "FourthPoint" : [40, 65, 1, 1],
                    "MeasuredByPercentage" : 1
                }
            }
        }
    ],
    "BarcodeReaderTaskSettingOptions" :
    [
        {
            "BarcodeFormatIds" :
            [
                "BF_DATAMATRIX"
            ],
            "ExpectedBarcodesCount" : 1,
            "Name" : "task-read-single-tiny-datamatrix"
        }
    ]
}

That generated template can then be validated against the target image with the same helper script:

python template-optimizer/tools/validate_dbr_template.py path/to/image.png --template-file path/to/generated-template.json

In this workspace, the custom template decoded the DataMatrix value DM-18Batch directly on the original image. That is the practical payoff of AI-generated template work: not a giant exported preset, but a small file that matches the failure mode you actually observed and solves it through template configuration alone.

Step 7: Load the Focused Template Back into the App

Once the focused template works, the desktop app can import it directly. This is why focused single-template files are especially useful in this project: the GUI explicitly uses the first template name in the JSON file.

Here is the import logic from main.py:

def _import_dbr_template(self):
    """Load a custom DBR template JSON file for all subsequent captures."""
    path, _ = QFileDialog.getOpenFileName(
        self, "Import DBR Template",
        self._active_template_path or "",
        "JSON files (*.json)"
    )
    if not path:
        return
    try:
        # Validate by trying to load into a temporary router
        router = CaptureVisionRouter()
        err, msg = router.init_settings_from_file(path)
        if err != 0:
            QMessageBox.warning(self, "Import Failed",
                                f"DBR error [{err}]: {msg}")
            return

        # Find the first template name inside the JSON so we use the right one
        with open(path, "r", encoding="utf-8") as f:
            data = json.load(f)
        templates = data.get("CaptureVisionTemplates", [])
        if templates:
            template_name = templates[0]["Name"]
        else:
            # Fallback: keep the default name
            template_name = EnumPresetTemplate.PT_READ_BARCODES.value

That small detail has a big architectural consequence: if you want predictable behavior in the GUI, generate a focused template whose first CaptureVisionTemplate is the one you actually want to run. AI-generated templates are much easier to keep clean than manually edited all-in-one exports.

Watch the Desktop Demo

The sample project already includes a short demo of the Python desktop tool that can import and test custom DBR templates:

Source Code

https://github.com/yushulx/python-barcode-qrcode-sdk/tree/main/examples/official/annotation_tool

Build a Dual-Engine Barcode Annotation Tool in Python to Create Ground-Truth Datasets

Xiao Ling — Wed, 15 Apr 2026 08:04:29 +0000

Evaluating the accuracy of a barcode SDK requires clean, human-verified ground-truth data — but building that dataset by hand is slow and error-prone. Running two independent engines in parallel, comparing their outputs automatically, and letting a developer resolve discrepancies is a far more efficient approach. This tutorial shows how to build exactly that: a PySide6 desktop annotation tool driven by Dynamsoft Barcode Reader Bundle and ZXing-C++, capable of producing a barcode-benchmark/1.0 JSON dataset in minutes.

What you'll build: A Python desktop annotation application that loads image folders, auto-detects barcodes with dual engines (Dynamsoft DBR + ZXing-C++), highlights cross-engine mismatches for human review, supports manual quad drawing and crop-and-detect, and exports a portable ground-truth JSON dataset.

Demo Video: Dual-Engine Barcode Annotation Tool in Action

Prerequisites

Python 3.9 or later
pip install PySide6>=6.5 opencv-python>=4.8 numpy>=1.24 zxing-cpp>=3.0.0 dynamsoft-barcode-reader-bundle>=11.4.2000
A Dynamsoft Barcode Reader license key (a 30-day trial key is bundled in the sample; replace it for production use)

Get a 30-day free trial license at dynamsoft.com/customer/license/trialLicense

Install all dependencies at once:

pip install -r requirements.txt

Then launch the annotation tool:

python main.py

Step 1: Initialise the Dynamsoft License

The SDK must be activated once at module level before any CaptureVisionRouter instance is created. The call is synchronous and caches the result, so subsequent launches work even without a network connection.

from dynamsoft_barcode_reader_bundle import (
    LicenseManager, CaptureVisionRouter, EnumPresetTemplate, EnumErrorCode
)

LICENSE_KEY = "LICENSE-KEY"

err, msg = LicenseManager.init_license(LICENSE_KEY)
if err not in (EnumErrorCode.EC_OK, EnumErrorCode.EC_LICENSE_CACHE_USED):
    print(f"[DBR] License warning: {msg}")

EC_LICENSE_CACHE_USED is a non-fatal status returned when the SDK uses its locally cached license after a successful prior activation, so both codes are treated as success.

Step 2: Run Dual-Engine Detection in a Background Thread

A QThread subclass runs both Dynamsoft DBR and ZXing-C++ on every image. The engines operate independently; their results are then merged by matching decoded text values. When both agree on the same text, the annotation is flagged match=True and routed to the Verified list automatically.

import zxingcpp
from PySide6.QtCore import QThread, Signal

class DetectionWorker(QThread):
    progress = Signal(int, int)    # done, total
    file_done = Signal(str, list)  # abs_path, annotations

    def __init__(self, file_paths, template_path=None, template_name=None, parent=None):
        super().__init__(parent)
        self._files = list(file_paths)
        self._template_path = template_path
        self._template_name = template_name or EnumPresetTemplate.PT_READ_BARCODES.value

    def run(self):
        router = CaptureVisionRouter()
        if self._template_path:
            err, msg = router.init_settings_from_file(self._template_path)
            if err != 0:
                print(f"[DBR] Template load failed ({err}): {msg}")
        total = len(self._files)
        for i, path in enumerate(self._files):
            anns = self._detect(router, path)
            self.file_done.emit(path, anns)
            self.progress.emit(i + 1, total)

    def _detect(self, router, path):
        # --- Dynamsoft DBR ---
        dbr_items = []
        try:
            result = router.capture(path, self._template_name)
            if result:
                dbr_r = result.get_decoded_barcodes_result()
                if dbr_r:
                    for item in (dbr_r.get_items() or []):
                        pts = item.get_location().points
                        dbr_items.append({
                            "text":   item.get_text(),
                            "format": item.get_format_string(),
                            "points": [(p.x, p.y) for p in pts],
                        })
        except Exception as exc:
            print(f"[DBR] {exc}")

        # --- ZXing-C++ ---
        zxing_items = []
        try:
            img = cv2.imread(path)
            if img is not None:
                for zx in zxingcpp.read_barcodes(img):
                    pos = zx.position
                    zxing_items.append({
                        "text":   zx.text,
                        "format": zx.format.name,
                        "points": [
                            (pos.top_left.x,     pos.top_left.y),
                            (pos.top_right.x,    pos.top_right.y),
                            (pos.bottom_right.x, pos.bottom_right.y),
                            (pos.bottom_left.x,  pos.bottom_left.y),
                        ],
                    })
        except Exception as exc:
            print(f"[ZXing] {exc}")

        # Merge: pair results by decoded text value
        zxing_text_set = {r["text"] for r in zxing_items}
        zxing_by_text  = {}
        for r in zxing_items:
            zxing_by_text.setdefault(r["text"], []).append(r)

        anns = []
        for r in dbr_items:
            match = r["text"] in zxing_text_set
            zxing_pair = zxing_by_text[r["text"]].pop(0) if match and zxing_by_text.get(r["text"]) else None
            anns.append({
                **r,
                "match":  match,
                "source": "dbr",
                "dbr":    {"text": r["text"], "format": r["format"], "points": r["points"]},
                "zxing":  {"text": zxing_pair["text"], "format": zxing_pair["format"],
                           "points": zxing_pair["points"]} if zxing_pair else None,
            })
        for r in zxing_items:
            if r["text"] not in {i["text"] for i in dbr_items}:
                anns.append({**r, "match": False, "source": "zxing", "dbr": None,
                             "zxing": {"text": r["text"], "format": r["format"], "points": r["points"]}})
        return anns

The worker is started from the main thread and its file_done signal is connected to the UI:

self._worker = DetectionWorker(
    paths,
    template_path=self._active_template_path,
    template_name=self._active_template_name,
)
self._worker.file_done.connect(self._on_file_detected)
self._worker.progress.connect(self._on_progress)
self._worker.finished.connect(self._on_detection_done)
self._worker.start()

Step 3: Classify and Display Results in Three Lists

Each image is classified as soon as its detection completes, without waiting for the full batch to finish. Images where every barcode was agreed on by both engines go straight to Verified (green), those with any mismatch go to Needs Review (red), and images where nothing was detected go to No Result (grey).

def _on_file_detected(self, path, annotations):
    self._files[path]["annotations"] = annotations
    filename = os.path.basename(path)

    item = QListWidgetItem(filename)
    item.setData(Qt.UserRole, path)
    if len(annotations) == 0:
        item.setForeground(QColor(120, 120, 120))
        self._list_noresult.addItem(item)
    elif all(a.get("match") for a in annotations):
        item.setForeground(QColor(0, 130, 0))
        self._list_verified.addItem(item)
    else:
        item.setForeground(QColor(180, 0, 0))
        self._list_review.addItem(item)

    self._grp_verified.setTitle(f"Verified ({self._list_verified.count()})")
    self._grp_review.setTitle(f"Needs Review ({self._list_review.count()})")
    self._grp_noresult.setTitle(f"No Result ({self._list_noresult.count()})")

    if self._current_path is None:
        self._select_file(path)

Step 4: Add and Edit Annotations Manually

Two complementary annotation modes handle cases where automatic detection fails.

Crop Mode — drag a rectangle over a region and both engines attempt to decode only that cropped area. Coordinates are translated back to the full image space before being stored:

def _decode_quad_region(self, path, quad_points):
    cv_img = load_image_cv(path)
    xs = [p[0] for p in quad_points]
    ys = [p[1] for p in quad_points]
    x1, y1 = max(0, int(min(xs))), max(0, int(min(ys)))
    x2, y2 = min(cv_img.shape[1], int(max(xs)) + 1), min(cv_img.shape[0], int(max(ys)) + 1)
    crop = cv_img[y1:y2, x1:x2]

    # Run DBR on the cropped region via a temp file
    import tempfile
    with tempfile.NamedTemporaryFile(suffix=".png", delete=False) as tmp:
        tmp_path = tmp.name
    cv2.imwrite(tmp_path, crop)
    router = CaptureVisionRouter()
    result = router.capture(tmp_path, self._active_template_name)
    os.unlink(tmp_path)
    # ... ZXing pass and coordinate offset back to full image omitted for brevity

Draw Mode — press D, click four corners of the barcode quad to draw a custom polygon, then enter the text and format in the BarcodeEditDialog. The annotation is stored with source="manual":

def _on_quad_drawn(self, quad_points):
    dlg = BarcodeEditDialog(text="", fmt="Unknown")
    if dlg.exec() != QDialog.Accepted:
        return
    ann = {
        "text":   dlg.text_edit.text(),
        "format": dlg.fmt_combo.currentText(),
        "points": quad_points,
        "match":  False,
        "source": "manual",
        "dbr":    None,
        "zxing":  None,
    }
    self._files[self._current_path]["annotations"].append(ann)
    self._redraw(self._current_path)

Every polygon corner is also a draggable DraggableVertex handle, so fine-tuning positions requires no mode switching — just click and drag.

Step 5: Export and Re-Import the Ground-Truth Dataset

Click Export JSON to save all loaded images and their annotations to a barcode-benchmark/1.0 JSON file. The schema is self-describing:

def _export_json(self):
    images_out = []
    total_barcodes = 0
    for path in self._all_paths:
        filename = os.path.basename(path)
        anns = (self._files.get(path) or {}).get("annotations") or []
        barcodes = [
            {"text": a["text"], "format": a["format"], "points": a["points"]}
            for a in anns
        ]
        total_barcodes += len(barcodes)
        images_out.append({"file": filename, "barcodes": barcodes})

    out = {
        "format":        "barcode-benchmark/1.0",
        "dataset":       "Annotated Collection",
        "total_images":  len(images_out),
        "total_barcodes": total_barcodes,
        "images":         images_out,
    }
    save_path, _ = QFileDialog.getSaveFileName(
        self, "Save Annotations", "annotations.json", "JSON (*.json)"
    )
    if save_path:
        with open(save_path, "w", encoding="utf-8") as f:
            json.dump(out, f, indent=2, ensure_ascii=False)

In a later session, click Import JSON to reload exported annotations directly onto the same image set — no re-detection is needed. Every imported barcode is immediately flagged as verified and appears in the Verified list:

annotations = [
    {
        "text":   bc.get("text", ""),
        "format": bc.get("format", "Unknown"),
        "points": [tuple(p) for p in bc.get("points", [])],
        "match":  True,
        "source": "imported",
        "dbr":    None,
        "zxing":  None,
    }
    for bc in raw_barcodes
]

Step 6: Use the Lightweight Read-Only Viewer

The companion viewer.py depends only on tkinter (Python built-in) and Pillow — no PySide6 or OpenCV required — making it easy to pass to teammates or testers who don't need to run the full annotation tool.

pip install Pillow
python viewer.py

Source Code

https://github.com/yushulx/python-barcode-qrcode-sdk/tree/main/examples/official/annotation_tool

iOS Barcode SDK Benchmark: Dynamsoft vs ML Kit, Apple Vision, and ZXing-CPP in SwiftUI

Xiao Ling — Mon, 13 Apr 2026 06:41:47 +0000

Developers integrating barcode scanning into iOS apps have several viable options — Dynamsoft Barcode Reader (commercial), Google ML Kit (free), Apple Vision (native), and ZXing-CPP (open source) — each with different accuracy, speed, and format coverage characteristics. This article walks through building a SwiftUI benchmark app that runs all four SDKs side-by-side against identical image and video inputs. The results show Dynamsoft Barcode Reader as the clear leader: it achieved the highest detection count on a real-world test set while remaining the only SDK to combine that accuracy with hardware-accelerated, production-viable speed.

What you'll build: An iOS 16+ SwiftUI app that benchmarks four barcode scanning SDKs (Dynamsoft, ML Kit, Apple Vision, ZXing-CPP) across image, video, and live camera modes with colored bounding-box overlays and an HTML report exportable from a built-in HTTP server.

Demo Video: iOS Barcode Scanner Benchmark in Action

Prerequisites

Xcode 15.0+ with Swift 5.0 support
iOS 16.0+ device or simulator
CocoaPods installed (sudo gem install cocoapods) — required for Google ML Kit
A Dynamsoft Barcode Reader trial license key (the other three SDKs are free)

Get a 30-day free trial license at dynamsoft.com/customer/license/trialLicense

Step 1: Install and Configure the SDKs

ML Kit is not available through Swift Package Manager, so it is installed via CocoaPods. From the project root:

cd BarcodeBenchmarkiOS
pod install

After installation, always open BarcodeBenchmark.xcworkspace, not the .xcodeproj file. DynamsoftCaptureVisionBundle (v11.4.1200) and zxing-cpp (v2.3.0) are declared in project.pbxproj and resolved automatically by Xcode on first build. Apple Vision requires no additional setup — it is part of the iOS SDK.

The Podfile targets iOS 16 and links ML Kit 8.0.0:

platform :ios, '16.0'

target 'BarcodeBenchmark' do
  use_frameworks!

  # Google ML Kit Barcode Scanning
  pod 'GoogleMLKit/BarcodeScanning', '8.0.0'
end

Step 2: Define a Shared BarcodeDetector Protocol

All four detectors implement the same two-method protocol. This keeps the benchmark logic SDK-agnostic and makes it straightforward to add or remove a backend in the future:

protocol BarcodeDetector {
    func detectBarcodes(in image: UIImage) async throws -> [BarcodeInfo]
    func detectBarcodes(in pixelBuffer: CVPixelBuffer) async throws -> [BarcodeInfo]
}

The UIImage overload is used for image and video-frame benchmarks; the CVPixelBuffer overload is called directly from the camera sample buffer delegate for live scanning.

Step 3: Implement the Dynamsoft Barcode Reader Detector

Dynamsoft uses CaptureVisionRouter from DynamsoftCaptureVisionBundle. License verification happens asynchronously on init; decoding is a single synchronous call to captureFromImage:

import DynamsoftCaptureVisionBundle
import DynamsoftBarcodeReaderBundle

class DynamsoftBarcodeDetector: NSObject, BarcodeDetector, LicenseVerificationListener {

    private let cvr: CaptureVisionRouter

    override init() {
        cvr = CaptureVisionRouter()
        super.init()
        LicenseManager.initLicense("YOUR_LICENSE_KEY_HERE", verificationDelegate: self)
    }

    func onLicenseVerified(_ isSuccess: Bool, error: Error?) {
        if !isSuccess {
            print("Dynamsoft license error: \(error?.localizedDescription ?? "Unknown error")")
        }
    }

    func detectBarcodes(in image: UIImage) async throws -> [BarcodeInfo] {
        let result = cvr.captureFromImage(image, templateName: "ReadBarcodes_Default")
        guard let items = result.decodedBarcodesResult?.items else { return [] }
        return items.map { item in
            let bounds: CGRect? = {
                guard image.size.width > 0, image.size.height > 0 else { return nil }
                let br = item.location.boundingRect
                return CGRect(
                    x: br.minX / image.size.width,
                    y: br.minY / image.size.height,
                    width: br.width / image.size.width,
                    height: br.height / image.size.height)
            }()
            return BarcodeInfo(format: item.formatString ?? "", text: item.text ?? "", decodeTimeMs: 0, normalizedBounds: bounds)
        }
    }

    func detectBarcodes(in pixelBuffer: CVPixelBuffer) async throws -> [BarcodeInfo] {
        let ciImage = CIImage(cvPixelBuffer: pixelBuffer)
        guard let cgImage = CIContext().createCGImage(ciImage, from: ciImage.extent) else {
            throw DetectionError.invalidImage
        }
        return try await detectBarcodes(in: UIImage(cgImage: cgImage))
    }
}

Step 4: Implement the ML Kit Barcode Detector

ML Kit uses an async-callback BarcodeScanner. The detector wraps the callback in withCheckedThrowingContinuation to fit the async throws protocol signature. Passing .all to BarcodeScannerOptions enables every supported format:

import MLKitBarcodeScanning
import MLKitVision

class MLKitBarcodeDetector: BarcodeDetector {

    private let scanner: MLKitBarcodeScanning.BarcodeScanner

    init() {
        let options = BarcodeScannerOptions(formats: .all)
        scanner = MLKitBarcodeScanning.BarcodeScanner.barcodeScanner(options: options)
    }

    func detectBarcodes(in image: UIImage) async throws -> [BarcodeInfo] {
        let visionImage = VisionImage(image: image)
        visionImage.orientation = image.imageOrientation
        let imgW = image.size.width
        let imgH = image.size.height
        return try await withCheckedThrowingContinuation { continuation in
            scanner.process(visionImage) { barcodes, error in
                if let error = error {
                    continuation.resume(throwing: DetectionError.detectionFailed(error.localizedDescription))
                    return
                }
                let results = (barcodes ?? []).map { barcode -> BarcodeInfo in
                    var bounds: CGRect?
                    if imgW > 0 && imgH > 0 {
                        let f = barcode.frame
                        bounds = CGRect(x: f.minX / imgW, y: f.minY / imgH,
                                        width: f.width / imgW, height: f.height / imgH)
                    }
                    return BarcodeInfo(format: barcode.format.description,
                                       text: barcode.rawValue ?? barcode.displayValue ?? "",
                                       decodeTimeMs: 0, normalizedBounds: bounds)
                }
                continuation.resume(returning: results)
            }
        }
    }

    func detectBarcodes(in pixelBuffer: CVPixelBuffer) async throws -> [BarcodeInfo] {
        let ciImage = CIImage(cvPixelBuffer: pixelBuffer)
        guard let cgImage = CIContext().createCGImage(ciImage, from: ciImage.extent) else {
            throw DetectionError.invalidImage
        }
        return try await detectBarcodes(in: UIImage(cgImage: cgImage))
    }
}

Step 5: Implement the Apple Vision Barcode Detector

Apple Vision requires no SDK installation. VNDetectBarcodesRequest is created per call and submitted via VNImageRequestHandler. Note that Vision's boundingBox origin is bottom-left, so the Y coordinate must be flipped before drawing overlays:

import Vision

class VisionBarcodeDetector: BarcodeDetector {

    func detectBarcodes(in image: UIImage) async throws -> [BarcodeInfo] {
        guard let cgImage = image.cgImage else {
            throw DetectionError.invalidImage
        }
        return try await withCheckedThrowingContinuation { continuation in
            let request = VNDetectBarcodesRequest { request, error in
                if let error = error {
                    continuation.resume(throwing: DetectionError.detectionFailed(error.localizedDescription))
                    return
                }
                guard let results = request.results as? [VNBarcodeObservation] else {
                    continuation.resume(returning: [])
                    return
                }
                let barcodes = results.map { observation -> BarcodeInfo in
                    let bb = observation.boundingBox
                    // boundingBox origin is bottom-left; flip Y for top-left screen coords
                    let bounds = CGRect(x: bb.minX, y: 1.0 - bb.maxY,
                                       width: bb.width, height: bb.height)
                    return BarcodeInfo(format: self.mapSymbology(observation.symbology),
                                       text: observation.payloadStringValue ?? "",
                                       decodeTimeMs: 0, normalizedBounds: bounds)
                }
                continuation.resume(returning: barcodes)
            }
            request.symbologies = [.qr, .code128, .code39, .code93, .ean8, .ean13,
                                    .upce, .pdf417, .aztec, .dataMatrix, .codabar, .itf14]
            let handler = VNImageRequestHandler(cgImage: cgImage, options: [:])
            do {
                try handler.perform([request])
            } catch {
                continuation.resume(throwing: DetectionError.detectionFailed(error.localizedDescription))
            }
        }
    }

    func detectBarcodes(in pixelBuffer: CVPixelBuffer) async throws -> [BarcodeInfo] {
        return try await withCheckedThrowingContinuation { continuation in
            let request = VNDetectBarcodesRequest { request, error in
                if let error = error {
                    continuation.resume(throwing: DetectionError.detectionFailed(error.localizedDescription))
                    return
                }
                guard let results = request.results as? [VNBarcodeObservation] else {
                    continuation.resume(returning: [])
                    return
                }
                let barcodes = results.map { observation -> BarcodeInfo in
                    let bb = observation.boundingBox
                    let bounds = CGRect(x: bb.minX, y: 1.0 - bb.maxY,
                                       width: bb.width, height: bb.height)
                    return BarcodeInfo(format: self.mapSymbology(observation.symbology),
                                       text: observation.payloadStringValue ?? "",
                                       decodeTimeMs: 0, normalizedBounds: bounds)
                }
                continuation.resume(returning: barcodes)
            }
            request.symbologies = [.qr, .code128, .code39, .code93, .ean8, .ean13,
                                    .upce, .pdf417, .aztec, .dataMatrix, .codabar, .itf14]
            let handler = VNImageRequestHandler(cvPixelBuffer: pixelBuffer, options: [:])
            do {
                try handler.perform([request])
            } catch {
                continuation.resume(throwing: DetectionError.detectionFailed(error.localizedDescription))
            }
        }
    }
}

Step 6: Implement the ZXing-CPP Barcode Detector

ZXing-CPP is integrated via SPM using the zxing-cpp package and its iOS ZXingCpp wrapper. The ZXIBarcodeReader class handles both CGImage and CVPixelBuffer directly, and bounding coordinates are extracted from the result's position quad:

import ZXingCpp

class ZXingCppBarcodeDetector: BarcodeDetector {

    private let reader: ZXIBarcodeReader

    init() {
        reader = ZXIBarcodeReader()
    }

    func detectBarcodes(in image: UIImage) async throws -> [BarcodeInfo] {
        guard let cgImage = image.cgImage else {
            throw DetectionError.invalidImage
        }
        let results = try reader.read(cgImage)
        let imgW = Int(image.size.width)
        let imgH = Int(image.size.height)
        return results.map { r in
            var bounds: CGRect?
            if imgW > 0 && imgH > 0 {
                let pos = r.position
                let xs = [pos.topLeft.x, pos.topRight.x, pos.bottomRight.x, pos.bottomLeft.x]
                let ys = [pos.topLeft.y, pos.topRight.y, pos.bottomRight.y, pos.bottomLeft.y]
                if let minX = xs.min(), let maxX = xs.max(),
                   let minY = ys.min(), let maxY = ys.max() {
                    bounds = CGRect(x: CGFloat(minX) / CGFloat(imgW),
                                    y: CGFloat(minY) / CGFloat(imgH),
                                    width: CGFloat(maxX - minX) / CGFloat(imgW),
                                    height: CGFloat(maxY - minY) / CGFloat(imgH))
                }
            }
            return BarcodeInfo(format: mapFormat(r.format), text: r.text,
                                decodeTimeMs: 0, normalizedBounds: bounds)
        }
    }

    func detectBarcodes(in pixelBuffer: CVPixelBuffer) async throws -> [BarcodeInfo] {
        let results = try reader.read(pixelBuffer)
        let bufW = CVPixelBufferGetWidth(pixelBuffer)
        let bufH = CVPixelBufferGetHeight(pixelBuffer)
        return results.map { r in
            var bounds: CGRect?
            if bufW > 0 && bufH > 0 {
                let pos = r.position
                let xs = [pos.topLeft.x, pos.topRight.x, pos.bottomRight.x, pos.bottomLeft.x]
                let ys = [pos.topLeft.y, pos.topRight.y, pos.bottomRight.y, pos.bottomLeft.y]
                if let minX = xs.min(), let maxX = xs.max(),
                   let minY = ys.min(), let maxY = ys.max() {
                    bounds = CGRect(x: CGFloat(minX) / CGFloat(bufW),
                                    y: CGFloat(minY) / CGFloat(bufH),
                                    width: CGFloat(maxX - minX) / CGFloat(bufW),
                                    height: CGFloat(maxY - minY) / CGFloat(bufH))
                }
            }
            return BarcodeInfo(format: mapFormat(r.format), text: r.text,
                                decodeTimeMs: 0, normalizedBounds: bounds)
        }
    }
}

Step 7: Extract and Benchmark Barcode Detection Over Video Frames

For video benchmarks, VideoFrameExtractor uses AVAssetImageGenerator to sample frames at a fixed interval (default 0.5 s). Each frame is decoded as a CGImage and wrapped into a UIImage before being passed to the detector chain. The tolerance is set to zero to ensure exact frame positions are decoded:

class VideoFrameExtractor {
    static func extractFrames(from videoURL: URL, interval: Double) async throws -> [UIImage] {
        let asset = AVAsset(url: videoURL)
        let duration = try await asset.load(.duration)
        let durationSeconds = CMTimeGetSeconds(duration)

        let imageGenerator = AVAssetImageGenerator(asset: asset)
        imageGenerator.appliesPreferredTrackTransform = true
        imageGenerator.requestedTimeToleranceBefore = .zero
        imageGenerator.requestedTimeToleranceAfter = .zero

        var frames: [UIImage] = []
        var currentTime: Double = 0

        while currentTime < durationSeconds {
            let cmTime = CMTime(seconds: currentTime, preferredTimescale: 600)
            if let cgImage = try? imageGenerator.copyCGImage(at: cmTime, actualTime: nil) {
                frames.append(UIImage(cgImage: cgImage))
            }
            currentTime += interval
        }
        return frames
    }
}

Once frames are extracted, each SDK runs sequentially. The benchmark records total processing time and accumulates unique barcode results (format:text as a deduplication key) across frames — so a barcode visible in five consecutive frames counts as one detection:

private func runDetectorBenchmark(
    detector: BarcodeDetector,
    engineName: String,
    frames: [UIImage],
    totalFrames: Int,
    startProgress: Double,
    endProgress: Double
) async {
    var allBarcodes: [BarcodeInfo] = []
    var uniqueBarcodes = Set<String>()
    var totalTimeMs: Int64 = 0

    for (index, frame) in frames.enumerated() {
        let startTime = Date()
        let barcodes = (try? await detector.detectBarcodes(in: frame)) ?? []
        totalTimeMs += Int64(Date().timeIntervalSince(startTime) * 1000)

        for barcode in barcodes {
            let key = "\(barcode.format):\(barcode.text)"
            if uniqueBarcodes.insert(key).inserted {
                allBarcodes.append(barcode)
            }
        }

        let frameProgress = Double(index + 1) / Double(totalFrames)
        let overallProgress = startProgress + (frameProgress * (endProgress - startProgress))
        await MainActor.run {
            progress = overallProgress
            statusMessage = "\(engineName): Frame \(index + 1)/\(totalFrames)"
        }
    }

    await MainActor.run {
        var result = BenchmarkResult(engineName: engineName)
        result.framesProcessed = totalFrames
        result.totalTimeMs = totalTimeMs
        result.barcodes = allBarcodes
        // assign to the appropriate viewModel property
    }
}

Step 8: Configure the Live Camera Feed with AVFoundation

CameraManager wraps AVCaptureSession, selects the rear wide-angle camera, and configures AVCaptureVideoDataOutput to push sample buffers to a per-SDK delegate. Resolution is selectable between 720p and 1080p from the home screen before starting any scanner:

func setupCamera(resolution: CameraResolution, delegate: AVCaptureVideoDataOutputSampleBufferDelegate) throws {
    let session = AVCaptureSession()

    switch resolution {
    case .hd720:
        session.sessionPreset = .hd1280x720
    case .hd1080:
        session.sessionPreset = .hd1920x1080
    }

    guard let videoDevice = AVCaptureDevice.default(.builtInWideAngleCamera, for: .video, position: .back) else {
        throw CameraError.noCameraAvailable
    }

    let videoDeviceInput = try AVCaptureDeviceInput(device: videoDevice)
    if session.canAddInput(videoDeviceInput) {
        session.addInput(videoDeviceInput)
    }

    videoOutput.setSampleBufferDelegate(delegate, queue: DispatchQueue(label: "videoQueue"))
    videoOutput.alwaysDiscardsLateVideoFrames = true
    if session.canAddOutput(videoOutput) {
        session.addOutput(videoOutput)
    }

    if let connection = videoOutput.connection(with: .video) {
        connection.videoOrientation = .portrait
    }

    self.session = session
}

Each live scanner view (DynamsoftScannerView, MLKitScannerView, VisionScannerView, ZXingCppScannerView) draws color-coded bounding-box overlays — blue, green, purple, and orange respectively — using normalized CGRect coordinates returned by the detectors.

Step 9: Run Remote Benchmarks via the Built-In HTTP Server

Why Remote Benchmarking Is Valuable for Mobile SDK Evaluation

Evaluating a barcode SDK properly requires a large, diverse image corpus — often dozens to hundreds of files. Transferring this dataset to a mobile device and triggering each test manually is slow and error-prone. The built-in HTTP server solves this: open the device's IP in a desktop browser, drag-and-drop files in batch, and let the device run detection autonomously using its real CPU, GPU, and Neural Engine. The advantage over emulators or simulator tests is that results reflect the actual hardware pipeline — including ML accelerator scheduling and thermal throttling — which matters when comparing SDKs with different compute backends.

A secondary benefit is reproducibility: the server processes files in a consistent order, runs each SDK on the exact same decoded image bytes, and returns a structured JSON response per file that the browser accumulates into a downloadable HTML report. This workflow was used to produce the benchmark data in this article.

The server starts with a single call and makes the device's local IP available as a URL:

func startWebServer() {
    guard webServer == nil else { return }
    do {
        let server = try BenchmarkWebServer(port: BenchmarkConfig.serverPort, viewModel: self)
        try server.start()
        webServer = server
        isWebServerRunning = true
        if let ip = getLocalIPAddress() {
            serverURL = "http://\(ip):\(BenchmarkConfig.serverPort)"
        }
    } catch {
        print("Failed to start web server: \(error)")
    }
}

For each uploaded file the server determines whether it is an image or video (via MIME type or byte sniffing), runs all four detectors in series, and returns a JSON object with per-SDK barcode counts and timing. Video files are sampled at up to 30 evenly-spaced frames. A real-time progress endpoint (GET /api/video-progress) lets the browser poll SDK-level progress during video processing.

Benchmark Results

All benchmark data was collected via the remote HTTP server using 83 images from the Dynamsoft Barcode Test Sheet — a publicly available set covering QR codes, Data Matrix, PDF417, Aztec, EAN/UPC, Code 128/39/93, Codabar, ITF, and other common linear and 2D symbologies at various image qualities and orientations.

Source Code

https://github.com/yushulx/ios-swiftui-barcode-mrz-document-scanner/tree/main/examples/BarcodeBenchmarkiOS

How to Build a Cross-Platform Barcode, MRZ, and Document Scanner with .NET MAUI Blazor

Xiao Ling — Fri, 10 Apr 2026 06:04:04 +0000

Extracting structured data from barcodes, passports, and physical documents typically demands separate native implementations for every platform. The Dynamsoft Capture Vision SDK bundles barcode reading, MRZ recognition, and document detection into one JavaScript-callable API, and .NET MAUI Blazor lets you host that API inside a cross-platform BlazorWebView — giving you a single Razor codebase that runs on Android, iOS, macOS, and Windows.

What you'll build: A .NET MAUI Blazor app named "Vision Scanner" that scans barcodes, reads MRZ fields from passports and IDs, and detects document boundaries with a perspective-correction editor, running on all four major platforms using Dynamsoft Capture Vision SDK v3.2.5000.

Demo Video: .NET MAUI Blazor Vision Scanner in Action

Prerequisites

.NET 8 SDK
Visual Studio 2022 17.8+ with the .NET MAUI workload installed
A Dynamsoft Capture Vision license key

Get a 30-day free trial license at dynamsoft.com/customer/license/trialLicense

Step 1: Add the Dynamsoft Capture Vision Bundle to the Blazor Host Page

The SDK ships as a single CDN bundle. Add it to wwwroot/index.html before the Blazor framework script so that the Dynamsoft global is available as soon as the WebView loads.

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no, viewport-fit=cover" />
    <title>Vision Scanner</title>
    <base href="/" />
    <link rel="stylesheet" href="css/bootstrap/bootstrap.min.css" />
    <link href="css/app.css" rel="stylesheet" />
    <link href="BarcodeScanner.styles.css" rel="stylesheet" />
    <script src="https://cdn.jsdelivr.net/npm/dynamsoft-capture-vision-bundle@3.2.5000/dist/dcv.bundle.min.js"></script>
    <script src="jsInterop.js"></script>
</head>
<body>
    <div class="status-bar-safe-area"></div>
    <div id="app">Loading...</div>
    <script src="_framework/blazor.webview.js" autostart="false"></script>
</body>
</html>

Step 2: Register Platform-Specific Services and Configure the Android WebView

The camera abstraction (ICameraService) is registered in MauiProgram.cs. On macOS, WKWebView does not support getUserMedia, so a native AVFoundation implementation is wired in instead. On Android, a custom BlazorWebViewHandler replaces the default WebChromeClient to grant the WebView camera permission at the OS level.

// MauiProgram.cs
using Microsoft.AspNetCore.Components.WebView.Maui;
using BarcodeScanner.Services;

#if ANDROID
using BarcodeScanner.Platforms.Android;
#endif

namespace BarcodeScanner;

public static class MauiProgram
{
    public static MauiApp CreateMauiApp()
    {
        var builder = MauiApp.CreateBuilder();
        builder
            .UseMauiApp<App>()
            .ConfigureFonts(fonts =>
            {
                fonts.AddFont("OpenSans-Regular.ttf", "OpenSansRegular");
            }).ConfigureMauiHandlers(handlers =>
            {
#if ANDROID
                handlers.AddHandler<BlazorWebView, MauiBlazorWebViewHandler>();
#endif
            });

        builder.Services.AddMauiBlazorWebView();
#if DEBUG
        builder.Services.AddBlazorWebViewDeveloperTools();
#endif

#if MACCATALYST
        builder.Services.AddSingleton<ICameraService, MacCameraService>();
#else
        builder.Services.AddSingleton<ICameraService, DefaultCameraService>();
#endif

        return builder.Build();
    }
}

The Android-specific MyWebChromeClient grants all WebView permission requests — including camera — so that getUserMedia resolves inside the Blazor page:

// Platforms/Android/MyWebChromeClient.cs
public class MyWebChromeClient : WebChromeClient
{
    private MainActivity _activity;

    public MyWebChromeClient(Context context)
    {
        _activity = context as MainActivity;
    }

    public override void OnPermissionRequest(PermissionRequest request)
    {
        try
        {
            request.Grant(request.GetResources());
            base.OnPermissionRequest(request);
        }
        catch (Exception ex)
        {
            Console.WriteLine(ex);
        }
    }

    public override bool OnShowFileChooser(global::Android.Webkit.WebView webView,
        IValueCallback filePathCallback, FileChooserParams fileChooserParams)
    {
        base.OnShowFileChooser(webView, filePathCallback, fileChooserParams);
        return _activity.ChooseFile(filePathCallback,
            fileChooserParams.CreateIntent(), fileChooserParams.Title);
    }
}

This handler is injected via the custom MauiBlazorWebViewHandler:

// Platforms/Android/MauiBlazorWebViewHandler.cs
public class MauiBlazorWebViewHandler : BlazorWebViewHandler
{
    protected override global::Android.Webkit.WebView CreatePlatformView()
    {
        var view = base.CreatePlatformView();
        view.SetWebChromeClient(new MyWebChromeClient(this.Context));
        return view;
    }
}

Step 3: Initialize the SDK with a License Key

The Index.razor page presents a license activation screen. When the user taps Activate, Blazor calls the JavaScript initSDK function via IJSRuntime. The function initializes the license, pre-loads the WASM modules for barcode (DBR), label recognition (DLR), and document detection (DDN), loads the MRZ character-recognition models, and creates a CaptureVisionRouter instance.

@* Index.razor — Activate button handler *@
private async Task Activate()
{
    if (string.IsNullOrWhiteSpace(licenseKey)) return;
    isActivating = true;
    statusMessage = "";
    StateHasChanged();

    try
    {
        isActivated = await JSRuntime.InvokeAsync<bool>(
            "jsFunctions.initSDK", objRef, licenseKey);
        statusMessage = isActivated
            ? "SDK activated successfully!"
            : "SDK activation failed.";
    }
    catch (Exception ex)
    {
        statusMessage = "Error: " + ex.Message;
    }

    isActivating = false;
    StateHasChanged();
}

The corresponding JavaScript receives the call and boots the SDK:

// wwwroot/jsInterop.js — initSDK
initSDK: async function (dotnetRef, licenseKey) {
    dotnetHelper = dotnetRef;
    toggleLoading(true);
    try {
        await Dynamsoft.License.LicenseManager.initLicense(licenseKey, true);
        Dynamsoft.Core.CoreModule.loadWasm(["DBR", "DLR", "DDN"]);

        parser = await Dynamsoft.DCP.CodeParser.createInstance();
        await Dynamsoft.DCP.CodeParserModule.loadSpec("MRTD_TD1_ID");
        await Dynamsoft.DCP.CodeParserModule.loadSpec("MRTD_TD2_FRENCH_ID");
        await Dynamsoft.DCP.CodeParserModule.loadSpec("MRTD_TD2_ID");
        await Dynamsoft.DCP.CodeParserModule.loadSpec("MRTD_TD2_VISA");
        await Dynamsoft.DCP.CodeParserModule.loadSpec("MRTD_TD3_PASSPORT");
        await Dynamsoft.DCP.CodeParserModule.loadSpec("MRTD_TD3_VISA");
        await Dynamsoft.CVR.CaptureVisionRouter.appendDLModelBuffer("MRZCharRecognition");
        await Dynamsoft.CVR.CaptureVisionRouter.appendDLModelBuffer("MRZTextLineRecognition");

        cvr = await Dynamsoft.CVR.CaptureVisionRouter.createInstance();

        isSDKReady = true;
        toggleLoading(false);
        return true;
    } catch (ex) {
        console.error(ex);
        toggleLoading(false);
        return false;
    }
},

Step 4: Decode Barcodes, MRZ, and Documents from Image Files

The Reader.razor page lets users pick an image from the device file system. After the file is selected, C# calls jsFunctions.loadAndDecodeFile, passing the <input> element ID and the active mode string ("barcode", "mrz", or "document"). The JavaScript side reads the file directly from the DOM, decodes it via cvr.capture(), and returns a JSON object containing the result text and any detected geometry.

The active template is selected from the mode string at call time:

// wwwroot/jsInterop.js — getTemplateName
function getTemplateName() {
    if (currentMode === 'barcode') return 'ReadBarcodes_Default';
    if (currentMode === 'mrz') return 'ReadMRZ';
    if (currentMode === 'document') return 'DetectDocumentBoundaries_Default';
    return 'ReadBarcodes_Default';
}

On the C# side, the Reader.razor component handles the file change event and parses the returned JSON to update the UI:

// Pages/Reader.razor — LoadImage
private async Task LoadImage(ChangeEventArgs e)
{
    result = "";
    showDocumentEditor = false;
    rectifiedImage = "";
    isLoading = true;
    StateHasChanged();

    try
    {
        // JS reads the file directly from the <input> element — no C# stream needed.
        var json = await JSRuntime.InvokeAsync<string>(
            "jsFunctions.loadAndDecodeFile", "reader_file_input", selectedMode);

        if (!string.IsNullOrEmpty(json))
        {
            using var doc = System.Text.Json.JsonDocument.Parse(json);
            var root = doc.RootElement;

            if (root.TryGetProperty("error", out var errProp))
            {
                result = "Error: " + errProp.GetString();
            }
            else
            {
                currentImageBase64 = root.TryGetProperty("image", out var imgProp)
                    ? imgProp.GetString() ?? "" : "";

                var decodeResult = root.TryGetProperty("result", out var resProp)
                    ? resProp.GetString() ?? "" : "";

For MRZ results, the JavaScript parser extracts structured fields from the raw MRZ string before returning them to C#:

// wwwroot/jsInterop.js — extractMrzInfo
function extractMrzInfo(result) {
    const parseResultInfo = {};
    parseResultInfo['Document Type'] = JSON.parse(result.jsonString).CodeType;
    parseResultInfo['Issuing State'] = result.getFieldValue("issuingState");
    parseResultInfo['Surname'] = result.getFieldValue("primaryIdentifier");
    parseResultInfo['Given Name'] = result.getFieldValue("secondaryIdentifier");
    let type = result.getFieldValue("documentCode");
    parseResultInfo['Passport Number'] = type === "P"
        ? result.getFieldValue("passportNumber")
        : result.getFieldValue("documentNumber");
    parseResultInfo['Nationality'] = result.getFieldValue("nationality");
    parseResultInfo["Gender"] = result.getFieldValue("sex");
    // ... date-of-birth and expiry fields
    return parseResultInfo;
}

Step 5: Stream the Live Camera Feed and Handle Platform Differences

Scanner.razor checks the injected ICameraService.RequiresNativeCamera flag to decide whether to use the standard getUserMedia web path or the macOS native AVFoundation path.

// Pages/Scanner.razor — OnAfterRenderAsync (first render)
if (useNativeCamera)
{
    await InitNativeCamera();
}
else
{
    await InitWebCamera();
}

Web camera path (Android, iOS, Windows): Blazor enumerates cameras via getUserMedia, opens the chosen device, and starts a requestAnimationFrame decode loop:

// Pages/Scanner.razor — InitWebCamera
private async Task InitWebCamera()
{
    cameraLabels = await JSRuntime.InvokeAsync<string[]>(
        "jsFunctions.initScanner", objRef, "camera_view", selectedMode);

    if (cameraLabels != null && cameraLabels.Length > 0)
    {
        StateHasChanged();
        await JSRuntime.InvokeVoidAsync("jsFunctions.openCamera", 0);
        await JSRuntime.InvokeVoidAsync("jsFunctions.startScanning", selectedMode);
    }
}

Native camera path (macOS): MacCameraService opens an AVCaptureSession, captures pixel buffers via AVCaptureVideoDataOutput, converts every second frame to a base64 JPEG, and exposes it through GetLatestFrame(). The Blazor component polls this method via a [JSInvokable] callback that the JavaScript timer calls on an interval:

// Pages/Scanner.razor — GetLatestFrame (JS-invokable poll endpoint)
[JSInvokable]
public string? GetLatestFrame()
{
    if (_disposed || !useNativeCamera) return null;
    return CameraService.GetLatestFrame();
}

Scan results from both paths flow back to C# through [JSInvokable] callbacks:

// wwwroot/jsInterop.js — processCameraResult (barcode branch)
if (currentMode === 'barcode' &&
    item.type === Dynamsoft.Core.EnumCapturedResultItemType.CRIT_BARCODE) {
    txts.push(item.text);
    globalPoints = item.location.points;
    if (overlayCtx) drawCameraOverlayQuad(overlayCtx, item.location.points, '#00ff00', 3);
}
// ...
if (currentMode === 'barcode' && txts.length > 0 && dotnetHelper) {
    dotnetHelper.invokeMethodAsync('OnScanResultReceived', txts.join('\n'));
}

Step 6: Rectify Captured Documents with a Browser-Side Homography

When Document mode is active and the user taps Capture, the frozen frame and the detected quad points are sent to C#. The Scanner.razor component opens an overlay editor where the user can drag the four corner handles to refine the boundary, then tap Rectify. The perspective warp is computed entirely in the browser using an inverse-mapping homography — no server round-trip.

// wwwroot/jsInterop.js — warpPerspective (inverse-mapping core)
function warpPerspective(srcCanvas, quadPts) {
    // ...
    const dstPts = [{ x: 0, y: 0 }, { x: W, y: 0 }, { x: W, y: H }, { x: 0, y: H }];
    const Hinv = buildHomography(dstPts, sp);   // inverse: dst → src

    for (let y = 0; y < H; y++) {
        for (let x = 0; x < W; x++) {
            const [sx, sy] = applyH(Hinv, x + 0.5, y + 0.5);
            // bilinear sample from source
            // ...
        }
    }
    dstCtx.putImageData(dstImg, 0, 0);
    return dstCanvas;
}

After rectification, the corrected image is displayed inline. The user can re-open the quad editor by tapping Edit, or export the image via jsFunctions.saveDocument, which triggers the native OS share sheet on Android/iOS or a file-save dialog on Windows and macOS.

Source Code

https://github.com/yushulx/dotnet-maui-blazor-barcode-mrz-document-scanner

Decode Barcodes and QR Codes from Uploaded Images in ASP.NET MVC 5

Xiao Ling — Tue, 07 Apr 2026 08:00:13 +0000

Reading barcodes and QR codes on the server side is a common requirement for document processing, inventory systems, and logistics applications. With the Dynamsoft Barcode Reader SDK and ASP.NET MVC 5 on .NET Framework 4.8, you can add accurate, multi-format barcode decoding to a web application in a few focused steps. The SDK's CaptureVisionRouter API handles the heavy lifting on the backend, while a lightweight JavaScript frontend previews the image and draws color-coded overlays over every detected code.

What you'll build: A responsive ASP.NET MVC 5 web application that accepts an uploaded image, decodes all barcodes and QR codes server-side using the Dynamsoft Barcode Reader SDK 11.4, and renders canvas overlays with format and text results in the browser.

See the ASP.NET MVC Barcode and QR Code Reader Demo

Prerequisites

.NET Framework 4.8 Developer Pack
Visual Studio 2019 or later with the ASP.NET and web development workload
NuGet CLI or the Visual Studio NuGet Package Manager
A Dynamsoft Barcode Reader license key

Get a 30-day free trial license

Step 1: Install and Configure the SDK in ASP.NET MVC 5

Add the Dynamsoft.DotNet.BarcodeReader.Bundle NuGet package to the project. The bundle ships with native binaries for both x64 and x86 Windows, which are automatically copied to the output directory via the included MSBuild targets file.

<!-- packages.config -->
<packages>
  <package id="Dynamsoft.DotNet.BarcodeReader.Bundle" version="11.4.2000" targetFramework="net48" />
  <package id="Microsoft.AspNet.Mvc"      version="5.2.9" targetFramework="net48" />
  <package id="Newtonsoft.Json" version="13.0.3" targetFramework="net48" />
</packages>

Restore packages from the command line before the first build:

nuget restore MvcBarcodeQRCodeFramework.csproj -PackagesDirectory packages

Step 2: Enable Attribute Routing for the Upload Endpoint

The barcode upload endpoint uses MVC attribute routing ([Route("upload")]). Enable it in RouteConfig.cs before the conventional route:

public static void RegisterRoutes(RouteCollection routes)
{
    routes.IgnoreRoute("{resource}.axd/{*pathInfo}");

    // Enable attribute routing so [Route("upload")] works in FileController
    routes.MapMvcAttributeRoutes();

    routes.MapRoute(
        name: "Default",
        url: "{controller}/{action}/{id}",
        defaults: new { controller = "Home", action = "Index", id = UrlParameter.Optional }
    );
}

Step 3: Initialize the License and Decode Barcodes Server-Side

FileController accepts a multipart/form-data POST, saves each uploaded file to ~/App_Data/uploads/, initializes the license once per request, then calls CaptureVisionRouter.Capture() for each file. The decoded results — text, format string, and four-corner location — are serialized to JSON and returned to the browser.

using Dynamsoft.DBR;
using Dynamsoft.License;
using Dynamsoft.CVR;
using Dynamsoft.Core;

public class FileController : Controller
{
    [HttpPost]
    [Route("upload")]
    public ActionResult Upload()
    {
        var files = Request.Files;
        var uploadPath = Server.MapPath("~/App_Data/uploads");

        if (!Directory.Exists(uploadPath))
            Directory.CreateDirectory(uploadPath);

        string errorMsg;
        int errorCode = LicenseManager.InitLicense(
            "LICENSE-KEY",
            out errorMsg);

        if (errorCode != (int)EnumErrorCode.EC_OK &&
            errorCode != (int)EnumErrorCode.EC_LICENSE_CACHE_USED)
        {
            return Json(new { error = "License error: " + errorMsg });
        }

        var barcodes = new List<object>();

        using (var cvr = new CaptureVisionRouter())
        {
            for (int i = 0; i < files.Count; i++)
            {
                HttpPostedFileBase file = files[i];
                if (file == null || file.ContentLength == 0) continue;

                string fileName = Path.GetFileName(file.FileName);
                string filePath = Path.Combine(uploadPath, fileName);
                file.SaveAs(filePath);

                CapturedResult result = cvr.Capture(filePath, PresetTemplate.PT_READ_BARCODES);
                DecodedBarcodesResult barcodesResult = result.GetDecodedBarcodesResult();
                if (barcodesResult == null) continue;

                foreach (BarcodeResultItem item in barcodesResult.GetItems())
                {
                    var loc = item.GetLocation();
                    barcodes.Add(new
                    {
                        text   = item.GetText(),
                        format = item.GetFormatString(),
                        points = new[]
                        {
                            new { x = loc.points[0][0], y = loc.points[0][1] },
                            new { x = loc.points[1][0], y = loc.points[1][1] },
                            new { x = loc.points[2][0], y = loc.points[2][1] },
                            new { x = loc.points[3][0], y = loc.points[3][1] },
                        }
                    });
                }
            }
        }

        return Json(new { barcodes });
    }
}

PresetTemplate.PT_READ_BARCODES selects the SDK's built-in multi-format template, which covers QR Code, Code 128, Code 39, PDF417, DataMatrix, Aztec Code, and many more without any additional configuration.

Step 4: Upload the Image and Draw Canvas Overlays in the Browser

The frontend uses a plain XMLHttpRequest to POST the file as multipart/form-data as soon as an image is selected or dropped. FileReader creates a data-URL preview, and once the server responds, each barcode's four points are scaled from the image's natural pixel dimensions to its rendered size and drawn as a filled, stroked quadrilateral on an absolutely-positioned <canvas> overlay.

function handleFile(file) {
    var reader = new FileReader();
    reader.addEventListener('load', function () {
        previewArea.style.display = 'flex';
        imageEl.src = reader.result;
        imageEl.onload = function () {
            syncCanvas();
            upload(file);
        };
    });
    reader.readAsDataURL(file);
}

function upload(file) {
    spinnerEl.style.display = 'flex';
    var formData = new FormData();
    formData.append('barcodeImage', file, file.name);

    var xhr = new XMLHttpRequest();
    xhr.open('POST', '/upload', true);
    xhr.onreadystatechange = function () {
        if (xhr.readyState !== 4) return;
        spinnerEl.style.display = 'none';
        if (xhr.status === 200) {
            try {
                showResults(JSON.parse(xhr.responseText));
            } catch (e) {
                showError('Failed to parse server response.');
            }
        } else {
            showError('Upload failed (HTTP ' + xhr.status + ').');
        }
    };
    xhr.send(formData);
}

var PALETTE = ['#00b4d8', '#ff6b6b', '#51cf66', '#fcc419', '#cc5de8', '#ff922b'];

function drawQuad(points, sx, sy, index) {
    var color = PALETTE[index % PALETTE.length];
    ctx.save();
    ctx.strokeStyle = color;
    ctx.lineWidth   = 2.5;
    ctx.fillStyle   = color + '30';
    ctx.beginPath();
    ctx.moveTo(points[0].x * sx, points[0].y * sy);
    for (var i = 1; i < points.length; i++)
        ctx.lineTo(points[i].x * sx, points[i].y * sy);
    ctx.closePath();
    ctx.fill();
    ctx.stroke();

    var cx = (points[0].x + points[1].x + points[2].x + points[3].x) / 4 * sx;
    var cy = (points[0].y + points[1].y + points[2].y + points[3].y) / 4 * sy;
    ctx.fillStyle = color;
    ctx.beginPath();
    ctx.arc(cx, cy, 12, 0, Math.PI * 2);
    ctx.fill();
    ctx.fillStyle = '#fff';
    ctx.font = 'bold 12px sans-serif';
    ctx.textAlign = 'center';
    ctx.textBaseline = 'middle';
    ctx.fillText(index + 1, cx, cy);
    ctx.restore();
}

sx and sy are scale factors computed as imageEl.offsetWidth / imageEl.naturalWidth (and the height equivalent), ensuring overlays remain in sync regardless of how the browser scales the image to fit the viewport.

Step 5: Build and Run with IIS Express

Build the project from the command line or open it in Visual Studio. To run without Visual Studio's debug launcher, call IIS Express directly:

msbuild MvcBarcodeQRCodeFramework.csproj /p:Configuration=Debug /t:Build
"C:\Program Files\IIS Express\iisexpress.exe" /path:"%CD%" /port:62873

Then open http://localhost:62873/ in a browser, drop or select an image with barcodes, and results appear instantly.

Source Code

https://github.com/yushulx/dotnet-barcode-qr-code-sdk/tree/main/example/official/aspnet-framework

How to Build a Flutter Document Scanner App with Edge Detection, Editing, and PDF Export

Xiao Ling — Thu, 02 Apr 2026 09:06:44 +0000

Paper forms, receipts, and ID cards are still part of many mobile workflows, but camera captures are often skewed, low quality, or inconsistent. This project solves that by combining real-time document detection, perspective correction, and multi-page export in one Flutter app. It runs on Android and iOS using the Dynamsoft Capture Vision SDK.

What you'll build: A Flutter mobile document scanner that detects and deskews pages in real time, supports quad editing and drag-and-drop page sorting, and exports results as images or PDF with Dynamsoft Capture Vision SDK.

Demo Video: Flutter Mobile Document Scanner in Action

Prerequisites

Flutter SDK 3.8+
Dart SDK 3.8+
Android API 21+ and iOS 13+
A valid Dynamsoft Capture Vision license key
Dependencies from pubspec.yaml, including dynamsoft_capture_vision_flutter: ^3.2.5000

Get a 30-day free trial license at dynamsoft.com/customer/license/trialLicense

Step 1: Install and Configure the SDK

Add the Dynamsoft package in pubspec.yaml and keep your license key in a shared constants file so initialization is centralized.

dependencies:
  flutter:
    sdk: flutter

  cupertino_icons: ^1.0.8

  dynamsoft_capture_vision_flutter: ^3.2.5000

  path_provider: ^2.1.5

  image_picker: ^1.1.2

  image_gallery_saver_plus: ^4.0.1

  pdf: ^3.11.1

  permission_handler: ^11.3.1

  open_filex: ^4.6.0

class AppConstants {
  AppConstants._();

  static const String licenseKey = 'LICENSE-KEY';
  static const String appName = 'Document Scanner';
  static const Duration snackBarDuration = Duration(seconds: 4);
  static const String exportFilePrefix = 'docscan_';
}

Step 2: Initialize Camera Input and Dynamsoft Router

The scanner page sets up CaptureVisionRouter, applies a cross-frame filter, initializes the license, then starts camera capture with the detectAndNormalizeDocument template.

Future<void> _initSdk() async {
  _cvr = CaptureVisionRouter.instance
    ..addResultFilter(
      MultiFrameResultCrossFilter()
        ..enableResultCrossVerification(
          EnumCapturedResultItemType.deskewedImage.value,
          true,
        ),
    );

  _receiver = CapturedResultReceiver()
    ..onProcessedDocumentResultReceived =
        (ProcessedDocumentResult result) async {
      if (_isProcessing || _cooldown) return;
      final items = result.deskewedImageResultItems;
      if (items == null || items.isEmpty) return;

      final item = items.first;

      _latestDeskewedItem = item;
      _latestOriginalImageHashId = result.originalImageHashId;

      if (_isBtnClicked) {
        _isBtnClicked = false;
        _captureTimeoutTimer?.cancel();
        _captureTimeoutTimer = null;
        _captureResult(item, result.originalImageHashId, autoCapture: false);
      } else if (item.crossVerificationStatus ==
          EnumCrossVerificationStatus.passed) {
        _quadStabilizer.feedQuad(item.sourceDeskewQuad);
      }
    };

  final (isSuccess, message) =
      await LicenseManager.initLicense(AppConstants.licenseKey);
  if (!isSuccess && mounted) {
    setState(() => _licenseError = message);
    return;
  }

  try {
    await _cvr.setInput(_camera);
    _cvr.addResultReceiver(_receiver);
    await _camera.open();
    await _cvr.startCapturing(_template);
    if (mounted) setState(() => _isSdkReady = true);
  } catch (e) {
    if (mounted) setState(() => _licenseError = e.toString());
  }
}

Step 3: Stabilize Auto-Capture and Add Manual Fallback

To avoid noisy captures, the app waits for stable quads across consecutive frames; manual capture also has a 500 ms timeout fallback to raw frame capture when no document is found.

class QuadStabilizer {
  double iouThreshold;
  double areaDeltaThreshold;
  int stableFrameCount;
  bool autoCaptureEnabled;

  Quadrilateral? _previousQuad;
  int _consecutiveStableFrames = 0;
  void Function()? onStable;

  QuadStabilizer({
    this.iouThreshold = 0.85,
    this.areaDeltaThreshold = 0.15,
    this.stableFrameCount = 3,
    this.autoCaptureEnabled = true,
    this.onStable,
  });

  void feedQuad(Quadrilateral quad) {
    if (!autoCaptureEnabled) return;

    if (_previousQuad == null) {
      _previousQuad = quad;
      _consecutiveStableFrames = 0;
      return;
    }

    final double iou = calculateIoU(_previousQuad!, quad);
    final double prevArea = _calculateQuadArea(_previousQuad!);
    final double currArea = _calculateQuadArea(quad);
    final double areaDelta =
        prevArea > 0 ? (currArea - prevArea).abs() / prevArea : 1.0;

    if (iou >= iouThreshold && areaDelta <= areaDeltaThreshold) {
      _consecutiveStableFrames++;
      if (_consecutiveStableFrames >= stableFrameCount) {
        onStable?.call();
        reset();
      }
    } else {
      _consecutiveStableFrames = 0;
    }

    _previousQuad = quad;
  }
}

void _onCapturePressed() {
  if (!_isSdkReady || _isProcessing || _cooldown) return;
  _isBtnClicked = true;
  _latestDeskewedItem = null;
  _latestOriginalImageHashId = null;
  _captureTimeoutTimer?.cancel();
  _captureTimeoutTimer = Timer(const Duration(milliseconds: 500), () {
    if (_isBtnClicked) {
      _isBtnClicked = false;
      _captureRawFrame();
    }
  });
}

Step 4: Import from Gallery and Handle No-Detection Cases

When scanning files from the gallery, the app tries captureFile first and gracefully falls back to the original image if no document is detected.

Future<void> _pickFromGallery() async {
  final picker = ImagePicker();
  final XFile? file = await picker.pickImage(source: ImageSource.gallery);
  if (file == null) return;

  if (mounted) setState(() => _isProcessing = true);

  try {
    final result = await _cvr.captureFile(file.path, _template);

    final docResult = result.processedDocumentResult;
    final items = docResult?.deskewedImageResultItems;

    if (items != null && items.isNotEmpty) {
      final item = items.first;
      final originalImage = await _cvr
          .getIntermediateResultManager()
          .getOriginalImage(result.originalImageHashId);

      final page = DocumentPage(
        originalImage: originalImage,
        normalizedImage: item.imageData!,
        quad: item.sourceDeskewQuad,
      );
      if (mounted) setState(() => _pages.add(page));
    } else {
      final originalImage = await ImageIO().readFromFile(file.path);
      if (originalImage != null && mounted) {
        ScaffoldMessenger.of(context).showSnackBar(
          const SnackBar(
              content: Text('No document detected. Using original image.')),
        );
        final page = DocumentPage(
          originalImage: originalImage,
          normalizedImage: originalImage,
          quad: null,
        );
        setState(() => _pages.add(page));
      }
    }
  } catch (e) {
    if (mounted) {
      ScaffoldMessenger.of(context).showSnackBar(
        SnackBar(content: Text('Failed to process image: $e')),
      );
    }
  } finally {
    if (mounted) setState(() => _isProcessing = false);
  }
}

Step 5: Add Interactive Quad Editing Before Export

The result page opens an edit screen for the current page, then applies cropAndDeskewImage using the updated quadrilateral selected by the user.

void _editQuad() {
  final page = _pages[_currentIndex];
  if (!page.hasOriginalImage || page.quad == null) return;

  Navigator.push<Map<String, dynamic>>(
    context,
    MaterialPageRoute(
      builder: (_) => EditPage(
        originalImageData: page.originalImage!,
        quad: page.quad!,
      ),
    ),
  ).then((result) {
    if (result != null && mounted) {
      setState(() {
        if (result['croppedImageData'] != null &&
            result['updatedQuad'] != null) {
          page.updateFromQuadEdit(
            result['croppedImageData'] as ImageData,
            result['updatedQuad'] as Quadrilateral,
          );
        }
      });
    }
  });
}

Future<void> _applyEdit() async {
  if (_controller == null || _isApplying) return;
  setState(() => _isApplying = true);

  try {
    final selectedQuad = await _controller!.getSelectedQuad();
    final croppedImageData = await ImageProcessor()
        .cropAndDeskewImage(widget.originalImageData, selectedQuad);

    if (mounted) {
      Navigator.pop(context, {
        'croppedImageData': croppedImageData,
        'updatedQuad': selectedQuad,
      });
    }
  } catch (e) {
    if (mounted) {
      setState(() => _isApplying = false);
      ScaffoldMessenger.of(context).showSnackBar(
        SnackBar(
          content: const Text(
            'The selected area is not a valid quadrilateral. '
            'Please drag the corners to form a proper rectangle.',
          ),
          duration: AppConstants.snackBarDuration,
        ),
      );
    }
  }
}

Step 6: Enable Drag-and-Drop Page Reordering

For multi-page workflows, the app opens a dedicated sorting view and uses ReorderableListView.builder so users can drag pages into the final PDF/image order.

void _sortPages() {
  Navigator.push<List<DocumentPage>>(
    context,
    MaterialPageRoute(
      builder: (_) => SortPagesPage(pages: List.from(_pages)),
    ),
  ).then((reordered) {
    if (reordered != null && mounted) {
      setState(() {
        _pages.clear();
        _pages.addAll(reordered);
        _currentIndex = 0;
        _pageController.jumpToPage(0);
      });
    }
  });
}

Expanded(
  child: ReorderableListView.builder(
    padding: const EdgeInsets.all(8),
    itemCount: _workingList.length,
    onReorder: (oldIndex, newIndex) {
      setState(() {
        if (newIndex > oldIndex) newIndex--;
        final item = _workingList.removeAt(oldIndex);
        _workingList.insert(newIndex, item);
      });
    },
    itemBuilder: (ctx, index) {
      return _SortPageItem(
        key: ValueKey(index),
        page: _workingList[index],
        pageNumber: index + 1,
      );
    },
  ),
)

Step 7: Export Scanned Pages to Gallery or PDF

The result screen supports two output paths: save each page to the system gallery as PNG, or generate a multi-page A4 PDF in the app documents directory.

Future<void> _exportImages() async {
  if (_isSaving) return;
  setState(() => _isSaving = true);
  try {
    if (Platform.isAndroid) {
      final status = await Permission.photos.request();
      if (!status.isGranted) {
        await Permission.storage.request();
      }
    }

    final timestamp = DateTime.now().millisecondsSinceEpoch;
    int saved = 0;

    for (int i = 0; i < _pages.length; i++) {
      final bytes = await _pages[i].getDisplayBytes();
      if (bytes == null) continue;

      final fileName = '${AppConstants.exportFilePrefix}${timestamp}_${i + 1}.png';
      final result = await ImageGallerySaverPlus.saveImage(
        bytes,
        quality: 95,
        name: fileName,
      );

      if (result != null && result['isSuccess'] == true) {
        saved++;
      }
    }

    if (mounted) {
      ScaffoldMessenger.of(context).showSnackBar(
        SnackBar(
          content: Text('$saved image(s) saved to gallery'),
          duration: AppConstants.snackBarDuration,
        ),
      );
    }
  } catch (e) {
    if (mounted) {
      ScaffoldMessenger.of(context).showSnackBar(
        SnackBar(content: Text('Failed to save images: $e')),
      );
    }
  } finally {
    if (mounted) setState(() => _isSaving = false);
  }
}

Future<void> _exportPdf() async {
  if (_isSaving) return;
  setState(() => _isSaving = true);
  try {
    final pdf = pw.Document();

    for (int i = 0; i < _pages.length; i++) {
      final bytes = await _pages[i].getDisplayBytes();
      if (bytes == null) continue;

      final image = pw.MemoryImage(bytes);

      pdf.addPage(
        pw.Page(
          pageFormat: PdfPageFormat.a4,
          build: (pw.Context context) {
            return pw.Center(
              child: pw.Image(image, fit: pw.BoxFit.contain),
            );
          },
        ),
      );
    }

    final dir = await getApplicationDocumentsDirectory();
    final documentsDir = Directory('${dir.path}/documents');
    if (!documentsDir.existsSync()) {
      documentsDir.createSync(recursive: true);
    }

    final timestamp = DateTime.now().millisecondsSinceEpoch;
    final fileName = '${AppConstants.exportFilePrefix}$timestamp.pdf';
    final file = File('${documentsDir.path}/$fileName');
    final pdfBytes = await pdf.save();
    await file.writeAsBytes(pdfBytes);

    if (mounted) {
      ScaffoldMessenger.of(context).showSnackBar(
        SnackBar(
          content: Text('PDF saved: ${file.path}'),
          duration: AppConstants.snackBarDuration,
        ),
      );
    }

    await OpenFilex.open(file.path);
  } catch (e) {
    if (mounted) {
      ScaffoldMessenger.of(context).showSnackBar(
        SnackBar(content: Text('Failed to export PDF: $e')),
      );
    }
  } finally {
    if (mounted) setState(() => _isSaving = false);
  }
}

Source Code

https://github.com/yushulx/flutter-barcode-mrz-document-scanner/tree/main/examples/multi_document_capture

How to Build a JavaScript Multi-Page Document Scanner Web App with Auto-Capture and PDF Export

Xiao Ling — Thu, 02 Apr 2026 06:21:23 +0000

Scanning physical documents with a phone or laptop camera—and getting a clean, de-skewed result instantly—is a common requirement in expense-management, healthcare, and contract-signing workflows. The Dynamsoft Capture Vision SDK handles the heavy lifting (boundary detection, perspective normalization) entirely in the browser via WebAssembly, so no server-side component is needed.

What you'll build: A vanilla-JavaScript web app that opens the device camera, automatically detects document boundaries frame-by-frame, captures and normalizes multi-page documents, lets users edit corners, apply image filters, reorder pages, and export the result as a PDF or individual PNGs — powered by Dynamsoft Capture Vision Bundle v3.2.5000.

Online Demo

https://yushulx.me/javascript-barcode-qr-code-scanner/examples/multi-document-capture/

Demo Video: Multi-Page Document Scanner in Action

Prerequisites

A modern browser with camera access (Chrome, Edge, Safari, Firefox).
Node.js and any static HTTP server (Python's http.server, npx serve, etc.) for local development.
A Dynamsoft Capture Vision license key.

Get a 30-day free trial license for Dynamsoft Capture Vision.

Step 1: Load the SDK via CDN

The entire SDK is delivered as a single bundle from jsDelivr — no npm install required. Add both the Dynamsoft Capture Vision bundle and jsPDF to the <head> of index.html:

<script src="https://cdn.jsdelivr.net/npm/dynamsoft-capture-vision-bundle@3.2.5000/dist/dcv.bundle.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/jspdf@2.5.1/dist/jspdf.umd.min.js"></script>

The bundle includes the Document Detection & Normalization (DDN) module, the CaptureVisionRouter, and the LicenseManager — everything needed to go from raw camera frames to normalized document images.

Step 2: Initialize the License and CaptureVisionRouter

After the user submits a license key on the start screen, initSDK activates the license, loads the DDN WebAssembly module, creates a CaptureVisionRouter instance, and loads the preset template file that defines the detection and normalization tasks:

const TEMPLATE_PATH = "./DBR_and_DDN_detect_PresetTemplates.json";
const DETECT_TEMPLATE = "DetectDocumentBoundaries_Default";
const NORMALIZE_TEMPLATE = "NormalizeDocument_Default";

async function initSDK(licenseKey) {
    updateInitStatus("Activating license...");
    await Dynamsoft.License.LicenseManager.initLicense(licenseKey, true);

    updateInitStatus("Loading modules...");
    await Dynamsoft.Core.CoreModule.loadWasm(["DDN"]);

    updateInitStatus("Initializing scanner...");
    cvr = await Dynamsoft.CVR.CaptureVisionRouter.createInstance();
    await cvr.initSettings(TEMPLATE_PATH);

    updateInitStatus("Opening camera...");
    await initCamera();
}

The initSettings call registers DetectDocumentBoundaries_Default and NormalizeDocument_Default as named tasks so they can be invoked by name later.

Step 3: Run the Frame-by-Frame Detection Loop

The scan loop calls cvr.capture() on each animation frame, passing the live <video> element and the detection template name. When a four-point quad is returned, it is drawn on an overlay <canvas> and fed into the stabilizer:

async function scanLoop() {
    if (!isScanning) return;

    if (isProcessingFrame || isCaptureInProgress) {
        scanLoopId = requestAnimationFrame(scanLoop);
        return;
    }

    isProcessingFrame = true;
    try {
        const result = await cvr.capture(videoElement, DETECT_TEMPLATE);
        let quad = null;

        for (const item of result.items || []) {
            if (item.location && item.location.points && item.location.points.length === 4) {
                quad = item.location.points;
                break;
            }
        }

        if (quad) {
            latestDetectedQuad = quad;
            drawOverlay(quad);
            // ... stabilizer logic and auto-capture
        } else {
            clearOverlay();
            resetStabilizer();
        }
    } catch (_) {
        clearOverlay();
    }

    isProcessingFrame = false;
    if (isScanning) {
        scanLoopId = requestAnimationFrame(scanLoop);
    }
}

The isProcessingFrame guard prevents concurrent captures on devices where WebAssembly inference takes longer than a single animation frame.

Step 4: Stabilize the Detected Quad Before Auto-Capture

Firing a capture the instant a quad is detected produces blurry or mis-framed results. The stabilizer compares consecutive quads using Intersection over Union (IoU) and an area-delta check; only after stableFrameCount consecutive stable frames is the capture triggered automatically:

const quadStabilizer = {
    enabled: true,
    iouThreshold: 0.85,
    areaDeltaThreshold: 0.15,
    stableFrameCount: 3,
};

function isQuadStable(current, previous) {
    if (!current || !previous || current.length !== 4 || previous.length !== 4) return false;
    const boxA = pointsToBoundingBox(current);
    const boxB = pointsToBoundingBox(previous);
    const iou = computeIoU(boxA, boxB);

    const areaA = polygonArea(current);
    const areaB = polygonArea(previous);
    const areaDelta = areaB === 0 ? 1 : Math.abs(areaA - areaB) / areaB;

    return iou >= quadStabilizer.iouThreshold && areaDelta <= quadStabilizer.areaDeltaThreshold;
}

All three thresholds (iouThreshold, areaDeltaThreshold, stableFrameCount) are exposed in an in-app settings panel so users can tune them for their environment.

Step 5: Normalize the Document with Perspective Correction

Once a stable quad is confirmed (or the user taps the capture button), normalizeDocument projects the detected boundary points back into cvr to perform perspective de-skew and returns the corrected image as a <canvas>:

async function normalizeDocument(frameCanvas, points) {
    try {
        const settings = await cvr.getSimplifiedSettings(NORMALIZE_TEMPLATE);
        settings.roi.points = points;
        settings.roiMeasuredInPercentage = 0;
        await cvr.updateSettings(NORMALIZE_TEMPLATE, settings);

        const normalizedResult = await cvr.capture(frameCanvas, NORMALIZE_TEMPLATE);
        for (const item of normalizedResult.items || []) {
            if (item.toCanvas && typeof item.toCanvas === "function") {
                return item.toCanvas();
            }
        }
        return null;
    } catch (_) {
        return null;
    }
}

Setting roiMeasuredInPercentage = 0 tells the router that settings.roi.points are pixel coordinates in the source frame, not percentages.

Step 6: Apply Image Filters Per Page

After capture, each page can be rendered in color, grayscale, or binary mode. The filter is applied client-side using getImageData / putImageData on a scratch canvas, leaving the original baseCanvas untouched:

function buildFilteredCanvas(page) {
    const src = page.baseCanvas;
    const out = document.createElement("canvas");
    out.width = src.width;
    out.height = src.height;
    const ctx = out.getContext("2d", { willReadFrequently: true });
    ctx.drawImage(src, 0, 0);

    if (page.filter === "color") {
        return out;
    }

    const image = ctx.getImageData(0, 0, out.width, out.height);
    const data = image.data;
    for (let i = 0; i < data.length; i += 4) {
        const gray = Math.round(data[i] * 0.299 + data[i + 1] * 0.587 + data[i + 2] * 0.114);
        if (page.filter === "grayscale") {
            data[i] = gray;
            data[i + 1] = gray;
            data[i + 2] = gray;
        } else {
            const binary = gray > 140 ? 255 : 0;
            data[i] = binary;
            data[i + 1] = binary;
            data[i + 2] = binary;
        }
    }
    ctx.putImageData(image, 0, 0);
    return out;
}

Step 7: Edit the Document Quad After Capture

If the auto-detected corners are inaccurate, the user can open the edit screen for any captured page, drag the four corner handles to the correct positions, and re-apply perspective normalization without recapturing. openEdit loads the stored originalCanvas and quadPoints into editState, then renderEditCanvas draws the image and overlays draggable 14 px circular handles:

function openEdit() {
    const page = pages[currentPageIndex];
    if (!page.originalCanvas || !page.quadPoints) {
        showToast("No quad data available for editing.");
        return;
    }
    editState = {
        originalCanvas: page.originalCanvas,
        quadPoints: page.quadPoints.map(p => ({ x: p.x, y: p.y })),
        draggingCorner: -1,
        imgRect: null,
    };
    resultScreen.classList.add("hidden");
    editScreen.classList.remove("hidden");
    requestAnimationFrame(renderEditCanvas);
}

function renderEditCanvas() {
    if (!editState) return;
    const rect = editCanvasEl.getBoundingClientRect();
    const cw = Math.round(rect.width);
    const ch = Math.round(rect.height);
    if (cw === 0 || ch === 0) return;
    editCanvasEl.width = cw;
    editCanvasEl.height = ch;

    const img = editState.originalCanvas;
    const scale = Math.min(cw / img.width, ch / img.height);
    const imgW = Math.round(img.width * scale);
    const imgH = Math.round(img.height * scale);
    const imgX = Math.round((cw - imgW) / 2);
    const imgY = Math.round((ch - imgH) / 2);
    editState.imgRect = { x: imgX, y: imgY, w: imgW, h: imgH };

    editCtx2.clearRect(0, 0, cw, ch);
    editCtx2.drawImage(img, imgX, imgY, imgW, imgH);

    const pts = editState.quadPoints.map(p => ({
        x: imgX + (p.x / img.width) * imgW,
        y: imgY + (p.y / img.height) * imgH,
    }));

    editCtx2.beginPath();
    editCtx2.moveTo(pts[0].x, pts[0].y);
    for (let i = 1; i < 4; i++) editCtx2.lineTo(pts[i].x, pts[i].y);
    editCtx2.closePath();
    editCtx2.fillStyle = "rgba(106,196,187,0.2)";
    editCtx2.fill();
    editCtx2.strokeStyle = "#6ac4bb";
    editCtx2.lineWidth = 2;
    editCtx2.stroke();

    const HANDLE_R = 14;
    pts.forEach((p, i) => {
        editCtx2.beginPath();
        editCtx2.arc(p.x, p.y, HANDLE_R, 0, Math.PI * 2);
        editCtx2.fillStyle = i === editState.draggingCorner ? "#fe8e14" : "#6ac4bb";
        editCtx2.fill();
        editCtx2.strokeStyle = "#fff";
        editCtx2.lineWidth = 2;
        editCtx2.stroke();
    });
}

When the user taps Apply, applyEdit calls normalizeDocument with the updated corner coordinates and replaces the stored baseCanvas in-place — the page is corrected without affecting any other page:

async function applyEdit() {
    if (!editState) return;
    editApplyBtn.disabled = true;
    try {
        const normalized = await normalizeDocument(editState.originalCanvas, editState.quadPoints);
        if (!normalized) {
            showToast("Normalization failed. Adjust corners and try again.");
            return;
        }
        pages[currentPageIndex].baseCanvas = copyCanvas(normalized);
        pages[currentPageIndex].quadPoints = editState.quadPoints.map(p => ({ x: p.x, y: p.y }));
        editState = null;
        editScreen.classList.add("hidden");
        resultScreen.classList.remove("hidden");
        renderResult();
        updateThumbnailBar();
    } catch (e) {
        console.error(e);
        showToast("Edit failed: " + (e?.message || "Unknown error"));
    } finally {
        editApplyBtn.disabled = false;
    }
}

Step 8: Reorder Pages with Drag-and-Drop

For multi-page sessions, the sort overlay lets users drag thumbnail cards into the desired order. The implementation supports both mouse (dragstart / drop) and touch (touchstart / touchmove / touchend) events. A floating clone follows the finger during a touch drag, and document.elementFromPoint identifies the drop target:

function openSort() {
    if (pages.length < 2) {
        showToast("Need at least 2 pages to reorder.");
        return;
    }
    const workingOrder = pages.map((_, i) => i);
    let dragIdx = -1;

    function rebuild() {
        sortList.innerHTML = "";
        workingOrder.forEach((pageIdx, pos) => {
            const item = document.createElement("div");
            item.className = "sort-item";
            item.draggable = true;
            item.dataset.pos = pos;

            item.addEventListener("dragstart", (e) => {
                dragIdx = pos;
                item.classList.add("dragging");
                e.dataTransfer.effectAllowed = "move";
            });
            item.addEventListener("drop", (e) => {
                e.preventDefault();
                item.classList.remove("drag-over");
                const dropPos = Number(item.dataset.pos);
                if (dragIdx >= 0 && dragIdx !== dropPos) {
                    const [moved] = workingOrder.splice(dragIdx, 1);
                    workingOrder.splice(dropPos, 0, moved);
                    rebuild();
                }
            });

            // Touch drag support
            let touchClone = null;
            item.addEventListener("touchstart", (e) => {
                dragIdx = pos;
                const touch = e.touches[0];
                touchClone = item.cloneNode(true);
                touchClone.style.cssText = `position:fixed;left:${touch.clientX - 30}px;top:${touch.clientY - 24}px;width:${item.offsetWidth}px;opacity:0.8;z-index:200;pointer-events:none`;
                document.body.appendChild(touchClone);
                item.classList.add("dragging");
            }, { passive: true });
            item.addEventListener("touchend", (e) => {
                item.classList.remove("dragging");
                if (touchClone) { touchClone.remove(); touchClone = null; }
                const touch = e.changedTouches[0];
                const overItem = document.elementFromPoint(touch.clientX, touch.clientY)?.closest(".sort-item");
                if (overItem) {
                    const dropPos = Number(overItem.dataset.pos);
                    if (dragIdx >= 0 && dragIdx !== dropPos) {
                        const [moved] = workingOrder.splice(dragIdx, 1);
                        workingOrder.splice(dropPos, 0, moved);
                        rebuild();
                    }
                }
                dragIdx = -1;
            });
            sortList.appendChild(item);
        });
    }

    rebuild();
    sortOverlay.classList.remove("hidden");

    sortDoneBtn.onclick = () => {
        pages = workingOrder.map(i => pages[i]);
        currentPageIndex = 0;
        sortOverlay.classList.add("hidden");
        renderResult();
        updateThumbnailBar();
    };
}

When the user taps Done, workingOrder (an index remapping array) is applied to pages in a single .map() call, making the reorder non-destructive until confirmed.

Step 9: Export the Scanned Pages as a PDF

All captured pages (with filters applied) are packed into a single multi-page PDF using jsPDF. Each page is centered and scaled to fit A4:

async function exportPdf() {
    if (!pages.length || !window.jspdf) return;
    const { jsPDF } = window.jspdf;
    const pdf = new jsPDF({ unit: "pt", format: "a4" });

    for (let i = 0; i < pages.length; i++) {
        const canvas = buildFilteredCanvas(pages[i]);
        const imageData = canvas.toDataURL("image/jpeg", 0.95);
        if (i > 0) pdf.addPage("a4", "portrait");

        const pageWidth = pdf.internal.pageSize.getWidth();
        const pageHeight = pdf.internal.pageSize.getHeight();
        const ratio = Math.min(pageWidth / canvas.width, pageHeight / canvas.height);
        const drawWidth = canvas.width * ratio;
        const drawHeight = canvas.height * ratio;
        const x = (pageWidth - drawWidth) / 2;
        const y = (pageHeight - drawHeight) / 2;

        pdf.addImage(imageData, "JPEG", x, y, drawWidth, drawHeight);
    }

    const blob = pdf.output("blob");
    const blobUrl = URL.createObjectURL(blob);
    window.open(blobUrl, "_blank", "noopener,noreferrer");
}

Individual pages can also be downloaded as PNG files via the Export as Images option, which calls canvas.toDataURL("image/png") for each page.

Source Code

https://github.com/yushulx/javascript-barcode-qr-code-scanner/tree/main/examples/multi-document-capture

How to Build an Android Document Scanner with Auto-Capture and PDF Export

Xiao Ling — Wed, 01 Apr 2026 07:36:21 +0000

Scanning physical documents with a phone camera sounds simple until you deal with skewed angles, inconsistent lighting, and multi-page workflows. The Dynamsoft Capture Vision SDK for Android handles real-time document boundary detection, perspective correction, and image normalization — letting you focus on the user experience instead of low-level image processing.

What you'll build: A full-featured Android document scanning app in Java that auto-captures documents via quad stabilization, supports gallery import, quad editing, image filters (color/grayscale/binary), page sorting, rotation, and multi-page PDF and JPEG export — all powered by Dynamsoft Capture Vision SDK.

Demo Video: Android Document Scanner in Action

Prerequisites

Android Studio (Arctic Fox or later)
Android SDK with compileSdk 35 and minSdk 23
Java 11 (source and target compatibility)
Dynamsoft Capture Vision SDK — com.dynamsoft:capturevisionbundle:3.4.1000
A physical Android device (camera-based features do not work on the emulator)

Get a 30-day free trial license for Dynamsoft Capture Vision SDK.

Step 1: Add the Dynamsoft Maven Repository and SDK Dependency

Dynamsoft packages are hosted on a custom Maven repository. Add it to your root build.gradle:

allprojects {
    repositories {
        google()
        mavenCentral()
        maven { url "https://download2.dynamsoft.com/maven/aar" }
    }
}

Then declare the SDK dependency in your module-level build.gradle:

dependencies {
    implementation "com.dynamsoft:capturevisionbundle:3.4.1000"

    implementation 'androidx.appcompat:appcompat:1.7.1'
    implementation 'com.google.android.material:material:1.11.0'
    implementation 'androidx.activity:activity:1.8.2'
    implementation 'androidx.viewpager2:viewpager2:1.0.0'
    implementation 'androidx.exifinterface:exifinterface:1.3.7'
}

Step 2: Initialize the License and Set Up the Camera

In your ScannerFragment, initialize the Dynamsoft license and set up CameraEnhancer with CaptureVisionRouter. The router connects to the camera as its input source and starts processing frames using the document detection template.

@Override
public View onCreateView(@NonNull LayoutInflater inflater, @Nullable ViewGroup container, @Nullable Bundle savedInstanceState) {
    PermissionUtil.requestCameraPermission(requireActivity());

    mViewModel = new ViewModelProvider(requireActivity()).get(DocumentScannerViewModel.class);
    mViewModel.actionBarTitle.setValue(requireContext().getString(R.string.scan_page_title));

    if (savedInstanceState == null) {
        LicenseManager.initLicense("LICENSE-KEY", (isSuccess, error) -> {
            if (!isSuccess && error != null) {
                error.printStackTrace();
            }
        });
    }
    return inflater.inflate(R.layout.fragment_scanner, container, false);
}

Wire up the camera and capture vision router in onViewCreated:

CameraView cameraView = view.findViewById(R.id.cameraView);
mCamera = new CameraEnhancer(cameraView, getViewLifecycleOwner());
mRouter = new CaptureVisionRouter();

MultiFrameResultCrossFilter filter = new MultiFrameResultCrossFilter();
filter.enableResultCrossVerification(EnumCapturedResultItemType.CRIT_DESKEWED_IMAGE, true);
mRouter.addResultFilter(filter);

try {
    mRouter.setInput(mCamera);
} catch (CaptureVisionRouterException e) {
    e.printStackTrace();
    return;
}

Start and stop capturing in sync with the fragment lifecycle:

@Override
public void onResume() {
    super.onResume();
    mCamera.open();
    mRouter.startCapturing(EnumPresetTemplate.PT_DETECT_AND_NORMALIZE_DOCUMENT, new CompletionListener() {
        @Override
        public void onSuccess() { }

        @Override
        public void onFailure(int errorCode, String errorString) {
            mViewModel.startCapturingError.postValue(errorString);
        }
    });
}

@Override
public void onPause() {
    super.onPause();
    mCamera.close();
    mRouter.stopCapturing();
}

Step 3: Receive Detection Results and Enable Auto-Capture

Register a CapturedResultReceiver to handle detected documents. Each frame that contains a deskewed document image is either consumed by manual capture or fed to a QuadStabilizer for automatic capture when the document boundary remains stable across consecutive frames.

mRouter.addResultReceiver(new CapturedResultReceiver() {
    @Override
    public void onProcessedDocumentResultReceived(@NonNull ProcessedDocumentResult result) {
        if (result.getDeskewedImageResultItems().length > 0) {
            DeskewedImageResultItem item = result.getDeskewedImageResultItems()[0];

            mLatestDeskewedItem = item;
            mLatestOriginalImageHashId = result.getOriginalImageHashId();

            if (mIsBtnClicked) {
                mIsBtnClicked = false;
                mHandler.removeCallbacks(mCaptureTimeoutRunnable);
                captureResult(item, result.getOriginalImageHashId());
            } else if (item.getCrossVerificationStatus() == EnumCrossVerificationStatus.CVS_PASSED) {
                Quadrilateral quad = item.getSourceDeskewQuad();
                if (quad != null) {
                    mQuadStabilizer.feedQuad(quad);
                }
            }
        }
    }
});

The QuadStabilizer compares consecutive quad detections using IoU (Intersection over Union) and area-delta thresholds. When the boundary stays stable for a configurable number of frames, it triggers auto-capture:

public void feedQuad(Quadrilateral quad) {
    if (!autoCaptureEnabled) {
        return;
    }

    if (previousQuad == null) {
        previousQuad = quad;
        consecutiveStableFrames = 0;
        return;
    }

    float iou = calculateIoU(previousQuad, quad);
    double prevArea = calculateQuadArea(previousQuad);
    double currArea = calculateQuadArea(quad);
    float areaDelta = prevArea > 0 ? (float) Math.abs(currArea - prevArea) / (float) prevArea : 1.0f;

    if (iou >= iouThreshold && areaDelta <= areaDeltaThreshold) {
        consecutiveStableFrames++;
        if (consecutiveStableFrames >= stableFrameCount && callback != null) {
            callback.onStable();
            reset();
        }
    } else {
        consecutiveStableFrames = 0;
    }

    previousQuad = quad;
}

Step 4: Capture and Store Normalized Document Pages

When a document is captured — manually or via auto-capture — the normalized image, original frame, and detected quad are bundled into a DocumentPage and stored in a shared ViewModel:

private void captureResult(DeskewedImageResultItem item, String originalImageHashId) {
    if (mCooldown) return;
    mCooldown = true;

    ImageData normalizedImage = item.getImageData();
    Quadrilateral quad = item.getSourceDeskewQuad();
    ImageData originalImage = mRouter.getIntermediateResultManager().getOriginalImage(originalImageHashId);

    DocumentPage page = new DocumentPage(originalImage, normalizedImage, quad);

    if (mIsRetakeMode && mRetakeIndex >= 0) {
        mViewModel.replacePage(mRetakeIndex, page);
        mViewModel.retakePageIndex.postValue(-1);
        mHandler.post(() -> requireActivity().getSupportFragmentManager().popBackStack());
    } else {
        mViewModel.addPage(page);
    }

    mQuadStabilizer.reset();
    mHandler.postDelayed(() -> mCooldown = false, 1500);
}

For manual capture, if no document boundary is detected within 500ms, a raw camera frame is captured as a fallback:

private void captureRawFrame() {
    if (mCooldown) return;
    mCooldown = true;

    try {
        ImageData frame = mCamera.getImage();
        if (frame != null) {
            DocumentPage page = new DocumentPage(frame, frame, null);
            if (mIsRetakeMode && mRetakeIndex >= 0) {
                mViewModel.replacePage(mRetakeIndex, page);
                mViewModel.retakePageIndex.postValue(-1);
                mHandler.post(() -> requireActivity().getSupportFragmentManager().popBackStack());
            } else {
                mViewModel.addPage(page);
            }
        }
    } catch (Exception e) {
        e.printStackTrace();
    }

    mHandler.postDelayed(() -> mCooldown = false, 1500);
}

Step 5: Edit Document Boundaries with the Quad Editor

Users can drag the detected document corners to adjust the crop region. The EditFragment uses Dynamsoft's ImageEditorView to display the original image overlaid with a draggable quad. On apply, a perspective transform produces the corrected output:

private void applyEdit() {
    DocumentPage page = mViewModel.getPage(mPageIndex);
    if (page == null || !page.hasOriginalImage()) {
        requireActivity().getSupportFragmentManager().popBackStack();
        return;
    }

    DrawingLayer layer = mEditorView.getDrawingLayer(DrawingLayer.DDN_LAYER_ID);
    List<DrawingItem> items = layer.getDrawingItems();

    Quadrilateral newQuad = null;
    for (DrawingItem item : items) {
        if (item instanceof QuadDrawingItem) {
            newQuad = ((QuadDrawingItem) item).getQuad();
            break;
        }
    }

    if (newQuad == null) {
        requireActivity().getSupportFragmentManager().popBackStack();
        return;
    }

    try {
        Bitmap originalBitmap = page.getOriginalImage().toBitmap();
        Bitmap deskewed = DocumentPage.perspectiveTransform(originalBitmap, newQuad);
        page.updateFromQuadEdit(deskewed, newQuad);
        mViewModel.notifyPagesChanged();
    } catch (CoreException e) {
        e.printStackTrace();
        Toast.makeText(requireContext(), "Failed to apply edit", Toast.LENGTH_SHORT).show();
    }

    requireActivity().getSupportFragmentManager().popBackStack();
}

Step 6: Apply Image Filters and Export to PDF or JPEG

Each DocumentPage supports color mode toggling between color, grayscale, and binary. The ResultFragment lets users apply filters, rotate pages, reorder via drag-and-drop, and export:

mBtnFilterColor.setOnClickListener(v -> applyFilter(EnumImageColourMode.ICM_COLOUR));
mBtnFilterGrayscale.setOnClickListener(v -> applyFilter(EnumImageColourMode.ICM_GRAYSCALE));
mBtnFilterBinary.setOnClickListener(v -> applyFilter(EnumImageColourMode.ICM_BINARY));

PDF export iterates over all captured pages and writes them into an Android PdfDocument, scaling to A4 dimensions:

public static Uri exportToPdf(Context context, List<DocumentPage> pages) throws IOException, CoreException {
    if (pages == null || pages.isEmpty()) return null;

    PdfDocument pdfDocument = new PdfDocument();

    try {
        for (int i = 0; i < pages.size(); i++) {
            DocumentPage page = pages.get(i);
            Bitmap bitmap = page.getDisplayBitmap();
            if (bitmap == null) continue;

            int pageWidth = bitmap.getWidth();
            int pageHeight = bitmap.getHeight();

            float scale = 1.0f;
            if (pageWidth > 2480 || pageHeight > 3508) {
                scale = Math.min(2480f / pageWidth, 3508f / pageHeight);
                pageWidth = Math.round(pageWidth * scale);
                pageHeight = Math.round(pageHeight * scale);
            }

            PdfDocument.PageInfo pageInfo = new PdfDocument.PageInfo.Builder(pageWidth, pageHeight, i + 1).create();
            PdfDocument.Page pdfPage = pdfDocument.startPage(pageInfo);

            Canvas canvas = pdfPage.getCanvas();
            if (scale != 1.0f) {
                canvas.scale(scale, scale);
            }
            canvas.drawBitmap(bitmap, 0, 0, null);

            pdfDocument.finishPage(pdfPage);
        }

        String timestamp = new SimpleDateFormat("yyyyMMdd_HHmmss", Locale.getDefault()).format(new Date());
        String fileName = "DocScan_" + timestamp + ".pdf";

        File documentsDir = new File(context.getFilesDir(), "documents");
        if (!documentsDir.exists()) {
            documentsDir.mkdirs();
        }

        File pdfFile = new File(documentsDir, fileName);
        FileOutputStream fos = new FileOutputStream(pdfFile);
        pdfDocument.writeTo(fos);
        fos.flush();
        fos.close();

        return FileProvider.getUriForFile(context,
                context.getPackageName() + ".fileprovider", pdfFile);
    } finally {
        pdfDocument.close();
    }
}

Step 7: Detect and Normalize Documents from Gallery Images

Users can import images from the gallery. The app reads EXIF orientation, applies rotation correction, and feeds the corrected bitmap to the capture vision router for document detection:

private void processGalleryImage(Uri imageUri) {
    try {
        int exifRotation = 0;
        try (InputStream exifStream = requireContext().getContentResolver().openInputStream(imageUri)) {
            if (exifStream != null) {
                ExifInterface exif = new ExifInterface(exifStream);
                int orientation = exif.getAttributeInt(
                        ExifInterface.TAG_ORIENTATION, ExifInterface.ORIENTATION_NORMAL);
                switch (orientation) {
                    case ExifInterface.ORIENTATION_ROTATE_90:  exifRotation = 90;  break;
                    case ExifInterface.ORIENTATION_ROTATE_180: exifRotation = 180; break;
                    case ExifInterface.ORIENTATION_ROTATE_270: exifRotation = 270; break;
                }
            }
        } catch (IOException ignored) { }

        Bitmap bitmap;
        try (InputStream decodeStream = requireContext().getContentResolver().openInputStream(imageUri)) {
            if (decodeStream == null) return;
            bitmap = BitmapFactory.decodeStream(decodeStream);
        }
        if (bitmap == null) return;

        if (exifRotation != 0) {
            android.graphics.Matrix m = new android.graphics.Matrix();
            m.postRotate(exifRotation);
            Bitmap rotated = Bitmap.createBitmap(bitmap, 0, 0, bitmap.getWidth(), bitmap.getHeight(), m, true);
            if (rotated != bitmap) bitmap.recycle();
            bitmap = rotated;
        }

        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        bitmap.compress(Bitmap.CompressFormat.JPEG, 95, baos);
        byte[] jpegBytes = baos.toByteArray();

        CapturedResult capturedResult = mRouter.capture(jpegBytes, EnumPresetTemplate.PT_DETECT_AND_NORMALIZE_DOCUMENT);
        if (capturedResult != null) {
            ProcessedDocumentResult docResult = capturedResult.getProcessedDocumentResult();
            if (docResult != null && docResult.getDeskewedImageResultItems().length > 0) {
                DeskewedImageResultItem deskewedItem = docResult.getDeskewedImageResultItems()[0];
                ImageData normalizedImage = deskewedItem.getImageData();
                Quadrilateral quad = deskewedItem.getSourceDeskewQuad();
                DocumentPage page = new DocumentPage(null, normalizedImage, quad);
                bitmap.recycle();
                mViewModel.addPage(page);
                return;
            }
        }

        Toast.makeText(requireContext(), R.string.no_document_detected, Toast.LENGTH_SHORT).show();
        DocumentPage page = new DocumentPage(bitmap);
        mViewModel.addPage(page);
    } catch (Exception e) {
        e.printStackTrace();
    }
}

Source Code

https://github.com/yushulx/android-camera-barcode-mrz-document-scanner/tree/main/examples/DynamsoftDocumentScanner

How to Build a React Native MRZ Passport Scanner for Android and iOS

Xiao Ling — Tue, 24 Mar 2026 02:15:19 +0000

Verifying travel documents manually is error-prone and slow. The Machine Readable Zone (MRZ) printed on passports, ID cards, and other ICAO-compliant travel documents encodes all key identity fields in a structured, machine-readable format — but extracting it on mobile requires both an accurate OCR engine and a live camera pipeline. The Dynamsoft MRZ Scanner SDK wraps both into a single React Native call that works on Android and iOS.

What you'll build: A React Native app (Android API 21+ / iOS 13+) that opens a live camera scanner, reads the MRZ from any TD1/TD2/TD3 travel document, and displays the parsed fields — name, document number, nationality, dates — using dynamsoft-mrz-scanner-bundle-react-native v3.2.5002.

Demo Video: React Native MRZ Scanner in Action

Prerequisites

Before you start, make sure you have the following:

Node.js ≥ 18
React Native 0.79 (uses the New Architecture)
Android Studio (latest) with a connected device or emulator running Android 5.0 (API 21)+
Xcode (latest, macOS only) for iOS 13.0+ builds
JDK 17
A Dynamsoft MRZ Scanner license key — the project ships with a time-limited trial key that requires a network connection.

Get a 30-day free trial license at dynamsoft.com/customer/license/trialLicense

Step 1: Install and Configure the SDK

Install the project dependencies with npm. The two Dynamsoft packages — the MRZ scanner bundle and the underlying Capture Vision engine — are declared in package.json and will be installed automatically.

npm install

The relevant SDK dependencies in package.json:

"dependencies": {
  "dynamsoft-capture-vision-react-native": "3.2.5002",
  "dynamsoft-mrz-scanner-bundle-react-native": "3.2.5002",
  "react": "19.0.0",
  "react-native": "0.79.0"
}

For iOS, install the CocoaPods dependencies after npm install:

cd ios && pod install && cd ..

Step 2: Configure the License and Launch the MRZ Scanner

Import MRZScanner, MRZScanConfig, and EnumResultStatus from the bundle package, then call MRZScanner.launch(config) with your license key. The SDK takes over the camera, presents its built-in scanning UI, and returns only when the user finishes or cancels.

import {
  EnumResultStatus,
  MRZScanConfig,
  MRZScanner,
} from 'dynamsoft-mrz-scanner-bundle-react-native';

const handleScan = async () => {
  setScanState({kind: 'scanning'});
  try {
    const config: MRZScanConfig = {
      // Trial license — network connection required.
      // Request an extension at:
      // https://www.dynamsoft.com/customer/license/trialLicense/?product=dcv&package=cross-platform
      license:
        'LICENSE-KEY',
    };
    const result = await MRZScanner.launch(config);
    // ... handle result
  } catch (e: unknown) {
    const message = e instanceof Error ? e.message : String(e);
    setScanState({kind: 'error', code: -1, message});
  }
};

Step 3: Handle Scan Result States

MRZScanner.launch() resolves with a result object whose resultStatus maps to one of three EnumResultStatus values. Check the status before accessing result.data to distinguish a successful scan from a user cancellation or an SDK-level error.

if (result.resultStatus === EnumResultStatus.RS_FINISHED) {
  setScanState({kind: 'result', data: result.data as Record<string, string>});
} else if (result.resultStatus === EnumResultStatus.RS_CANCELED) {
  setScanState({kind: 'cancelled'});
} else {
  setScanState({
    kind: 'error',
    code: result.errorCode ?? -1,
    message: result.errorString ?? 'An unknown error occurred.',
  });
}

Step 4: Display Parsed MRZ Fields in the UI

When the status is RS_FINISHED, result.data is a flat key-value map of parsed MRZ fields. The sample app promotes high-value fields — firstName, lastName, documentNumber, documentType — to a "hero" section and lists the rest in a scrollable card.

const HERO_FIELDS = ['firstName', 'lastName', 'documentNumber', 'documentType'];

function ResultCard({data}: ResultCardProps) {
  const heroEntries = HERO_FIELDS.map(k => [k, data[k]] as [string, string]).filter(
    ([, v]) => v != null && v !== '',
  );
  const otherEntries = Object.entries(data).filter(
    ([k, v]) => !HERO_FIELDS.includes(k) && v != null && v !== '',
  );

  return (
    <View style={styles.resultCard}>
      {heroEntries.length > 0 && (
        <View style={styles.heroSection}>
          {heroEntries.map(([key, value]) => (
            <View key={key} style={styles.heroRow}>
              <Text style={styles.heroLabel}>{formatLabel(key)}</Text>
              <Text style={styles.heroValue}>{value}</Text>
            </View>
          ))}
        </View>
      )}
      {/* other fields omitted for brevity */}
    </View>
  );
}

Step 5: Handle the Android Hardware Back Button

On Android the hardware back button exits the React Native app by default. The sample app intercepts it with BackHandler so users return to the idle state rather than closing the app mid-workflow.

useEffect(() => {
  const subscription = BackHandler.addEventListener('hardwareBackPress', () => {
    if (scanState.kind !== 'idle') {
      setScanState({kind: 'idle'});
      return true; // event handled — prevent default (app exit)
    }
    return false; // let the system handle it (idle → exit as normal)
  });
  return () => subscription.remove();
}, [scanState.kind]);

Source Code

https://github.com/yushulx/android-camera-barcode-mrz-document-scanner/tree/main/examples/react-native-mrz-scanner

How to Build a React Native Document Scanner with Auto-Detection, Crop, and Export

Xiao Ling — Mon, 23 Mar 2026 06:57:26 +0000

Turning a phone camera into a reliable document scanner requires accurate edge detection, perspective correction, and clean image output — all in real time. The Dynamsoft Capture Vision React Native SDK handles the heavy lifting, letting you ship a cross-platform document scanner for Android and iOS without writing any native code yourself.

What you'll build: A React Native app that auto-detects document edges from the live camera feed, lets users fine-tune the crop with draggable corner handles, apply color modes (full color, grayscale, binary), and export the result as a high-quality PNG.

Demo Video: React Native Document Scanner in Action

Prerequisites

Before starting, make sure you have:

Node.js 18+ and npm
React Native CLI (not Expo) with React Native 0.79+
Android Studio with an emulator or physical device (Android)
Xcode 15+ with CocoaPods (iOS)
A Dynamsoft Capture Vision trial or full license key

Get a 30-day free trial license at dynamsoft.com/customer/license/trialLicense

Step 1: Create the React Native Project and Install Dependencies

Start by creating a new React Native project (or use an existing one) and installing the required packages:

npx @react-native-community/cli init ScanDocument
cd ScanDocument
npm install dynamsoft-capture-vision-react-native @react-navigation/native @react-navigation/native-stack react-native-safe-area-context react-native-screens react-native-fs

For iOS, install the native pods:

cd ios && pod install && cd ..

The key dependency is dynamsoft-capture-vision-react-native (v3.4.1000), which bundles the camera enhancer, capture vision router, and document normalizer into a single React Native package.

Step 2: Initialize the License and Configure Navigation

The app entry point registers the root component in index.js:

import {AppRegistry} from 'react-native';
import App from './src/App';
import {name as appName} from './app.json';

AppRegistry.registerComponent(appName, () => App);

In App.tsx, set up a stack navigator with four screens and initialize the Dynamsoft license when the home screen mounts:

import {Quadrilateral, ImageData, LicenseManager} from 'dynamsoft-capture-vision-react-native';
import {createNativeStackNavigator, NativeStackScreenProps} from '@react-navigation/native-stack';
import {NavigationContainer} from '@react-navigation/native';

export type ScreenNames = ['Home', 'Scanner', 'Editor', 'NormalizedImage'];
export type RootStackParamList = Record<ScreenNames[number], undefined>;
export type StackNavigation = NativeStackScreenProps<RootStackParamList>;

const Stack = createNativeStackNavigator<RootStackParamList>();

function App(): React.JSX.Element {
  return (
    <SafeAreaProvider>
      <NavigationContainer>
        <Stack.Navigator initialRouteName="Home">
          <Stack.Screen name="Home" component={HomeScreen} options={{headerShown: false}} />
          <Stack.Screen name="Scanner" component={Scanner} options={{headerShown: false}} />
          <Stack.Screen name="Editor" component={Editor}
            options={{title: 'Adjust & Crop', headerStyle: {backgroundColor: '#2563EB'}, headerTintColor: '#fff'}} />
          <Stack.Screen name="NormalizedImage" component={NormalizedImage}
            options={{title: 'Review & Export', headerStyle: {backgroundColor: '#2563EB'}, headerTintColor: '#fff'}} />
        </Stack.Navigator>
      </NavigationContainer>
    </SafeAreaProvider>
  );
}

License initialization happens inside the HomeScreen component. Replace the license string with your own key:

useEffect(() => {
  LicenseManager.initLicense('LICENSE-KEY')
    .then(() => setLicenseReady(true))
    .catch(e => {
      console.error('Init license failed: ' + e.message);
      setError('License initialization failed.\n' + e.message);
      setLicenseReady(true);
    });
}, []);

Step 3: Detect and Capture Documents from the Camera Feed

The Scanner screen opens the camera, runs real-time document detection, and auto-captures when a stable document boundary is confirmed. The SDK's CameraEnhancer, CaptureVisionRouter, and MultiFrameResultCrossFilter work together:

import {
  CameraEnhancer,
  CameraView,
  CaptureVisionRouter,
  EnumCapturedResultItemType,
  EnumCrossVerificationStatus,
  EnumPresetTemplate,
  MultiFrameResultCrossFilter,
} from 'dynamsoft-capture-vision-react-native';

Open the camera when the screen is focused and close it when it loses focus:

const cameraRef = useRef<CameraEnhancer>(CameraEnhancer.getInstance());
const cvrRef = useRef<CaptureVisionRouter>(CaptureVisionRouter.getInstance());

useFocusEffect(
  React.useCallback(() => {
    const camera = cameraRef.current;
    camera.open();
    return () => {
      camera.close();
    };
  }, []),
);

Wire the camera to the capture vision router and enable cross-frame verification to filter out false positives:

if (!sdkInitialized) {
  cvr.setInput(camera);
  const filter = new MultiFrameResultCrossFilter();
  filter.enableResultCrossVerification(EnumCapturedResultItemType.CRIT_DESKEWED_IMAGE, true);
  cvr.addFilter(filter);
  sdkInitialized = true;
}

Register a result receiver that fires when a deskewed document image is ready. The capture triggers either through cross-verification passing or a manual shutter tap:

receiverRef.current = cvr.addResultReceiver({
  onProcessedDocumentResultReceived: result => {
    if (
      result.deskewedImageResultItems &&
      result.deskewedImageResultItems.length > 0 &&
      (ifBtnClick.current || result.deskewedImageResultItems[0].crossVerificationStatus === EnumCrossVerificationStatus.CVS_PASSED)
    ) {
      ifBtnClick.current = false;
      global.originalImage = cvr.getIntermediateResultManager().getOriginalImage(result.originalImageHashId) as ImageData;
      global.deskewedImage = result.deskewedImageResultItems[0].imageData;
      global.sourceDeskewQuad = result.deskewedImageResultItems[0].sourceDeskewQuad;
      if (global.originalImage.width > 0 && global.originalImage.height > 0) {
        navigation.navigate('NormalizedImage');
      }
    }
  },
});

cvr.startCapturing(EnumPresetTemplate.PT_DETECT_AND_NORMALIZE_DOCUMENT);

Start capturing with the built-in PT_DETECT_AND_NORMALIZE_DOCUMENT template — no custom template configuration needed.

Step 4: Fine-Tune the Document Crop with Draggable Corners

The Editor screen uses ImageEditorView to display the original image with a draggable quadrilateral overlay. Users drag the corner handles to fine-tune the document boundary before confirming:

import {
  EnumDrawingLayerId,
  ImageData,
  ImageEditorView,
  ImageProcessor,
} from 'dynamsoft-capture-vision-react-native';

export function Editor({navigation}: StackNavigation) {
  const editorView = useRef<ImageEditorView>(null);

  useEffect(() => {
    if (editorView.current) {
      editorView.current.setOriginalImage(global.originalImage);
      editorView.current.setQuads([global.sourceDeskewQuad], EnumDrawingLayerId.DDN_LAYER_ID);
    }
  }, []);

When the user confirms, extract the selected quad and re-deskew the image:

const getSelectedQuadAndDeskew = async (): Promise<ImageData | null | undefined> => {
  if (!editorView.current) {
    return null;
  }
  const quad = await editorView.current.getSelectedQuad().catch(e => {
    console.error('getSelectedQuad error: ' + e.message);
    return null;
  });
  if (quad) {
    global.sourceDeskewQuad = quad;
    return new ImageProcessor().cropAndDeskewImage(global.originalImage, quad);
  } else {
    Alert.alert('No selection', 'Please select a quad to confirm.');
    return null;
  }
};

The ImageProcessor.cropAndDeskewImage() method applies perspective correction based on the four-corner quadrilateral, producing a clean, rectangular document image.

Step 5: Apply Color Modes and Export the Document as PNG

The NormalizedImage screen displays the deskewed result and provides three actions: edit (re-open the quad editor), change color mode, and export.

Convert between color, grayscale, and binary outputs using ImageProcessor:

import {
  ImageIO,
  ImageProcessor,
  imageDataToBase64,
} from 'dynamsoft-capture-vision-react-native';

const changeColorMode = (mode: string) => {
  if (global.showingImage && global.showingImage !== global.deskewedImage) {
    global.showingImage.release();
  }
  switch (mode) {
    case ColorMode.color:
      global.showingImage = global.deskewedImage;
      break;
    case ColorMode.grayscale:
      global.showingImage = new ImageProcessor().convertToGray(global.deskewedImage) ?? global.deskewedImage;
      break;
    case ColorMode.binary:
      global.showingImage =
        new ImageProcessor().convertToBinaryLocal(
          global.deskewedImage,
          /*blockSize = */ 0,
          /*compensation = */ 10,
          /*invert = */ false,
        ) ?? global.deskewedImage;
      break;
  }
  setBase64(imageDataToBase64(global.showingImage) ?? '');
};

Export the current image as a PNG using ImageIO.saveToFile(), writing to the platform-appropriate directory:

import {
  ExternalCachesDirectoryPath,
  TemporaryDirectoryPath,
} from 'react-native-fs';

const imageIO = new ImageIO();
const savedPath =
  (Platform.OS === 'ios'
    ? TemporaryDirectoryPath
    : ExternalCachesDirectoryPath) + `/document_${Date.now()}.png`;
imageIO.saveToFile(global.showingImage, savedPath, true);
Alert.alert('Saved ✓', 'Image saved to:\n' + savedPath);

Source Code

https://github.com/yushulx/android-camera-barcode-mrz-document-scanner/tree/main/examples/react-native-document-scanner

Compare AI CLI Responses Side by Side with a Python PySide6 Desktop App

Xiao Ling — Mon, 16 Mar 2026 05:55:23 +0000

Choosing between AI coding assistants is hard when you can only test one at a time. Developers who have Qwen Code, GitHub Copilot CLI, OpenCode, and Gemini CLI installed need a fast way to send the same prompt to all of them and read the results in parallel. This project builds a Python desktop app with PySide6 that does exactly that — one prompt in, four streaming response panels out.

What you'll build: A PySide6 desktop GUI that dynamically detects installed AI CLI tools, sends a single prompt to all of them concurrently, and streams each response into its own panel with a Markdown/rendered-HTML toggle for easy reading.

Prerequisites

Python 3.10+
PySide6 (pip install PySide6)
markdown (pip install markdown)
At least one of the following AI CLI tools installed and authenticated:

CLI	Install command	Documentation
Qwen Code	`npm install -g @qwen-code/qwen-code`	github.com/QwenLM/qwen-code
GitHub Copilot CLI	`npm install -g @github/copilot`	docs.github.com
OpenCode	`npm install -g opencode-ai`	opencode.ai
Gemini CLI	`npm install -g @google/gemini-cli`	github.com/google-gemini/gemini-cli

Install Dependencies

Create a requirements.txt with the two Python dependencies and install them:

PySide6>=6.6.0
markdown>=3.5

pip install -r requirements.txt

Define the CLI Tool Registry

Each AI CLI has its own binary name, non-interactive invocation flags, and brand color. The app stores these as a list of dictionaries so panels can be generated dynamically at startup.

CLI_DEFS = [
    {
        "id": "qwen",
        "name": "Qwen Code",
        "cmd": lambda p: ["qwen", p],
        "color": "#4A9EEB",
        "download_url": "https://github.com/QwenLM/qwen-code",
        "install_hint": "npm install -g @qwen-code/qwen-code",
    },
    {
        "id": "copilot",
        "name": "GitHub Copilot CLI",
        "cmd": lambda p: [_REAL_COPILOT, "-p", p] if _REAL_COPILOT else ["copilot", "-p", p],
        "color": "#9B6FE8",
        "download_url": "https://docs.github.com/en/copilot/github-copilot-in-the-cli",
        "install_hint": "gh extension install github/gh-copilot",
    },
    {
        "id": "opencode",
        "name": "OpenCode",
        "cmd": lambda p: ["opencode", "run", p],
        "color": "#E8623A",
        "download_url": "https://opencode.ai",
        "install_hint": "npm install -g opencode-ai",
    },
    {
        "id": "gemini",
        "name": "Gemini CLI",
        "cmd": lambda p: ["gemini", "-p", p],
        "color": "#34A853",
        "download_url": "https://github.com/google-gemini/gemini-cli",
        "install_hint": "npm install -g @google/gemini-cli",
    },
]

Each CLI's non-interactive invocation syntax differs:

CLI	Command pattern
Qwen Code	`qwen "<prompt>"`
GitHub Copilot CLI	`copilot -p "<prompt>"`
OpenCode	`opencode run "<prompt>"`
Gemini CLI	`gemini -p "<prompt>"`

Handle Windows Subprocess

On Windows, npm-installed CLI tools are .CMD wrapper scripts that subprocess.Popen cannot spawn directly — they need to be executed through cmd /c. The build_cmd helper handles this transparently, while skipping the wrapper for real .exe binaries:

_IS_WINDOWS = platform.system() == "Windows"

def build_cmd(args: list[str]) -> list[str]:
    """On Windows .CMD/.BAT scripts cannot be spawned directly; wrap with cmd /c.
    If the binary is already a .exe, no wrapping is needed."""
    if _IS_WINDOWS and not args[0].lower().endswith(".exe"):
        return ["cmd", "/c"] + args
    return args

GitHub Copilot CLI has an additional complication: the VS Code extension installs a PowerShell wrapper (copilot.ps1) that performs an interactive version check using Read-Host. When stdin is /dev/null, this causes the process to exit immediately without running the query. The _find_real_copilot function resolves the actual copilot.exe binary by temporarily removing the wrapper's directory from PATH:

def _find_real_copilot() -> str | None:
    """Find the real copilot binary, skipping the VS Code PS1/BAT wrapper."""
    wrapper_path = shutil.which("copilot")
    if not wrapper_path:
        return None

    if wrapper_path.lower().endswith(".exe"):
        return wrapper_path

    wrapper_dir = os.path.dirname(os.path.abspath(wrapper_path))
    filtered = [p for p in os.environ.get("PATH", "").split(os.pathsep)
                if os.path.normcase(os.path.abspath(p)) != os.path.normcase(wrapper_dir)]
    old_path = os.environ["PATH"]
    os.environ["PATH"] = os.pathsep.join(filtered)
    try:
        real = shutil.which("copilot")
    finally:
        os.environ["PATH"] = old_path
    return real

Stream CLI Output with a Background QThread Worker

Each CLI runs in its own QThread to keep the UI responsive. The CLIWorker class spawns a subprocess, reads stdout line by line, strips ANSI escape codes, and emits each chunk as a Qt signal:

_ANSI_RE = re.compile(r"\x1B(?:[@-Z\\-_]|\[[0-?]*[ -/]*[@-~])")

def strip_ansi(text: str) -> str:
    return _ANSI_RE.sub("", text)

class CLIWorker(QThread):
    output_chunk = Signal(str)
    finished = Signal(bool, str)

    def __init__(self, cmd: list[str]):
        super().__init__()
        self._cmd = cmd
        self._cancelled = False
        self._process: subprocess.Popen | None = None

    def cancel(self):
        self._cancelled = True
        if self._process and self._process.poll() is None:
            self._process.kill()

    def run(self):
        try:
            self._process = subprocess.Popen(
                self._cmd,
                stdout=subprocess.PIPE,
                stderr=subprocess.STDOUT,
                stdin=subprocess.DEVNULL,
                text=True,
                encoding="utf-8",
                errors="replace",
            )
            for line in iter(self._process.stdout.readline, ""):
                if self._cancelled:
                    self._process.kill()
                    self.finished.emit(False, "Cancelled")
                    return
                self.output_chunk.emit(strip_ansi(line))
            self._process.stdout.close()
            self._process.wait()
            rc = self._process.returncode
            if rc == 0:
                self.finished.emit(True, "")
            else:
                self.finished.emit(False, f"Process exited with code {rc}")
        except FileNotFoundError:
            self.finished.emit(False, f"Command not found: {self._cmd[0]}")
        except Exception as exc:
            self.finished.emit(False, str(exc))

Key design decisions:

stdin=subprocess.DEVNULL prevents CLI tools from blocking on interactive prompts.
stderr=subprocess.STDOUT merges error output into the panel so nothing is silently lost.
Line-by-line iteration (readline) enables real-time streaming rather than waiting for the process to finish.

Build per-CLI Response Panels with Markdown Rendering

Each CLIPanel is a QFrame containing a header (CLI name, toggle button, status indicator) and two stacked content areas — a monospace raw-text view and a rendered HTML view. The toggle button switches between them:

def _on_toggle(self, checked: bool):
    self._rendered_mode = checked
    if checked:
        self._toggle_btn.setText("✎ Markdown")
        self.render_area.setHtml(markdown_to_html(self._raw_text))
        self.text_area.setVisible(False)
        self.render_area.setVisible(True)
    else:
        self._toggle_btn.setText("⟳ Render")
        self.render_area.setVisible(False)
        self.text_area.setVisible(True)

The Markdown-to-HTML conversion uses the Python markdown library with extensions for fenced code blocks, tables, and line breaks, wrapped in a dark-theme CSS stylesheet:

_MD_EXTENSIONS = ["fenced_code", "tables", "nl2br", "sane_lists"]

def markdown_to_html(text: str) -> str:
    body = md_lib.markdown(text, extensions=_MD_EXTENSIONS)
    return f"""<!DOCTYPE html><html><head><meta charset='utf-8'>{_MD_CSS}</head>
<body>{body}</body></html>"""

When a CLI is not installed, the panel displays a styled info card with a clickable download URL and suggested install command instead of an empty text area:

def _show_download_info(self):
    self._set_status("● Not installed", "#f44336")
    url = self.cli_def["download_url"]
    hint = self.cli_def["install_hint"]
    name = self.cli_def["name"]
    color = self.cli_def["color"]
    html = f"""
<div style="color:#e0e0e0; font-family:'Segoe UI',sans-serif; padding:16px;">
  <p style="font-size:16px; font-weight:bold; color:{color};">{name}</p>
  <p style="color:#f44336; font-size:13px;">&#9888; Not installed on this system</p>
  <p style="font-size:12px; color:#aaa; margin-top:16px;">Download / Documentation:</p>
  <p style="margin-top:4px;">
    <a href="{url}" style="color:#4A9EEB; font-size:13px;">{url}</a>
  </p>
</div>
"""
    self.text_area.setHtml(html)

Assemble the Main Window and Prompt Bar

The MainWindow dynamically checks for installed CLIs using shutil.which, creates a CLIPanel for each one, and arranges them in a horizontal QSplitter. The prompt input area at the bottom supports both a Send button and a Ctrl+Enter keyboard shortcut:

# ── CLI panels ───────────────────────────────────────────────────────
splitter = QSplitter(Qt.Orientation.Horizontal)

available_count = 0
for cli_def in CLI_DEFS:
    avail = shutil.which(cli_def["id"]) is not None
    panel = CLIPanel(cli_def, avail)
    splitter.addWidget(panel)
    self._panels.append(panel)
    if avail:
        available_count += 1

When the user clicks Send, the prompt text is dispatched to every installed panel concurrently:

def _send_prompt(self):
    prompt = self._prompt_input.toPlainText().strip()
    if not prompt:
        return
    self._prompt_input.clear()
    for panel in self._panels:
        panel.start_query(prompt)

The Ctrl+Enter shortcut is implemented via a QEvent filter on the prompt input:

def eventFilter(self, obj, event):
    if obj is self._prompt_input and event.type() == QEvent.Type.KeyPress:
        key_ev: QKeyEvent = event
        ctrl = Qt.KeyboardModifier.ControlModifier
        if (
            key_ev.key() == Qt.Key.Key_Return
            and key_ev.modifiers() & ctrl
        ):
            self._send_prompt()
            return True
    return super().eventFilter(obj, event)

Source Code

https://github.com/yushulx/multi-ai-cli-comparison