Forem: Mininglamp

How Mano-P Achieves #1 on OSWorld: Architecture, Benchmarks, and Edge Deployment

Mininglamp — Tue, 14 Apr 2026 12:15:51 +0000

Open-source GUI agents have been gaining traction, but most still rely on cloud inference, DOM parsing, or CLI hooks. Mano-P takes a different approach: pure vision-driven GUI automation that runs entirely on edge devices. And the benchmark results back it up — #1 on OSWorld among specialized models.

This article breaks down the architecture, benchmark data, and edge deployment performance, all from the project's public README and technical report.

The Numbers

OSWorld: #1 Among Specialized Models

OSWorld is the standard benchmark for GUI agent evaluation. Mano-P 1.0-72B achieves a 58.2% success rate, ranking first among all specialized GUI agent models. For context:

Model	OSWorld Success Rate
Mano-P 1.0-72B	58.2%
opencua-72b	45.0%
Gap	+13.2 percentage points

That's not a marginal improvement — it's a 29% relative gain over the second-place model.

WebRetriever: Beating Cloud Giants

On the WebRetriever Protocol I benchmark, Mano-P scores 41.7 NavEval, which puts it ahead of:

Model	NavEval Score
Mano-P 1.0	41.7
Gemini 2.5 Pro Computer Use	40.9
Claude 4.5 Computer Use	31.3

Worth noting: Gemini and Claude are cloud-based services with massive compute budgets. Mano-P achieves comparable or better results while running on local hardware.

13 Benchmarks, SOTA Across the Board

Beyond OSWorld and WebRetriever, Mano-P holds SOTA positions across 13 benchmarks spanning GUI grounding, perception & cognition, context learning, and pruning efficiency. The full benchmark data is available in the README.

The Architecture: Why Pure Vision?

Most GUI agents fall into one of these categories:

Approach	How It Works	Limitation
DOM/HTML Parsing	Read page structure directly	Web-only, breaks on native apps
CDP + CLI	Chrome DevTools Protocol + shell commands	Browser-dependent, fragile
Cloud Computer Use	Send screenshots to cloud API	Privacy concerns, latency, API costs
Pure Vision (Mano-P)	See the screen, understand it, act	Requires capable on-device model

Mano-P chose pure vision. No DOM access, no browser hooks, no platform-specific APIs. The model looks at the screen — the same pixels a human sees — and decides what to click, type, or scroll.

This is harder to build, but the payoff is generality: the same model works across any GUI application, any platform, without integration work per app.

Training Methodology: Mano-Action

The technical backbone is Mano-Action, a bidirectional self-reinforcement learning framework. The training follows three stages:

Stage 1: Supervised Fine-Tuning (SFT)
Starting from a base vision-language model, fine-tune on curated GUI interaction datasets.

Stage 2: Offline Reinforcement Learning
Learn from recorded interaction trajectories, optimizing action quality without live environment access.

Stage 3: Online Reinforcement Learning
The model interacts with real GUI environments, receiving feedback and iterating. This is where the "think-act-verify" loop reasoning mechanism comes in — the model plans an action, executes it, verifies the result, and adjusts.

The bidirectional aspect means Text→Action and Action→Text consistency are both optimized, creating a tighter loop between understanding and execution.

Edge Optimization: Running on Apple M4

The 72B model delivers SOTA benchmarks, but the edge story is equally important. Through mixed-precision quantization and a novel visual token pruning technique called GSPruning, Mano-P achieves practical performance on consumer hardware.

GSPruning: Preserving What Matters

GSPruning (Global Spatial Pruning) is designed specifically for vision-language models processing high-resolution interfaces. It:

Preserves global spatial structure through anchor points
Identifies semantic outliers for critical UI elements
Achieves 2-3× throughput speedup with minimal performance loss

On the Online-Mind2Web benchmark, GSPruning at 25% token retention achieves a success rate of 0.400 on Qwen3VL-4B, compared to 0.425 at full tokens — only a 6% drop while running significantly faster.

M4 Pro Performance

The 4B quantized model (w4a16) on Apple M4 Pro with 64GB RAM:

Metric	Value
Prefill Speed	476 tokens/s
Decode Speed	76 tokens/s
Peak Memory	4.3 GB
Prefill Time (4K context)	8.6s

4.3 GB peak memory means this runs comfortably alongside other applications. No dedicated GPU server required.

Hardware Requirements

Two deployment options:

Direct: Mac mini or MacBook with Apple M4 chip, 32GB+ RAM
Computing Stick: Any Mac + Mano-P computing stick via USB 4.0+

Data Privacy: The Edge Advantage

In Local Mode, all processing happens on-device:

✅ Screenshots never leave the device
✅ Task descriptions stay local
✅ No cloud API calls
✅ Full source code is open for audit

Cloud Mode is available as a fallback (screenshots sent to mano.mininglamp.com), but the local-first architecture means sensitive workflows can run with zero data exposure.

Getting Started

Three usage forms are available:

CLI (for terminal users):

brew tap HanningWang/tap
brew install mano-cua

mano-cua run "Open WeChat and tell FTY the meeting is postponed"

Python SDK (planned):

from mano_client import ManoClient
client = ManoClient()
client.run("Search for AI news on Xiaohongshu")

ClawHub Skill (for AI agents):

clawhub install mano-cua

The Skill form is designed for AI agents like Claude Code or OpenClaw — the agent automatically invokes Mano-P when GUI operations are needed.

What's Next

Mano-P is being released in three phases:

Phase 1 (now): Mano-CUA Skills — for agent enthusiasts to build CUA task workflows
Phase 2 (coming): Local models + SDK — for developers with high security requirements
Phase 3 (planned): Training methods + pruning techniques — for researchers who want to train their own GUI-VLA models

The project is Apache 2.0 licensed. Full source, benchmarks, and documentation: github.com/Mininglamp-AI/Mano-P

Open-Sourcing Mano-P Today: Pure Vision GUI Agent, OSWorld #1, Apache 2.0

Mininglamp — Mon, 13 Apr 2026 10:29:19 +0000

Article

Mano-P: Open-Sourcing the #1 GUI Agent on OSWorld — Pure Vision, On-Device, Apache 2.0

Today we're open-sourcing Mano-P under Apache 2.0.

GitHub: https://github.com/Mininglamp-AI/Mano-P

What is Mano-P?

Mano-P is a pure-vision GUI agent. It looks at your screen — literally a screenshot — understands the UI elements, and performs actions. No CDP protocol. No HTML parsing. No accessibility API. Just vision.

The name: Mano is Spanish for "hand." Current AI agents can interact with computers, but most of them operate like a lobster's claw — functional, but clumsy. Mano-P is designed to give agents a proper hand.

The P stands for four things:

Power — 13 multimodal benchmark SOTAs, #1 among proprietary models on OSWorld
Private — Runs on-device, your data never leaves your machine
Public — Apache 2.0, fully open source starting today
Personal — A foundation for building your own personalized AI

Why Does This Matter?

There are broadly four approaches to GUI automation today:

Approach	Examples	Limitation
Traditional RPA	UiPath	Coordinate-based, breaks when UI changes
Browser CUA	CDP-based agents	Limited to browsers
Cloud Computer Use	Claude CU, etc.	Requires uploading screen data to the cloud
Pure Vision GUI	Mano-P	Sees the screen directly, works everywhere, runs locally

Most current GUI agents either depend on browser protocols (limiting them to Chrome) or run in the cloud (requiring you to stream your screen to a remote server). Mano-P takes a different approach: it processes raw screen captures through a vision model to understand and interact with any GUI — desktop apps, browsers, 3D tools, professional software, anything with a graphical interface.

Benchmarks

Numbers first.

OSWorld: #1 Among Proprietary Models

Mano-P 72B: 58.2% success rate
Second place (opencua-72b): 45.0%
Gap: +13.2 percentage points
Ranks 5th overall (the four models above it are 100B+ general-purpose LLMs like GPT, Claude, and Gemini)

13 Multimodal Benchmark SOTAs

Selected results:

Benchmark	Score
ScreenSpot-V2	93.5
MMBench	87.5
UI-Vision	46.6

Web Navigation

On WebRetriever Protocol I NavEval:

Mano-P: 41.7
Gemini 2.5 Pro CU: 40.9
Claude 4.5 CU: 31.3

On-Device Performance

The 4B quantized model (w4a16) running on Apple M4 Pro:

Metric	Value
Prefill	476 tokens/s
Decode	76 tokens/s
Peak memory	4.3 GB

Your screenshots, your instructions, your data — all processed locally. Nothing uploaded to any cloud server.

For anyone dealing with sensitive environments (enterprise systems, personal data, financial interfaces), this isn't a nice-to-have. It's a requirement.

Key Technical Highlights

Pure-Vision GUI Understanding: No CDP, no DOM parsing. The model directly processes screen captures to identify UI elements, understand layout, and locate interaction targets.

Mano-Action Bidirectional Self-Enhancement: Inspired by cycle consistency in GANs. The training loop runs in both directions — Text→Action and Action→Text — enforcing consistency between them. This significantly improves generalization to unseen UIs.

Three-Stage Training Pipeline:

SFT (Supervised Fine-Tuning): Learn basic screen-to-action mapping from human demonstrations
Offline RL: Learn from collected trajectories (both successes and failures) without live interaction
Online RL: Direct interaction with real GUI environments for continued policy improvement

GSPruning (Guided Structural Pruning): Compresses visual token retention to 约 25%, boosting throughput 2-3× with minimal accuracy loss. This is what makes on-device inference practical.

Think-Act-Verify Loop: Instead of generating a full action sequence upfront, Mano-P executes one step at a time: analyze the screen → perform an action → verify the result → plan the next step. Far more robust than one-shot planning.

Open-Source Roadmap

We're releasing Mano-P in three stages:

Phase	What	Status
Phase 1	Mano-CUA Skill — ready to use	✅ Today
Phase 2	Local model + SDK — zero cloud dependency	🔜 Coming soon
Phase 3	Training methods + GSPruning + quantization	📋 Planned

The end goal: the entire stack — training, pruning, quantization, deployment — fully open to the community.

Get Started

Option 1: CLI (mano-cua)

brew tap HanningWang/tap && brew install mano-cua

Then run:

mano-cua run "Open WeChat and tell FTY the meeting is postponed"

Option 2: Agent Skill (mano-skill)

clawhub install mano-cua

Once installed, your agent can autonomously invoke Mano-P for GUI operations — no manual triggering needed. The agent decides when GUI interaction is required and calls Mano-P automatically.

Option 3: Python SDK (mano-client)

Coming soon. Watch the repo for updates.

What Can You Do With It?

Mano-afk: Fully Autonomous App Building

Give it a natural language requirement. Mano-P autonomously handles the entire pipeline: requirements analysis → architecture → code → test → fix → verify. No human in the loop.

Personal AI: Learning Your Habits

Mano-P can learn and adapt to your personal operational style. A fun example: it can play mahjong following your specific strategy — not the optimal strategy, but your style. That's what Personal AI means in practice.

Why "Personal AI"?

When a capable GUI agent runs on your local device, handles your data without uploading anything, and operates all the software you use daily — it stops being a "tool" and starts becoming a truly personal AI.

With Mano-P open-sourced under Apache 2.0, every developer, every team, every organization can build their own version. This is the beginning.

⭐ Star, Clone, Try It

https://github.com/Mininglamp-AI/Mano-P

Apache 2.0. Issues, PRs, and contributions welcome.

SOTA on 13 Benchmarks: Mininglamp Open-Sources GUI-VLA Model Mano-P 1.0

Mininglamp — Mon, 13 Apr 2026 04:41:27 +0000

SOTA on 13 Benchmarks: Mininglamp Open-Sources GUI-VLA Model Mano-P 1.0

Original article: WeChat (Chinese)
GitHub: Mano-P

Mininglamp Technology has officially open-sourced Mano-P 1.0, an in-house GUI-aware agent model. Mano-P handles GUI perception, understanding, planning, action, and verification — all through pure vision. It can directly understand and operate desktop applications, web interfaces, and complex graphical workflows, and it runs locally on Apple M4 devices.

Mano-P moves AI beyond "look but don't touch." It executes complex tasks across platforms directly in real graphical interfaces. The project is released under Apache 2.0 with full source code available for audit, commercial use, and modification.

By combining pure visual understanding with local execution, Mano-P enables individual developers and organizations to build personalized AI at low cost while maintaining full data sovereignty.

Vision-Only: Solving the Last Mile for Complex Workflows

Most automation today depends on underlying API calls, the CDP protocol, or HTML parsing. These approaches fall short when dealing with non-standard applications or cross-system workflows. Mano-P takes a fundamentally different approach: pure visual understanding as the core paradigm. It requires no external APIs or protocols, and can directly understand and operate desktop software, 3D applications, and specialized professional tools — breaking free from the browser-centric ecosystem.

Mano-P also serves as an execution backbone for existing agent ecosystems. It integrates seamlessly as a skill into AI agents like OpenClaw. With this integration, agents can navigate across multiple windows and cross-application workflows, performing clicks, text input, window switching, and visual verification in a closed loop.

This addresses a long-standing bottleneck in agent workflows: the need for human intervention. Mano-P enables not just automated build-and-test pipelines, but autonomous execution of complex business scenarios end-to-end.

SOTA on 13 Benchmarks: Raising the Bar for GUI-Specific Models

Mano-P ships in two versions: a full 72B model that pushes the performance ceiling, and a 4B quantized model (w4a16) optimized for on-device deployment.

The 72B version achieves SOTA results across 13 authoritative multimodal benchmarks, covering GUI Grounding, CUA (Computer Use Agent), multimodal perception and cognition, video understanding, and long-context learning. It sets a new performance standard for on-device GUI agents.

On the OSWorld proprietary model benchmark, Mano-P 72B reaches a 58.2% task success rate — ranking first globally and leading the second-place opencua-72b (45.0%) by 13.2 percentage points. It also tops ScreenSpot-V2, MMBench, UI-Vision, and other evaluation suites.

These results are driven by architectural innovation. Mano-P uses a three-stage progressive training pipeline: SFT (supervised fine-tuning), offline reinforcement learning, and online reinforcement learning. Combined with a proprietary GSPruning visual token pruning technique, this delivers a significant leap in on-device inference efficiency.

On Apple M4 Pro hardware, the 4B quantized model achieves 476 tokens/s prefill speed and 76 tokens/s decode speed, with peak memory usage of just 4.3 GB — well within the constraints of mainstream edge devices.

On-Device Deployment: Air-Gapped Data Protection

As AI moves deeper into core business processes, data privacy and compliance become critical. Mano-P supports fully local on-device deployment with zero cloud data transmission. Its "pure vision + local execution" architecture enables physical isolation between data processing and external networks.

In local mode, the model runs directly on Mac mini or MacBook (M4 chip or later, 32 GB+ RAM), or via a Mano-P compute stick connected over USB 4.0. Screenshots, business data, and task instructions all stay in a local closed loop, eliminating cloud transmission risk at the source.

Mano-P also handles long-running tasks autonomously offline. Even without network access, it can independently drive complex business workflows, including mid-process decision-making and error correction.

Full Open-Source Strategy: Accelerating the Personalized AI Ecosystem

Mano-P is released under Apache 2.0 with complete client source code open for audit, commercial use, and derivative work.

To lower the barrier to entry, Mano-P offers three ready-to-use deployment modes covering different tech stacks. No complex API key setup required — users can build high-performance GUI agents with minimal configuration.

As a first step, Mininglamp is open-sourcing the Mano-CUA core skill. Users can configure it into OpenClaw or Claude Code to build smarter CUA task workflows and eliminate human-intervention bottlenecks.

The Mano-CUA local model and SDK components are expected to be open-sourced within the month, targeting developers with high-security requirements. Users will be able to call locally deployed GUI-VLA models to build custom skills and tools, with all CUA operations executing on local Mac hardware — nothing uploaded to external servers.

Looking ahead, Mininglamp plans to fully open-source the underlying training methods, token pruning techniques, and mixed-precision quantization schemes behind Mano-P, enabling developers to build custom local GUI-VLA models tailored to their own use cases.

From technical breakthroughs to ecosystem building, Mano-P tightly integrates GUI perception, visual operation, local execution, and open-source collaboration. It establishes a solid technical foundation for on-device agents and charts a concrete path toward Personalized AI.

GitHub: https://github.com/Mininglamp-AI/Mano-P

Original article (Chinese): https://mp.weixin.qq.com/s/eWnQTvY0OiuHzujJE32kPw

The Evolution of GUI Agents: From RPA Scripts to AI That Sees Your Screen

Mininglamp — Thu, 09 Apr 2026 07:12:14 +0000

In 2020, if you wanted to automate a desktop app, you'd write an RPA script — record mouse movements, hardcode coordinates, and pray the UI never changed.

In 2024, if you wanted an AI to operate a browser, you'd use a CDP-based agent — one that reads the DOM, parses HTML, and executes tasks inside Chrome.

In 2026, there's a model that looks at a screenshot, understands the interface, and clicks, types, and switches windows like a human — no API needed, no HTML parsing, no knowledge of the underlying tech stack.

These three stages represent three paradigm shifts in GUI automation over the past few years.

Let's break down how we got here.

Generation 1: RPA — Record and Replay

Traditional RPA (UiPath, Blue Prism, Automation Anywhere) boils down to one idea: record what a human does, then replay it.

Under the hood, it's simulating mouse and keyboard events at the OS level. Early versions used coordinate-based targeting — change the resolution and everything breaks. Later iterations added control tree recognition (Windows UI Automation, macOS Accessibility API) and image matching.

RPA still powers automation at banks, insurance companies, and government systems today. But for developers, it has structural problems:

Brittle: Change one pixel in the UI and the script breaks
Zero understanding: It doesn't know what it's doing — just mechanically repeating
High maintenance: Every UI change requires re-recording
Limited scope: Cross-application, cross-platform workflows are painful

RPA was always "automation for non-technical users," not something that excited developers.

Generation 2: Browser CUA — DOM-Based Agents

In 2024–2025, LLMs got good enough to understand web pages. A new class of solutions emerged:

Use Chrome DevTools Protocol (CDP) to grab the page DOM
Feed DOM/HTML fragments to an LLM for comprehension
LLM outputs action instructions (click element X, fill form Y)
Execute via CDP

The improvement was real: LLMs brought understanding instead of mechanical replay. But the limitations were equally clear:

Locked inside the browser: CDP is a Chrome protocol. Desktop apps, native apps, games, 3D tools — none of them work
Depends on HTML structure: Complex or dynamically rendered pages produce massive, unreliable DOM trees
Data security: DOM content (including your login state and sensitive data) gets sent to a cloud LLM

For developers, this solved "browser automation" but not "general GUI automation."

Generation 3: Pure-Vision GUI Agents — See the Screen, Not the Code

Starting in late 2025, a fundamentally different approach matured: models that take a screenshot as input and output actions like "click at (x, y)" or "type 'hello world'."

The key difference from everything before: no dependency on any underlying protocol or interface. No CDP, no Accessibility API, no need to know what framework the app was built with. Input is a screenshot. Output is an action.

Coverage is theoretically unlimited — any application with a graphical interface can be operated. Desktop software, browsers, games, 3D modeling tools, even apps inside a remote desktop session.

The technical challenges are significant:

GUI Grounding: The model needs to precisely locate and understand interface elements
Multi-step planning: Complex tasks require sequences of actions with memory
Error recovery: When something goes wrong, the model needs to detect the anomaly and self-correct

This approach splits into two paths — cloud (screenshots sent to remote servers) and on-device (inference runs locally). Same technique, completely different data flow.

On-Device Pure-Vision: Where It Gets Interesting

Let me use a concrete example to show where on-device GUI agents stand today.

Mano-P 1.0 is a GUI-VLA (Vision-Language-Action) agent model purpose-built for on-device deployment. Pure vision, no CDP, no HTML parsing.

Benchmark results

On OSWorld — the academic community's standard benchmark for desktop GUI agents — the Mano-P 72B model achieved 58.2% success rate, ranking #1 among proprietary models globally.

For context: the other four models in the top 5 are all 100B+ general-purpose models. A 72B model purpose-built for GUI scenarios beating them says something about the efficiency of specialized models vs. the brute-force approach.

Across a broader evaluation, Mano-P hit SOTA on 13 benchmark leaderboards spanning GUI grounding, perception, video understanding, and in-context learning.

On-device performance

The 4B quantized model (w4a16) runs at 476 tokens/s prefill, 76 tokens/s decode on Apple M4 Pro, with peak memory of just 4.3GB.

That means on an M4 Mac mini or MacBook with 32GB RAM, you can run an OSWorld-champion-level GUI agent entirely on-device. No data ever leaves your machine.

One command to install:

brew install mano-cua

No API key. No cloud config. No worrying about where your screenshots end up.

The Comparison Table Developers Actually Want

Dimension	Traditional RPA	Browser CUA	Cloud Computer Use	On-Device GUI Agent (Mano-P)
Perception	Coordinates / control tree / image matching	DOM / HTML parsing	Cloud screenshot + vision model	Local screenshot + vision model
Coverage	Single app	Browser only	Theoretically all platforms	All platforms
Understanding	None	Yes (HTML-based)	Yes (vision-based)	Yes (vision-based)
Data flow	Local	DOM sent to cloud	Screenshots uploaded to cloud	Data never leaves device
Robustness	Low (breaks on UI change)	Medium (depends on DOM stability)	High	High
Deployment	Local RPA engine	Browser + API	Cloud API + network	Local device (e.g., M4 Mac + 32GB)

There's a frequently overlooked distinction here: cloud Computer Use and on-device GUI agents use the same technique (pure vision), but the data flow is completely different.

Cloud solutions send your screenshots — everything on your screen, including code, emails, and credentials — to a remote server. For many developers, that's a non-starter.

On-device solutions run inference locally. Screenshots processed locally. Actions executed locally. This isn't "we added encryption" level security — it's physically eliminating the possibility of data leakage.

Why On-Device Only Became Possible Now

Two changes made this viable:

Hardware: Apple's M4 unified memory architecture gave consumer devices the foundation to run medium-scale models. M4 + 32GB unified memory + high-bandwidth memory bus — this was workstation-grade hardware two years ago.

Model compression: Mano-P's GSPruning visual token pruning + w4a16 quantization keeps the 4B model at 4.3GB peak memory with 476 tokens/s throughput. That's a fully usable inference speed.

What's the Endgame?

When an AI agent can see any screen, understand intent, and operate any graphical interface, it has the same software-usage capability as a human user. It doesn't need APIs, doesn't wait for integrations, doesn't learn each tool's SDK.

The implications:

Long-tail software gets activated: Millions of professional tools with no API can suddenly be operated by agents
Cross-application workflows become possible: Design in Figma, compile in Terminal, deploy in browser — all via GUI
The walls between software break down: No data export/import needed — the agent just operates at the interface level

With benchmark scores above 50% on complex desktop tasks, we're watching GUI agents cross from "lab demo" to "developer-usable."

Try It

Mano-P 1.0 is open source under Apache 2.0.

brew install mano-cua

👉 GitHub: github.com/Mininglamp-AI/Mano-P

What's your take — is on-device the right path for GUI agents, or is cloud compute still the pragmatic choice? Drop your thoughts below.