Forem: Takahiro Sato

Compose Your Markdown Workflow: mq Subcommands + Unix Pipes

Takahiro Sato — Sat, 14 Feb 2026 14:01:49 +0000

Piping mq Subcommands Together

mq is a jq-like CLI for Markdown. Beyond its query engine, mq comes with a set of external subcommands:

Subcommand	What it does
`mq conv`	Convert Excel, PDF, Word, HTML, etc. to Markdown
`mq edit`	Terminal-based Markdown editor
`mq view`	Markdown viewer
`mq task`	Markdown-based task runner
`mq tui`	Interactive TUI for trying queries

They all work with stdin/stdout, so you can pipe them together.

A Few Examples

Pull headers out of an Excel file:

mq conv report.xlsx | mq edit | mq '.h'

Extract h2s from a Word doc and open them in the editor:

mq conv meeting-notes.docx | mq '.h2' | mq edit

Grab headings from a web page and view them:

curl -s https://mqlang.org | mq -I html 'select(.h1 || .h2)' | mq view

Merge multiple files with separators:

mq -A -S '"---"' 'identity' chapter1.md chapter2.md chapter3.md | mq view

Each subcommand has a single job:

mq conv  → convert to Markdown
mq       → query and transform
mq edit  → edit
mq view  → display

Custom Subcommands

Drop an executable named mq-* into ~/.mq/bin/ or your PATH, and it becomes a subcommand.

mq --list  # see all available subcommands

mx: Turn Your Documentation into Executable Tasks

Takahiro Sato — Fri, 10 Oct 2025 14:17:57 +0000

mx is a task runner that executes code blocks directly from Markdown files based on section titles.

What is mx?

mx is a command-line task runner that treats your Markdown files as executable documentation. Instead of maintaining separate task scripts and documentation, you can keep everything in one place—typically your README.md—and execute tasks directly from there.

Built on top of mq, a jq-like command-line tool for Markdown processing, mx parses your Markdown documents and executes the code blocks within specific sections.

Why mx?

Traditional task runners require you to maintain separate files from your documentation. With mx, your documentation is your task runner. This approach offers several benefits:

Single source of truth: No synchronization issues between docs and scripts
Self-documenting: Your tasks include their own documentation naturally
Discoverable: New team members can find and understand tasks easily
Multi-language support: Automatically detects and executes code in different languages
Zero configuration: Works out of the box, with optional customization

Quick Start

Installation

The fastest way to install mx is using the installation script:

curl -sSL https://raw.githubusercontent.com/harehare/mx/refs/heads/main/bin/install.sh | bash

Alternatively, if you have Rust installed:

cargo install --git https://github.com/harehare/mx.git

Basic Usage

Create a README.md with some tasks:

## Build

\`\`\`bash
cargo build --release
\`\`\`

## Test

\`\`\`bash
cargo test
\`\`\`

## Deploy

\`\`\`bash
./scripts/deploy.sh production
\`\`\`

List available tasks:

mx

Run a specific task:

mx Build

That's it! mx will execute the bash code block under the "Build" section.

Key Features

1. Multi-Language Runtime Support

One of mx's most powerful features is its ability to automatically detect and execute code blocks in different programming languages. Simply specify the language in your code block, and mx will use the appropriate runtime:

## Python Analysis

\`\`\`python
import pandas as pd
df = pd.read_csv('data.csv')
print(df.describe())
\`\`\`

## Node.js Build

\`\`\`javascript
const esbuild = require('esbuild');
esbuild.build({
  entryPoints: ['src/index.ts'],
  bundle: true,
  outfile: 'dist/bundle.js',
});
\`\`\`

## Go Benchmark

\`\`\`go
package main
import (
    "fmt"
    "time"
)
func main() {
    start := time.Now()
    // Your code here
    fmt.Printf("Execution time: %v\n", time.Since(start))
}
\`\`\`

## Ruby Script

\`\`\`ruby
require 'json'
data = JSON.parse(File.read('config.json'))
puts "Config loaded: #{data.keys.join(', ')}"
\`\`\`

Run any of these with:

mx "Python Analysis"
mx "Node.js Build"
mx "Go Benchmark"
mx "Ruby Script"

mx automatically uses the correct runtime for each language—no need to specify or configure anything!

2. Configurable Runtimes

While mx works out of the box with sensible defaults, you can customize runtime environments to match your needs:

mx init

This creates an mx.toml configuration file where you can specify custom runtimes:

heading_level = 2

[runtimes]
bash = "bash"
sh = "sh"
python = "python3"
ruby = "ruby"
node = "node"
javascript = "node"
js = "node"
php = "php"
perl = "perl"

[runtimes.go]
command = "go run"
execution_mode = "file"

[runtimes.mq]
command = "mq"
execution_mode = "arg"

Simple Runtime Configuration

For most languages, you can use a simple string format:

[runtimes]
python = "pypy3"

This uses the default execution mode (stdin), where code is passed via standard input.

Advanced Runtime Configuration

Some languages require different execution modes. You can specify these using the detailed configuration format:

[runtimes.go]
command = "go run"
execution_mode = "file"

Available Execution Modes:

stdin (default): Code is passed via standard input (works for Python, Ruby, Node.js, etc.)
file: Code is written to a temporary file and the file path is passed as an argument (required for Go)
arg: Code is passed directly as a command-line argument (used for specialized tools like mq)

This flexibility means you can:

Use different Python versions (python3.11, pypy, etc.)
Switch between Node.js and Deno
Use custom interpreters or compilers
Configure the appropriate execution mode for each language
Add support for any language with a CLI

3. Executable Documentation

mx turns your documentation into a living, executable artifact. Instead of documenting commands that might become outdated, you document runnable tasks:

## Setup Development Environment

Install all dependencies and initialize the database.

\`\`\`bash
# Install npm dependencies
npm install

# Setup database
npx prisma migrate dev

# Generate API client
npm run generate:api
\`\`\`

## Start Development Server

Runs the application in development mode with hot reloading enabled.

\`\`\`bash
npm run dev
\`\`\`

## Run Tests with Coverage

Executes the full test suite and generates a coverage report.

\`\`\`bash
npm run test:coverage
open coverage/index.html
\`\`\`

Now your documentation is always in sync with what actually runs, because they're the same thing!

Real-World Use Case: Polyglot Projects

mx really shines in projects that use multiple programming languages. Here's an example from a typical web application:

## Setup

Install all dependencies across the stack.

\`\`\`bash
npm install
pip install -r requirements.txt
go mod download
\`\`\`

## Frontend Build

Bundle and optimize the React application.

\`\`\`javascript
const esbuild = require('esbuild');
esbuild.build({
  entryPoints: ['frontend/src/index.tsx'],
  bundle: true,
  minify: true,
  outfile: 'dist/app.js',
}).catch(() => process.exit(1));
\`\`\`

## Backend Test

Run the Python backend test suite.

\`\`\`python
import pytest
import sys

result = pytest.main(['-v', 'tests/'])
sys.exit(result)
\`\`\`

## Data Migration

Execute Go-based database migration.

\`\`\`go
package main
import (
    "database/sql"
    "log"
    _ "github.com/lib/pq"
)
func main() {
    db, err := sql.Open("postgres", "...")
    if err != nil {
        log.Fatal(err)
    }
    // Migration logic here
}
\`\`\`

## Generate API Docs

Create API documentation from OpenAPI spec.

\`\`\`ruby
require 'yaml'
require 'erb'

spec = YAML.load_file('openapi.yaml')
template = ERB.new(File.read('docs/template.erb'))
File.write('docs/api.html', template.result(binding))
\`\`\`

With mx, you can run all these tasks seamlessly:

mx Setup
mx "Frontend Build"
mx "Backend Test"
mx "Data Migration"
mx "Generate API Docs"

Each task automatically uses the appropriate runtime based on the code block language!

Advanced Features

Multiple Documentation Files

Organize complex projects with multiple task files:

mx -f docs/deployment.md "Deploy to Production"
mx -f docs/maintenance.md "Database Backup"

Custom Heading Levels

Prefer using ### for tasks? Configure it in mx.toml:

heading_level = 3

Task Discovery

New to a project? List all available tasks to see what you can do:

mx
# or
mx list --file docs/tasks.md

Use Cases

mx is perfect for:

Open Source Projects: Make it easy for contributors to discover and run common tasks
Polyglot Microservices: Manage tasks across services written in different languages
Documentation Sites: Keep your documentation examples executable and tested
Onboarding: New developers learn about and run tasks from one place
CI/CD Workflows: Document deployment procedures that can be executed locally or in pipelines
Data Science: Combine Python analysis, R scripts, and bash data processing in one place

Current Status

⚠️ Note: mx is currently under active development. While it's functional and usable, the API and features may change as the project evolves.

Contributing

mx is open source and welcomes contributions! Check out the GitHub repository to:

Report issues
Suggest features
Submit pull requests
Star the project ⭐

Conclusion

mx brings together documentation and automation in a natural way. By treating Markdown as executable documentation and providing seamless multi-language runtime support, it eliminates the gap between what you document and what you actually run.

If you're working on projects that use multiple programming languages and want your documentation to stay in sync with your actual workflows, give mx a try!

Processing Markdown in Python with mq

Takahiro Sato — Sun, 27 Jul 2025 13:29:53 +0000

The markdown-query package brings the functionality of the mq Markdown processor to Python. mq uses a jq-like syntax to filter, transform, and extract data from Markdown documents.

Installation

pip install markdown-query

Basic Usage

The primary interface is the run() function, which takes three parameters: a query string, markdown content, and optional configuration.

import mq

markdown = """# Title

This is a paragraph.

## Section

- Item 1
- Item 2

``javascript
console.log("hello");
``
"""

# Extract all headings
result = mq.run("select(.h1 || .h2)", markdown, None)
print(result.values)
# ['# Title', '## Section']

# Get heading text without markdown formatting
result = mq.run("select(.h1 || .h2) | to_text()", markdown, None)
print(result.values)
# ['Title', 'Section']

Working with Different Input Formats

The library supports multiple input formats through the Options class:

import mq

# Process HTML input
html = '<h1>Title</h1><p>Content</p><ul><li>Item</li></ul>'

options = mq.Options()
options.input_format = mq.InputFormat.HTML

result = mq.run(".h1 | upcase()", html, options)
print(result.values)
# ['# TITLE']

Available input formats:

InputFormat.MARKDOWN (default)
InputFormat.HTML
InputFormat.MDX
InputFormat.TEXT

Query Examples

Extract Code Blocks

# Get all code blocks
code_blocks = mq.run("select(.code)", markdown, None)

# Get code block content as text
code_text = mq.run("select(.code) | to_text()", markdown, None)

Filter by Content

# Find headings containing specific text
headings = mq.run('select(.h1 || .h2) | select(test("Section"))', markdown, None)

# Extract list items
items = mq.run(".[] | to_text()", markdown, None)

Transform Content

# Convert headings to uppercase
upper_headings = mq.run("select(.h1, .h2) | upcase()", markdown, None)

# Replace text in paragraphs
modified = mq.run('select(.paragraph) | gsub("paragraph"; "text")', markdown, None)

Accessing Result Data

The run() function returns an MQResult object with a values list:

result = mq.run("select(.h1)", markdown, None)

# Access all values
for value in result.values:
    print(value)

# Access individual items
first_heading = result[0]
print(first_heading.text)           # Text content
print(first_heading.markdown_type)  # MarkdownType enum

# Iterate over results
for item in result:
    print(f"Type: {item.markdown_type}, Text: {item.text}")

Integration with Other Tools

The library works well with other Python markdown processing tools:

from markitdown import MarkItDown
import mq

# Convert web pages to markdown, then process with mq
markitdown = MarkItDown()
result = markitdown.convert("https://example.com")

# Extract specific content
code_samples = mq.run(".code | to_text()", result.text_content, None)
all_links = mq.run(".link | to_html()", result.text_content, None)

Configuration Options

The Options class provides additional configuration:

options = mq.Options()
options.input_format = mq.InputFormat.HTML
options.list_style = mq.ListStyle.PLUS      # Use + for lists
options.link_title_style = mq.TitleSurroundStyle.SINGLE
options.link_url_style = mq.UrlSurroundStyle.ANGLE

result = mq.run("select(.list)", content, options)

Error Handling

Queries that fail will raise a PyRuntimeError:

try:
    result = mq.run("invalid_query", markdown, None)
except RuntimeError as e:
    print(f"Query failed: {e}")

Performance

The library is built on Rust and compiled to native code, providing fast processing for large markdown files.

Resources

Support

🐛 Report bugs
💡 Request features
⭐ Star the project if you find it useful!

The markdown-query package provides a straightforward way to apply mq's markdown processing capabilities in Python applications, from simple content extraction to complex document transformations.

Markdown Processing in AI Applications with mq-mcp

Takahiro Sato — Sun, 13 Jul 2025 10:17:43 +0000

AI assistants frequently work with Markdown content from documentation, blogs, and technical articles. Processing this content programmatically requires extracting specific elements, transforming structure, and filtering based on criteria. The Model Context Protocol (MCP) provides a standardized way for AI systems to access external tools and data sources.

mq-mcp combines the Markdown processing capabilities of the mq tool with MCP, allowing AI assistants to perform content analysis through a standard interface.

Model Context Protocol Integration

MCP enables AI assistants to interact with external tools using JSON-RPC 2.0. mq-mcp implements an MCP server that exposes four primary tools:

html_to_markdown: Converts HTML to Markdown with optional query processing
extract_markdown: Processes Markdown content using mq queries
available_functions: Lists all available mq functions with descriptions
available_selectors: Returns available element selectors

Configuration Setup

// Claude Desktop
{
  "mcpServers": {
    "mq": {
      "command": "/path/to/mq",
      "args": ["mcp"]
    }
  }
}

# Claude Code
claude mcp add mq-mcp -- mq mcp

Content Processing Examples

Extracting Code Blocks

{
  "markdown": "# API Reference\n\n```

javascript\nconst api = require('./api');\n

```\n\n```

python\nimport api\n

```",
  "query": ".code(\"javascript\")"
}

Filtering Headings

{
  "markdown": "# Main Title\n## Section A\n### Subsection\n## Section B",
  "query": "select(or(.h1, .h2))"
}

Link Extraction

{
  "markdown": "See [API docs](./api.md) and [external guide](https://example.com/guide)",
  "query": ".link | select(startsWith(\"./\"))"
}

Advanced Processing

// Transform text
{
  "markdown": "# getting started\n## basic usage",
  "query": ".h | upcase()"
}

// Filter with conditions
{
  "markdown": "```

bash\nnpm install\n

```\n\n```

bash\nyarn add\n

```",
  "query": ".code | select(contains(\"npm\"))"
}

HTML to Markdown Conversion

The html_to_markdown function provides comprehensive HTML processing with optional query filtering. This enables AI systems to extract structured content from web pages, documentation sites, and HTML-based content.

Basic HTML Conversion

Convert HTML to clean Markdown format:

{
  "html": "<div><h1>API Documentation</h1><p>This describes the REST API endpoints.</p><ul><li>GET /users</li><li>POST /users</li></ul></div>"
}

Output:

# API Documentation

This describes the REST API endpoints.

- GET /users
- POST /users

Complex HTML Structure Processing

Handle nested HTML with tables and formatted content:

{
  "html": "<article><header><h1>Database Schema</h1></header><table><tr><th>Column</th><th>Type</th></tr><tr><td>id</td><td>INTEGER</td></tr><tr><td>name</td><td>VARCHAR</td></tr></table><footer><p>Updated: 2024</p></footer></article>",
  "query": "select(or(.h1, .table))"
}

Output:

# Database Schema

| Column | Type    |
| ------ | ------- |
| id     | INTEGER |
| name   | VARCHAR |

Web Scraping Integration

Extract content from live web pages:

{
  "html": "<div class='documentation'><h2>Installation</h2><pre><code>npm install package</code></pre><h2>Usage</h2><pre><code>const pkg = require('package');</code></pre></div>",
  "query": "nodes | .code | first()"
}

Output:

npm install package

This extracts only the installation code block, filtering out usage examples.

HTML Element Mapping

The conversion process maps HTML elements to Markdown equivalents:

HTML Element	Markdown Output	Notes
`<h1>` - `<h6>`	`#` - `######`	Header levels preserved
`<p>`	Text blocks	Paragraph spacing maintained
`<ul>`, `<ol>`	`-`, `1.`	List structure preserved
`<table>`	Pipe tables	Column alignment detected
`<code>`	`code`	Inline code formatting
`<pre><code>`	code blocks	Code blocks with language detection
`<blockquote>`	`>`	Quote block formatting
`<a>`	`[text](url)`	Links with titles preserved
`<img>`	`![alt](src)`	Images with alt text

Processing Pipeline

The HTML to Markdown conversion follows this sequence:

HTML Parsing: DOM tree construction from input HTML
Structure Analysis: Identify semantic elements and nesting
Markdown Generation: Convert elements to Markdown syntax
Query Processing: Apply mq queries to resulting structure
Output Formatting: Return filtered and formatted content

Installation

Install the mq CLI tool which includes the MCP server:

# Using Cargo
cargo install --git https://github.com/harehare/mq.git mq-cli

# Using Homebrew
brew install harehare/tap/mq

# Direct binary download
curl -L https://github.com/harehare/mq/releases/latest/download/mq-linux-x86_64 -o mq
chmod +x mq

Integration Benefits

Structured Processing: jq-like queries enable precise content extraction
Format Flexibility: Handles both HTML and Markdown inputs
Discovery Features: Built-in function and selector documentation

Resources

Support

🐛 Report bugs
💡 Request features
⭐ Star the project if you find it useful!

Efficient HTML to Markdown Conversion for LLM Input with mq-crawler

Takahiro Sato — Sat, 05 Jul 2025 15:13:30 +0000

LLM applications require clean, structured input data. Web content often contains HTML markup, navigation elements, and irrelevant content that degrades LLM performance. Converting HTML to clean Markdown while extracting only relevant content remains a common challenge.

mq-crawler addresses this by combining web crawling with HTML-to-Markdown conversion and content filtering through mq queries.

Single Binary Deployment

mq-crawler (distributed as mqcr) is a standalone binary. The single executable includes the complete web crawler, HTML parser, Markdown converter, and mq-lang query processor. This eliminates dependency management and environment setup issues common with multi-component solutions.

# Download and run immediately - no installation required
curl -L https://github.com/harehare/mq/releases/latest/download/mqcr-linux-x86_64 -o mqcr
chmod +x mqcr
./mqcr https://docs.example.com

Core Functionality

mq-crawler crawls websites, converts HTML to Markdown, and processes content with mq-lang queries. The tool respects robots.txt, implements rate limiting, and supports concurrent processing.

# Basic crawling with markdown output
mqcr https://docs.example.com

# Extract specific content sections
mqcr -q '.h2 | select(contains("API"))' https://docs.example.com

# Parallel crawling with 4 workers
mqcr -c 4 -o ./output https://docs.example.com

# Crawl only the specified URL (no recursion)
mqcr --depth 0 https://docs.example.com

HTML to Markdown Conversion

The conversion process handles complex HTML structures:

Table Processing

HTML tables convert to properly formatted Markdown tables with column alignment detection:

# Input HTML
<table>
  <tr><th>Method</th><th>Endpoint</th><th>Description</th></tr>
  <tr><td>GET</td><td>/api/users</td><td>List users</td></tr>
</table>

# Output Markdown
| Method | Endpoint   | Description |
| ------ | ---------- | ----------- |
| GET    | /api/users | List users  |

Concurrent Processing

The crawler supports parallel processing for efficiency:

# Process multiple pages concurrently
mqcr -c 8 --crawl-delay 1.0 https://docs.example.com

# Configure timeouts for different operations
mqcr --page-load-timeout 30 --script-timeout 10 --implicit-timeout 5 https://example.com

Timeout options control different aspects:

--page-load-timeout: Full page loading (default: 30s)
--script-timeout: JavaScript execution (default: 10s)
--implicit-timeout: Element finding (default: 5s)

Content Filtering with mq Queries

mq-crawler processes content through mq-lang queries for targeted extraction:

Code Example Collection

# Extract all code examples with context
mqcr -q '
  .code |
  {
    "language": attr("lang"),
    "code": to_text(),
  }
' https://tutorial.example.com

Ethical Crawling Features

robots.txt Compliance

# Respect robots.txt automatically
mqcr https://example.com

# Use custom robots.txt
mqcr --robots-path ./custom-robots.txt https://example.com

Rate Limiting

# Configure crawl delays
mqcr --crawl-delay 2.0 https://example.com

# Respectful concurrent crawling
mqcr -c 3 --crawl-delay 1.5 https://example.com

Installation and Setup

Package Installation

# Install via Homebrew
brew install harehare/tap/mqcr

# Download pre-built binary directly
curl -L https://github.com/harehare/mq/releases/latest/download/mqcr-linux-x86_64 -o mqcr
chmod +x mqcr

# Or build from source
cargo install https://github.com/harehare/mq.git mq-crawler

Results and Benefits

The mq-crawler approach provides:

Single Binary Deployment: immediate execution
Clean Markdown Output: Structured content without HTML noise
Targeted Extraction: Query-based filtering for relevant content
Ethical Compliance: Automated robots.txt respect and rate limiting
Scalable Processing: Concurrent crawling with configurable limits
LLM-Ready Format: Properly formatted Markdown

This combination reduces manual preprocessing overhead while maintaining content quality for LLM applications.

Installation

# Using Homebrew
brew install harehare/tap/mqcr

# Direct binary download (no dependencies)
curl -L https://github.com/harehare/mq/releases/latest/download/mqcr-linux-x86_64 -o mqcr
chmod +x mqcr

For other installation methods, including Docker and pre-built binaries, check the official installation guide.

Resources

Support

🐛 Report bugs
💡 Request features
⭐ Star the project if you find it useful!

In-Place Markdown Editing with mq's --update Flag

Takahiro Sato — Wed, 02 Jul 2025 15:07:59 +0000

When managing documentation across multiple markdown files, manual editing becomes inefficient. The mq command-line tool includes an --update flag that modifies files in place while preserving their markdown structure.

In-Place File Modification

Most markdown processors output filtered results without modifying source files. The -U or --update flag changes this behavior by updating the original markdown structure, replacing specific nodes with query results.

# Basic syntax for updates
mq --update 'your_query_here' file.md

Update Examples

1. Batch URL Updates

Replace URLs across multiple files:

# Update specific URLs across all markdown files
mq -U 'if (and(or(.link, .definition), matches_url("old-domain.com"))):
  update(replace("old-domain.com", "new-domain.com"))' docs/**/*.md

Before:

See our [API docs](https://old-domain.com/api)
[Reference]: https://old-domain.com/reference

After:

Check out our [API docs](https://new-domain.com/api)
[Reference]: https://new-domain.com/reference

2. Content Standardization

Standardize code block languages across your documentation:

# Convert all 'js' code blocks to 'javascript'
mq -U '.code | if (attr("lang") == "js"): set_attr("lang", "javascript")' *.md

3. Header Consistency

Ensure consistent header formatting:

# Remove trailing colons from headers
mq -U '.h | if (ends_with(":")): update(rtrimstr(":"))' documentation.md

Before:

# Getting Started:
## Installation:

After:

# Getting Started
## Installation

4. Link Reference Cleanup

Update broken internal links:

# Fix internal link references
mq -U '.link | if (starts_with("#old-section")): update(replace("#old-section", "#new-section"))' README.md

Advanced Update Patterns

Conditional Updates

Update content based on complex conditions:

# Update deprecated badges
mq -U 'if (and(.image, contains("deprecated"))): update(replace("deprecated", "legacy"))' CHANGELOG.md

Selective Text Replacement

Replace text only in specific contexts:

# Update version numbers only in code blocks
mq -U '.code | if (contains("v1.0")): update(replace("v1.0", "v2.0"))' docs/*.md

Nested Element Updates

Modify content within complex structures:

# Update URLs inside link text
mq -U '.link | if (matches_url("example.com")): update("newexample.com")' documentation/*.md

Update Safety

Advanced Patterns

Update + Custom Functions

Create reusable update patterns:

# Save to custom.mq
def modernize_badges(badge_text):
  if (contains("build-passing")):
    replace("build-passing", "build%20passing")
  else:
    badge_text;

# Use it
mq -U 'include "custom" | .image | modernize_badges()' README.md

Update + File Context

Use file information in updates:

# Add file-specific prefixes
mq -U 'let filename = __FILE__ | .h1 | let text = to_text() | update(s"[${filename}] ${text}")' docs/*.md

Update Scenarios

Documentation Maintenance

# Remove outdated warning boxes
mq -U 'select(.blockquote) | if (contains("deprecated")): update("")' docs/*.md

# Update installation commands
mq -U '.code("bash") | if (contains("npm install old-package")): update("npm install new-package")' tutorials/*.md

Link Management

# Convert relative to absolute URLs
mq -U '.link | if (starts_with("./")): update(replace("./", "https://docs.example.com/"))' *.md

# Update image CDN URLs
mq -U '.image | if (matches_url("old-cdn.com")): update(replace("old-cdn.com", "new-cdn.com"))' assets/*.md

Implementation Details

Structure Preservation

mq parses markdown into an abstract syntax tree, unlike text replacement tools:

# This won't accidentally modify code blocks or URLs
mq -U '.text | if (contains("API")): update("Interface")' docs.md

Element Targeting

Queries target specific markdown elements:

# Only update headers, not body text
mq -U '.h | if (contains("v1")): update(replace("v1", "v2"))' changelog.md

Format Preservation

Updates maintain markdown formatting conventions:

# Preserves link formatting styles
mq -U --link-title-style=double '.link | update_text("New Title")' docs.md

Integration with Workflows

CI/CD Automation

# GitHub Actions
- uses: harehare/setup-mq@v1
- name: Update Documentation URLs
  run: |
    mq -U '.link | if (contains("staging.example.com")):
      update(replace("staging.example.com", "example.com"))' docs/*.md

Installation

The easiest way to install mq is through Cargo:

cargo install --git https://github.com/harehare/mq.git mq-cli --tag v0.2.7

For other installation methods, including Homebrew, Docker, and pre-built binaries, check the official installation guide.

Summary

The --update flag modifies mq behavior from read-only filtering to in-place file editing. The feature handles markdown transformations by operating on parsed syntax trees rather than raw text, preserving document structure during modifications.

Start with simple text replacements, then implement conditional logic for complex transformations. This approach reduces manual editing overhead in documentation maintenance workflows.

Support

🐛 Report bugs
💡 Request features
⭐ Star the project if you find it useful!

Resources

Advanced mq Techniques: From Simple Queries to Complex Workflows

Takahiro Sato — Mon, 30 Jun 2025 00:58:57 +0000

Unleash the full power of mq for professional Markdown processing

Beyond Basic Queries

In my previous article, we covered the basics of mq. Now let's dive into advanced techniques that make mq indispensable for professional workflows.

Advanced Selectors and Filtering

Conditional Logic

mq supports sophisticated conditional logic:

# Extract different content based on header level
mq 'let text = to_text(self)
    | if (is_h1()):
      s"# Table of Contents for ${text}"
    elif (is_h2()):
      s"## ${text}"
    else:
      None' documentation.md

Complex Filtering

# Find code blocks that contain specific patterns
mq '.code | select(or(contains("async"), contains("function")))' api-docs.md

# Extract headers that don't contain "deprecated"
mq '.h | select(not(contains("deprecated")))' changelog.md

# Find lists with more than 3 items
mq '.[] | select(len() > 3)' requirements.md

String Interpolation and Variables

Using Variables

# Define reusable values
mq -I null 'let project_name = "my-awesome-project" |
    let version = "1.0.0" |
    s"# ${project_name} v${version}\n\nWelcome to ${project_name}!"'

Dynamic Content Generation

# Generate badges dynamically
mq -I null 'let lang = "rust" |
    let repo = "harehare/mq" |
    s"[![${lang}](https://img.shields.io/badge/language-${lang}-orange.svg)](https://github.com/${repo})"'

Working with Multiple Files

Batch Processing

# Process all markdown files in a directory
mq '.h1 | to_text()' docs/**/*.md

# Combine multiple files with separators
mq -S 's"\n---\n# ${__FILE__}\n---\n"' 'identity()' docs/*.md

# Extract code examples from all files
mq '.code("javascript") | to_text()' tutorials/**/*.md

File-Specific Processing

# Add file context to extracted content
mq 'let filename = __FILE__
    | let text = to_text(self)
    | .code
    | s"From ${filename}:\n${text}\n"' examples/*.md

Content Transformation Workflows

Documentation Processing

# Generate API documentation index
mq 'select(.h2) |
    let title = to_text(self) |
    let anchor = to_link(s"#${title}", title, "") |
    to_md_list(anchor, 1)' api/*.md > api-index.md

Content Migration

# Convert HTML to Markdown and extract specific sections
mq 'select(or(.h1, .h2))
  | let text = to_text(self)
  | if (is_h1()):
      s"# ${text}"
    else:
      s"## ${text}"' legacy-docs.html

LLM and AI Workflows

Prompt Engineering

# Extract code examples for training data
mq '.code |
    let code_content = to_text(self) |
    s"Code:\n${code_content}\n---"' tutorials/*.md

Content Preparation

# Prepare content for embedding
mq 'select(or(.h, .text)) |
    let type = to_md_name() |
    let content = to_text(self) |
    s"${type}: ${content}"' knowledge-base/*.md

Advanced Table Processing

CSV to Markdown

# Convert CSV data to markdown tables
mq 'nodes | csv2table_with_header()' data.csv

Custom Functions and Modules

Creating Reusable Logic

Create a file custom.mq:

def format_api_endpoint(method, path, description):
  s"**${method}** `${path}`\n\n${description}\n";

def extract_endpoints():
  select(or(select(starts_with("GET"), starts_with("POST"))));

Use it in your queries:

mq 'include "custom" | .code | extract_endpoints() | format_api_endpoint("GET", self, "API endpoint")' api-docs.md

Performance Optimization

Parallel Processing

# Process large numbers of files in parallel
mq -P 5 '.h1 | to_text()' docs/**/*.md

Streaming Processing

# Process large files without loading everything into memory
mq --unbuffered '.code | to_text()' large-documentation.md

Integration Patterns

CI/CD Integration

# GitHub Actions example
- name: Extract API Changes
  run: |
    mq 'select(and(.h2, contains("API"))) | to_text()' CHANGELOG.md > api-changes.txt

Build Tool Integration

# Extract version from changelog
VERSION=$(mq 'select(.h2) | to_text() | match("v?([0-9.]+)")' CHANGELOG.md)
echo "Building version: $VERSION"

Real-World Use Cases

Documentation Maintenance

# Find outdated documentation sections
mq 'select(or(contains("TODO"), contains("FIXME")))
    | s"File: ${__FILE__}\nIssue: ${to_text(self)}"' docs/**/*.md

Content Audit

# Check for broken internal links
mq '.link | select(starts_with("#")) |
    let target = slice(1) |
    s"Link target: ${target}"' *.md

Conclusion

mq excels at complex Markdown processing tasks that would be difficult or impossible with traditional text processing tools. By mastering these advanced techniques, you can build sophisticated documentation workflows, automate content processing, and integrate Markdown manipulation into your development pipeline.

The combination of structural understanding, powerful filtering, and transformation capabilities makes mq an useful tool for any developer working with Markdown at scale.

Support

🐛 Report bugs
💡 Request features
⭐ Star the project if you find it useful!

Resources

mq: The Missing Link Between jq and Markdown

Takahiro Sato — Sun, 29 Jun 2025 13:47:38 +0000

Transform Markdown with the power of jq-like queries

Introduction

Have you ever found yourself needing to extract specific content from Markdown files, manipulate documentation structures, or process content for LLM workflows? If you're familiar with jq for JSON processing, you'll love mq - a powerful command-line tool that brings the same level of querying sophistication to Markdown documents.

What is mq?

mq is a Rust-based command-line tool that processes Markdown using a syntax similar to jq. It allows you to slice, filter, map, and transform Markdown content with ease, making it an essential tool for developers working with documentation, content management, and modern AI workflows.

Why mq Matters

In today's development landscape, Markdown is everywhere:

Documentation: README files, API docs, technical specifications
LLM Workflows: Processing prompts and outputs for AI applications
Content Management: Blog posts, articles, and knowledge bases
Static Site Generation: Processing content for Jekyll, Hugo, and similar tools

Traditional text processing tools like grep, sed, and awk work at the text level, but Markdown has structure that these tools can't understand. mq understands Markdown's semantic structure, allowing you to work with headers, code blocks, lists, and tables as first-class objects.

Getting Started

Installation

The easiest way to install mq is through Cargo:

cargo install --git https://github.com/harehare/mq.git mq-cli --tag v0.2.7

For other installation methods, including Homebrew, Docker, and pre-built binaries, check the official installation guide.

Your First Query

Let's start with a simple example. Create a file called example.md:


# My Project

This is a sample project with multiple sections.

## Features

- Fast processing
- Easy to use
- Extensible

## Code Examples

``javascript
console.log("Hello, World!");
``

``python
print("Hello, World!")
``

## Installation

Run the following command:jj

``bash
npm install my-project
``

Now, let's extract all the code blocks:

mq '.code' example.md

This will output all code blocks from the file. The .code selector specifically targets code block elements in the Markdown structure.

Basic Selectors

mq provides several built-in selectors:

.code - Select all code blocks
.h1, .h2, .h3, etc. - Select headers by level
.[] - Select list items
.[][] - Select table cells

Let's try a few more examples:

# Extract all headers
mq '.h' example.md

# Extract only level 2 headers
mq '.h2' example.md

# Extract all list items
mq '.[]' example.md

# Extract JavaScript code blocks only
mq '.code("javascript")' example.md

Filtering and Transforming

One of mq's strengths is its ability to filter and transform content:

# Find headers containing "install"
mq '.h | select(contains("install"))' example.md

# Convert all headers to text
mq '.h | to_text()' example.md

# Extract code blocks and convert to text
mq '.code | to_text()' example.md

Practical Examples

Generate Table of Contents

mq 'select(or(.h1, .h2, .h3)) | let link = to_link(add("#", to_text(self)), to_text(self), "") | if (is_h1()): to_md_list(link, 1) elif (is_h2()): to_md_list(link, 2) elif (is_h3()): to_md_list(link, 3) else: None' example.md

Extract All Code Examples

mq '.code | to_text()' example.md

Find Specific Content

# Find all sections mentioning "installation"
mq 'select(contains("installation"))' example.md

Interactive Development

For experimentation and learning, mq provides a REPL (Read-Eval-Print Loop):

mq repl

This allows you to interactively test queries and explore your Markdown structure.

IDE Support

mq comes with excellent tooling support:

VSCode Extension: Available on the Visual Studio Marketplace
Language Server Protocol (LSP): For custom function development
Syntax Highlighting: For .mq files

What's Next?

This introduction covers the basics of mq, but there's much more to explore:

Advanced selectors and filtering
Custom functions and modules
Batch processing multiple files
Integration with CI/CD pipelines
Web API and Python bindings

mq transforms how you work with Markdown, making complex document processing tasks simple and intuitive. Whether you're managing documentation, processing content for AI workflows, or building content pipelines, mq provides the tools you need.

Resources

Start exploring mq today and discover how it can streamline your Markdown workflows!

Support

🐛 Report bugs
💡 Request features
⭐ Star the project if you find it useful!

Extract AI Rules from Markdown Files with mq

Takahiro Sato — Thu, 26 Jun 2025 15:16:23 +0000

Managing separate AI rule files for different systems? mq can generate them from one master file.

The Problem

Each AI system needs its own rule file. Managing multiple separate Markdown files gets messy quickly.

Simple Solution

Keep all AI rules in one master file, then generate individual files:

$ mq 'nodes | sections(2) | filter(fn(section): let head = first(section) | contains(head, "Claude Rules");) | first() | map(decrease_header_level) | to_markdown_string()' master-ai-rules.md > claude-rules.md

Generate Multiple AI Rule Files

# Extract different AI system rules from one master file
$ mq 'nodes | sections(2) | filter(fn(s): contains(first(s), "GPT Rules")) | first() | map(decrease_header_level) | to_markdown_string()' master-ai-rules.md > gpt-rules.md
$ mq 'nodes | sections(2) | filter(fn(s): contains(first(s), "Gemini Rules")) | first() | map(decrease_header_level) | to_markdown_string()' master-ai-rules.md > gemini-rules.md
$ mq 'nodes | sections(2) | filter(fn(s): contains(first(s), "Claude Rules")) | first() | map(decrease_header_level) | to_markdown_string()' master-ai-rules.md > claude-rules.md

Why This Approach Works

Single source of truth: All AI rules in one master file
Easy maintenance: Update rules in one place
Automatic generation: Create individual files with one script
Version control friendly: Track changes in one file
No vendor lock-in: Generated files are plain Markdown, work anywhere

Try It

$ brew install harehare/tap/mq
$ mq 'nodes | sections(2) | filter(fn(s): contains(first(s), "Claude Rules")) | first() | map(decrease_header_level) | to_markdown_string()' master-ai-rules.md > claude-rules.md

Now you can manage all AI rules in one file and generate individual rule files as needed!

Support

🐛 Report bugs
💡 Request features
⭐ Star the project if you find it useful!

Optimize NotebookLM Input with mq: Efficient Markdown File Merging

Takahiro Sato — Thu, 19 Jun 2025 14:54:54 +0000

Uploading multiple Markdown files to NotebookLM one by one is tedious. mq solves this by merging and filtering files into a single, optimized input for better AI analysis.

Basic File Merging

Merge all content from multiple files:

mq --separator 's"\nFILE: ${__FILE__}\n"' 'identity()' notes/*.md > merged.md

Extract Key Information

Keep only headers and links for focused analysis:

mq --separator 's"\nFILE: ${__FILE__}\n"' 'select(or(.h, .link))' notes/*.md > merged.md

Include code blocks too:

mq --separator 's"\nFILE: ${__FILE__}\n"' 'select(or(.h, .link, .code))' notes/*.md > merged.md

Common Use Cases

Documentation: Combine README, API docs, and guides
Research: Merge papers, notes, and references
Projects: Consolidate specs, meeting notes, and plans

Upload the generated merged.md to NotebookLM for streamlined AI-powered analysis and Q&A.

Support

🐛 Report bugs
💡 Request features
⭐ Star the project if you find it useful!

Extracting Markdown Headers and Links for LLMs with mq

Takahiro Sato — Sat, 07 Jun 2025 14:51:13 +0000

There may be times when you want to extract just the headers or links from Markdown files, rather than working with the entire document.
You can also use Large Language Models (LLMs) with only the extracted headers or links, instead of always providing the full document.

Here's how you could try this with mq, a jq-like CLI tool for Markdown:

Command

Note: The following command assumes you are running it in zsh.

mq '.h | let h = to_text() | s"- [${h}](${__FILE__})"' **/*.md

.h selects all headers
let h = to_text() gets header text
s"- [${h}](${__FILE__})" outputs a Markdown link with the file path

Example

Suppose you have:

docs/intro.md

# Introduction
## Getting Started

docs/usage.md

# Usage
## Advanced

Run:

mq '.h | s"- [${self}](${__FILE__})"' docs/*.md

Output:

- [# Introduction](docs/intro.md)
- [## Getting Started](docs/intro.md)
- [## Example Code Block](docs/intro.md)
- [# Usage](docs/usage.md)
- [## Advanced](docs/usage.md)

Run:

mq '.h | let h = to_text() | s"- [${h}](${__FILE__})"' docs/*.md

Output:

- [Introduction](docs/intro.md)
- [Getting Started](docs/intro.md)
- [Example Code Block](docs/intro.md)
- [Usage](docs/usage.md)
- [Advanced](docs/usage.md)

You could use this list as a table of contents, or provide it to an LLM as a hint for summarization or exploration, instead of providing the full document.
This approach might be useful in some scenarios.

Install

cargo install --git https://github.com/harehare/mq.git mq-cli
# or
brew install harehare/tap/mq

Support

🐛 Report bugs
💡 Request features
⭐ Star the project if you find it useful!

mq makes it easy to extract and format Markdown structure—such as headers and links—which may be helpful in some cases as an alternative to inputting the entire document.