Forem: Marc Gille-Sepehri

Self-Starting Processes: Timer Events, Mailbox Polling, and Why Your Agentic Workflows Should Launch Themselves

Marc Gille-Sepehri — Thu, 16 Apr 2026 21:55:48 +0000

Follow-up to Why We Keep Process Data Outside the Engine

In the first article, we made the case for keeping process data outside the engine. The instanceId is the only binding key. Your code handles the intelligence. The engine handles the orchestration.

But there was a gap in that story. Someone still had to start the process.

A REST call. A button click. A cron job in a separate service calling startInstance(). The engine could orchestrate anything — once a human or an external trigger told it to begin. The process itself had no agency over its own lifecycle.

That changes now.

Processes That Start Themselves

in-concert now supports timer start events and message start events — standard BPMN elements that let a process definition declare when and why it should launch, without any external trigger.

A timer start event says: run this process every hour. Or every weekday at 8:30. Or on the last Friday of every month. Or three times at 10-minute intervals, then stop.

A message start event says: run this process when an email arrives in this mailbox.

Deploy the BPMN. The engine takes it from there. No Lambda functions. No external scheduler. No webhook plumbing. The process definition is self-sufficient.

Timer Start Events — Every Flavour of "When"

Put a timer on a start event and the engine creates a persistent schedule. A background worker fires it, starts an instance, advances the schedule, and goes back to sleep. If the server restarts, the schedule is in MongoDB — nothing is lost.

<bpmn:startEvent id="TimerStart" name="Daily compliance check">
  <bpmn:timerEventDefinition>
    <bpmn:timeCycle>R/P1D</bpmn:timeCycle>
  </bpmn:timerEventDefinition>
</bpmn:startEvent>

That is a process that runs once a day, forever, until you pause it. The engine supports five expression formats:

Format	Example	What it does
ISO 8601 repeating interval	`R/PT1H`, `R3/PT10M`	Every hour (unbounded), or 3 times at 10-min intervals
ISO 8601 duration	`PT30M`	Once, 30 minutes after deploy
ISO 8601 date-time	`2026-12-25T00:00:00Z`	Once, at that exact moment
Cron (5-field)	`30 8 * * 1-5`	Weekdays at 8:30
RRULE (RFC 5545)	`FREQ=MONTHLY;BYDAY=FR;BYSETPOS=-1`	Last Friday of every month

RRULE is the format behind Outlook and Google Calendar recurrence. Any pattern you can set in a calendar invitation, you can use to schedule a process. FREQ, INTERVAL, BYDAY, BYMONTHDAY, BYMONTH, BYSETPOS, COUNT, UNTIL — all supported. Zero external dependencies.

<bpmn:startEvent id="TimerStart" name="Last Friday of every month">
  <bpmn:timerEventDefinition>
    <bpmn:timeCycle>DTSTART:20260130T090000Z
RRULE:FREQ=MONTHLY;BYDAY=FR;BYSETPOS=-1</bpmn:timeCycle>
  </bpmn:timerEventDefinition>
</bpmn:startEvent>

Pause and resume any schedule at runtime via the SDK or REST API. The schedule is a first-class object — queryable, manageable, observable.

const schedules = await client.listTimerSchedules({ definitionId });
await client.pauseTimerSchedule(schedules[0]._id);
// ... later
await client.resumeTimerSchedule(schedules[0]._id);

Why This Matters for Agentic Workflows

Agentic systems are not request-response. They are continuous. A compliance monitoring agent should check every morning whether anything changed overnight. A portfolio rebalancing agent should evaluate positions on a schedule. A reporting agent should assemble and distribute summaries at the end of every week.

These are not one-off tasks triggered by a user. They are standing processes with their own heartbeat. Timer start events give them that heartbeat — expressed in standard BPMN, persisted in the engine, surviving restarts and deployments.

Message Start Events — Email as a Process Trigger

Timer events handle "when." Message events handle "what happened."

A message start event with the graph-mailbox connector tells the engine: poll this Microsoft 365 mailbox, and when an unread email arrives, start a process instance.

<bpmn:message id="Msg_Inbox" name="inbox-poll"
  tri:connectorType="graph-mailbox"
  tri:mailbox="support@your-company.com" />

<bpmn:startEvent id="Start" name="Email received">
  <bpmn:messageEventDefinition messageRef="Msg_Inbox" />
</bpmn:startEvent>

Two tri: extension attributes on the <bpmn:message> element identify the connector type and the mailbox. The Graph API credentials are configured once as engine settings — environment variables or SDK init() — and never appear in the BPMN.

Deploy the process. The engine polls. An email arrives. A process instance is created.

The onMailReceived Callback

Here is where the "data outside the engine" principle from the first article meets the real world.

The engine creates the process instance — so you have an instanceId — but does not advance a single token until your callback returns. Your code receives the full email: subject, sender, body, and attachment metadata. You store it in your domain. You decide whether to proceed.

client.init({
  connectors: {
    'graph-mailbox': {
      tenantId: process.env.GRAPH_TENANT_ID,
      clientId: process.env.GRAPH_CLIENT_ID,
      clientSecret: process.env.GRAPH_CLIENT_SECRET,
    },
  },

  onMailReceived: async ({ mailbox, email, instanceId, getAttachmentContent }) => {
    // Store the email in your domain, bound to the process instance
    await myStore.saveEmail(instanceId, {
      subject: email.subject,
      from: email.from.address,
      body: email.body.content,
      receivedAt: email.receivedDateTime,
    });

    // Download attachments on demand — metadata is already there, content is lazy
    for (const att of email.attachments) {
      const buffer = await getAttachmentContent(att.id);
      await myStorage.upload(instanceId, att.name, buffer, att.contentType);
    }

    // Return { skip: true } to terminate the instance without running
    if (isSpam(email)) return { skip: true };
  },

  onServiceCall: async ({ instanceId, payload }) => {
    // Your agentic logic — LLM calls, tool invocations, etc.
  },
});

Attachments are not pre-loaded into memory. The callback receives metadata — name, content type, size — and a getAttachmentContent() function that downloads a single attachment on demand. A 40 MB zip does not sit in your Node process unless you explicitly ask for it.

The { skip: true } return value terminates the instance. Spam filter, sender allowlist, duplicate detection — your code, your rules. The engine created the instance so you have an instanceId to correlate against. If you skip, it is cleanly terminated. If you proceed, the process runs.

Why This Matters for Agentic Workflows

Email is the entry point for most business processes in the real world. Customer requests, supplier invoices, regulatory notifications, internal approvals — they arrive as emails with attachments, and someone has to triage them, extract data, route them, and act.

This is exactly what agentic workflows do. The BPMN process models the routing. The LLM handles the triage and extraction. The human task is the escalation point when the AI is uncertain. And now the trigger — the email itself — is part of the process definition.

No middleware. No separate polling service. No Azure Function glue. The BPMN file declares the mailbox. The engine polls it. Your onMailReceived callback stores the data. The process runs.

A support email arrives → the agent extracts the intent → checks the knowledge base → drafts a response → routes to a human reviewer if confidence is low → sends the reply. All modelled in BPMN. All starting from an email. All running without anyone clicking "start."

The Architecture — Same Pattern, Different Triggers

Both timer and message start events follow the same internal pattern we use for the continuation worker:

Deploy creates a persistent schedule document in MongoDB
Worker loop polls for due schedules, claims with an optimistic lease
Fire calls startInstance() — same as if you called it yourself via the API
Advance updates the schedule (next fire time, or mark as exhausted)

Multi-instance safe. Survives restarts. No in-memory state. The schedule is a MongoDB document with an index — the same infrastructure the engine already uses for continuations and outbox delivery.

Come Build With Us

Timer and message start events are available now in @the-real-insight/in-concert on npm. The full documentation — RRULE expressions, cron, Graph mailbox setup, onMailReceived callback reference — is in the SDK usage guide.

If you are building agentic workflows and want your processes to have their own lifecycle — starting on a schedule, reacting to emails, running continuously without external triggers — this is the engine layer for that.

Star the repo. Try it on a real process. Open an issue if something does not work the way you expect. The BPMN subset is growing, and the patterns we are building — timer-driven agents, email-triggered workflows, LLM-routed decisions — are where #agenticbpm gets practical.

We are The Real Insight GmbH, and we believe BPMN is the orchestration backbone for the agentic era. Processes should not wait to be told when to start. They should know.

→ github.com/The-Real-Insight/in-concert
→ npmjs.com/package/@the-real-insight/in-concert
→ the-real-insight.com

Powered by The Real Insight GmbH BPMN Engine — the-real-insight.com

Why We Keep Process Data Outside the Engine — and Why It Changes Everything for Agentic BPM

Marc Gille-Sepehri — Sat, 11 Apr 2026 11:19:37 +0000

There is a design decision at the heart of in-concert that surprises people when they first encounter it: the engine knows nothing about your data.

No domain objects stored inside the engine. No variable interpolation in BPMN expressions. No built-in scripting that reaches into your database. When a process instance is running, the engine holds exactly one thing that belongs to you: an instanceId. Everything else — documents, application state, business context, AI responses — lives in your systems, bound to that id.

This is not an oversight. It is the central architectural choice, and it shapes everything else about how in-concert works. And this is the beginning of #agenticbpm.

The Problem with Data-Coupled Engines

Traditional BPM engines — Camunda, Flowable, Activiti — manage process variables alongside process state. You deploy a BPMN, you pass in variables, and the engine stores them, interpolates them into conditions, and threads them through the execution. It is convenient at first. Then reality arrives.

Your process needs to evaluate a condition against data that lives in your ERP. Or the "variable" is actually a 40-page document. Or the service task needs to call an LLM with context assembled from five different sources. Or your security model requires that PII never leaves your own database.

Suddenly the engine is not a neutral orchestrator. It has become a data store you did not ask for, a security boundary you have to manage, and an integration point that does not understand the shape of your actual domain.

We built in-concert after running into exactly these problems. The solution was radical simplicity: the engine does not store your data, period.

The Three Touch Points — and Why They Are All Fuzzy

In any BPMN process, there are three places where execution intersects with the outside world. In a traditional engine, these are handled by scripting, expression languages, and built-in connectors. In in-concert, they are handled by your code — deliberately, explicitly, and with full access to everything you know.

1. Service Tasks

A service task means "call something external and continue." In a classic engine, you write a connector or a script that runs inside the engine's JVM or Node.js process. The engine manages the call, handles the result, and stores output variables.

In in-concert, the engine calls your handler and waits:

client.init({
  onServiceCall: async ({ instanceId, payload }) => {
    // Full control. Assemble context from anywhere.
    const context = await myDataStore.getContextFor(instanceId);
    const result = await myLLM.invoke(payload.extensions?.toolId, context);
    await client.completeExternalTask(instanceId, payload.workItemId, { result });
  },
});

The handler receives the instanceId and whatever metadata you put on the BPMN node (tri:toolId, tri:toolType, custom extensions). It completes when it is done — whether that is 50ms or 50 minutes later, whether via a direct response, a message queue reply, or a webhook. The engine waits. It does not care how long.

But "call an LLM" understates what the handler can actually do. Consider what happens in a real agentic workflow:

The LLM as context mapper. Your process node carries a tri:toolId that identifies an MCP tool — say, search-crm or generate-proposal. The tool has a defined input schema. Your application data has its own shape. The LLM's job here is not to answer a question — it is to map your domain objects into the tool's input format, invoke the tool, and map the structured output back into whatever your process needs next.

onServiceCall: async ({ instanceId, payload }) => {
  const toolId = payload.extensions?.toolId;       // e.g. "search-crm"
  const tool = mcpClient.getTool(toolId);           // get tool + schema
  const context = await myDataStore.getContextFor(instanceId);

  // LLM maps context → tool input schema
  const toolInput = await llm.mapToToolInput(tool.inputSchema, context);

  // Invoke the MCP tool
  const toolOutput = await mcpClient.invoke(toolId, toolInput);

  // LLM maps tool output → domain result
  const result = await llm.mapFromToolOutput(tool.outputSchema, toolOutput, context);

  await client.completeExternalTask(instanceId, payload.workItemId, { result });
},

This pattern — LLM as the mapping and reasoning layer around structured tool calls — is where BPMN and agentic AI (i.e. #agenticbpm) meet most naturally. The process definition models the flow and the intent. The BPMN node identifies which tool to use. The LLM handles the fuzzy work of bridging between your data model and the tool's contract. And because all of this happens in your code, you can swap models, adjust prompts, and iterate on the mapping logic without touching the process definition at all.

The handler is also where long-running integration lives. Publish a job to a queue, return immediately, and complete the task when the consumer acknowledges. Poll an external system until it is ready. Wait for a webhook. None of this requires anything special from the engine — it simply waits until your handler calls completeExternalTask.

2. Transition Conditions — the XOR Gateway

This is where the fuzziness gets interesting. In a BPMN XOR gateway, one of several outgoing flows is selected based on a condition. In a traditional engine, you write an expression: ${amount > 1000}, or a FEEL expression, or a Groovy script. The engine evaluates it against stored variables.

But what if the condition is not a clean boolean expression? What if it is "does this application look fraudulent?" or "is this document complete enough to proceed?" or "based on the conversation so far, which department should handle this?"

These are not expressions. They are judgements — and judgements require context, and often require an LLM.

In in-concert, gateway decisions are routed to your handler:

client.init({
  onDecision: async ({ instanceId, payload }) => {
    // payload.transitions is the list of outgoing flows with names and conditions
    // You evaluate — using your data, your rules, your LLM
    const context = await myDataStore.getContextFor(instanceId);
    const selected = await myRouter.evaluate(payload.transitions, context);
    await client.submitDecision(instanceId, payload.decisionId, {
      selectedFlowIds: [selected.flowId],
    });
  },
});

The engine gives you the transition options. You choose. The evaluation logic — however simple or sophisticated — belongs to you. You can use a simple if/else. You can call an LLM with the full application context. You can run a rules engine. The engine does not prescribe how you decide; it only records that you did.

3. Human Tasks — the Worklist

User interaction is the most obviously fuzzy of the three. A human task is not deterministic. The user brings judgement, context, domain knowledge, and occasionally the wrong answer. The task might be "review this contract," "approve this expense," or "assess whether this customer qualifies."

In in-concert, human tasks are projected to a queryable worklist. Your UI queries it, filtered by role, by claimed status, by instance. The user sees the task, opens your application where the full document and context live, makes a decision, and your code calls completeUserTask() with the result.

The engine never renders a form. It never stores the contract. It never knows what the user saw. It only knows that a human task at a given node in a given process instance was completed with a given result — and it advances accordingly.

This lets you build any interaction model: cherry-picking worklists, supervisor assignment, AI-assisted pre-screening before human review. The engine is the backbone, not the bottleneck.

What This Unlocks for Agentic BPM

Here is the part that we find genuinely exciting.

AI agents need orchestration. A single LLM call is not a workflow — it is a function. Useful, but limited. Real agentic systems involve sequences of steps, parallel branches, human checkpoints, error handling, retries, long-running waits. They need state across time. They need the ability to hand off between AI and human. They need audit trails.

BPMN is a remarkably good fit for this. It has been modelling complex, long-running processes for decades. It handles parallelism, subprocesses, boundary events, timers, and message correlation out of the box. And it is visual — a BPMN diagram is something a business analyst and a developer can read together.

in-concert brings BPMN to agentic systems with a clean separation: the engine handles the orchestration; your code handles the intelligence.

Service tasks become LLM invocations. Gateway decisions become LLM evaluations against your domain context. Human tasks become the checkpoints where a person reviews or overrides what the AI decided. And because all data and logic live outside the engine, you can iterate on your prompts, your models, and your routing logic without touching the process definition.

The instanceId is the thread that holds it together. Every LLM call, every database query, every human task can be correlated to a specific process instance. You know exactly where in the process you are, what decisions were made, and what the audit trail looks like — because in-concert records all of that.

Try It

in-concert is open source, MIT-licensed (with attribution), and published on npm:

npm install @the-real-insight/in-concert

The SDK works in two modes. For microservice deployments, run the engine as a standalone service and connect via REST and WebSocket. For embedded or test use, run it directly in-process against MongoDB — same API, no server needed.

import { BpmnEngineClient } from '@the-real-insight/in-concert/sdk';

// REST mode
const client = new BpmnEngineClient({ mode: 'rest', baseUrl: 'http://localhost:3000' });

// Local / embedded mode
const client = new BpmnEngineClient({ mode: 'local', db });

The full quick start, API reference, and BPMN conformance matrix are in the GitHub repository. The README documents every SDK method with accurate type signatures pulled directly from the package.

Come Build With Us

in-concert is early. The BPMN subset is intentionally focused — we implement what production workflows actually need, and we fail loudly on anything we do not support yet. There is meaningful work to be done on the conformance surface, the developer experience, and the agentic integration patterns.

If this resonates with you — if you have built on BPM engines and felt the friction of data-coupled orchestration, or if you are thinking about how to bring structure to agentic AI workflows — we would love to have you involved.

Star the repo. Try it on a real process. Open an issue. Submit a PR. The contribution guide is in docs/contributing.md and there are good first issue labels for anyone who wants to start small.

We are The Real Insight GmbH, and we are building the engine layer for #agenticbpm. This is just the beginning.

→ github.com/The-Real-Insight/in-concert
→ npmjs.com/package/@the-real-insight/in-concert
→ the-real-insight.com

Powered by The Real Insight GmbH BPMN Engine — the-real-insight.com