Forem: MongoDB Guests

Whistleblower for Database: Set up an internal informant who exposes performance

MongoDB Guests — Thu, 07 May 2026 10:44:20 +0000

This tutorial was written by Darshan Jayarama.

As DBAs, waking up at 2 AM because production is down or slow is a nightmare, and none of us like it. However, we all know that it is not instant karma; it is like a pressure cooker where the database has reached its breaking point. Database whistleblowers constantly give signals that we ignore before it explodes into a Production Issue.

Whistleblower 1: Performance Informant

High CPU utilization?

CPU utilization is not just an alert, it is a future prediction with regard to the database. When you get constant High CPU utilization alerts, think of it as a Whistleblower reviewing the database logs, metrics, and processes that are consuming CPU. If possible, shift those processes to a different server and let your Database breathe without pressure. Ignoring high CPU utilization can lead to high latency.

Query <5 seconds?

A query that was blazing fast is now causing a slow query alert since the last data load. It is a clear indication of a Missing or Unoptimized index. If an index is missing, create one. If unoptimized, then redesign a better index. Still facing the same? It needs a deeper and wider investigation.

Cache hit ratio <80%?

A lower cache hit ratio means higher disk reads, which tends to cause high IO Wait, and an Increased disk queue depth. It can happen if your query is unoptimized (we had addressed this earlier), or if lower memory/cache is assigned. Increase the memory if there is nothing pending in Database tuning.

Whistleblower 2: Cost Informant

Storage >80%:

Set your whistleblower when storage utilization hits >80% its your duty now to analyze storage to archive any old logs, old data, or remove unwanted processes that consume your space.

IOPS > Budget?

We usually get an alert for a high document-scanned-to-returned ratio, which indicates that queries are performing more IO than required. Optimize them. Reduce IOPS consumption by rewriting queries, redesigning the schema, and implementing a better Index strategy.

Unexpected auto scale?

MongoDB Atlas has a feature that monitors system load and automatically scales up or down as needed. Implement the solution. In case of a scale down, it does require 24-hour monitoring, but if you are sure there are no more surprises, you can scale down yourself.

Whistleblower 3: Reliability Informant

Replication lag >10 Seconds?

Investigate where it is lagging. Is it in primary? Must be loaded from writes, which lowers the read request priority. It usually triggers the built-in flow-control logic. If secondary is the issue, then is it at writing or reading? A detailed understanding of flow can help resolve this data durability. Ask developers to set writeConcern to match the application's requirements.

Oplog <24Hours?

A lower oplog size means lower retention of replication events. Set at least 24 hours, resize as per the application write load minOplogRetention.

Connection >80%?

Each connection can consume 1 MB of memory; the more connections, the more memory is consumed. Maintain connection hygiene, use a connection pool, and close connections when no longer needed.

Conclusion: Your production incident is one missed alert away; hire your whistleblower. Choose: Pressure cooker explosion → OR → Proactive DBA legend?

Vacuum Seal Your MongoDB: Cuts Cost Down 50% Today

MongoDB Guests — Tue, 05 May 2026 10:10:10 +0000

This article was written by Darshan Jayarama.

I was recently packing for a short trip, and I personally hate carrying 2–3 pieces of luggage. One for my laptop, another for clothes. So a person like me has definitely used a “vacuum sealer,” allowing me to pack as many clothes as I want.

During the trip, I had an idea. Which vacuum sealers do we have to help store efficiently and reduce billing? Here, I have jotted down some…

Online Archive

Does your dataset contain cold data? For a supply chain company, data older than three months is considered infrequently accessed. For an e-commerce website or a courier company, data that has already been delivered is accessed by users very rarely. We can group them using a query, such as lastAccess > 30, and create the Online archive store in cloud object storage. Worried about how to access this when needed? Relax — it’s all covered, there are no access restrictions. Storage costs drop from $4/GB/Month to $0.02/GB/Month.

Table Compact

In a write-intensive application where you frequently delete data, storage watermarks always stay high. You can run the below command to check:

db.COLLNAME.stats().freeStorageSize

The above command outputS how many bytes are available to be reused. Even though wiredTiger uses this space organically over time, if the storage utilization is effectively essential, you can run compact on the collection.

We can run compact with the following syntax:

use mydb // switch to the database 
db.runCommand( { compact: "myCollection" } ) // mention the collection name

As it also has many other options, I would recommend visiting the official documentation of compact.

Resync Secondary:

Running compact is fine if you’ve got just a handful of collections. But when you’re dealing with many collections with high watermarks, especially large ones, running compact isn't ideal.

You can resync a member in rolling fashion so that all members can release that space back to the operating system and utilize the storage efficiently.

TTL Index:

So far, we have discussed the reactive approach for storage utilization. We are preparing the plan once the storage hits the ceiling. But do we have any such proactive approach? Yes, here is where TTL comes into play. If you believe the data is no longer needed after a certain time, set the TTL index to expire, and the record will be deleted.

This approach will be useful for the application that generates large logs. Set the TTL index for 3 days, and the log entry will be removed from the collection.

Query Optimization:

Being a query enthusiast, I circle back to query optimization, as a single bad query can eat up your IOPS, Network, and Compute.

Tune the query.
Create a compound index covering the query and the projection
Reduce the network round-trip and IOPS
If possible, precompute the value and store it in a collection.

Don't know where to start? Use/Abuse the Atlas Performance Advisor for index recommendations, inefficient queries, and inefficient schema advice. Or visit my previous blogs:

Flex Cluster:

Planning to perform some dev/API test? Go with the Flex cluster rather than the M10+ cluster for non-prod deployment. Same API, essentially Pause/Resume when not needed.

Conclusion:

MongoDB gives tons of storage-saving options — but only smart DBAs use them. Your cluster is bleeding $$$ right now. Online Archive, TTL indexes, compaction, proper sizing = 60% instant savings.

Pick 3 techniques → Deploy → Watch bill shrink. Simple as that. 💰

My first “local” Vector Search: MongoDB community edition

MongoDB Guests — Thu, 30 Apr 2026 10:56:11 +0000

This article was written by Darshan Jayarama.

Ever since I received an email about the vector search auto-Embedding feature released in MongoDB community edition, my palms have been itching to test it out. And now.. I’m happily writing this blog after witnessing its power firsthand.

Vector Search is one of the most admired, powerful, and amazing products of MongoDB. I felt pity for MongoDB Community/On-prem customers, as they were unable to use this because Vector search and Atlas search were only available in MongoDB Atlas. But this is no longer the case!

How did I test this?

I am sure by now you might be thinking, “Enough of this product endorsement, show me the code!”

I’m getting there :). Initially, I followed what the MongoDB documentation suggests, but I struggled a lot as there is a disconnection when setting up the Docker containers:

As per the Installation documentation, select Docker as your operating system.
When the MongoDB community server starts, it starts with hostname as mongod.search-community:27017. Make a note of this.
When we asked to start the MongoDB Search container, in that yml file, it sets the syncsource as mongot-community.search-community:27017
- Don't just copy/paste the yml from there. Make sure you are setting the correct syncSource, which should be the Community server container name mongod.search-community:27017. To ensure your container hostname, run db.isMaster().me, it should print you the hostname.
Next, there is mongot username and password entry that should be made in the yml file. Whileyou are creating a user, the instructions specify mongot user as mongot, but in the yml file, it is mentioned as mongotUser. If you created it with another name, correct the section.

After these corrections, you can see the 2 containers running happily (I literally jumped out of my chair over joy). Next, connect to the mongod using mongosh to test the power.

I created an index on the favorite movies collection, plot field:

db.movies.createSearchIndex("vector_index", "vectorSearch", {
 "fields": [
   {
     "type": "autoEmbed",
     "modality": "text",
     "path": "plot",
     "model": "voyage-4"
   },
   {
     "type": "filter",
     "path": "year"
   }
 ]
})

You can use voyage-4-lite, but as per the voyage usage stats, both voyage-4 and voyage-4-lite both have the same 10000TPM 3RPM limitation. My preferred choice is voyage-4.

Then I wanted to see the results based on the context. So I ran the below query:

db.movies.aggregate([
 {
   "$vectorSearch": {
     "index": "vector_index",
     "path": "plot",
     "query": {
       "text": "bullied boy learns karate"
     },"model": "voyage-4-lite" ,
           "numCandidates": 10000,
     "limit": 10
   }
 },
 {
   "$project": {
     "_id": 0,
     "title": 1,
     "year":1,
     "plot": 1,
     "score": { $meta: "vectorSearchScore" }
   }
 }
])

The above query should retrieve movies with context of “bullied boy learns karate” (here we expect ‘karate-kid’):

The answers amused me:

[
 {
   plot: 'A love-struck weakling must pretend to be boxer in order to gain respect from the family of the girl he loves.',
   title: 'Battling Butler',
   year: 1926,
   score: 0.6800339221954346
 },
 {
   plot: 'Two young brothers become the leaders of a gang of kids in their neighborhood. Their father is an office clerk who tries for advancement by playing up his boss. When the boys visit the boss...',
   title: 'I Was Born, But...',
   year: 1932,
   score: 0.6692402362823486
 },
 {
   plot: 'A living puppet, with the help of a cricket as his conscience, must prove himself worthy to become a real boy.',
   title: 'Pinocchio',
   year: 1940,
   score: 0.6628834009170532
 },
 {
   plot: 'An idealistic adolescent, suffering under the thumb of a sadistic schoolmaster, falls in love with a loose girl who is bullied and tormented by another lover.',
   title: 'Torment',
   year: 1944,
   score: 0.6620646715164185
 },
 {
   plot: `Against all odds Father Flanagan starts "Boys' Town" after hearing a convict's story. Whitey Marsh comes there. He runs away but, hungry, returns. He runs away again but, when friend Pee ...`,
   title: 'Boys Town',
   year: 1938,
   score: 0.6570459604263306
 },
 {
   plot: 'When three thuggish men are responsible for the death of his father and the crippling of his brother, young David must choose between supporting his family or risking his life and exacting vengeance.',
   title: "Tol'able David",
   year: 1921,
   score: 0.6559526920318604
 },
 {
   plot: "Fight promoter Nick Donati grooms a bellhop as a future champ, but has second thoughts when the 'kid' falls for his sister.",
   title: 'Kid Galahad',
   year: 1937,
   score: 0.6525536179542542
 },
 {
   plot: 'In a repressive boarding school with rigid rules of behavior, four boys decide to rebel against the direction on a celebration day.',
   title: 'Zero for Conduct',
   year: 1933,
   score: 0.651709794998169
 },
 {
   plot: 'While at a ski lodge, Larry Blake sees instructor Karin Borg and decides to sign up for private lessons. The next thing he knows, she is Mrs. Blake. When he announces that he is going back ...',
   title: 'Two-Faced Woman',
   year: 1941,
   score: 0.6513179540634155
 },
 {
   plot: 'To reconcile with his girlfriend, a bookish college student tries to become an athlete.',
   title: 'College',
   year: 1927,
   score: 0.6459328532218933
 }
]

Even though the expected result was not there, all the results related to karate, boxing, or being bullied.

Below are the commands I have used to complete the setup of this.

Pulling docker images;

docker pull mongodb/mongodb-community-search:latest
docker pull mongodb/mongodb-community-server:latest

Create internal docker network to communicate

docker network create search-community

Starting community server container

echo '
net:
  port: 27017
  bindIpAll: true  # Equivalent to --bind_ip_all

replication:
  replSetName: rs0
systemLog:
  destination: file
  path: "/var/log/mongodb/mongod.log"
  logAppend: true

setParameter:
  searchIndexManagementHostAndPort: mongot-community.search-community:27028
  mongotHost: mongot-community.search-community:27028
  skipAuthenticationToSearchIndexManagementServer: false
  useGrpcForSearch: true

# Security configuration
security:
  authorization: enabled  # Equivalent to --auth
  keyFile: /keyfile' > mongod.conf

mkdir ./data/db

openssl rand -base64 756 > keyfile
chmod 400 keyfile

docker run -d --rm --name mongod -e MONGODB_INITDB_ROOT_USERNAME=root -e MONGODB_INITDB_ROOT_PASSWORD=rootpass -v ./mongod.conf:/etc/mongod.conf:ro -v ./data/db:/data/db -v ./keyfile:/keyfile -p 27017:27017 --network search-community mongodb/mongodb-community-server:latest --config /etc/mongod.conf

Initiate replication

mongosh -u root -p rootpass --eval 'rs.initiate(); sleep(10); rs.status()'
mongosh -u root -p rootpass --eval "db.getSiblingDB('admin').createUser(
 {
   user: 'mongot',
   pwd: 'mongotPassword',
   roles: ['searchCoordinator']
 }
)"

Load sample movies collection data
Downloaded movies from https://github.com/neelabalan/mongodb-sample-dataset/blob/main/sample_mflix/movies.json

mongoimport  -d sample_mflix -c movies /Users/darshanj/Downloads/movies.json -u root -p rootpass --authenticationDatabase admin

Prepare search process

mkdir mongot_data

Creating mongot config file

hostname=`mongosh -u root -p rootpass --eval "db.isMaster().me"`
cat << EOF > mongot.config
syncSource:
  replicaSet:
     hostAndPort: "$hostname" 
     username: "mongot"
     passwordFile: "/passwordFile"
     authSource: "admin"
     tls: false
     readPreference: primaryPreferred
storage:
  dataPath: "/data/mongot"
server:
  grpc:
     address: "mongot-community.search-community:27028"
     tls:
        mode: "disabled"
metrics:
  enabled: true
  address: "mongot-community.search-community:9946"
healthCheck:
  address: "mongot-community.search-community:8080"
logging:
  verbosity: INFO
embedding:
  queryKeyFile: /etc/mongot/voyage-api-query-key
  indexingKeyFile: /etc/mongot/voyage-api-indexing-key
  providerEndpoint: https://api.voyageai.com/v1/embeddings
  isAutoEmbeddingViewWriter: true

echo -n "mongotPassword" > passwordFile
chmod 400 passwordFile

Go to https://dashboard.voyageai.com/organization/api-keys API key, copy secret

printf "<your-voyage-api-query-key>" > voyage-api-query-key

Repeat one more time for index API. Go to https://dashboard.voyageai.com/organization/api-keys API key, copy secret

printf "<your-voyage-api-index-key>" > voyage-api-indexing-key

Starting MongoDB Search

docker run -d --rm --name mongot-community -v ./mongot_data:/data/mongot -v ./mongot.config:/mongot-community/config.default.yml -v ./passwordFile:/passwordFile:ro -v ./voyage-api-indexing-key:/etc/mongot/voyage-api-indexing-key:ro -v ./voyage-api-query-key:/etc/mongot/voyage-api-query-key:ro --network search-community -p 8080:8080 -p 9946:9946 mongodb/mongodb-community-search:latest --internalListAllIndexesForTesting=true

Check the docker status

docker ps

Now connect to the mongosh and create the index and run the test query

mongosh -u root -p rootpass

use sample_mflix
db.movies.createSearchIndex("vector_index", "vectorSearch", {
 "fields": [
   {
     "type": "autoEmbed",
     "modality": "text",
     "path": "plot",
     "model": "voyage-4"
   },
   {
     "type": "filter",
     "path": "year"
   }
 ]
})

db.movies.aggregate([
 {
   "$vectorSearch": {
     "index": "vector_index",
     "path": "plot",
     "query": {
       "text": "bullied boy learns karate"
     },"model": "voyage-4-lite" ,
           "numCandidates": 10000,
     "limit": 10
   }
 },
 {
   "$project": {
     "_id": 0,
     "title": 1,
     "year":1,
     "plot": 1,
     "score": { $meta: "vectorSearchScore" }
   }
 }
])

I know we can make one single Docker compose file to make it easier, but I prefer this way to make debugging easier, and the understanding of each step will be clearer.

Congratulations! You have successfully created your first local vector search.

PS: To be safe, delete the API keys when no longer needed, just like I did :).

How schema anti-patterns in MongoDB can cost you $$$$

MongoDB Guests — Tue, 28 Apr 2026 13:38:52 +0000

This article was written by Darshan Jayarama.

MongoDB is a flexible schema database. This means you have the flexibility to modify the structure of the data store. One collection can have 5 fields in one row(document), 10 in another row, 1000 in another row, and 1 in another row. It doesn't matter how the structure has been aligned in the collection.

If you are a developer with a SQL background, you may find this to be a fantasy, like a science fiction movie. However, that is the flexibility you can achieve with the help of MongoDB. A good percentage of people choose MongoDB for this flexible schema, but let it be known that abusing this feature could cost you big time.

I would like to share some of the story from when I was working with customers to tune the application's performance.

Massive boundless array

Problem: One customer had ping data from an IOT device stored in an array field. In the first few hours of the day, document writes were working fine, but as time passed, writes started to become expensive.

Reason: As the document gets inflated with the growing array field, the findOneAndUpdate() method gets heavy in the network for each document to fetch →update →Save them.

Solution: As the ping frequency is every minute for 1000s of IoT devices, we created a new collection that maps device and ping data to deviceID, hence throughout the day, write was stabilized.

Timeseries saved time

Problem: For the same customer, write was stabilized with a referential model, but each read started costing them time as it had to map the same device ping throughout the day while creating the report.

Reason: Each deviceID data was spread across storage in different pages, so for 100 records, it was technically fetching 100 pages, keeping the required data, and throwing away the rest. Which increased the pagein-pageout.

Solution: MongoDB offers a timeseries collection, which utilizes a bucketing pattern, it has been explained in detail in the video series.

We have created a timeseries collection to store the ping data with deviceID as the metaField and timestamp as the timeField.

When multitenant became the villain

Problem: A customer from a webhosting platform was using a different DB for a different customer with a customer-identification suffix like billing_cust1, data_cust2… Started growing a customer base, became a pain in their database.

Reason: Customer was using MongoDB ChangeStream, which increased the number of databases and caused a delay in the start of their application collections. As explained in the documentation, changestream uses oplog collection in which we cannot utilize the indexes, hence, opening a high number of targeting streams can impact the performance.

Solution: We converted multi-tenancy to a monolithic database structure, storing all customer data we stored in a single collection with customerID in each document. So whenever we query for a specific customer, we append another filter with customerID to perform a targeted query and ensure data safety. This reduced 10000s of collections into 5–10 collections.

$lookup is designed badly

Problem: The application slowed down during the delivery time while verifying the product to be delivered.

Reason: A famous e-commerce company was storing productID in the orders collection. When a scan was performed via the QR code, it was performing multiple lookups in the products collection to fetch the product details.

Solution: Because the number of products was limited, we remodeled the table’s schema and stored the product details in the order collection. As the order details would no longer be accessed regularly after delivery, we archive orders that are 3–4 months old.

Closing thoughts: Performance tuning is a never-ending task, analyze the queries → tune → repeat.

How MongoDB Executes a find() Query: A Complete Lifecycle Guide

MongoDB Guests — Thu, 23 Apr 2026 08:30:25 +0000

This article was written by Darshan Jayarama.

When you type something like db.orders.find({ status: "pending", customerId: 1042 }) and the results come back in milliseconds, it feels simple… almost instant.

But behind that one line, MongoDB is doing a lot more than just “searching a collection.”

During my time as a Senior TSE at MongoDB, I spent most of my days deep in query performance and indexing issues, working closely with both customers and the engineering team. It may look simple, but internally, a lot is happening when a find() executes.

Most engineers understand it at a surface level. But once you start digging into what actually happens under the hood, that’s when things change. You begin to see why some queries are fast, why others are painfully slow, and how indexes truly make or break performance.

So let’s break it down — step by step — what really happens inside MongoDB when a query runs. No shortcuts, nothing skipped.

The full journey of a `find()` query (quick overview)

Before we go step by step, here’s the big picture. Every find() query goes through a series of steps in the mentioned order.

There are also two “fast paths” (shown as green arrows):
• When MongoDB can reuse a plan from the plan cache
• When the data is already in memory (WiredTiger cache)

Getting your queries to use these fast paths is what good MongoDB performance tuning is all about.

Stage 1: The Client Sends a Query Over the Wire

Before MongoDB even sees your query, your driver prepares it.

Whether you’re using PyMongo, the Node.js driver, or mongosh, your query is converted into BSON (a binary version of JSON) and sent over a network connection to MongoDB.

MongoDB uses the Wire Protocol to communicate, which is basically a structured way of sending messages between your app and the database.

Since MongoDB 3.6, most operations use a format called OP_MSG.

This message includes:
• which database and collection you’re querying
• the actual filter (your query conditions)
• extra options like projection, sort, limit, skip, hints, and read preferences

# PyMongo serializes this to BSON OP_MSG internally

db.orders.find(
   { "status": "pending", "customerId": 1042 },
   projection={"_id": 0, "amount": 1},
   sort=[("createdAt", -1)],
   limit=10
)

The query lands at the mongod process. The network listener accepts the connection and hands it off to a worker thread from the connection pool.

Stage 2: Authentication Check

The first thing MongoDB does — even before looking at any data — is check who is making the request.

MongoDB supports different ways to authenticate users, but in most real-world cases, this step is already handled. Drivers usually keep connections open and authenticated, so this check happens almost instantly

MongoDB just verifies the session linked to that connection.

If authentication fails, the query stops right there.
You’ll see an error like: “Authentication failed”, and the query never even reaches the database engine.

Stage 3: Authorization — Can You Read This Collection?

Authentication tells MongoDB who you are. Authorization decides what you’re allowed to do. MongoDB uses role-based access control (RBAC). It checks whether your user has permission to run a find() on that specific database and collection.

This could come from built-in roles like read or readAnyDatabase, or from custom roles with specific permissions.

If you don’t have access, the query stops there. You’ll see an error like: “not authorized to execute command”. The good part? This check is extremely fast — it’s just a quick in-memory lookup, not a database scan.

Stage 4: Query Parsing and BSON Validation

At this point, your query is sitting in memory as a BSON document. Now MongoDB starts understanding it.

It does two main things:

1. Checks if the query is valid

MongoDB makes sure your query is written correctly — valid operators like $eq, $in, $gt, $elemMatch, proper structure, and no invalid combinations. If something is wrong, the query fails right here with an error.

2. Converts it into an internal format:

MongoDB rewrites your query into a standard internal structure (think of it like a tree). This is why the order of fields doesn’t matter.

For example, these two are treated exactly the same:
{ a: 1, b: 2 }
{ b: 2, a: 1 }

Internally, MongoDB sees it more like:

AND
├── status = "pending"
└── customerId = 1042

In short, MongoDB first checks that your query is valid, then rewrites it into a format it can efficiently work with.

Stage 5: The Query Planner Enumerates Candidate Plans

This is where MongoDB actually starts making smart decisions. The query planner takes your parsed query and determines the best way to get the data. It doesn’t just pick one way — it tries out multiple options.

For every useful index, MongoDB creates a possible plan. It also always keeps one backup option: scanning the whole collection (COLLSCAN).

Each plan is basically a step-by-step approach to fetch the data.

For example:

Using index on status:

 FETCH
  └─ scan index { status: 1 }
Using index on customerId:
 FETCH
  └─ scan index { customerId: 1 }
Using compound index (status + customerId):
 scan index { status:1, customerId:1 }
 (no FETCH needed — everything is in the index)
No index:
 scan entire collection

MongoDB also thinks about a few important things here:
• Can it avoid sorting by using an index?
• Can it return results directly from the index (covered query)?
• Can it combine multiple indexes if needed?

In short, MongoDB tries different ways to run your query and prepares multiple plans before choosing the best one.

Stage 6: Plan Cache Lookup — The Fast Path

Before MongoDB tries out different plans, it first checks something called the plan cache.

The plan cache stores the best plan from previous runs of similar queries. So if MongoDB has already seen a query like yours before, it can skip all the extra work and reuse the same plan.

What matters here is the query shape — basically the structure of the query, not the actual values.

For example, these two queries are treated the same:

db.orders.find({ status: "pending", customerId: 1042 })

db.orders.find({ status: "shipped", customerId: 9999 })

Because structurally, they are identical:
{ status: <eq>, customerId: <eq> }

Now two things can happen:
• Cache hit → MongoDB already knows the best plan, so it skips all the trial work and runs it directly (this is the fast path)
• Cache miss → MongoDB doesn’t have a saved plan, so it tries multiple options to find the best one

A few important things to know about the plan cache:
• It’s stored in memory (not on disk)
• It’s maintained per collection
• It gets cleared when MongoDB restarts
• It’s also reset if the indexes change or the collection changes a lot

You can even inspect or clear it manually:

// See cached plans
db.orders.getPlanCache().list()
// Clear cache (forces MongoDB to re-evaluate plans)
db.orders.getPlanCache().clear()

In short, if your query has been seen before, MongoDB can skip straight to execution — which is why repeated queries are usually much faster 🚀

Stage 7: Multi-Plan Trial — The Index Race

If MongoDB doesn’t find a plan in the cache, it tries something really interesting.

Instead of guessing the best plan, it tests all possible plans simultaneously. This is called the multi-plan stage.

Each plan is given a chance to run for a few steps, one after the other — kind of like a race.

For example:
Round 1:
Plan A → small progress
Plan B → small progress
Plan C → small progress

Round 2:
Plan A → more progress
Plan B → more progress
Plan C → more progress

This continues until one plan clearly outperforms the others.

The winner is the plan that:
• returns the first ~100 results fastest, or
• finishes scanning the data quickest

Once a plan wins, MongoDB:
→ uses it for the current query
→ saves it in the plan cache for next time

What about full collection scans (COLLSCAN)?

They usually lose… but not always.

• If there are no useful indexes → COLLSCAN wins
• If the collection is very small → COLLSCAN can actually be faster than using an index

If you want to see how this decision was made, you can run:

db.orders.find({ status: "pending", customerId: 1042 })
        .explain("allPlansExecution")

Stage 8: Index Scan (IXSCAN) or Collection Scan (COLLSCAN)

Now that MongoDB has picked the best plan, it finally executes the query.

If an index is used (IXSCAN)

MongoDB uses indexes that work like a tree structure. It quickly navigates this tree to find matching entries.

Once it finds a match, it uses a reference (called RecordId) to go and fetch the actual document from the collection.

Think of it like:
— find the entry in the index
— then go grab the full document

There’s also something called a covered query: If all the fields you need are already in the index, MongoDB doesn’t even need to look at the actual documents.

It returns results directly from the index
This is the fastest possible way to read data

If no index is used (COLLSCAN)

If there’s no useful index, MongoDB has no choice — it reads every document one by one.

So if your collection has 10 million documents, it will scan all 10 million.

That’s definitely slow.

How to spot a problem

When you run explain(), watch for this:

db.orders.find({ status: "pending" }).explain("executionStats")
// Red flags to look for:
// "stage": "COLLSCAN"          ← no index used
// "totalDocsExamined": 9847321  ← scanned 9.8M docs
// "nReturned": 142              ← returned only 142
// Ratio: 69,000:1               ← extremely inefficient

Red flags:

"stage": "COLLSCAN" → no index is being used
Very high "totalDocsExamined"
Very low "nReturned"
scanned ~9.8 million documents
returned only 142

That’s a huge gap, and a clear sign your query is inefficient

Simple rule
If MongoDB is reading way more documents than it returns, you probably need a better index.

Stage 9: The Storage Layer — WiredTiger Cache and Disk

Once MongoDB knows which documents it needs, the next question is: Where does it actually read the data from?

There are three possible sources, and the speed depends on where the data is found.

1. WiredTiger cache (in-memory — fastest)
MongoDB uses WiredTiger as its storage engine, which keeps frequently accessed data in memory.

By default, it uses about 50% of the available RAM.

If the required data is already in this cache, MongoDB can return it almost instantly. This is the ideal scenario.

2. OS page cache (still fast)
If the data is not in MongoDB’s own cache, it checks the operating system’s page cache.

The OS may already have the data in memory from recent reads. Since MongoDB memory-maps its data files, this check is efficient.

This is slightly slower than the WiredTiger cache, but still very fast.

3. Disk (slowest)
If the data is not present in either cache, MongoDB has to read it from disk.

The performance here depends on the type of storage:

NVMe SSD: fastest among disks
SATA SSD: moderate
HDD: significantly slower

At scale, disk access becomes the main bottleneck, especially when queries access data randomly across large datasets. If your working data set fits in memory, queries remain fast. If MongoDB frequently needs to read from disk, query performance drops significantly. This is why working set size is critical in MongoDB performance tuning — your frequently accessed data should ideally fit in RAM.

Checking cache performance

You can inspect cache behavior using:

db.serverStatus().wiredTiger.cache
// 'pages read into cache'       ← total cache misses (disk reads)
// 'pages requested from cache'  ← total requests
// Hit ratio = 1 - (read/requested)
// Target: > 95%

Key metrics:

pages read into cache → disk reads (cache misses)
pages requested from cache → total requests

A good system typically has a cache hit ratio above 95%.

Stage 10: Results Returned to the Client

Here’s what happens:

Applies projection: Removes any fields you didn’t ask for and keeps only what’s needed
Applies skip and limit: Skips the first N documents (if specified) and limits how many results are returned
Converts back to BSON: Turns the in-memory document into a format that can be sent over the network
Creates response batches: MongoDB doesn’t send everything at once. The first batch contains up to 101 documents or 16MB of data (whichever comes first)
Sends the response: The data is sent back to your application over the same connection

What if there’s more data?

If your query returns more than one batch, MongoDB doesn’t send everything in one go.

Instead, it returns a cursor ID.

Your driver then automatically requests the next batch using getMore commands. This happens behind the scenes, so you usually don’t notice it.

The Complete Picture — All 10 Stages

Closing notes: Key Takeaways for Production

Always check your explain() output - Use explain("executionStats") to see exactly how your query ran — which plan was used, how many documents were scanned vs returned, and whether the plan came from cache.
Design indexes based on query patterns, not just fields -Think about how your queries are written. For example, a compound index like { status: 1, customerId: 1 } is usually much more effective than having separate indexes on each field. It can even avoid extra steps like fetching documents if the query is covered.
Keep an eye on cache efficiency - Monitor your WiredTiger cache hit ratio. If it drops below ~90%, it usually means your frequently accessed data no longer fits in memory. At that point, you may need to add more RAM, shard the data, or rethink how your application accesses data.
Be aware of plan cache issues - If a query suddenly becomes slow after something like a bulk insert, the plan cache might be the cause. MongoDB may have picked a suboptimal plan after re-evaluating. In such cases, clear the cache and check the query again with explain().

// Your daily driver for query diagnostics
db.orders.find({ status: "pending" }).explain("executionStats")

// Key fields to read:
// executionStats.executionTimeMillis    ← total time
// executionStats.totalDocsExamined      ← docs touched
// executionStats.totalKeysExamined      ← index keys scanned
// executionStats.nReturned              ← docs returned
// queryPlanner.winningPlan.stage        ← IXSCAN or COLLSCAN

Introduction to Events using MongoDB

MongoDB Guests — Wed, 22 Apr 2026 21:52:27 +0000

This article was written by Otavio Santana.

Keeping software simple is one of the hardest challenges for any software architect or senior engineer. Not because of technology, but because of decisions we often overlook—especially coupling. Many systems start simple, but as new requirements emerge, the same code begins to orchestrate multiple responsibilities. What once felt clean becomes fragile, and small changes require touching multiple parts of the system. This is how architectures silently degrade. The good news is that there are ways to address this problem. In this tutorial, we will explore an architectural approach that helps reduce coupling and improve extensibility, and how MongoDB can effectively support this style.

In this tutorial, you’ll:

Model a simple payment system
Write events sync and async events
Explore how MongoDB can help you on archive Event-driven design.

You can find all the code presented in this tutorial in the GitHub repository:

git clone git@github.com:soujava/helidon-mongodb-event-driven.git

Prerequisites

For this tutorial, you’ll need:

Java 21.
Maven.
A MongoDB cluster.
MongoDB Atlas (Option 1)
Docker (Option 2)

You can use the following Docker command to start a standalone MongoDB instance:

docker run --rm -d --name mongodb-instance -p 27017:27017 mongo

When we discuss event-driven design, we change the approach or the question: Instead of asking “who should I call next?”, we start asking: “What just happened?”

This change allows modules to interact without direct dependencies. Different parts of the system can react independently—either synchronously or asynchronously—without modifying the original flow.
As a first step, we need to understand what an event is. An event represents a fact that has already happened in the system. It is immutable and carries enough information for other components to react. Instead of telling the system what to do, events describe what occurred, allowing behavior to emerge through reactions.

In Domain-Driven Design (DDD), events take on a more specific role. They represent meaningful events in the business domain, using the same language as the business itself. These domain events can also act as integration points, enabling different parts of the system, including across bounded contexts—or even external systems—to respond to business changes. Event-driven goes beyond DDD, allowing facts on both the technical and the domain side.

In this tutorial, we will create an over-simplified payment system where, given a payment for a product, we will issue a payment request to an external provider, which will start an async event. Based on that, we will generate an event depending on whether the payment succeeded or failed. The whole process starts at the JAX-RS endpoint, generating a sequence of events and states inside the payment.

In this tutorial, we will simplify using CDI events, but you can naturally enhance it or use another service to handle those events, such as Kafka or a JMS Broker. Thus, we will use MongoDB and Helidon. Creating a Helidon project is simple: use the Helidon Starter, and in the wizard progress flow, you will select Helidon MP, QuickStart, and JSON-B. Feel free to define the groupid, artifactId, version, and package name as you wish. After downloading, the next step is including the MongoDB integration, in this case, Eclipse JNoSQL, in the pom.xml file in the root of the downloaded project:

<dependency>
   <groupId>org.eclipse.jnosql.databases</groupId>
   <artifactId>jnosql-mongodb</artifactId>
   <version>1.1.13</version>
</dependency>

With the project defined, the next step is to set the database configuration. You can either use a local database or explore MongoDB Atlas; either is fine. We will use locally, thus, run this Docker command to start a MongoDB instance, include those new properties:

# configure the MongoDB client for a replica set of two nodes
jnosql.mongodb.url=mongodb://localhost:27017
# mandatory: define the database name
jnosql.document.database=ecommerce

Step 1: Create the entities

With the project defined, the next step is to create the entities and repositories that will support it.

In the src/main/java directory, create a Product class:

package com.acme;

import jakarta.nosql.Column;
import jakarta.nosql.Embeddable;

import java.util.UUID;

@Embeddable(Embeddable.EmbeddableType.GROUPING)
public record Product(@Column UUID code, @Column String name) {

}

Create a PaymentCounter class:

package com.acme.statistics;

import com.acme.Product;
import com.acme.infraestructure.JsonFieldStrategy;
import jakarta.json.bind.annotation.JsonbVisibility;
import jakarta.nosql.Column;
import jakarta.nosql.Entity;
import jakarta.nosql.Id;

import java.util.UUID;

@Entity
@JsonbVisibility(JsonFieldStrategy.class)
public class PaymentCounter {

    @Id
    private UUID productCode;

    @Column
    private Product product;

    @Column
    private int successfulPayments;

    @Column
    private int failedPayments;

    PaymentCounter(Product product) {
        this.productCode = product.code();
        this.product = product;
    }

    PaymentCounter() {
    }

    public void paymentFailed() {
        this.failedPayments++;
    }

    public void paymentSucceeded() {
        this.successfulPayments++;
    }

    @Override
    public String toString() {
        return "PaymentCounter{" +
                "productCode=" + productCode +
                ", successfulPayments=" + successfulPayments +
                ", failedPayments=" + failedPayments +
                '}';
    }
}

And the Payment and PaymentStatus classes:

package com.acme.payment;

import com.acme.Product;
import com.acme.infraestructure.JsonFieldStrategy;
import jakarta.json.bind.annotation.JsonbVisibility;
import jakarta.nosql.Column;
import jakarta.nosql.Entity;
import jakarta.nosql.Id;

import java.math.BigDecimal;
import java.util.Objects;
import java.util.UUID;

@Entity
@JsonbVisibility(JsonFieldStrategy.class)
public class Payment {

    @Id
    private UUID id;

    @Column
    private BigDecimal amount;

    @Column
    private PaymentStatus status;

    @Column
    private Product product;

    Payment() {
    }

    Payment(Product product, BigDecimal amount, PaymentStatus status) {
        this.id = UUID.randomUUID();
        this.amount = amount;
        this.status = status;
        this.product = product;
    }

    public UUID getId() {
        return id;
    }

    @Override
    public boolean equals(Object o) {
        if (!(o instanceof Payment payment)) {
            return false;
        }
        return Objects.equals(id, payment.id);
    }

    @Override
    public int hashCode() {
        return Objects.hashCode(id);
    }

    public Product getProduct() {
        return product;
    }

    @Override
    public String toString() {
        return "Payment{" +
                "id='" + id + '\'' +
                ", amount=" + amount +
                ", status=" + status +
                ", product=" + product +
                '}';
    }

    public void failed() {
        this.status = PaymentStatus.FAILED;
    }

    public void confirmed() {
        this.status = PaymentStatus.CONFIRMED;
    }
}


//different file

package com.acme.payment;

public enum PaymentStatus {
    PENDING,
    CONFIRMED,
    FAILED
}

Explanation of annotations:

@entity: Marks the Room class as a database entity for management by Jakarta NoSQL.
@id: Indicates the primary identifier for the entity, uniquely distinguishing each document in the MongoDB collection.
@column: Maps fields (roomNumber, type) for reading from or writing to MongoDB.

With the entities created, the next step is creating the repositories for each entity. In our tutorial, we simplified Product as a Value Object.

package com.acme.payment;

import jakarta.data.repository.BasicRepository;
import jakarta.data.repository.Repository;

import java.util.UUID;

@Repository
public interface PaymentRepository extends BasicRepository<Payment, UUID> {
}

//different file
package com.acme.statistics;

import jakarta.data.repository.BasicRepository;
import jakarta.data.repository.Repository;

import java.util.UUID;

@Repository
public interface PaymentCounterRepository extends BasicRepository<PaymentCounter, UUID> {
}

With both entities and repositories defined, the next step is defining the events that we will handle on this system.

Step 2: Create the events

In this sample, we will have three events to represent three states of the payment: when started (pending) ,because it requires operation on an external payment service; Payment failed; or confirmed based on the response from this external payment provider.

In the src/main/java directory, create a Product class:

package com.acme.payment;

public record PaymentFailedEvent(Payment payment) {
}
//different file
package com.acme.payment;

public record PaymentConfirmedEvent(Payment payment) {
}
//different file
package com.acme.payment;

public record PaymentRequestedEvent(Payment payment) {
}

Step 3: Generate the listeners

At this event, we will have listeners or classes that respond to some facts or events in our system. Using CDI that is pretty simple, we just need to have a parameter with the class to be observed and put either Observes or ObservesAsync to observe synchronous or asynchronous simultaneously.

We will start creating a service to update the status based on both confirmed or failed payment. This service will find the payment by id and then update the status.

In the src/main/java directory, create a PaymentStatusService class:

package com.acme.payment;

import jakarta.enterprise.context.ApplicationScoped;
import jakarta.enterprise.event.Observes;
import jakarta.inject.Inject;

import java.util.logging.Logger;

@ApplicationScoped
class PaymentStatusService {

    private static final Logger LOGGER = Logger.getLogger(PaymentStatusService.class.getName());

    private final PaymentRepository repository;

    @Inject
    PaymentStatusService(PaymentRepository repository) {
        this.repository = repository;
    }

    PaymentStatusService() {
        this.repository = null;
    }

    void payed(@Observes PaymentConfirmedEvent event) {
        LOGGER.info("Payment " + event.payment().getId() + " was payed");
        var payment = repository.findById(event.payment().getId()).orElseThrow();
        payment.confirmed();
        repository.save(payment);
        LOGGER.info("Payment " + payment.getId() + " was confirmed");
    }

    void errorOnPayment(@Observes PaymentFailedEvent event) {
        LOGGER.info("Payment " + event.payment().getId() + " failed");
        var payment = repository.findById(event.payment().getId()).orElseThrow();
        payment.failed();
        repository.save(payment);
        LOGGER.info("Payment " + payment.getId() + " was failed");
    }
}

The next step is the counter that will register success and failure based on the product code.

In the src/main/java directory, create a PaymentCounterService class:

package com.acme.statistics;

import com.acme.Product;
import com.acme.payment.Payment;
import com.acme.payment.PaymentFailedEvent;
import com.acme.payment.PaymentConfirmedEvent;
import jakarta.data.Order;
import jakarta.data.Sort;
import jakarta.data.page.Page;
import jakarta.data.page.PageRequest;
import jakarta.enterprise.context.ApplicationScoped;
import jakarta.enterprise.event.Observes;
import jakarta.inject.Inject;

import java.util.List;
import java.util.function.Consumer;
import java.util.logging.Logger;

@ApplicationScoped
public class PaymentCounterService {

    private static final Order<PaymentCounter> PRODUCT_CODE = Order.by(Sort.asc("productCode"));

    private static final Logger LOGGER = Logger.getLogger(PaymentCounterService.class.getName());

    private final PaymentCounterRepository repository;

    @Inject
    PaymentCounterService(PaymentCounterRepository repository) {
        this.repository = repository;
    }

    PaymentCounterService() {
        this.repository = null;
    }

    void success(@Observes PaymentConfirmedEvent event) {
        LOGGER.info("Payment successful, incrementing counter: " + event.payment().getId());
        execute(PaymentCounter::paymentSucceeded, event.payment().getProduct());
    }


    void success(@Observes PaymentFailedEvent event) {
        LOGGER.info("Payment failed, incrementing counter: " + event.payment().getId());
        execute(PaymentCounter::paymentFailed, event.payment().getProduct());
    }

    void execute(Consumer<PaymentCounter> action, Product product) {
        PaymentCounter counter = repository.findById(product.code()).orElseGet(() -> new PaymentCounter(product));
        LOGGER.info("Counter found: " + counter);
        action.accept(counter);
        LOGGER.info("Counter incremented: " + counter);
        repository.save(counter);
    }

    public List<PaymentCounter> findAll(PageRequest pageRequest) {
        LOGGER.info("Finding all counters, page: " + pageRequest.page());
        Page<PaymentCounter> counters = repository.findAll(pageRequest, PRODUCT_CODE);
        return counters.content();
    }
}

Finally, we will have the service that will handle an external provider. To simplify, we will set a wait of 2 seconds, and then based on the counter defined as either success or failure, this one uses an asynchronous event, as you can see in the annotation.

In the src/main/java directory, create a PaymentProvider class:

package com.acme.payment.provider;

import com.acme.payment.PaymentFailedEvent;
import com.acme.payment.PaymentRequestedEvent;
import com.acme.payment.PaymentConfirmedEvent;
import jakarta.enterprise.context.ApplicationScoped;
import jakarta.enterprise.event.Event;
import jakarta.enterprise.event.ObservesAsync;
import jakarta.inject.Inject;

import java.util.concurrent.atomic.AtomicLong;
import java.util.logging.Logger;

@ApplicationScoped
class PaymentProvider {

    private static final Logger LOGGER = Logger.getLogger(PaymentProvider.class.getName());

    private final AtomicLong counter = new AtomicLong();

    private final Event<PaymentConfirmedEvent> successEvent;

    private final Event<PaymentFailedEvent> errorEvent;

    @Inject
    PaymentProvider(Event<PaymentConfirmedEvent> successEvent, Event<PaymentFailedEvent> errorEvent) {
        this.successEvent = successEvent;
        this.errorEvent = errorEvent;
    }

    PaymentProvider() {
        this.successEvent = null;
        this.errorEvent = null;
    }

    void receivePayment(@ObservesAsync PaymentRequestedEvent event) {
        LOGGER.info("Processing payment: " + event.payment().getId());
        try {
            Thread.sleep(2000);
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
        }
        if(counter.get() % 2 == 0) {
            LOGGER.warning("Payment failed: " + event.payment().getId());
            errorEvent.fire(new PaymentFailedEvent(event.payment()));
        } else {
            LOGGER.info("Payment successful: " + event.payment().getId());
            successEvent.fire(new PaymentConfirmedEvent(event.payment()));
        }
        LOGGER.info("Payment processed: " + event.payment().getId() + " - Confirmation #" + counter.incrementAndGet());
    }
}

Step 4: Generating the rest endpoint

The observers are done, but we need one way to start the whole process. In this case, we will use a rest end-point. This resource will have two methods, one to create a payment and another to paginate the current payments.

At src/main/java, create the PaymentResource class.

package com.acme.payment;

import jakarta.data.page.PageRequest;
import jakarta.inject.Inject;
import jakarta.ws.rs.DefaultValue;
import jakarta.ws.rs.GET;
import jakarta.ws.rs.POST;
import jakarta.ws.rs.Path;
import jakarta.ws.rs.QueryParam;
import jakarta.ws.rs.core.Response;

import java.util.List;

@Path("/payments")
public class PaymentResource {


    private final PaymentService service;

    @Inject
    public PaymentResource(PaymentService service) {
        this.service = service;
    }

    PaymentResource() {
        this.service = null;
    }

    @POST
    public Response create(PaymentRequest request) {
        return Response.accepted(service.create(request)).build();
    }

    @GET
    public List<Payment> getAll(@QueryParam("page") @DefaultValue("1") int page) {
        var pageRequest = PageRequest.ofPage(page);
        return service.findAll(pageRequest);
    }
}

The resource itself needs the PaymentService that will create a payment as pending status, it will start an asynchronous event, and also, it will provide the payment status.

At src/main/java, create the PaymentService class.

package com.acme.payment;

import com.acme.Product;
import jakarta.data.Order;
import jakarta.data.Sort;
import jakarta.data.page.Page;
import jakarta.data.page.PageRequest;
import jakarta.enterprise.context.ApplicationScoped;
import jakarta.enterprise.event.Event;
import jakarta.inject.Inject;

import java.math.BigDecimal;
import java.util.List;
import java.util.logging.Logger;

@ApplicationScoped
public class PaymentService {

    private static final Logger LOGGER = Logger.getLogger(PaymentService.class.getName());
    private static final Order<Payment> PAYMENT_ORDER = Order.by(Sort.asc("id"));

    @Inject
    private PaymentRepository repository;

    @Inject
    private Event<PaymentRequestedEvent> event;

    public Payment create(PaymentRequest request) {

        LOGGER.info("Creating payment for " + request.productCode());
        Product product = new Product(
                request.productCode(),
                request.productName()
        );

        BigDecimal total = request.unitPrice()
                .multiply(BigDecimal.valueOf(request.quantity()));

        Payment payment = new Payment(
                product,
                total,
                PaymentStatus.PENDING
        );

        repository.save(payment);
        LOGGER.info("Payment created: " + payment);
        event.fireAsync(new PaymentRequestedEvent(payment));
        return payment;
    }

    public List<Payment> findAll(PageRequest pageRequest) {
        LOGGER.info("Finding all payments, page: " + pageRequest.page());
        Page<Payment> payments = repository.findAll(pageRequest, PAYMENT_ORDER);
        return payments.content();
    }
}

The last resource for today is the one to show the statistics of our payment based on the product code.

At src/main/java, create the PaymentCounterResource class.

package com.acme.statistics;

import com.acme.payment.Payment;
import jakarta.data.page.PageRequest;
import jakarta.inject.Inject;
import jakarta.ws.rs.DefaultValue;
import jakarta.ws.rs.GET;
import jakarta.ws.rs.Path;
import jakarta.ws.rs.QueryParam;

import java.util.List;

@Path("/payment-counter")
public class PaymentCounterResource {

    private final PaymentCounterService service;

    @Inject
    public PaymentCounterResource(PaymentCounterService service) {
        this.service = service;
    }

    PaymentCounterResource() {
        this.service = null;
    }

    @GET
    public List<PaymentCounter> getAll(@QueryParam("page") @DefaultValue("1") int page) {
        var pageRequest = PageRequest.ofPage(page);
        return service.findAll(pageRequest);
    }

}

Conclusion

We got it! In this tutorial, we learned a new software architecture pattern and its motivation: the event-driven design, which changes our way of thinking about the system by focusing on the events that occur within it, rather than worrying about orchestrating multiple services and eventually handling coupling. We could explore Domain-Driven design concepts with a lightweight event-driven approach, separating responsibilities, handling asynchronous processes naturally, and keeping the core logic simple and extensible.

Ready to explore the benefits of MongoDB Atlas? Get started now by trying MongoDB Atlas.
Access the source code used in this tutorial.
Any questions? Come chat with us in the MongoDB Community Forum.

References:

Source code

Detective mode: Fixing the MongoDB Aggregation pipeline

MongoDB Guests — Tue, 21 Apr 2026 09:32:31 +0000

This article was written by Darshan Jayarama.

It was a Friday, and I was quickly wrapping up my week, finishing the tasks that were assigned. Suddenly, the board became red — another S1. I was like, not now!!!

The application team started complaining about the API responses from the database, going from 600ms to 17 seconds. Booooom!!! That’s insane. No other choice — since it was production, I had to jump in. The war room was ready. The monitoring dashboard was nothing but a heart patient’s ECG — spikes, timeouts, and the main app was choking.

What's the Mystery?

From the dashboard, we identified a query that was asked by the VP to add a breakdown by region to the dashboard. It was just one more field. But it was ONE.MORE.FIELD.

Innocent query, which was harmless for years:

db.orders.aggregate([
 {
   $match: {
     status: "completed",
     created_at: { $gte: new Date("2025-01-31") }
   }
 },
 {
   $group: {
     _id: "$customer_id",
     total_orders: { $sum: 1 },
     total_revenue: { $sum: "$amount" },
     avg_order_value: { $avg: "$amount" }
   }
 },
 {
   $sort: { total_revenue: -1 }
 },
 {
   $limit: 100
 }
])

Simple, right? top 100 customer revenue dashboard.

Execution time 500–700ms, Docs Examined ~2.5M.

We then added just one more field query:

db.orders.aggregate([
 {
   $match: {
     status: "completed",
     created_at: { $gte: new Date("2024-01-01") }
   }
 },
 {
   $lookup: { 
     from: "customers",
     localField: "customer_id",
     foreignField: "_id",
     as: "customer"
   }
 },
 {
   $unwind: "$customer"
 },
 {
   $group: {
     _id: {
       customer_id: "$customer_id",
       region: "$customer.region"
     },
     total_orders: { $sum: 1 },
     total_revenue: { $sum: "$amount" },
     avg_order_value: { $avg: "$amount" }
   }
 },
 {
   $sort: { total_revenue: -1 }
 },
 {
   $limit: 100
 }
])

Execution time >15000ms, Docs Examined 2.5M X 2.5M, memory exploded.

The Investigation: Agent Darshan reporting

Evidence #1:

MongoDB profiler, setting the status to 2, helps us collect all the queries that are being run on the given database. Set this level only when needed, else you’re gonna blow up the log file:

db.setProfilingLevel(2, { slowms: 100 })
db.system.profile.find().sort({ ts: -1 }).limit(1).pretty()

I could see no mercy in our query, brutally attacking the MongoDB cluster:

{
 "op": "command",
 "ns": "production.orders",
 "command": { /* our aggregation */ },
 "keysExamined": 2489234,      // ✅ Used index on status
 "docsExamined": 2489234,      // But still examined all matched docs
 "numYield": 18943,
 "nreturned": 100,
 "responseLength": 8234,
 "millis": 647,
 "planSummary": "IXSCAN { status: 1 }",  // ✅ Used status index
 "execStats": {
   "stage": "FETCH",
   "filter": {
     "created_at": { "$gte": ISODate("2025-01-31") }  // 👈 Filter applied AFTER index
   },
   "inputStage": {
     "stage": "IXSCAN",
     "keyPattern": { "status": 1 },
     "indexName": "status_1",
     "keysExamined": 2489234,
     "seeks": 1,
     "dupsTested": 0,
     "dupsDropped": 0,
     "direction": "forward"
   },
   "totalDocsExamined": 2489234,
   "totalKeysExamined": 2489234,
   "executionTimeMillisEstimate": 447
 }
}

Red Flags:

DocsExamined = ~2.5M, whereas nReturened 100
PlanSummary: Single key index on boolean status_1.
High yield.

Evidence #2:

Need to review and explain the plan to make the query confess to the brutal attack:

{
 "stages": [
   {
     "$cursor": {
       "queryPlanner": {
         "winningPlan": {
          "stage": "IXSCAN",
           "keyPattern": { "status": 1 },
           "filter": {
             "status": { "$eq": "completed" },
             "created_at": { "$gte": new Date("2025-01-31") }
           }
         }
       },
       "executionStats": {
         "executionTimeMillis": 647,  // $match stage: 647ms
         "totalDocsExamined": 2489234,
         "totalKeysExamined": 2489234,  //
         "nReturned": 2489234  // Passed 2.48M docs to next stage
       }
     }
   },
   {
     "$lookup": {  // 💀💀💀 THE SERIAL KILLER
       "from": "customers",
       "as": "customer",
       "executionTimeMillis": 13934,  // 13.9 seconds just for $lookup!
       "totalDocsExamined": 2489234,  // Looked up 2.48M customers
       "stage": "NESTED_LOOP_JOIN",  // No index on customers._id
       "indexesUsed": []  // 👈 DISASTER
     }
   },
   {
     "$unwind": {
       "executionTimeMillis": 823
     }
   },
   {
     "$group": {
       "executionTimeMillis": 1156,
       "usedDisk": true,  // 💀 Spilled to disk (>100MB memory)
       "spilledRecords": 847293
     }
   },
   {
     "$sort": {
       "executionTimeMillis": 87,
       "sortPattern": { "total_revenue": -1 },
       "usedDisk": false  // Lucky, only 100 docs to sort after $limit
     }
   }
 ],
 "executionTimeMillis": 16847
}

Time breakdown:

$match: 647ms → $lookup: 13.9 seconds → $unwind: 823ms → $group: 1156ms → $sort: 87ms → Total 16.9 seconds

Suspects list

Suspect #1:

Unoptimized index for $match. Even though we had a single index on status:1 and created_at:1, we didn't have a compound index on both.

Motive: Index inefficiency
Proof: was high nReturned to the next stage.

Suspect #2:

The $lookup brutalness, for each customer_id from the previous stage, finding the join on the customers collection with an absent element in indexUsed. Isn’t that brutal?

Motive: just one more field request
Proof: empty indexUsed and 13.9s execution time.

Suspect #3:

$group surrendered with memory hog, spilled ~847k documents to disk as it exceeded 100MB per stage MongoDB limit.

Motive: Grouping 2.4M records with useDisk true
Proof: 847k spilledRecords.

Suspect #4:

We were performing an expensive $lookup operation before we could filter down to 100 records.

Motive: Not thinking about pipeline optimization
Proof: Processing 2.48M docs when we only needed 100

Recapping all mistakes to fix!

We were doing $match: 647ms(inefficient index)→ $lookup: 13.9 seconds(massive 2.4M join) → $unwind: 823ms → $group: 1156ms(memory overflow) → $sort: 87ms → Total 16.9 seconds

We redesigned as,

$match with compound index for efficient filtering
$group by customer_id reduce the dataset
$sort and $limit top 100 customer and we have only 100 records now.
$lookup for the 100 customer data we wanted.
$project to add region information.

Does it fix the issues?

The Solution:

2.5M orders
↓ $match (INDEXED — fast!)
2.48M orders
↓ $group (by customer_id only)
~500K customer groups
↓ $sort + $limit
100 top customers
↓ $lookup (only 100 lookups!) ← ✅ FIXED
100 results with customer data
↓ $project (extract region)
100 final results

The best method of using $lookup is not using it!

db.orders.aggregate([
 {
   $match: {
     status: "completed",
     created_at: { $gte: new Date("2024-01-01") }
   }
 },
 {
   $group: {
     _id: "$customer_id",  // Group BEFORE lookup
     total_orders: { $sum: 1 },
     total_revenue: { $sum: "$amount" },
     avg_order_value: { $avg: "$amount" }
   }
 },
 {
   $sort: { total_revenue: -1 }
 },
 {
   $limit: 100  // Reduce to 100 BEFORE lookup
 },
 {
   $lookup: {  // Now only looking up 100 customers
     from: "customers",
     localField: "_id",
     foreignField: "_id",
     as: "customer"
   }
 },
 {
   $unwind: "$customer"
 },
 {
   $project: {
     customer_id: "$_id",
     region: "$customer.region",
     total_orders: 1,
     total_revenue: 1,
     avg_order_value: 1
   }
 }
])

Curious to know the metrics now? I am excited to display as well:

Execution time 847ms, docsExamined 2.5M, lookup performed 100 documents.

✅ Issue resolved

Root cause: $lookup before aggregation.
Fix: Aggregate first, then join the top 100
Performance: 17s → 0.8s (20x improvement)
Added indexes: (status, created_at)

The Costly Lessons Learned:

One field can kill you. That “simple” region addition cost us $24,000 in downtime.
$lookup is not free. It’s actually very expensive. Treat it like a loaded gun.
Order matters. Aggregate → Limit → Join. Not Join → Aggregate → Limit.
Indexes aren’t optional. They’re the difference between 1.8 seconds and 0.15 seconds.
Explain is your friend. If you’re not running, explain, you’re flying blind.
The 100MB limit is real. When $group says “usedDisk: true”, you’re already in trouble.
Test with production data volumes. 1,000 docs is not the same as 2,500,000 docs.

FastAPI With LangChain and MongoDB

MongoDB Guests — Fri, 17 Apr 2026 08:48:54 +0000

This article was written by Carlos Barboza.

I'm happy to welcome you to this tutorial! This demonstration provides a fully functional example of integrating FastAPI, LangChain, and MongoDB, and I'm eager to share it with all technology enthusiasts.

The value of this guide is that it provides a working example—you can copy and paste the code, and it will run seamlessly without any further configuration. It will also give you a clear, practical overview of the kinds of applications you can build using these powerful tools.

I am developing a suggestion application designed to assist customers globally in generating new ideas. This involves providing personalized recommendations and checking for existing companies that offer comparable services.

Prerequisites

Python, virtualenv basics
Basic understanding of REST APIs
MongoDB basics (collections, documents)
Accounts/keys
- MongoDB Atlas or local MongoDB
- LLM provider API key

Project Architecture

FastAPI is a Python framework for building APIs quickly, efficiently, and with very little code.

It’s known for being fast (as the name implies), easy to use, and perfect for modern applications like AI, microservices, and backend APIs. In this example, it will handle the API queries for the backend.

MongoDB is a NoSQL, document-based database that stores data in flexible JSON-like documents instead of rigid tables like SQL. This makes it fast to develop, easy to scale, and great for evolving data structures. For our case, it will store all the information.

LangChain is a framework for building applications powered by LLMs (like GPT, LLaMA, Ollama, and Mistral) that lets you connect models with tools, memory, external data, and multi-step reasoning. It provides the necessary tools in Python for utilizing AI models.

Project Setup

MongoDB Atlas

The simplest way to obtain a MongoDB instance for this guide is to register on the official site. This will provide you with an M0 instance, which is entirely sufficient for executing this guide.

Docker

Should you not have Docker installed, the official link provides installation instructions based on your operating system.

The docker-compose file is configured to set up a MongoDB container. This container uses the official version 7.0, is named mongodb, and uses the credentials root as the username and example as the password. Furthermore, a volume is defined to ensure data persistence across container reloads.

version: "3.3"
services:
  mongo:
    image: mongo:7.0
    container_name: mongodb
    restart: unless-stopped
    ports:
      - "27017:27017"
    environment:
      MONGO_INITDB_ROOT_USERNAME: root
      MONGO_INITDB_ROOT_PASSWORD: example
    volumes:
      # Named volume for persistent data
      - mongo_data:/data/db

volumes:
  mongo_data:

Execute the following commands to execute the docker:

docker compose up -d

Finally, you can connect to the MongoDB instance using the connection string. In your case, it’s mongodb://root:example@localhost:27017.

LLMs

Large language models (LLMs) are AI models trained on massive amounts of text that can understand language, generate text, write code, answer questions, and hold conversations like a human.

They don’t think—they predict the next word intelligently based on patterns they've learned. For our case, we are going to use Ollama so you can execute the example on your local machine.
Alternatively, you may choose to use a different LLM, such as OpenAI, though this may incur additional costs.

Ollama installation

Depending on your operating system, you must follow the official instructions. At the end of the day, you should be able to execute Ollama commands in your terminal.

ollama list # Shows all the models available in your terminal
ollama pull llama3.2 # Download the llama3.2 model

Development

Requirements.txt

These are all the libraries needed for the project:

aiohappyeyeballs==2.6.1
aiohttp==3.13.2
aiosignal==1.4.0
annotated-doc==0.0.4
annotated-types==0.7.0
anyio==4.12.0
attrs==25.4.0
bcrypt==3.2.2
certifi==2025.11.12
cffi==2.0.0
charset-normalizer==3.4.4
click==8.3.1
colorama==0.4.6
cryptography==46.0.3
dataclasses-json==0.6.7
distro==1.9.0
dnspython==2.8.0
ecdsa==0.19.1
email-validator==2.3.0
fastapi==0.123.0
frozenlist==1.8.0
greenlet==3.2.4
h11==0.16.0
httpcore==1.0.9
httpx==0.28.1
httpx-sse==0.4.3
idna==3.11
jiter==0.12.0
jsonpatch==1.33
jsonpointer==3.0.0
langchain==1.1.0
langchain-classic==1.0.0
langchain-community==0.4.1
langchain-core==1.1.0
langchain-openai==1.1.0
langchain-text-splitters==1.0.0
langgraph==1.0.4
langgraph-checkpoint==3.0.1
langgraph-prebuilt==1.0.5
langgraph-sdk==0.2.10
langsmith==0.4.49
marshmallow==3.26.1
motor==3.7.1
multidict==6.7.0
mypy_extensions==1.1.0
numpy==2.3.5
openai==2.8.1
orjson==3.11.4
ormsgpack==1.12.0
packaging==25.0
passlib==1.7.4
propcache==0.4.1
pyasn1==0.6.1
pycparser==2.23
pydantic==2.12.5
pydantic-settings==2.12.0
pydantic_core==2.41.5
pymongo==4.15.4
python-dotenv==1.2.1
python-jose==3.5.0
python-multipart==0.0.20
PyYAML==6.0.3
regex==2025.11.3
requests==2.32.5
requests-toolbelt==1.0.0
rsa==4.9.1
six==1.17.0
sniffio==1.3.1
SQLAlchemy==2.0.44
starlette==0.50.0
tenacity==9.1.2
tiktoken==0.12.0
tqdm==4.67.1
typing-inspect==0.9.0
typing-inspection==0.4.2
typing_extensions==4.15.0
urllib3==2.5.0
uvicorn==0.38.0
xxhash==3.6.0
yarl==1.22.0
zstandard==0.25.0

Environment file

It’s recommended to have an .env file with sensitive information like the MongoDB URI, JWT Secret Key, and other settings. This file should be excluded from the git.ignore and must be configured on the main location.

MONGO_URI=mongodb+srv://admin:admin@atlascluster.f3spasq.mongodb.net/
MONGO_DB_NAME=fastapi_demo
JWT_SECRET_KEY=super-secret-key
JWT_ALGORITHM=HS256
ACCESS_TOKEN_EXPIRE_MINUTES=60
OLLAMA_HOST=http://host.docker.internal:11434 # Only if fastAPI is running with docker

.gitignore

# ---------------------
# Environment variables
# ---------------------
.env
.env.local
.env.*.local

# ---------------------
# Python
# ---------------------
__pycache__/
*.pyc
*.pyo
*.pyd
*.pdb
*.pkl
*.db

# Virtual environments
venv/
.venv/

# Byte-compiled
*.py[cod]
*$py.class

# ---------------------
# FastAPI / Uvicorn logs
# ---------------------
*.log

# ---------------------
# IDE & editor files
# ---------------------
.vscode/
.idea/
*.swp

# ---------------------
# OS files
# ---------------------
.DS_Store
Thumbs.db

# ---------------------
# Docker
# ---------------------
/data/
docker-data/
*.pid

# ---------------------
# Optional cache folders
# ---------------------
cache/
logs/

Schema folder

chat.py
from pydantic import BaseModel

class ChatRequest(BaseModel):
    question: str

class ChatResponse(BaseModel):
    answer: str
    used_context: bool

item.py
from pydantic import BaseModel

class Item(BaseModel):
    id: str
    name: str
    price: float

user.py
from pydantic import BaseModel, EmailStr

class UserCreate(BaseModel):
    email: EmailStr
    password: str

class UserPublic(BaseModel):
    id: str
    email: EmailStr

class Token(BaseModel):
    access_token: str
    token_type: str = "bearer"

class UserInDB(BaseModel):
    id: str
    email: EmailStr
    hashed_password: str

notes.py
from pydantic import BaseModel

class NotesCreate(BaseModel):
    text: str

class NotesInDB(BaseModel):
    id: str
    user_id: str
    text: str

Core folder

config.py
import os

MONGO_URI = os.getenv("MONGO_URI", "mongodb+srv://admin:admin@atlascluster.f3spasq.mongodb.net/")
MONGO_DB_NAME = os.getenv("MONGO_DB_NAME", "fastapi_demo")

JWT_SECRET_KEY = os.getenv("JWT_SECRET_KEY", "change-me")
JWT_ALGORITHM = os.getenv("JWT_ALGORITHM", "HS256")
ACCESS_TOKEN_EXPIRE_MINUTES = int(os.getenv("ACCESS_TOKEN_EXPIRE_MINUTES", "60"))

llm.py
import os
from langchain_community.chat_models import ChatOllama

OLLAMA_HOST = os.getenv("OLLAMA_HOST", "http://localhost:11434")

llm = ChatOllama(
    model="llama3.2"
    ,base_url=OLLAMA_HOST,
)

security.py
from datetime import datetime, timedelta, timezone
from typing import Optional

from fastapi.security import OAuth2PasswordBearer
from jose import jwt
from passlib.context import CryptContext

from app.core.config import JWT_SECRET_KEY, JWT_ALGORITHM, ACCESS_TOKEN_EXPIRE_MINUTES

pwd_context = CryptContext(schemes=["bcrypt"], deprecated="auto")
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="auth/login")

def hash_password(password: str) -> str:
    return pwd_context.hash(password)

def verify_password(plain_password: str, hashed_password: str) -> bool:
    return pwd_context.verify(plain_password, hashed_password)

def create_access_token(
    data: dict,
    expires_delta: Optional[timedelta] = None,
) -> str:
    to_encode = data.copy()
    expire = datetime.now(timezone.utc) + (
        expires_delta or timedelta(minutes=ACCESS_TOKEN_EXPIRE_MINUTES)
    )
    to_encode.update({"exp": expire})
    encoded_jwt = jwt.encode(to_encode, JWT_SECRET_KEY, algorithm=JWT_ALGORITHM)
    return encoded_jwt

db folder

from motor.motor_asyncio import AsyncIOMotorClient
from app.core import config as core_config

client = AsyncIOMotorClient(core_config.MONGO_URI)
db = client[core_config.MONGO_DB_NAME]

users_collection = db["users"]
notes_collection = db["notes"]
items_collection = db["items"]

depsFolder

auth.py
from typing import Optional, Annotated

from fastapi import Depends, HTTPException, status
from jose import JWTError, jwt

from app.core.config import JWT_SECRET_KEY, JWT_ALGORITHM
from app.core.security import oauth2_scheme, hash_password, verify_password
from app.db.mongo import users_collection
from app.schemas.user import UserInDB, UserCreate

async def get_user_by_email(email: str) -> Optional[UserInDB]:
    doc = await users_collection.find_one({"email": email})
    if not doc:
        return None
    return UserInDB(
        id=str(doc["_id"]),
        email=doc["email"],
        hashed_password=doc["hashed_password"],
    )

async def create_user(user: UserCreate) -> UserInDB:
    existing = await get_user_by_email(user.email)
    if existing:
        raise HTTPException(
            status_code=status.HTTP_400_BAD_REQUEST,
            detail="Email already registered",
        )
    hashed = hash_password(user.password)
    res = await users_collection.insert_one(
        {"email": user.email, "hashed_password": hashed}
    )
    return UserInDB(id=str(res.inserted_id), email=user.email, hashed_password=hashed)

async def get_current_user(
    token: Annotated[str, Depends(oauth2_scheme)]
) -> UserInDB:
    credentials_exception = HTTPException(
        status_code=status.HTTP_401_UNAUTHORIZED,
        detail="Could not validate credentials",
        headers={"WWW-Authenticate": "Bearer"},
    )
    try:
        payload = jwt.decode(token, JWT_SECRET_KEY, algorithms=[JWT_ALGORITHM])
        email: str = payload.get("sub")
        if email is None:
            raise credentials_exception
    except JWTError:
        raise credentials_exception

    user = await get_user_by_email(email)
    if user is None:
        raise credentials_exception
    return user

Routers folder

auth.py
from typing import Annotated

from fastapi import APIRouter, Depends, HTTPException, status
from fastapi.security import OAuth2PasswordRequestForm

from app.core.security import create_access_token, verify_password
from app.deps.auth import create_user, get_user_by_email, get_current_user
from app.schemas.user import UserCreate, UserPublic, Token, UserInDB

router = APIRouter(prefix="/auth", tags=["auth"])

@router.post("/register", response_model=UserPublic)
async def register(user: UserCreate):
    new_user = await create_user(user)
    return UserPublic(id=new_user.id, email=new_user.email)

@router.post("/login", response_model=Token)
async def login(form_data: Annotated[OAuth2PasswordRequestForm, Depends()]):
    # I am using email as the username field in the OAuth2 form.
    user = await get_user_by_email(form_data.username)
    if not user or not verify_password(form_data.password, user.hashed_password):
        raise HTTPException(
            status_code=status.HTTP_401_UNAUTHORIZED,
            detail="Incorrect email or password",
        )
    access_token = create_access_token(data={"sub": user.email})
    return Token(access_token=access_token)

@router.get("/me", response_model=UserPublic)
async def get_me(
    current_user: Annotated[UserInDB, Depends(get_current_user)]
):
    return UserPublic(id=current_user.id, email=current_user.email)

chat.py
from typing import Annotated

from fastapi import APIRouter, Depends
from langchain_core.messages import HumanMessage, SystemMessage

from app.core.llm import llm
from app.db.mongo import notes_collection
from app.deps.auth import get_current_user
from app.schemas.user import UserInDB
from app.schemas.chat import ChatRequest, ChatResponse

router = APIRouter(prefix="/chat", tags=["chat"])

@router.post("/", response_model=ChatResponse)
async def chat_with_ai(
    body: ChatRequest,
    current_user: Annotated[UserInDB, Depends(get_current_user)],
    # Uses LangChain + OpenAI + some context from MongoDB.
):
    print(current_user)

    notes_cursor = (
        notes_collection.find({"user_id": current_user.id})
        .sort("_id", -1)
        .limit(3)
    )
    notes = await notes_cursor.to_list(length=3)

    context_text = "\n".join(
        note.get("text", "") for note in notes if note.get("text")
    )
    used_context = bool(context_text)

    system_prompt = (
        "You are an assistant helping the user with their notes."
        " "
        "Use the following context if relevant; if not, just answer normally.\n\n"
        f"Context:\n{context_text or '(no context available)'}"
    )
    messages = [
        SystemMessage(content=system_prompt),
        HumanMessage(content=body.question),
    ]

    response = await llm.ainvoke(messages)

    return ChatResponse(answer=response.content, used_context=used_context)

items.py
from typing import List, Annotated

from fastapi import APIRouter, Depends

from app.db.mongo import items_collection
from app.schemas.item import Item
from app.schemas.user import UserInDB
from app.deps.auth import get_current_user

router = APIRouter(prefix="/items", tags=["items"])

@router.get("/", response_model=List[Item])
async def list_items(
    current_user: Annotated[UserInDB, Depends(get_current_user)]
):
    # Example async Mongo query using Motor.
    # Demo: seed items if collection is empty
    count = await items_collection.count_documents({})
    if count == 0:
        await items_collection.insert_many(
            [
                {"name": "Book", "price": 10.5},
                {"name": "Laptop", "price": 999.99},
            ]
        )

    docs = await items_collection.find().to_list(length=100)
    return [
        Item(id=str(doc["_id"]), name=doc["name"], price=float(doc["price"]))
        for doc in docs
    ]

notes.py
from typing import Annotated

from fastapi import APIRouter, Depends
from langchain_core.messages import HumanMessage, SystemMessage

from app.core.llm import llm
from app.db.mongo import notes_collection
from app.deps.auth import get_current_user
from app.schemas.user import UserInDB
from app.schemas.chat import ChatRequest, ChatResponse
from app.schemas.notes import NotesCreate, NotesInDB

router = APIRouter(prefix="/notes", tags=["notes"])

@router.post("/", response_model=NotesInDB)
async def create_note(body: NotesCreate,current_user: Annotated[UserInDB, Depends(get_current_user)]):
    print(current_user)

    note = await notes_collection.insert_one({"text": body.text, "user_id": current_user.id})
    return NotesInDB(id=str(note.inserted_id), text=body.text, user_id=current_user.id)

Main.py

from fastapi import FastAPI

from app.routers import auth, items, chat, notes

app = FastAPI(title="FastAPI + MongoDB + JWT + LangChain Example")

@app.get("/")
async def root():
    return {"message": "FastAPI + MongoDB + JWT + LangChain example"}

# Include routers
app.include_router(auth.router)
app.include_router(items.router)
app.include_router(chat.router)
app.include_router(notes.router)

Application execution

On the main folder, you can execute the following command:

uvicorn app.main:app --reload
INFO: Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)

If you access the page http://127.0.0.1:8000/docs#, you can find the Swagger documentation created by default.

Users creation

You can execute the following command to create a new user:

curl -X 'POST' \
  'http://127.0.0.1:8000/auth/register' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "email": "user@example.com",
  "password": "string" 
}'

Or if you have vscode Rest Api Extension you can execute
POST http://127.0.0.1:8000/auth/register
Accept: application/json
Content-Type: application/json

{
  "email": "user@example.com",
  "password": "string"
}

The above command will create a new user with hashed password:

Users authentication

In order to authenticate the user and retrieve the bearer token, please execute the following command:

curl -X 'POST' \
  'http://127.0.0.1:8000/auth/login' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  -d 'grant_type=password&username=user%40example.com&password=string&scope=&client_id=string&client_secret=string'
Rest API extension
POST http://127.0.0.1:8000/auth/login
Content-Type: application/x-www-form-urlencoded
accept: application/json

grant_type=password&username=test@example.com&password=123&scope=&client_id=string&client_secret=string

After a successful authentication, you will receive the following output. The token is important to authenticate on the remaining routes.

HTTP/1.1 200 OK
date: Tue, 02 Dec 2025 17:40:11 GMT
server: uvicorn
content-length: 180
content-type: application/json
Connection: close

{
  "access_token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJ0ZXN0QGV4YW1wbGUuY29tIiwiZXhwIjoxNzY0NzAwODExfQ.riXj1vvLZSyzyU4T7kG3ygYaUU8sgvgCjK_Os_uggvo",
  "token_type": "bearer"
}

Idea creation

The main purpose of the application is to make suggestions based on ideas that the end users have. The following request will create those notes.

POST http://127.0.0.1:8000/notes/
Content-Type: application/json
Authorization: Bearer {{token}}

{
  "text": "testing123"
}

Chat interaction

The application now offers advice based on the notes stored by users in the database. To enable interaction with the LLM, we define the chat's behavior in the chat.py file. This definition is an example of prompt engineering, and this is the section where you can modify the prompt's language and content, if needed.

system_prompt = (
        "You are an assistant helping the user with their notes."
        " "
        "Use the following context if relevant; if not, just answer normally.\n\n"
        f"Context:\n{context_text or '(no context available)'}"
    )

Execution example:

POST http://127.0.0.1:8000/chat/
Content-Type: application/json
Authorization: Bearer {{token}}

{
  "question": "What i need to do"
}

Execution output:

{
  "answer": "To help you achieve your goals of saving more money and finishing your project, here are some actionable steps:\n\n**Saving More Money:**\n\n1. **Track your expenses**: Write down every single transaction, no matter how small, for a week or two to understand where your money is going.\n2. **Create a budget**: Based on your income and expenses, allocate a specific amount for savings each month.\n3. **Automate your savings**: Set up an automatic transfer from your checking account to your savings account.\n4. **Cut back on unnecessary expenses**: Identify areas where you can cut back on unnecessary spending, such as dining out or subscription services.\n5. **Consider a savings challenge**: Try a savings challenge like the \"52-week savings challenge\" where you save an amount equal to the number of the week (e.g., Week 1: Save $1, Week 2: Save $2 etc.).\n\n**Finishing Your Project:**\n\n1. **Break down your project into smaller tasks**: Divide your project into manageable tasks to make it feel less overwhelming.\n2. **Create a schedule**: Set a specific deadline for each task and stick to it.\n3. **Prioritize your tasks**: Focus on the most critical tasks first, and then move on to less important ones.\n4. **Remove distractions**: Identify potential distractions (e.g., social media, email, phone notifications) and eliminate them while you work.\n5. **Take regular breaks**: Take short breaks to recharge and maintain productivity.\n\nWhich of these steps do you want to start with?",
"used_context": true
}

Docker (optional)

Our application is currently functional. However, to execute it in a different environment, we must install the necessary dependencies. Docker can facilitate and streamline this deployment process.

Create Dockerfile and paste the following code:

# Dockerfile
FROM python:3.11-slim

# Set work directory inside the container
WORKDIR /app

# Install system dependencies (optional but useful for many libs)
RUN apt-get update && apt-get install -y --no-install-recommends \
    build-essential \
    && rm -rf /var/lib/apt/lists/*

# Copy only requirements first (for better Docker cache)
COPY requirements.txt .

# Install Python dependencies
RUN pip install --no-cache-dir -r requirements.txt

# Copy the rest of your project
COPY . .

# Expose the FastAPI port
EXPOSE 8000

# Default command: run uvicorn
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

Create the docker-compose.yml:

version: "3.8"
services:
  fastapi:
    build: .
    container_name: fastapi-langchain
    ports:
      - "8000:8000"
    env_file:
      - .env # optional: if you have one
    extra_hosts:
      - "host.docker.internal:host-gateway"
    restart: unless-stopped

Now, we can build and execute our docker:

docker-compose build
docker-compose up -d

Conclusion

This tutorial has provided a complete, practical demonstration of building a modern, data-driven application by seamlessly integrating FastAPI, LangChain, and MongoDB.

We have successfully covered:

FastAPI for creating a robust, fast, and documented REST API, including user registration and JWT-based authentication.
MongoDB (using Motor) for asynchronous and scalable document storage, handling user data and application-specific notes.
LangChain (with Ollama) for enabling powerful AI capabilities, using the stored user notes as dynamic context to generate personalized, helpful suggestions and answers.

The example application, an idea generation assistant, serves as a clear blueprint for how to leverage this stack for complex projects requiring data persistence, secure user access, and cutting-edge large language model integration. By using a local LLM via Ollama, we also ensured the entire stack is executable on a local machine, making it accessible for rapid prototyping and development.

This integration strategy—combining the speed of FastAPI, the flexibility of MongoDB, and the intelligence of LangChain—is a highly effective pattern for developing the next generation of intelligent web applications. We hope this guide serves as a solid foundation for your future AI-powered projects.

Async PyMongo in FastAPI

MongoDB Guests — Tue, 14 Apr 2026 14:53:07 +0000

This tutorial was written by Gourav Bhabesh.

1. Introduction

Why modern APIs need asynchronous I/O

Modern APIs need to be robust and be able to serve calls on the go, and minimize the instances of timeouts and server overloads while providing a good user experience during long-running jobs. Asynchronous I/O can solve one of the key issues making modern APIs: staying fast and reliable. It lets the user know the progress of their task while the API handles the requests in the background and is available for other requests. One of the key challenges in the API calls to the database is that some queries can be returned in a few milliseconds but others can take seconds (even with well indexed queries). Without asynchronous I/O in the API, this can lead to long request queues.

You can imagine this as a bakery storefront serving different goods at the same time to multiple customers. We want the orders to be filled asap without the queue becoming too big. In our example, we have multiple servers (scalable infrastructure) who take the orders which can take varying time to process and each customer with longer order times is given a status bar to know where their order’s current status. So a customer with an order that can be instantly filled (picking up a cookie) does not have to wait behind such a customer with a long processing order (baking a cheese cake).

FastAPI’s design philosophy around async

The building block of the FastAPI is ASGI (Asynchronous Server Gateway Interface) standard, unlike the older WSGI (Web Server Gateway Interface) which was synchronous in nature by default. In short, it uses Python’s async/await syntax, yielding control to the event loop, allowing servers to process other requests in the meantime, rather than blocking an entire thread. Using an event loop lets FastAPI handle thousands of concurrent requests with fewer resources. This leads to higher throughput per thread and lower latency. This automatically encourages the developer to use the async ecosystem to fully leverage the performance benefits of async programming.

Database bottlenecks & why async DB drivers matter

One of the key challenges that any database faces is the uncertainty in response times, in which the queries will be answered by the database server. Some queries can take milli-seconds and some can take seconds. If the architecture is not well built to handle this uncertainty, it can cause severe database bottlenecks. To handle this challenge, async database drivers ensure that longer requests do not become a bottleneck for other incoming requests. Initially, the async MongoDB driver was built in the library Motor. But in late 2023, Pymongo team officially decided to build the native async solution, with the goal to replace Motor as the long-term solution.

Introducing the new Async API in PyMongo

The Pymongo team has been working on the goal to replace the Motor library to give native async support in the pymongo library itself. This gives confidence on the client side to use the official library with async support. The async support was first provided in the version 4.9 in Pymongo and it has been further stabilised to the current version of 4.15.0. The new API capabilities bring the asyncio support directly to the official MongoDB driver, allowing developers to use awaitable database operations without external libraries like Motor. Combined with the FastAPI framework, which already supports await/async capability, Pymongo can now be natively used to build high-throughput, non-blocking services.

2. What is PyMongo?

Overview of the official MongoDB Python driver

MongoDB has provided Pymongo as the official Python driver to allow the applications to communicate with MongoDB clusters, providing production-tested API to perform CRUD operations, manage indexes, run aggregation pipelines and handle advanced features such as transactions and change streams. Pymongo was traditionally built on a synchronous I/O model, meaning every database call made will block the executing thread until the MongoDB server does not return a response leading to database bottlenecks.

Traditional synchronous PyMongo behavior

The key behaviours of the traditional Pymongo library, which we should be aware of, to appreciate the addition of the native support of asynchronous, are that synchronous operations block Python threads until the operation completes or a network error or timeout occurs.. This leads to another behaviour - sequential execution, blocking all requests until and unless the previous request is not complete. This made the MongoClient “Thread-Safe” allowing for multi-threading applications to be written, but each thread still performs blocking database operations.

In our example, of the baking shop, it would be like each server is concurrently serving the customers but each server is waiting for the order to complete rather than handing a token to wait, while serving the next customer request which might be served in much less time.

Blocking vs non-blocking I/O explained simply

Blocking I/O means the application must wait for the database operation to finish or network timeout before it will allow anything else to continue. On the other hand, non-blocking I/O allows requests to continue flowing to the database while previous requests are still being processed. In addition, the async drivers provide a way to receive the status of the request, giving power to the developer to handle long operations and create a better user-experience.

Limitations of synchronous DB operations in async frameworks

Using synchronous DB operations in the async framework is one of the key mistakes made by many clients of MongoDB, part of the reason was the non-availability of official Python drivers that actually supported asynchronous behavior. If the synchronous Pymongo driver is used inside an async framework such as FastAPI, these blocking calls freeze the event loop, preventing the server from processing other incoming requests and significantly reducing concurrency. As a result, applications that rely on synchronous database drivers inside async frameworks often experience increased latency, request queuing, and poor scalability under load, even if the rest of the code is written using async and await.

Imagine our baking shop upgraded its entire framework to be non-synchronous. The cooks can cook multiple dishes at the same time, but the servers are still the same, processing sequential orders and not taking new orders until the previous order has been delivered. The cook is having an easy day!!

3. What is Async PyMongo?

Overview of PyMongo’s new async API (`pymongo.async_`)

The official MongoDB team started working on the native-async support in the Pymongo driver in version4.9, where it was first released as beta version in late 2023. As of writing this article, the pymongo version 4.15 has been released and has been stable since version 4.13. With the pymongo.With async support, we can create an AsyncMongoClient object similar to the MongoClient object we created in the previous synchronous call. The Client provides fully awaitable database operations such as insert_one, find_one, update_one, and aggregation pipelines, allowing MongoDB I/O to integrate natively with Python’s event loop.

Key advantages over synchronous API

Compared to the traditional synchronous API, async PyMongo significantly improves scalability and responsiveness under concurrent workloads. Because database calls no longer block the event loop, asynchronous framework applications can continue handling new requests while waiting for MongoDB responses, resulting in lower latency, higher throughput, and better CPU utilization—especially in high-traffic microservices

Relationship to Motor (Motor’s role before async PyMongo existed)

Before async Pymongo, motor was the de facto library to be used when developers needed asynchronous behavior when communicating with the MongoDB clusters. Motor did this by exposing a coroutine-based API, allowing applications to switch context and perform other tasks while waiting for I/O operations. However, Motor was not a separate driver from scratch; it was designed as a wrapper around the core synchronous PyMongo library. It used Tornado’s or asyncio’s event loop, wrapping Pymongo’s operations asynchronously, effectively allowing the main event loop to remain unblocked. Motor was specifically created to support asynchronous web frameworks, initially Tornado and later adding support for asyncio. Performance comparison between motor and async pymongo can be found on the official documentation here.

How to migrate from motor to async-pymongo and sync-pymongo to async-pymongo

Migrating from the motor-based asynchronous library to the async-pymongo library is simple and is well documented by MongoDB here. In simple terms, one needs to change the driver call to the database using the AsyncMongoClient and add the keyword await in front of the database operations.

When migrating from sync Pymongo to async Pymongo, keep the following important points from the official MongoDB documentation in mind:
· To convert an AsyncCursor to a list, you must use the asynchronous cursor.to_list() method.
· The AsyncCollection.find() method in the PyMongo Async API is synchronous, but returns an AsyncCursor.
To iterate through the cursor, you must use an async for loop.
· The AsyncMongoClient object does not support the connect keyword argument.
· You cannot share AsyncMongoClient objects across threads or event loops.
· To access a property or method of a result returned by an asynchronous call, you must properly wrap the call in parentheses, as shown in the following example:

id = (await posts.insert_one(doc)).inserted_id

4. Key Features of Async PyMongo

True async interface with awaitable operations

The biggest advantage of the async Pymongo library for developers is its awaitable operations. Functions such as insert_one(), find_one(), update_one() and aggregate() return objects that can wait rather than blocking the results. This fundamentally decouples the event loop from the operation, meaning the database operation running on the database side yields the control back to the event loop, allowing FastAPI to process the other incoming requests without blocking the thread.

In our baking shop example, the server has been decoupled from the chef, while the previous order is getting cooked in the kitchen, the server takes in a new order without ceasing the operation and blocking the thread and allowing for incoming requests to be fulfilled !! This model of working is fundamentally different from the traditional synchronous PyMongo, where every database call ties up a worker thread until completion, blocking the thread from serving the next call when under load.

Connection pooling behavior

The connection pool behaviour is quite an interesting one here. On the surface it appears both the synchronous and the asynchronous Pymongo connection-poling implementation look the same (parameters available - maxPoolSize, minPoolSize, waitQueueTimeoutMS etc.) and indeed rely on the same MongoDB C driver. However, async applications are actually able to spawn thousands of concurrent operations and grab many connections from the pool at once, whereas, in the synchronous application each request occupies a python worker thread, holding a connection from the pool waiting for MongoDB. This naturally limits concurrency as the application is limited by the number of workers, effectively meaning the full pool size can be never used optimally.

Imagining in our world of baking shop, we have an army of chefs to process the orders, the async model allows the server to grab hold of any available chef to accept a new order and move on to the next one, whereas in the synchronous setup the server will wait for the chef to finish the previous order before moving to the new order.

Async cursors and non-blocking iteration

One of the primary challenges that any database faces is returning results to query with a large result set. The traditional PyMongo driver already returns a cursor object rather than loading all the documents into the memory at once. In async PyMongo, these cursors are fully asynchronous in nature as well, resulting in iteration without blocking the event loop. Developers now have the choice of using the await cursor.to_list() or iterating over the cursor object using the async for. This will allow data to be streamed in batches while the application is handling incoming requests. This feature is of significant importance when we have endpoints that return large datasets, background processing jobs and real-time analytics workloads.

5. What is FastAPI?

FastAPI’s async-first architecture

FastAPI is a high-performance web framework that is production-ready and designed for building APIs in python, with roots embedded in asynchronous programming. It embraces Python’s asyncio model as its core principle rather than treating it as optional. This design choice gives FastAPI leverage for I/O bound workloads such as db access, external API calls and real-time data streaming.

Other traditional web frameworks default to synchronous request handling, whereas FastAPI encourages using asynchronous endpoints (async def) wherever possible. When an endpoint is declared using the async keyword, FastAPI uses the event loop execution rather than a worker thread. Now as long as those requests involve the non-blocking I/O operations, a single worker process can handle many requests concurrently. This is where FastAPI’s async-first architecture shines. When the entire request path - including the database access - is non blocking, each request becomes super efficient. Any stack in the request that blocks the event loop will make the FastAPI’s concurrency model diminish significantly.

Event loop and concurrency model

At the heart of FastAPI’s concurrency model is Python’s asyncio event loop. The event loop manages the execution of many coroutines, switching between them whenever one is waiting for I/O, such as a network response from MongoDB. This cooperative multitasking enables a single process to efficiently handle thousands of simultaneous requests without spawning thousands of threads. However, this model only works as intended when all long-running operations are awaitable. If a synchronous operation blocks the event loop — for example, a traditional synchronous PyMongo database call — the entire server becomes unresponsive to other requests until that operation completes.

Why FastAPI pairs naturally with non-blocking database drivers

As explained above the core principle of FastAPI is to yield the control back to the event loop and any blocking I/O operation in the chain of the operations sent by the request will be like throwing a wrench on the gears. With the pairing of the non-blocking db driver such as the async pymongo, when a FastAPI endpoint awaits a MongoDB operation, control is yielded back to the event loop, allowing for the next set of requests to be processed in parallel. With the alignment between the db driver and the base principle of FastAPI to be async, we eliminate the friction and make the full request truly asynchronous in nature and hence enable FastAPI to scale efficiently under high concurrency.

Surprising performance differences when mixing sync DB code into async routes

A common pitfall for developers writing code in FastAPI is mixing the synchronous PyMongo calls inside the async def endpoints. On the surface it may appear to work, but it breaks the core principle of the FastAPI to be truly asynchronous. One single blocking db call and the entire event loop gets stalled as other requests will have to queue behind it - even when the server has many workers available.

In practice, this means that an API written with FastAPI but using synchronous PyMongo can perform worse under load than a fully synchronous application with multiple worker threads. By contrast, pairing FastAPI with Async PyMongo preserves the framework’s intended concurrency benefits, leading to better throughput, lower latency for typical requests, and more predictable scaling behavior. In large applications, having the capability to predict the scalability of infrastructure and have a control on the concurrency is of paramount importance.

6. Why Use Async PyMongo with FastAPI?

Avoid event-loop blocking

Using FastAPI with Asynchronous Pymongo correctly is essential if we want to avoid event-loop blocking. The requests from the async-first framework like FastAPI, share the same event-loop within each worker process. Meaning any operation in this request if becomes synchronized will lead to blocking of the entire event-loop, which high-concurrency systems want to avoid at all cost.

By using Async Pymongo, each db operation becomes awaitable rather than blocking in nature, ensuring a slow db operation does not freeze the entire application, allowing other requests to get processed.

Handle thousands of concurrent requests

FastAPI is capable of handling thousands of concurrent requests per worker process, but designing the entire chain of operations in an asynchronous manner is a must to achieve this. One of the primary mistakes that are made in the coding side is missing the async and await keywords in the db operation call, this breaks the core principle of FastAPI leading to slowness. With correct implementation we can make the infrastructure more cost-effective and efficient as compared to a purely synchronous system which would not be scalable in the long run as more threads are required to handle the huge volume of requests.

Reduce latency under load

With modern tech stacks, both horizontal and vertical scaling is a must and systems built with such capabilities can withstand the test of time. And today's system should be capable enough to handle varying loads. With FastAPI - async nature is built in-core principle of the library itself and when combined with Async Pymongo it allows for the potential blocking db operations to be handled asynchronously. This leads to lower median latency for requests, especially I/O intensive ones. In section 8, we will see how the sync/async jobs specifically prove this point, when we compare the median latency of thousands of concurrent requests.

Improve throughput for high-traffic microservices

In the world of microservices where responsiveness and scalability are critical Async Pymongo dramatically improves the performance by removing the blocker of such requests to the db layer. Combined with async frameworks such as FastAPI, Async Pymongo is superbly placed to serve real-time applications, analytics dashboards or user-facing services.

Better CPU utilization and scaling on limited infrastructure

One of the vital nerves of any IT infrastructure is cost-efficiency and with Devops team’s mantra - “doing more with less”, fits perfectly with the Async Pymongo. Making the most of the available resources, async architecture uses the same thread without blocking incoming requests by handing over the control back to the single event loop. Instead of maintaining hundreds or thousands of idle threads waiting on the I/O, FastAPI with Async Pymongo exploits this single event loop allowing it to switch between tasks as needed. In cloud based infrastructure, where compute resources are directly tied to cost, it can lead to meaningful savings without sacrificing performance.

7. Async CRUD Operations with Examples

This section demonstrates the common MongoDB operations asynchronously using the pymongo.async_ inside FastAPI.

All examples assume:

from pymongo.async_ import AsyncMongoClient 
from bson import ObjectId 
client = AsyncMongoClient (MONGO_URI) db = client["appdb"] 
users = db["users"]

Insert one / insert many (async versions)

Insert one document example:

@app.post("/users") 
async def create_user(user: dict):
  result = await users.insert_one (user) 
  return {"inserted_id": str(result.inserted_id)}

Insert many documents example:

@app.post("/users/bulk") 
async def create_many(users_in: list[dict]): 
  result = await users.insert_many (users_in) 
  return {"inserted_ids": [str(i) for i in result.inserted_ids]}

Find one / find many with async cursors

Find one document example:

@app.get("/users/{email}")
async def get_user(email: str): 
 return await users.find_one ({"email": email})

Find many documents example:

@app.get("/users") 
async def list_users():
  cursor = users.find ({}) 
  return await cursor.to_list(length=100)

Async pagination using cursors

@app.get("/users/page") 
async def paginated_users(last_id: str | None = None, limit: int = 20): 
  query = {"_id": {"$gt": ObjectId(last_id)}} if last_id else {} 
  cursor = users.find(query).sort("_id", 1).limit (limit) 
  results = await cursor.to_list(length=limit) 

  return { 
    "results": results,
    "next_cursor": str(results[-1]["_id"]) if results else None, 
  }

(Avoid skip() for large collections; use _id cursor pagination instead.)

Update and replace operations

Update one document example:

@app.patch("/users/{email}")
async def update_user(email: str, changes: dict):
  result = await users.update_one(
    {"email": email},
    {"$set": changes},
  ) 
  return {"matched": result.matched_count, "modified":
result.modified_count}

Replace one document example:

@app.put("/users/{email}")
async def replace_user(email: str, new_doc: dict):
  result = await users.replace_one({"email": email}, new_doc) 
  return {"matched": result.matched_count}

Delete one/many operations

Delete one document example:

@app.delete("/users/{email}") 
async def delete_user(email: str):
  result = await users.delete_one ({"email": email}) 
  return {"deleted": result.deleted_count}

Delete many documents example:

@app.delete("/users/inactive")
async def delete_inactive():
  result = await users.delete_many ({"active": False}) 
  return {"deleted": result.deleted_count}

Async aggregation pipelines

@app.get("/users/stats")
async def user_stats():
  pipeline = [
    {"$group": {"_id": "$country", ""count"": {"$sum": 1}}},
    {"$sort": {"count": -1}},
  ] 
cursor = await users.aggregate (pipeline)
return await cursor.to_list(length=100)

Error handling (duplicate keys, timeouts, network issues)

Duplicate keys:

from pymongo.errors import DuplicateKeyError
from fastapi import HTTPException

try: 
  await users.insert_one({"email": "a@b.com"})
except DuplicateKeyError:
  raise HTTPException(status_code=400, detail="User already exists")

Timeout/network issues:

from pymongo.errors import ExecutionTimeout, ConnectionFailure

try:
  await users.find_one({"email": "a@b.com"})
except ExecutionTimeout:
  raise HTTPException(status_code=504, detail="DB timeout")
except ConnectionFailure:
  raise HTTPException(status_code=503, detail="DB unreachable")

8. Performance Comparison: Sync PyMongo vs Async PyMongo

In this section, we break down the high-stakes battle between the traditional Synchronous (Sync) driver and the modern Asynchronous (Async) driver using the large-scale sample_airbnb dataset. This isn't just a syntax preference; it's the difference between a server that remains responsive under fire and one that collapses under its own weight.

Example endpoints and profiling results

To push these drivers to the limit, we implemented a Complex Aggregation task. Instead of a simple "Ping," we forced each request to filter high-rated Airbnb listings, sort them by price, and project complex nested fields.

Sync Logic: Utilizes list(users.aggregate(pipeline)), which occupies a physical thread for the duration of the database round-trip and data serialization.

Async Logic: Utilizes await (await users.aggregate(pipeline)).to_list().
The server yields control during the database "wait time," allowing the single-threaded event loop to process other incoming users.

Locust Benchmark setup

We subjected both implementations to a "Trial by Fire" using Locust:

User Load: Scaled up to 500 concurrent users.

Hardware Constraint: We intentionally limited the Sync thread pool to 1 thread to simulate extreme resource exhaustion and observe the "Queueing Effect."

Target: listingsAndReviews collection in MongoDB Atlas.

Understanding event-loop blocking

The results highlight a fundamental architectural divide. In the Sync model, when the thread pool is exhausted (in our case, limited to 1), every new user must wait for the person in front of them to finish their entire database operation. This creates a "Single Lane Bridge" bottleneck. In the Async model, even with thousands of users, the event loop acts as a high-speed scheduler. It sends the request to MongoDB and immediately moves to the next user, effectively "stacking" thousands of concurrent database operations without needing a thread for each one.

Graphs and metrics discussion (Locust latency plots)

Table: Sync stats showing the performance on one thread
Graph: Sync response times

Table: Async stats showing the performance

Graph: Async response times

Looking at our Locust metrics, the disparity is stark:
| Metric | Sync (1 Thread) | Async (Event Loop) |
|-------------------|-----------------|--------------------|
| Max RPS | ~48.8 | ~113.4 |
| Median Latency | ~8,700 ms | ~2,800 ms |
| 95th-Percentile | ~9,400 ms | ~4,100 ms |
| Throughput Driver | Thread-bound | DB / Network-bound |

Sync Graphs: Notice the staircase effect in response times.
As users increase, the latency climbs linearly because users are waiting in a queue. The RPS is "hard-capped" because the single thread can only process so many documents per second.

Async Graphs: The response times remain significantly lower and more stable.
Even as we ramped to 500 users, the 95th percentile hovered around 4 seconds compared to nearly 10 seconds for Sync. This proves that Async manages high-concurrency significantly better, even when the individual database query is complex.

Full working snippet for readers to run

Ready to see these results on your own machine?
We have provided a complete repository containing the FastAPI implementation, the AnyIO thread limiter configuration, and the Locust scripts used for these benchmarks.

Demo work - https://github.com/gouravmongodb/fastapi_pymongo_demo

9. Common Pitfalls & How to Avoid Them

Even with Async PyMongo, it is easy to introduce subtle performance or correctness issues if you are not careful. Below are the most common pitfalls developers encounter and how to avoid them.

Forgetting await on Database Calls

Because Async PyMongo methods are awaitable, forgetting await will not raise an immediate syntax error but will return a coroutine object instead of executing the operation. This can lead to silent failures or unexpected behavior in your API responses. Always double-check that every database operation inside an async def route is properly awaited.

Creating a New Client for Every Request

Instantiating AsyncMongoClient inside each request handler is a major anti-pattern. It creates unnecessary TCP connections, increases latency, and quickly exhausts resources under load. Instead, create a single shared client at application startup (using FastAPI lifespan or startup events) and reuse it across all requests.

Returning Non-JSON-Serializable Types

MongoDB types like ObjectId, datetime, or Decimal128 are not directly JSON serializable. Returning raw MongoDB documents from FastAPI will cause serialization errors. The recommended approach is to either convert these types to strings or map your documents to Pydantic models that handle serialization correctly.

Exhausting Connection Pools Under Heavy Load

Async applications can issue far more concurrent database operations than synchronous ones, which makes connection pool tuning critical. If maxPoolSize is too low, requests will start queuing inside the driver, increasing latency. Monitor Atlas metrics and adjust maxPoolSize, minPoolSize, and maxConnecting based on your workload.

Mixing Synchronous and Asynchronous Code

Using synchronous PyMongo calls inside async def routes (or vice versa) undermines FastAPI’s concurrency model and can block the event loop. Choose one model—preferably Async PyMongo for FastAPI—and stick to it consistently throughout your application.

10. Conclusion

Asynchronous database access is a cornerstone of modern, scalable API design. In an async-first framework like FastAPI, using a non-blocking database driver is not a luxury—it is a necessity for handling high concurrency, reducing latency, and making efficient use of resources.

With the stabilization of pymongo.async_ in PyMongo 4.13, Async PyMongo has become the recommended and officially supported way to use MongoDB with FastAPI. It provides a native, awaitable interface that aligns with FastAPI’s event loop, simplifies dependencies, and brings the future of the MongoDB Python ecosystem under a single driver.

Synchronous PyMongo still has a place in the ecosystem — particularly for simple scripts, batch jobs, or applications that are not built around async frameworks. However, for production web services, real-time applications, and microservices that must scale, Async PyMongo is the better choice.

Ultimately, performance depends not just on “sync vs async,” but on how you design your workload, tune your connection pool, index your collections, and scale your MongoDB cluster. When used thoughtfully, Async PyMongo with FastAPI offers a powerful, maintainable, and future-proof foundation for building high-performance Python APIs.

Research work

· https://pymongo.readthedocs.io/en/stable/changelog.html#changes-in-version-4-9-2024-09-18
· From Pymongo driver v4.9.0 introduced the beta version of async function.
· From Pymongo driver v4.13.0 - asynchronous API is now stable and no longer in beta.
· https://www.mongodb.com/docs/languages/python/pymongo-driver/current/reference/migration/#migrate-to-pymongo-async
· https://www.mongodb.com/docs/languages/python/pymongo-driver/current/reference/migration/#performance-benchmarks

Comparing Python ODMs for MongoDB

MongoDB Guests — Mon, 13 Apr 2026 15:45:57 +0000

This article was written by Fabio Cionini.

ORM and ODM: Relational vs. document

You might already be familiar with object-relational mapping (ORM) techniques that convert relational database data into objects suitable for use in a programming language. Libraries like Sequelize (for Node.js) or, sticking with Python, SQLAlchemy, allow us to easily interact with data structures that are very different from the object metaphor we use in modern software development. They save us from writing boilerplate code and help encapsulate business rules and data validation.

When working with a document-oriented, non-relational database like MongoDB — whose data structure more closely resembles the objects we manipulate with our code — these techniques and libraries are often called object-document mapping (ODM).

The MongoDB document model

The MongoDB data structure closely resembles the objects we usually manipulate with our code. It uses JSON (in practice, BSON under the hood), similar to JavaScript objects, as the preferred human-readable data interchange format.

Because we can embed objects and arrays as values in documents, it becomes easier to integrate the data layer with the application layer. MongoDB’s flexible schema also allows rapid prototyping and fast iteration without the need to manage migrations and database schema versions as strictly as in traditional relational databases.

From this perspective, you might think that ODMs are less necessary than ORMs. Although the official MongoDB drivers already “speak” the language of objects, ODMs can be very useful to:

Speed up development by generating boilerplate code for CRUD operations and object mapping.
Make data access code more abstract and portable.
Integrate data validation in a consistent way.

When coupled with a data validation library like Pydantic, ODMs can help guarantee data consistency by enforcing validation rules at the application level.

In this article, we’ll explore the main features offered by some of the most used and loved object-document mapping libraries for Python and see how they complement the native capabilities of the official MongoDB drivers.

Python ODMs

Beanie

Beanie is an asynchronous ODM whose data models are based on Pydantic, the most widely used data validation library for Python. It is lightweight, very intuitive, and has been around for several years, making it one of the most used and loved ODMs available.

You can use modern Python language features such as type hints, decorators, and of course asynchronous calls (using non-blocking access to MongoDB provided by Async PyMongo, which is built on top of Python’s asyncio library).

Installation is easily accomplished using PIP:

py
pip install beanie

or Poetry:

py
poetry add beanie

Beanie uses Async PyMongo as an async database engine. You provide an AsyncMongoClient instance and a list of your document models to the init_beanie(...) function.

In Beanie, each database collection is represented by a class that extends the Document base class. By extending that class, you define the schema and perform all the read and write operations on the corresponding collection.

You can also use pydantic.BaseModel directly by extending its class for embedded documents. For example:

py
import asyncio
from typing import Optional
from pydantic import BaseModel
from beanie import init_beanie, Document, Indexed
from pymongo import AsyncMongoClient

class Category(BaseModel):
name: str
description: str

class Product(Document):
name: str
description: Optional[str] = None
price: Indexed(float)
category: Category

async def main():
client = AsyncMongoClient("mongodb://root:root@localhost:27017/")
await init_beanie(database=client.shop, document_models=[Product])

coffee = Category(
    name="Coffee",
    description="A preparation of roasted and ground coffee seeds.",
)
espresso = Product(
    name="Italian Espresso",
    price=2.95,
    category=coffee,
)

await espresso.insert()
if name == "main":
asyncio.run(main())

This defines two models:

Category, based on Pydantic, used as an embedded document inside Product.
Product, the actual document that will be inserted into the corresponding MongoDB collection.

Notice that price is defined as an indexed field via Indexed(float): this will create an index on price in the underlying collection. The insert() method stores the Product instance in MongoDB.

The Beanie website provides a clear, easy-to-follow tutorial on defining documents along with comprehensive documentation for more advanced use cases.

MongoEngine

MongoEngine is a mature and widely adopted ODM. It has been around for many years and is considered one of the most stable options in the Python ecosystem.

MongoEngine follows a synchronous programming model and offers a very Django-like developer experience, with:

Explicit field definitions.
Built-in validation.
Rich query capabilities.

Similarly to Beanie, each collection is represented by a class that extends the Document base class. Fields are declared explicitly using typed field classes, and embedded documents are modeled by extending EmbeddedDocument.

Installation is done with the usual pip command: pip install mongoengine

A simple example looks like this:

py
from mongoengine import (
connect,
Document,
EmbeddedDocument,
StringField,
FloatField,
EmbeddedDocumentField,
)

class Category(EmbeddedDocument):
name = StringField(required=True)
description = StringField()

class Product(Document):
name = StringField(required=True)
description = StringField()
price = FloatField(required=True)
category = EmbeddedDocumentField(Category)

connect(db="shop", host="mongodb://localhost:27017/shop")

coffee = Category(
name="Coffee",
description="A preparation of roasted and ground coffee seeds.",
)
espresso = Product(
name="Italian Espresso",
price=2.95,
category=coffee,
)
espresso.save()

Calling save() persists the document to MongoDB, automatically handling inserts or updates as needed.

The synchronous nature of MongoEngine offers a concise syntax and fine-grained control over the sequence of operations. For workloads where you don’t need concurrency at scale, this can make application logic easier to reason about, even if asynchronous ODMs may use resources more efficiently under heavy I/O load.

MongoEngine’s online documentation provides both a comprehensive API reference and a getting started tutorial.

ODMantic

ODMantic is a modern asynchronous ODM built on top of Pydantic and Motor, designed to provide strong typing, data validation, and native asyncio support.

Its design philosophy is quite close to Beanie: both libraries offer “Pythonic” ways to interact with MongoDB. ODMantic, however, is somewhat less focused on data migration and more on ergonomic models and async workflows.

Here is a basic example:

py
import asyncio
from typing import Optional
from odmantic import Model, Field, EmbeddedModel, AIOEngine
from motor.motor_asyncio import AsyncIOMotorClient

class Category(EmbeddedModel):
name: str
description: str

class Product(Model):
name: str
description: Optional[str] = None
price: float = Field(index=True)
category: Category

async def main():
client = AsyncIOMotorClient("mongodb://localhost:27017")
engine = AIOEngine(client=client, database="shop")

coffee = Category(
    name="Coffee",
    description="A preparation of roasted and ground coffee seeds.",
)
espresso = Product(
    name="Italian Espresso",
    price=2.95,
    category=coffee,
)

await engine.save(espresso)
if name == "main":
asyncio.run(main())

The model definitions and asynchronous syntax strongly resemble Beanie, but the way you persist data (engine.save(...)) feels similar to MongoEngine and many other ODMs/ORMs.

ODMantic can also interact with MongoDB synchronously in specific scenarios, but its primary strength lies in asynchronous code paths.

Documentation is available on the ODMantic website, including a concise quick-start example.

Django

Django is not an ODM by design, but through third-party integrations it can be used to model and interact with MongoDB using a familiar ORM-like interface.

Libraries such as Djongo and the MongoEngine–Django integration allow developers to reuse Django’s model layer while targeting MongoDB as the backing datastore. This approach is often chosen when MongoDB is introduced into an existing Django project and you want to minimize changes to the application layer.

Inside an existing Django backend, you can use a MongoDB-backed Django model like this (using Djongo):

py
from djongo import models
class Category(models.Model):
name = models.CharField(max_length=200)
description = models.TextField()

class Meta:
    abstract = True
class Product(models.Model):
name = models.CharField(max_length=200)
description = models.TextField(null=True, blank=True)
price = models.FloatField(db_index=True)
category = models.EmbeddedField(
model_container=Category,
)

coffee = Category(
name="Coffee",
description="A preparation of roasted and ground coffee seeds.",
)
espresso = Product.objects.create(
name="Italian Espresso",
price=2.95,
category=coffee,
)

The Meta class definition with abstract = True tells Django not to create a separate collection for Category. It is just an abstract class that will be used for embedded objects. The objects.create(...) call then inserts the document into MongoDB using Django’s familiar query API.

Documentation for Djongo is available on its website, while the Django–MongoEngine integration can be found on GitHub.

Asynchronous vs. synchronous

In the overview above, we’ve seen both synchronous and asynchronous ODM libraries. So what is the difference, and which one is best for your use case?

At a high level:

Synchronous code executes operations one after another. Each I/O operation (like a database call) blocks the current thread until it finishes.
Asynchronous code is non-blocking. When an I/O operation is waiting on the network or disk, the event loop can switch to other tasks instead of blocking.

In Python, a mechanism called the Global Interpreter Lock (GIL) prevents true parallel execution of Python bytecode in multiple threads at the same time, ensuring memory consistency and state predictability. This is a separate concern from async I/O, but it’s worth keeping in mind when considering concurrency strategies.

Python’s asyncio module is designed for managing asynchronous operations using coroutines and an event loop. Python also supports concurrency via the threading module and can leverage multiple CPUs via multiprocessing, which bypasses the GIL by using separate processes.

As mentioned earlier, ODM libraries like Beanie and ODMantic provide asynchronous capabilities using asyncio, while Django typically relies on the ASGI gateway and its ecosystem for async support.

Another important piece in the async story is Motor, the official asynchronous driver for MongoDB in Python. Motor has historically been the go-to choice for async access in Python, but it was deprecated in May 2025 in favor of the PyMongo Async API in the pymongo library (in GA at the time of writing). MongoDB will not add new features to Motor and will provide only bug fixes until its end of life on May 14, 2026.

From a performance standpoint:

Asynchronous libraries tend to shine when tasks are frequently blocked by I/O and do not use much CPU — for example, web applications that handle a high volume of concurrent client requests. When a task is waiting for input/output, it can yield control so other tasks can proceed.
Synchronous programming can still be a great fit when there are relatively few concurrent requests and tasks are CPU-bound. In these cases, the overhead of managing an event loop might not pay off, and a straightforward synchronous approach can be simpler and efficient enough.

Conclusion

We have explored different approaches to ODMs in Python:

Beanie and ODMantic are modern, lightweight, and support asynchronous programming, making them attractive for new projects, microservices architectures, and applications that need to handle many concurrent requests efficiently.
MongoEngine offers a solid foundation for synchronous software and a mature ecosystem, with a familiar, Django-like style.
Given the popularity and historical importance of Django in the Python web ecosystem, we also looked at how to integrate MongoDB via Django-friendly tools like Djongo and the Django–MongoEngine integration, which can be particularly useful when you are adding MongoDB to an existing Django project.

Choosing the “best” Python ODM for MongoDB depends on your context:

If you are building a new async-first API or microservice, Beanie or ODMantic are strong candidates.
If you prefer a synchronous style and value maturity and stability, MongoEngine is a solid choice.
If your project already relies heavily on Django, leveraging Djongo or the Django–MongoEngine integration lets you take advantage of MongoDB while preserving familiar patterns.

Ultimately, all of these tools help you align Python’s object-oriented programming model with MongoDB’s flexible, document-oriented data model — so you can focus more on your application’s logic and less on boilerplate data-access code.

Best Practices Cheat Sheet for Flask-PyMongo

MongoDB Guests — Wed, 08 Apr 2026 11:17:13 +0000

This tutorial was written by João Araújo.

Introduction

Starting a project is always a challenge. Naming the first things, deciding on a project structure, what stack to use, and even where we want that project to go. While it is important to have most of it planned, if we are unsure or lack visibility into the project's direction, we should use the right tools to navigate any challenges that may arise. The ability to build applications that evolve over time is a strength of both MongoDB and Flask. If you are starting a project today and are not sure how fast it will grow, or if you just need quick prototyping, MongoDB performance aligned with Flask should be considered. If you are here just for the cheat sheet, it is at the end - but consider reading through to understand the “whys” behind it.

What is MongoDB?

MongoDB is an open-source NoSQL database that stores and manages data using flexible document-oriented structures rather than the traditional tables and rows of relational databases. In MongoDB, data is stored in JSON-like documents (BSON) organized into databases and collections, allowing schemas to be dynamic and easily changed over time, making it highly adaptable for handling diverse, structured, and unstructured data. It’s designed for scalability, flexibility, and ease of development, and is widely used in modern web and mobile applications where rapid iteration and large data volumes are common.

What is Flask?

Flask is a micro web framework written in Python that provides the core essentials for building web applications without imposing a lot of structure or extra components, making it lightweight and easy to use. Flask’s biggest strengths are its simplicity and flexibility. Because it doesn’t enforce a specific project layout or include unnecessary features by default, developers can structure applications as they see fit and choose the best tools for tasks such as database access, authentication, and form handling. Additionally, Flask has a large ecosystem of add-ons that provide features such as authentication, authorization, login, sessions, and more.

Why Choose MongoDB with Flask?

Choosing MongoDB with Flask is a great combination for Python web projects, as both tools are built around flexibility and simplicity. MongoDB’s document-oriented, schema-less storage means you don’t have to define rigid table structures ahead of time. Data can reside in flexible JSON-like documents, making it easy to get started and adapt as your app evolves. Another big plus of using MongoDB with Flask is that JSON-like data structures (e.g., Python dictionaries) map naturally to BSON documents in the database, so developers can work with familiar Python types without complex transformations. We could go on, but let's get this cheat sheet started.

Setup and Project Structure

Configure MongoDB connection parameters (URI, host, port, credentials) via environment variables or configuration files.

Always use configuration files or environment variables to define database connections and overall credentials. This ensures there is no hard-coded information across production and other environments.

In the environment file, there isn't much to see here. I added information on how and what access we have to our database, as well as the environment we run on.

.env
FLASK_ENV=development
FLASK_APP=app.py

MONGODB_URI=mongodb://localhost:27017
MONGODB_DB=my_database

Use a single, globally shared `MongoClient` (or an appropriately scoped one) instead of creating a new connection per request.

Avoid using multiple database clients within the same application. You should create a single connection and reuse it. MongoDB Drivers manage a connection pool and can be configured to allow more or fewer active connections. When a new connection is required, the driver will use an available connection or establish a new one. This ensures authentication occurs only once, so the time spent resolving the request is dedicated to the actual query rather than to authentication and operation execution. With Flask's built-in extension management, we can instantiate it once and reuse it later.

python
import os
from pymongo import MongoClient
from flask import current_app
from dotenv import load_dotenv

# load environment variables
load_dotenv()

# defining my mongo connection
class Mongo:
    def __init__(self):
        self.client = None
        self.db = None

    def init_app(self, app):
        uri = os.getenv("MONGODB_URI")
        db_name = os.getenv("MONGODB_DB")

        if not uri or not db_name:
            raise RuntimeError("MongoDB environment variables are not set")

        self.client = MongoClient(uri)
        self.db = self.client[db_name]

        # store reference inside Flask app as extension
        app.extensions["mongo"] = self

    @staticmethod
    def get_db():
        mongo = current_app.extensions.get("mongo")
        if not mongo:
            raise RuntimeError("Mongo extension not initialized")
        return mongo.db


mongo = Mongo()

Keep your database access logic, application logic (Flask routes), and configuration separate. Helps with upgrades, testing, and debugging.

By keeping your database access logic, configuration, and business rules separate, you can support each point of view independently without having to search your project to identify the root cause. This is especially useful when you need to understand the application workload and determine which index to use on MongoDB.

The example below shows an example project:

.md
project/
├── .env
├── app.py
├── extensions.py        # Mongo connection
├── routes/
│   └── example.py      # Route logic
├── repositories/
│   └── example_repo.py # Mongo collection queries
└── requirements.txt

extensions.py

In the following file, I am instantiating a MongoDB client. I will instantiate it later in my Flask app by calling the init_app function and passing the Flask instance. This way, whenever my “main” is called, it will create my connection.

python
import os
from pymongo import MongoClient
from flask import current_app
from dotenv import load_dotenv

# load environment variables
load_dotenv()

# defining my mongo connection
class Mongo:
    def __init__(self):
        self.client = None
        self.db = None

    def init_app(self, app):
        uri = os.getenv("MONGODB_URI")
        db_name = os.getenv("MONGODB_DB")

        if not uri or not db_name:
            raise RuntimeError("MongoDB environment variables are not set")

        self.client = MongoClient(uri)
        self.db = self.client[db_name]

        # store reference inside Flask app as extension
        app.extensions["mongo"] = self

    @staticmethod
    def get_db():
        mongo = current_app.extensions.get("mongo")
        if not mongo:
            raise RuntimeError("Mongo extension not initialized")
        return mongo.db


mongo = Mongo()

repositories/example_repo.py

Here, I am simply defining how I interact with my database. If required, I would also define indexes here (not creating them). Having both the indexes and your interactions with that collection in the same folder makes it easier to evaluate new indexes, remove unnecessary ones, or upgrade indexes as new requirements arise.

python
from extensions import Mongo

COLLECTION_NAME = "example"

def get_all():
    db = Mongo.get_db()
    return list(db[COLLECTION_NAME].find({}, {"_id": 0}))

def insert_message(message: str):
    db = Mongo.get_db()
    result = db[COLLECTION_NAME].insert_one({"message": message})
    return str(result.inserted_id)

routes/example.py

Finally, I'm using Flask Blueprints to define my routes. Blueprints are a powerful, built-in mechanism for structuring applications into modular, reusable components rather than placing all code in one file.

python
from flask import Blueprint, jsonify, request
from repositories.example_repo import get_all, insert_message

example_bp = Blueprint("example", __name__, url_prefix="/example")

@example_bp.route("/", methods=["GET"])
def list_messages():
    return jsonify(get_all())

@example_bp.route("/", methods=["POST"])
def create_message():
    payload = request.get_json(silent=True) or {}
    message = payload.get("message")

    if not message:
        return jsonify({"error": "message is required"}), 400

    inserted_id = insert_message(message)
    return jsonify({"inserted_id": inserted_id}), 201

app.py

python
from flask import Flask
from routes.example import example_bp
from extensions import mongo

def create_app():
    app = Flask(__name__)

    # init extensions
    mongo.init_app(app)

    # register routes
    app.register_blueprint(example_bp)

    return app

app = create_app()

if __name__ == "__main__":
    app.run(debug=True)

Data Modeling & Schema Design

Maintain consistent field names and types in the application.

Maintaining consistent field names and data types across the application is critical to avoid subtle bugs, simplify queries, and keep your data model predictable over time. When field names or types change inconsistently (for example, userId versus user_id, or a date stored sometimes as a string and sometimes as a Date), it increases cognitive load, breaks indexes, complicates aggregations, and can cause runtime errors that are hard to trace. Data consistency makes your application easier to maintain, safer to refactor, and more reliable when multiple services, developers, or analytics tools interact with the same data. If you need to have multiple versions of the same document, consider using Document Versioning pattern.

You can use the MongoDB schema validator to define and validate documents inserted into a collection after the schema definition.

javascript
db.createCollection("users", {
  validator: {
    $jsonSchema: {
      bsonType: "object",
      required: ["userId", "email", "isActive", "createdAt"],
      properties: {
        userId: {
          bsonType: "string",
          description: "Must be a string and is required"
        },
        email: {
          bsonType: "string",
          pattern: "^.+@.+$",
          description: "Must be a valid email string."
        },
        isActive: {
          bsonType: "bool",
          description: "Must be a boolean."
        },
        createdAt: {
          bsonType: "date",
          description: "Must be a BSON date."
        },
        optionalField: {
         bsonType: “string”,
         description: “This field is optional.”
       }
      }
    }
  },
  validationLevel: "strict",
  validationAction: "error"
})

Design your document model according to query and access patterns.

Designing your document model based on query and access patterns means shaping your MongoDB documents around how the application actually reads and writes data, not just how the data appears conceptually. By embedding frequently accessed data together and avoiding unnecessary joins, you reduce query complexity, minimize network round-trips, and improve performance. A model optimized for real access patterns is faster, simpler to query, and scales more predictably than one designed only from a relational mindset.

In MongoDB, data that is read together should usually live together. That’s the core idea behind modeling for access patterns.

Model not aligned with access patterns(reference-heavy):

javascript
// orders
{ "_id": 1, "customer_id": 10 }

// order_items
{ "order_id": 1, "product": "Book", "qty": 2 }

// customers
{ "_id": 10, "name": "Alice" }

Query (multiple round trips / $lookup):

db.orders.aggregate([
  { $match: { _id: 1 } },
  { $lookup: {
      from: "order_items",
      localField: "_id",
      foreignField: "order_id",
      as: "items"
  }},
  { $lookup: {
      from: "customers",
      localField: "customer_id",
      foreignField: "_id",
      as: "customer"
  }}
])

Model aligned with access patterns (embedded):

javascript
# orders
{
  "_id": 1,
  "customer": {
    "id": 10,
    "name": "Alice"
  },
  "items": [
    { "product": "Book", "qty": 2 },
    { "product": "Pen", "qty": 1 }
  ]
}

Query (single, simple read):

python
db.orders.findOne({ _id: 1 })

Performance & Query Optimization

Use indexes for fields that you frequently query, filter, or sort by.

Using indexes on frequently queried, filtered, or sorted fields is essential for efficient MongoDB queries. A well-designed compound index can support multiple query patterns when it follows the ESR rule (Equality → Sort → Range), meaning equality filters should come first, followed by sort fields, and then range conditions. By ordering fields correctly, a single index can satisfy multiple query patterns without creating multiple indexes:

“user”
“user” and “age”
“user”, “age”, and “workedHours”

Index creation:

python
db.example.createIndex({ user: 1, age: 1, workedHours: 1 })

Query examples:

python
db.example.find({ user: "joao" })

db.example.find({ user: "joao", age: "28" })
# or
db.example.find({ user: "joao" }).sort({age: 1})

db.example.find({
  user: "joao",
  workedHours: { $gte: 100, $lte: 200 }
}).sort({age: 1})

For potentially large result sets, use pagination (_id skip/limit).

For potentially large result sets, always use pagination to avoid loading too much data into memory and to keep response times predictable. In MongoDB, we can rely on the _id field for pagination over large skip values because it scales efficiently as collections grow and avoids unnecessary data scans. By filtering on _id and combining it with the $limit stage, each page starts where the previous one ended, providing consistent performance and avoiding the cost of scanning and skipping documents. This is possible because ObjectIds are unique and grow monotonically by default.

python
# First page
db.logs.find({})
  .sort({ _id: 1 })
  .limit(10)

# Next page (use last _id from previous page)
db.logs.find({ _id: { $gt: ObjectId("65a123...") } }) // change to $lt if using descending
  .sort({ _id: 1 })
  .limit(10)

Security

Always validate and sanitize user input before inserting it into MongoDB.

Always validate and sanitize user input before inserting it into MongoDB, especially when the input comes from a public endpoint, since HTTP requests should be treated as untrusted by default. Proper validation ensures required fields are present and have the correct data types, while sanitization prevents malformed data, unexpected structures, or NoSQL injection (e.g. $gt, $ne) from reaching the database.

Additionally, if you store sensitive data, such as passwords, never store it in plaintext. Always hash them with an algorithm, such as bcrypt. This protects data integrity, reduces security risks, and keeps your MongoDB collections consistent and secure, even when accessed.

Another protection layer is using Binary subtype 8. This subtype will replace its content in log files for ###. Making sure that even if someone gets access to your MongoDB logs, they will be redacted on that specific field.

python
from flask import request, jsonify
import bcrypt
from bson import Binary

@app.route("/users", methods=["POST"])
def create_user():
    payload = request.get_json(silent=True) or {}

    # Validation
    if not isinstance(payload.get("email"), str):
        return jsonify({"error": "email must be a string"}), 400
    if not isinstance(payload.get("password"), str):
        return jsonify({"error": "password must be a string"}), 400

    salt = bcrypt.gensalt()
    hashed_pwd = bcrypt.hashpw(payload["password"].encode('utf-8'), salt)

    # Sanitization
    user = {
        "email": payload["email"].strip().lower(),
        # Hash password before storing
        "password": Binary(hashed_pwd, 8),
        "is_active": bool(payload.get("is_active", True))
    }

    db.users.insert_one(user)
    return jsonify({"status": "created"}), 201

Conclusion

This article outlines best practices for using MongoDB with Flask. This is not an exhaustive list, but a few common mistakes I've seen after many consulting sessions. Hopefully, this has shed some light and will help you avoid those common mistakes. The best way to learn MongoDB is through its extensive documentation, which is comprehensive and always up to date. If you are unsure or couldn't find anything, there are always the forums and community to help out!

And as promised, here's the Cheat sheet!

Area	Best Practice	Key Action / Code Example
Setup & Structure	Configure connection via `environment variables`.	Use environment variables for credentials.
Setup & Structure	Use a single, globally shared `MongoClient`.	Instantiate `MongoClient` once in an extension and reuse it.
Setup & Structure	Separate logic (routes, repo, config).	Use Flask Blueprints for routes and dedicated repository files for database interactions.
Data Modeling	Maintain consistent field names and types.	Enforce consistency with MongoDB's `$jsonSchema` validator when required.
Data Modeling	Design model for query & access patterns.	Embed data that is read together (e.g., customer info inside the order document).
Performance	Use indexes for frequent query, filter, or sort fields.	Follow the ESR Rule for compound indexes: Equality, Sort, Range.
Performance	Use _id field for pagination.	Use `find({ _id: { $gt: last_id } }).sort({ _id: 1 }).limit(N)` to avoid large `skip` values.
Security	Validate and sanitize all user input.	Check data types and strip/lower strings before insertion.
Security	Never store sensitive data in plaintext.	Always hash passwords before saving them to the database.

Create a workspace scheduler using Bryntum Scheduler Pro and MongoDB

MongoDB Guests — Wed, 01 Apr 2026 11:40:43 +0000

This Tutorial was written by Arsalan Khattak.

Bryntum Scheduler Pro is a scheduling UI component for the web. With features such as a scheduling engine, constraints, and a resource utilization view, it simplifies managing complex schedules.

In this tutorial, we'll use Bryntum Scheduler Pro and MongoDB, the popular document database, to build a workspace booking app for meeting rooms, desk banks, and coworking lounges. We'll use MongoDB Atlas, the fully managed MongoDB cloud service.

We'll do the following:

Set up MongoDB Atlas and get the connection string.
Create an npm workspaces monorepo for the backend and frontend code.
Seed the MongoDB database.
Create the backend server and the Bryntum load and sync endpoints.
Create a Vite vanilla TypeScript client.
Add Bryntum Scheduler Pro to the client.

Here's what we'll build:

You can find the code for the completed tutorial in our GitHub repository: Workspace scheduler using Bryntum Scheduler Pro and MongoDB.

Prerequisites

To follow along, you need Node.js version 20.19+, installed on your system. You'll also need a MongoDB Atlas account, you can register for an Atlas account using your GitHub account, your Google account, or your email address.

Setting Up MongoDB Atlas

We'll use the MongoDB Atlas CLI to create the organization, project, and cluster from the terminal. Using the CLI instead of the Atlas Cloud UI also works well if you're following along with an AI coding agent. It provides a better agent experience.

First, install the MongoDB Atlas CLI. You can install it on macOS or Linux with Homebrew:

brew install mongodb-atlas

atlas auth login

Select UserAccount when the CLI asks how you want to authenticate.

Create a new Atlas organization for this tutorial:

atlas organizations create bryntum --output json

Copy the returned organization ID, then create a new Atlas project:

atlas projects create schedulerpro --orgId <orgId> --output json

Copy the returned project ID, and use it to set the default org_id for Atlas:

atlas config set org_id <orgId

Set the default project_id to your project ID as well:

atlas config set project_id <projectId>

Create an Atlas cluster using the following command:

atlas setup --projectId <projectId> --currentIp --connectWith skip --skipSampleData

Take note of the database username, database password, and connection string.

To find the allowed IP list in your MongoDB Atlas dashboard, go to Security in the left navigation menu and select Database & Network Access. Under Network Access, select IP Access List.

Make sure your IP address is on the allowed list; otherwise, your Express server won't be able to connect to the database. Click the + Add IP Address button to add your IP address.

Setting up the monorepo

Create an empty project folder:

mkdir bryntumschedulerpro-mongodb
cd bryntumschedulerpro-mongodb

Create a package.json file in the project root folder and add the following JSON object:

{
  "name": "bryntumschedulerpro-mongodb",
  "version": "1.0.0",
  "private": true,
  "workspaces": [
    "client",
    "server"
  ],
  "scripts": {
    "dev": "concurrently 'npm run dev --workspace server' 'npm run dev --workspace client'",
    "build": "npm run build --workspace server && npm run build --workspace client",
    "seed": "npm run seed --workspace server",
    "start": "npm run start --workspace server"
  },
  "devDependencies": {
    "@types/node": "22.13.10",
    "concurrently": "9.2.1",
    "nodemon": "3.1.14",
    "typescript": "5.9.3",
    "vite": "7.3.1"
  }
}

This root package defines an npm workspaces TypeScript monorepo with a server and a client. The dev script starts both the Express server and the Vite client together. We'll use the seed script to populate the MongoDB database with example scheduler data.

Install the shared root dependencies:

npm install

Create a tsconfig.base.json file in the root folder:

{
  "compilerOptions": {
    "target": "ES2022",
    "strict": true,
    "skipLibCheck": true,
    "esModuleInterop": true,
    "allowSyntheticDefaultImports": true,
    "forceConsistentCasingInFileNames": true,
    "resolveJsonModule": true
  }
}

This base TypeScript config keeps the shared compiler settings in one place, and both workspaces extend it.

Create a .gitignore file in the root folder and add the following lines to it:

# Logs
logs
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*
pnpm-debug.log*
lerna-debug.log*

# Dependencies
node_modules
client/node_modules
server/node_modules

# Build output
**/dist
.vite

# Local config
*.local
.env

# Editor directories and files
.vscode/*
!.vscode/extensions.json
.idea
.DS_Store
*.suo
*.ntvs*
*.njsproj
*.sln
*.sw?

Create a .env file in the root folder, add the following environmental variables to it:

PORT=1339
MONGODB_URI=mongodb+srv://<dbUsername>:<dbPassword>@<clusterName>.<uniqueId>.mongodb.net/?appName=devrel-tutorial-javascript-bryntum
MONGODB_DB=bryntum-schedulerpro

Add your MongoDB connection string to the MONGODB_URI and add your database username and password. You can get the clusterName and uniqueId by running the following command:

atlas clusters connectionStrings describe <clusterName>

Creating the backend Express Server

Create the server folders:

mkdir -p server/src/lib server/src/routes server/src/scripts

Create a package.json file in the server folder and add the following JSON object to it:

{
  "name": "server",
  "private": true,
  "type": "module",
  "scripts": {
    "dev": "nodemon --watch src --ext ts --exec \"npm run serve\"",
    "serve": "npm run build && node dist/server.js",
    "build": "tsc -p tsconfig.json",
    "start": "node dist/server.js",
    "seed": "npm run build && node dist/scripts/seed.js"
  }
}

This workspace compiles the TypeScript server to dist and reruns the built server with nodemon during development.

Install the server dependencies:

npm install express@4.21.2 dotenv@16.4.7 mongodb@6.15.0 --workspace=server
npm install --save-dev @types/express@5.0.1 --workspace=server

Create a tsconfig.json file in the server folder and add the following configuration object:

{
  "extends": "../tsconfig.base.json",
  "compilerOptions": {
    "module": "NodeNext",
    "moduleResolution": "NodeNext",
    "rootDir": "src",
    "outDir": "dist",
    "lib": ["ES2022"],
    "types": ["node"]
  },
  "include": ["src/**/*.ts"]
}

This config compiles the server as Node.js ES modules. Because of this, the TypeScript source files will use .js import paths that match the compiled output.

Create a loadEnv.ts file in the server/src/lib folder and add the following lines of code to it:

import dotenv from 'dotenv';
import { existsSync } from 'node:fs';
import path from 'node:path';

export function loadEnv(): void {
    const candidates = [
        path.resolve(process.cwd(), '.env'),
        path.resolve(process.cwd(), '../.env')
    ];

    const envPath = candidates.find(existsSync);

    if (envPath) {
        dotenv.config({ path : envPath });
        return;
    }

    dotenv.config();
}

This helper loads environment variables whether we run the server scripts from the monorepo root or from the server workspace.

Now create a mongo.ts file in the server/src/lib folder and add the following code to it:

import { MongoClient, type MongoClientOptions, ServerApiVersion } from 'mongodb';

export function createMongoClient(uri: string): MongoClient {
    const isAtlasUri = uri.startsWith('mongodb+srv://') || uri.includes('.mongodb.net');
    const options: MongoClientOptions = isAtlasUri
        ? {
            serverApi : {
                version           : ServerApiVersion.v1,
                strict            : true,
                deprecationErrors : true
            }
        }
        : {};

    return new MongoClient(uri, options);
}

This helper uses the Atlas Server API options when the connection string points at MongoDB Atlas. Local MongoDB connections keep the default client options.

Create a cors.ts file in the server/src/lib folder and add the following code to it:

import type { RequestHandler } from 'express';

type CorsConfig = {
    origin : string
};

export default function cors(config: CorsConfig): RequestHandler {
    return (req, res, next) => {
        res.setHeader('Access-Control-Allow-Origin', config.origin);
        res.setHeader('Access-Control-Allow-Methods', 'GET,POST,PUT,PATCH,DELETE,OPTIONS');
        res.setHeader('Access-Control-Allow-Headers', 'Content-Type, Authorization');

        if (req.method === 'OPTIONS') {
            res.sendStatus(204);
            return;
        }

        next();
    };
}

This middleware allows the Vite client to call the API directly during local development.

Next, create a schedulerCrud.ts file in the server/src/lib/ folder to define the structure for the load and sync API routes. In this file, add the following imports, collection names, and TypeScript types that describe the request and response shapes:

import { randomUUID } from 'crypto';
import { Collection } from 'mongodb';

export const COLLECTIONS = {
    resources    : 'resources',
    events       : 'events',
    assignments  : 'assignments',
    calendars    : 'calendars',
    dependencies : 'dependencies'
} as const;

export type CrudRequestId = string | number;
export type GenericRecord = { _id?: string; [key: string]: any };
export type StoreKey = keyof typeof COLLECTIONS;
export type IdMap = Map<string, string>;
export type IdMaps = Record<StoreKey, IdMap>;
export type Normalizer = (record: GenericRecord) => GenericRecord;

export type StoreChanges = {
    added?: GenericRecord[];
    updated?: GenericRecord[];
    removed?: Array<{ id?: CrudRequestId }>;
};

export type SyncRequestBody = {
    requestId?: CrudRequestId | null;
    resources?: StoreChanges;
    events?: StoreChanges;
    assignments?: StoreChanges;
    calendars?: StoreChanges;
    dependencies?: StoreChanges;
};

export type StoreResponseSection = {
    rows: GenericRecord[];
    removed?: GenericRecord[];
};

export type SyncResponseBody = {
    success: boolean;
    requestId: CrudRequestId | null;
    resources?: StoreResponseSection;
    events?: StoreResponseSection;
    assignments?: StoreResponseSection;
    calendars?: StoreResponseSection;
    dependencies?: StoreResponseSection;
    message?: string;
};

The collections persist in the Bryntum data stores.

We'll use the Bryntum Crud manager to load the MongoDB data into the Bryntum Scheduler Pro and sync data changes to it. The Crud Manager expects a specific sync request structure and sync response structure, which you can see in the SyncRequestBody and SyncResponseBody types.

Add the following functions to the bottom of the file:

export function createIdMaps(): IdMaps {
    return {
        resources    : new Map(),
        events       : new Map(),
        assignments  : new Map(),
        calendars    : new Map(),
        dependencies : new Map()
    };
}

export function normalizeDate(value: unknown): unknown {
    if (value == null) {
        return value;
    }

    if (value instanceof Date) {
        return value.toISOString();
    }

    return value;
}

export function normalizeEventFields(record: GenericRecord): GenericRecord {
    if (!record) {
        return record;
    }

    const normalized = { ...record };

    if ('startDate' in record) {
        normalized.startDate = normalizeDate(record.startDate);
    }

    if ('endDate' in record) {
        normalized.endDate = normalizeDate(record.endDate);
    }

    return normalized;
}

In this code:

The createIdMaps function builds a phantom-to-permanent ID map for each store.
- When Bryntum creates a new record, it sends a temporary phantom ID that's generated on the client. The sync response then replaces the phantom ID with a permanent ID from the database. When the sync request sends multiple new records at once, one record might reference another that was also just created.
- Consider the example of a new dependency referencing a new event: The server inserts the event first, and maps its phantom ID to the new MongoDB _id. Then, when processing the dependency, resolveId looks up the event's phantom ID in the map to get the database-assigned _id.
The normalizeEventFields function converts Date objects to ISO strings before the server returns them.

Add the following ID resolver and normalizer functions to the bottom of the file:

export function resolveId(id: CrudRequestId, map: IdMap): string {
    const asString = String(id);

    return map.get(asString) || asString;
}

export function normalizeAssignmentFields(record: GenericRecord, idMaps: IdMaps): GenericRecord {
    if (!record) {
        return record;
    }

    const eventId = record.eventId ?? record.event;
    const resourceId = record.resourceId ?? record.resource;

    const normalized = {
        ...record
    };

    if (eventId != null) {
        normalized.eventId = resolveId(eventId, idMaps.events);
    }

    if (resourceId != null) {
        normalized.resourceId = resolveId(resourceId, idMaps.resources);
    }

    delete normalized.event;
    delete normalized.resource;

    return normalized;
}

export function normalizeDependencyFields(record: GenericRecord, idMaps: IdMaps): GenericRecord {
    if (!record) {
        return record;
    }

    const fromEvent = record.fromEvent ?? record.from;
    const toEvent = record.toEvent ?? record.to;

    const normalized = {
        ...record
    };

    if (fromEvent != null) {
        normalized.fromEvent = resolveId(fromEvent, idMaps.events);
    }

    if (toEvent != null) {
        normalized.toEvent = resolveId(toEvent, idMaps.events);
    }

    delete normalized.from;
    delete normalized.to;

    return normalized;
}

These two normalizers map Bryntum's event and resource references to permanent IDs using the ID maps built earlier in the same sync request.

Add the following helper functions that remove MongoDB internal fields:

export function stripInternalFields(record: GenericRecord): GenericRecord {
    if (!record) {
        return record;
    }

    const { _id, ...clean } = record;

    return { ...clean, id : String(_id) };
}

export function collectStoreRows(rows: GenericRecord[]): GenericRecord[] {
    return rows.map(row => stripInternalFields(row));
}

The stripInternalFields function maps MongoDB's _id field to Bryntum's id field before sending data to the client. The collectStoreRows function applies this cleanup to every row in a collection result.

Finally, add the applyStoreChanges() function that handles the add, update, and remove sync requests for every store:

export async function applyStoreChanges({
    collection,
    changes,
    idMap,
    normalizeAdded = value => value,
    normalizeUpdated = value => value
}: {
    collection: Collection<GenericRecord>;
    changes: StoreChanges;
    idMap: IdMap;
    normalizeAdded?: Normalizer;
    normalizeUpdated?: Normalizer;
}): Promise<GenericRecord[]> {
    const rows: GenericRecord[] = [];

    if (Array.isArray(changes.added)) {
        for (const rawRecord of changes.added) {
            const {
                $PhantomId,
                ...data
            } = rawRecord;

            const normalized = normalizeAdded(data);
            const id = String(normalized.id || randomUUID());
            delete normalized.id;
            const recordToStore = {
                ...normalized,
                _id : id
            };

            await collection.insertOne(recordToStore);

            if ($PhantomId) {
                idMap.set(String($PhantomId), id);
                rows.push({ $PhantomId, id });
            }
            else {
                rows.push({ id });
            }
        }
    }

    if (Array.isArray(changes.updated)) {
        for (const rawRecord of changes.updated) {
            const {
                id,
                ...updatedFields
            } = rawRecord;

            if (!id) {
                continue;
            }

            const normalized = normalizeUpdated(updatedFields);

            await collection.updateOne(
                { _id : String(id) },
                { $set : normalized }
            );
        }
    }

    if (Array.isArray(changes.removed) && changes.removed.length) {
        const ids = changes.removed
            .map(record => record.id)
            .filter(Boolean)
            .map(id => String(id));

        if (ids.length) {
            await collection.deleteMany({
                _id : {
                    $in : ids
                }
            });
        }
    }

    return rows;
}

For added records, the function strips the Bryntum $PhantomId, normalizes the data, and maps the phantom to the real ID so that dependent stores can resolve their references in the same sync request.

In MongoDB, every document needs a unique _id field as its primary key. If omitted, the MongoDB driver auto-generates an ObjectId, which consists of a timestamp, machine and process identifier, and counter. However, _id can be any unique value. We use a randomUUID() from Node.js's crypto module instead of ObjectId for consistency with our seed data. The seed script uses human-readable string IDs like 'booking-1' and 'room-terrace' so that cross-references between stores are easy to follow. For example, an assignment's eventId: 'booking-1' pointing to the event with an _id of 'booking-1'.

Creating a database seed script

Now let's create a seed script to populate the MongoDB Atlas database with example data. Create a seed.ts file in the server/src/scripts folder. Add the following imports, database connection setup, and the resources array:

import { loadEnv } from '../lib/loadEnv.js';
import { createMongoClient } from '../lib/mongo.js';

loadEnv();

const MONGODB_URI = process.env.MONGODB_URI || 'mongodb://127.0.0.1:27017';
const MONGODB_DB = process.env.MONGODB_DB || 'bryntum-schedulerpro';

const client = createMongoClient(MONGODB_URI);

type GenericRecord = Record<string, unknown>;

const resources: GenericRecord[] = [
    {
        _id           : 'room-terrace',
        name          : 'Terrace Briefing Room',
        workspaceType : 'Meeting room',
        neighborhood  : 'Docklands',
        floor         : 'Level 2',
        capacity      : 8,
        maxUnits      : 100,
        city          : 'Cape Town',
        amenities     : 'Display wall, VC bar'
    },
    {
        _id           : 'room-harbor',
        name          : 'Harbor Strategy Suite',
        workspaceType : 'Meeting room',
        neighborhood  : 'Harbor Wing',
        floor         : 'Level 5',
        capacity      : 16,
        maxUnits      : 100,
        city          : 'Cape Town',
        amenities     : 'Board table, dual screens'
    },
    {
        _id           : 'room-sprint',
        name          : 'Sprint Garage',
        workspaceType : 'Meeting room',
        neighborhood  : 'Atrium North',
        floor         : 'Level 3',
        capacity      : 6,
        maxUnits      : 100,
        city          : 'Cape Town',
        amenities     : 'Whiteboards, pin-up wall'
    },
    {
        _id           : 'desk-orbit-1',
        name          : 'Orbit Desk Bank A',
        workspaceType : 'Desk bank',
        neighborhood  : 'Docklands',
        floor         : 'Level 4',
        capacity      : 6,
        maxUnits      : 180,
        city          : 'Cape Town',
        amenities     : 'Sit-stand desks, lockers'
    },
    {
        _id           : 'desk-orbit-2',
        name          : 'Orbit Desk Bank B',
        workspaceType : 'Desk bank',
        neighborhood  : 'Docklands',
        floor         : 'Level 4',
        capacity      : 6,
        maxUnits      : 180,
        city          : 'Cape Town',
        amenities     : 'Phone booths, lockers'
    },
    {
        _id           : 'desk-focus',
        name          : 'Focus Pods',
        workspaceType : 'Desk bank',
        neighborhood  : 'Atrium North',
        floor         : 'Level 2',
        capacity      : 4,
        maxUnits      : 140,
        city          : 'Cape Town',
        amenities     : 'Quiet zone, task lights'
    },
    {
        _id           : 'coworking-canvas',
        name          : 'Canvas Coworking Lounge',
        workspaceType : 'Coworking lounge',
        neighborhood  : 'Harbor Wing',
        floor         : 'Ground',
        capacity      : 24,
        maxUnits      : 300,
        city          : 'Cape Town',
        amenities     : 'Cafe seating, soft booths'
    },
    {
        _id           : 'coworking-sky',
        name          : 'Skyline Lounge',
        workspaceType : 'Coworking Lounge',
        neighborhood  : 'Atrium North',
        floor         : 'Rooftop',
        capacity      : 18,
        maxUnits      : 260,
        city          : 'Cape Town',
        amenities     : 'Outdoor tables, power rails'
    }
];

Each resource represents a bookable workspace with a type, location, capacity, and amenities. The maxUnits field sets the resource's percentage capacity. It works together with the assignment units field: if the sum of all assignment units for a resource exceeds its maxUnits, the utilization panel marks that resource as overallocated.

Meeting rooms use 100 by default, so that a single 100% booking fills the room.
Desk banks use 180, so two overlapping bookings at 80% each would still be within capacity.
Coworking lounges use 300, allowing several concurrent partial bookings before hitting the limit.

Now add the events array. Each event represents a booking with a category, host, team, and time window:

const events: GenericRecord[] = [
    {
        _id             : 'booking-1',
        name            : 'Weekly launch review',
        bookingCategory : 'Meeting',
        host            : 'Ava',
        team            : 'Product',
        attendees       : 7,
        notes           : 'Mirror a client-facing property review flow with no overlaps.',
        startDate       : '2026-03-16T08:30:00',
        endDate         : '2026-03-16T11:30:00'
    },
    {
        _id             : 'booking-2',
        name            : 'Quiet desk block',
        bookingCategory : 'Desk',
        host            : 'Jules',
        team            : 'Growth',
        attendees       : 5,
        notes           : 'Focus time for a temporary landing team.',
        startDate       : '2026-03-16T09:00:00',
        endDate         : '2026-03-16T17:00:00'
    },
    {
        _id             : 'booking-3',
        name            : 'Sales deal room',
        bookingCategory : 'Meeting',
        host            : 'Maya',
        team            : 'Sales',
        attendees       : 12,
        notes           : 'Large suite booking for end-of-quarter negotiations.',
        startDate       : '2026-03-16T11:00:00',
        endDate         : '2026-03-16T14:00:00'
    },
    {
        _id             : 'booking-4',
        name            : 'Design sprint desks',
        bookingCategory : 'Desk',
        host            : 'Niko',
        team            : 'Design',
        attendees       : 6,
        notes           : 'Desk bank reserved as one inventory block.',
        startDate       : '2026-03-17T08:00:00',
        endDate         : '2026-03-17T18:00:00'
    },
    {
        _id             : 'booking-5',
        name            : 'Founder coffee session',
        bookingCategory : 'Coworking',
        host            : 'Lebo',
        team            : 'Community',
        attendees       : 14,
        notes           : 'Shared-capacity lounge booking with flexible units.',
        startDate       : '2026-03-17T10:00:00',
        endDate         : '2026-03-17T13:00:00'
    },
    {
        _id             : 'booking-6',
        name            : 'Board prep workshop',
        bookingCategory : 'Meeting',
        host            : 'Dylan',
        team            : 'Operations',
        attendees       : 5,
        notes           : 'Smaller room blocked for a prep workshop.',
        startDate       : '2026-03-17T12:30:00',
        endDate         : '2026-03-17T14:30:00'
    },
    {
        _id             : 'booking-7',
        name            : 'Product all-hands overflow',
        bookingCategory : 'Coworking',
        host            : 'Rae',
        team            : 'Product',
        attendees       : 20,
        notes           : 'Lounge used as shared overflow for team gathering.',
        startDate       : '2026-03-18T09:00:00',
        endDate         : '2026-03-18T12:00:00'
    },
    {
        _id             : 'booking-8',
        name            : 'Engineering hot-desk rotation',
        bookingCategory : 'Desk',
        host            : 'Theo',
        team            : 'Engineering',
        attendees       : 4,
        notes           : 'Desk cluster reserved to mirror a rotating occupancy block.',
        startDate       : '2026-03-18T08:00:00',
        endDate         : '2026-03-18T16:00:00'
    }
];

The bookings span three days and cover all three workspace types.

Finally, add the assignments, calendar, dependencies, and the seed function:

const assignments: GenericRecord[] = [
    { _id : 'assign-1', eventId : 'booking-1', resourceId : 'room-terrace', units : 100 },
    { _id : 'assign-2', eventId : 'booking-2', resourceId : 'desk-orbit-1', units : 80 },
    { _id : 'assign-3', eventId : 'booking-3', resourceId : 'room-harbor', units : 100 },
    { _id : 'assign-4', eventId : 'booking-4', resourceId : 'desk-orbit-2', units : 100 },
    { _id : 'assign-5', eventId : 'booking-5', resourceId : 'coworking-canvas', units : 55 },
    { _id : 'assign-6', eventId : 'booking-6', resourceId : 'room-sprint', units : 100 },
    { _id : 'assign-7', eventId : 'booking-7', resourceId : 'coworking-sky', units : 70 },
    { _id : 'assign-8', eventId : 'booking-8', resourceId : 'desk-focus', units : 65 }
];

const calendars: GenericRecord[] = [
    {
        _id       : 'workspace-hours',
        name      : 'Workspace Hours',
        intervals : [
            {
                recurrentStartDate : 'every weekday at 07:00',
                recurrentEndDate   : 'every weekday at 20:00',
                isWorking          : true
            }
        ]
    }
];

const dependencies: GenericRecord[] = [
    { _id : 'dep-1', fromEvent : 'booking-1', toEvent : 'booking-6' },
    { _id : 'dep-2', fromEvent : 'booking-4', toEvent : 'booking-8' },
    { _id : 'dep-3', fromEvent : 'booking-5', toEvent : 'booking-7' }
];

async function seed(): Promise<void> {
    await client.connect();

    const db = client.db(MONGODB_DB);
    const collectionNames = ['resources', 'events', 'assignments', 'calendars', 'dependencies'];

    await Promise.all(collectionNames.map(name => db.collection(name).deleteMany({})));
    await db.collection('resources').insertMany(resources);
    await db.collection('events').insertMany(events);
    await db.collection('assignments').insertMany(assignments);
    await db.collection('calendars').insertMany(calendars);
    await db.collection('dependencies').insertMany(dependencies);

    console.log(`Seeded ${MONGODB_DB} with workspace scheduler data.`);

    await client.close();
}

void seed().catch(error => {
    console.error(error);
    process.exit(1);
});

Assignments connect each booking to a workspace. The units field sets the utilization percentage for each assignment, which the Bryntum Scheduler Pro utilization widget will display.

The seed script clears all five collections before inserting the example data, so every run resets the database to the same state.

Now seed the database:

npm run seed

You can see the created collections and documents in your MongoDB Atlas dashboard. In the left navigation menu, under Database, select Clusters. Click on the Browse Collections to open the Data Explorer:

Creating the load and sync routes

Now let's create the load and sync routes that will load data into the Bryntum Scheduler Pro and save data changes to the MongoDB Atlas database.

Create a scheduler.ts file in the server/src/routes folder. Add the following imports, router factory, and the /load request route:

import { type Request, type Response, Router } from 'express';
import type { Db } from 'mongodb';
import {
    COLLECTIONS,
    applyStoreChanges,
    collectStoreRows,
    createIdMaps,
    normalizeAssignmentFields,
    normalizeDependencyFields,
    normalizeEventFields,
    type GenericRecord,
    type SyncRequestBody,
    type SyncResponseBody
} from '../lib/schedulerCrud.js';

type GetDb = () => Db;

export function createSchedulerRouter(getDb: GetDb): Router {
    const router = Router();

    router.get('/load', async(_req: Request, res: Response) => {
        try {
            const database = getDb();
            const [resources, events, assignments, calendars, dependencies] = await Promise.all([
                database.collection<GenericRecord>(COLLECTIONS.resources).find({}).sort({ neighborhood : 1, floor : 1, name : 1 }).toArray(),
                database.collection<GenericRecord>(COLLECTIONS.events).find({}).sort({ startDate : 1 }).toArray(),
                database.collection<GenericRecord>(COLLECTIONS.assignments).find({}).toArray(),
                database.collection<GenericRecord>(COLLECTIONS.calendars).find({}).toArray(),
                database.collection<GenericRecord>(COLLECTIONS.dependencies).find({}).toArray()
            ]);

            res.json({
                success      : true,
                resources    : { rows : collectStoreRows(resources) },
                events       : { rows : collectStoreRows(events).map(normalizeEventFields) },
                assignments  : { rows : collectStoreRows(assignments) },
                calendars    : { rows : collectStoreRows(calendars) },
                dependencies : { rows : collectStoreRows(dependencies) }
            });
        }
        catch (error) {
            console.error(error);

            res.status(500).json({
                success : false,
                message : 'Could not load workspace scheduler data'
            });
        }
    });

The /load route queries all five MongoDB collections in parallel and returns them in the structure that Bryntum's ProjectModel expects. Resources are sorted by neighborhood, floor, and name so the grid rows appear in a logical order.

Add the /sync request route below the /load route:

router.post('/sync', async(req: Request, res: Response) => {
        const {
            requestId,
            resources,
            events,
            assignments,
            calendars,
            dependencies
        } = req.body as SyncRequestBody;

        const idMaps = createIdMaps();

        try {
            const database = getDb();
            const response: SyncResponseBody = {
                success   : true,
                requestId : requestId || null
            };

            if (resources) {
                const rows = await applyStoreChanges({
                    collection : database.collection<GenericRecord>(COLLECTIONS.resources),
                    changes    : resources,
                    idMap      : idMaps.resources
                });

                if (rows.length) {
                    response.resources = { rows };
                }
            }

            if (events) {
                const rows = await applyStoreChanges({
                    collection       : database.collection<GenericRecord>(COLLECTIONS.events),
                    changes          : events,
                    idMap            : idMaps.events,
                    normalizeAdded   : normalizeEventFields,
                    normalizeUpdated : normalizeEventFields
                });

                if (rows.length) {
                    response.events = { rows };
                }
            }

            if (assignments) {
                const rows = await applyStoreChanges({
                    collection : database.collection<GenericRecord>(COLLECTIONS.assignments),
                    changes    : assignments,
                    idMap      : idMaps.assignments,
                    normalizeAdded(record) {
                        return normalizeAssignmentFields(record, idMaps);
                    },
                    normalizeUpdated(record) {
                        return normalizeAssignmentFields(record, idMaps);
                    }
                });

                if (rows.length) {
                    response.assignments = { rows };
                }
            }

            if (calendars) {
                const rows = await applyStoreChanges({
                    collection : database.collection<GenericRecord>(COLLECTIONS.calendars),
                    changes    : calendars,
                    idMap      : idMaps.calendars
                });

                if (rows.length) {
                    response.calendars = { rows };
                }
            }

            if (dependencies) {
                const rows = await applyStoreChanges({
                    collection : database.collection<GenericRecord>(COLLECTIONS.dependencies),
                    changes    : dependencies,
                    idMap      : idMaps.dependencies,
                    normalizeAdded(record) {
                        return normalizeDependencyFields(record, idMaps);
                    },
                    normalizeUpdated(record) {
                        return normalizeDependencyFields(record, idMaps);
                    }
                });

                if (rows.length) {
                    response.dependencies = { rows };
                }
            }

            res.json(response);
        }
        catch (error) {
            console.error(error);

            res.status(500).json({
                success   : false,
                requestId : requestId || null,
                message   : 'Could not sync workspace scheduler data'
            } satisfies SyncResponseBody);
        }
    });

    return router;
}

The /sync route processes each store in order: resources and events first, then assignments and dependencies. This order matters because assignments reference event and resource IDs, so those ID maps need to be populated before the assignment normalizer runs. For added rows, the response includes the database-assigned IDs so the Bryntum Scheduler Pro can replace its temporary phantom IDs with the database IDs.

Creating the Express server entry point

Create a server.ts in the server/srcfolder and add the following code to it:

import express from 'express';
import { type Db } from 'mongodb';
import cors from './lib/cors.js';
import { loadEnv } from './lib/loadEnv.js';
import { createMongoClient } from './lib/mongo.js';
import { createSchedulerRouter } from './routes/scheduler.js';

loadEnv();

const PORT = Number(process.env.PORT || 1339);
const FRONTEND_URL = process.env.FRONTEND_URL || 'http://localhost:5173';
const MONGODB_URI = process.env.MONGODB_URI || 'mongodb://127.0.0.1:27017';
const MONGODB_DB = process.env.MONGODB_DB || 'workspace_scheduler2';

const app = express();
app.use(cors({
    origin : FRONTEND_URL
}));
app.use(express.json({ limit : '2mb' }));

const client = createMongoClient(MONGODB_URI);
let db: Db | undefined;

function getDb(): Db {
    if (!db) {
        throw new Error('Database connection is not ready yet');
    }

    return db;
}

app.use('/api', createSchedulerRouter(getDb));

async function start(): Promise<void> {
    await client.connect();
    db = client.db(MONGODB_DB);
    console.log(`MongoDB ready for database "${MONGODB_DB}"`);

    app.listen(PORT, () => {
        console.log(`Workspace Scheduler server running on http://localhost:${PORT}`);
    });
}

start().catch((error) => {
    console.error('Failed to start server:', error);
    process.exit(1);
});

This server loads the environment variables, connects to MongoDB, and mounts the scheduler routes under /api.

Creating the Vite client app

Generate the Vite vanilla TypeScript client app using the following command:

npm create vite@latest client -- --template vanilla-ts

Select Install with npm and start now in the CLI options.

Replace the "scripts" object in the client/package.json with the following scripts:

"scripts": {
  "dev": "vite",
  "build": "tsc -p tsconfig.json && vite build"
}

Replace the configuration in client/tsconfig.json with the following code:
{
  "extends": "../tsconfig.base.json",
  "compilerOptions": {
    "module": "ESNext",
    "moduleResolution": "Bundler",
    "lib": ["ES2022", "DOM", "DOM.Iterable"],
    "types": ["vite/client"],
    "noEmit": true
  },
  "include": ["src/**/*.ts", "vite.config.ts"]
}

This config uses the Vite bundler module resolution strategy and does not emit JavaScript during the TypeScript check.

Create a vite.config.ts file in the client folder and add the following lines of code to it:

import { defineConfig } from 'vite';

export default defineConfig({
    server : {
        port  : 5173,
        proxy : {
            '/api' : 'http://localhost:1339'
        }
    },
    build : {
        outDir : 'dist'
    }
});

This config proxies /api requests to the Express server running on port 1339, so the client can use relative API URLs during local development.

Replace client/index.html with the following code:

<!doctype html>
<html lang="en">
<head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Workspace Scheduler</title>
</head>
<body>
    <main id="app">
        <div id="scheduler" class="scheduler-host"></div>
        <div id="utilization" class="utilization-host"></div>
    </main>
    <script type="module" src="/src/main.ts"></script>
</body>
</html>

This HTML creates two mount points: one for the Scheduler Pro, and one for the linked resource utilization view.

Delete the Vite starter files that we do not use:

rm -r client/public/icons.svg client/src/assets client/src/counter.ts

Installing the Bryntum Scheduler Pro

If you’re using the free trial, install the public Bryntum trial package:

npm install @bryntum/schedulerpro@npm:@bryntum/schedulerpro-trial  --workspace=client

If you have a Bryntum license, refer to our npm Repository Guide and install the licensed package:

npm install @bryntum/schedulerpro --workspace=client

Configuring the Bryntum Scheduler Pro

Create a schedulerProConfig.ts file in the client/src folder. Add the following imports, custom BookingModel, color map, filter dropdown options, and options objects:

import {
    DateHelper,
    EventModel,
    ProjectModel,
    StringHelper,
    type ProjectModelConfig,
    type ResourceUtilizationConfig,
    type SchedulerEventModel,
    type SchedulerProConfig
} from '@bryntum/schedulerpro';

class BookingModel extends EventModel {
    static override fields = [
        { name : 'bookingCategory', type : 'string' },
        { name : 'host', type : 'string' },
        { name : 'team', type : 'string' },
        { name : 'attendees', type : 'number' },
        { name : 'notes', type : 'string' }
    ];
}

const bookingColors: Record<string, string> = {
    Meeting   : '#ea6d4f',
    Desk      : '#267b8a',
    Coworking : '#d69a27'
};

export const workspaceTypeOptions = [
    { value : 'all', text : 'All types' },
    { value : 'Meeting room', text : 'Meeting room' },
    { value : 'Desk bank', text : 'Desk bank' },
    { value : 'Coworking lounge', text : 'Coworking lounge' }
] as const;

export const neighborhoodOptions = [
    { value : 'all', text : 'All neighborhoods' },
    { value : 'Docklands', text : 'Docklands' },
    { value : 'Atrium North', text : 'Atrium North' },
    { value : 'Harbor Wing', text : 'Harbor Wing' }
] as const;

export const filterState = {
    workspaceType : 'all',
    neighborhood  : 'all'
};

The BookingModel adds custom fields to the Bryntum EventModel. The bookingColors map gives each booking category a distinct color on the timeline.

Add the project configuration and instantiation:

export const projectConfig: Partial<ProjectModelConfig> = {
    eventModelClass : BookingModel,
    calendar        : 'workspace-hours',
    transport : {
        load : {
            url : '/api/load'
        },
        sync : {
            url : '/api/sync'
        }
    },
    autoLoad         : true,
    autoSync         : true,
    validateResponse : true
};

export const project = new ProjectModel(projectConfig);

The projectConfig transport property configures the load and sync URLs that the Bryntum Scheduler Pro Crud Manager uses. These URLs are the Express API endpoints that we created. Because autoLoad and autoSync are both enabled, the scheduler fetches data on page load and posts changes back to the API whenever data is modified in the scheduler.

Add the following scheduler configuration:

export const schedulerConfig: Partial<SchedulerProConfig> = {
    appendTo   : 'scheduler',
    project,
    startDate  : '2026-03-16T07:00:00',
    endDate    : '2026-03-18T20:00:00',
    viewPreset : 'hourAndDay',
    rowHeight  : 62,
    barMargin  : 7,
    multiEventSelect : true,
    columns          : [
        {
            type  : 'resourceInfo',
            text  : 'Workspace',
            field : 'name',
            width : 240
        },
        {
            text  : 'Type',
            field : 'workspaceType',
            width : 145
        },
        {
            text  : 'Neighborhood',
            field : 'neighborhood',
            width : 150
        },
        {
            text  : 'Floor',
            field : 'floor',
            width : 90
        },
        {
            text   : 'Capacity',
            field  : 'capacity',
            width  : 100,
            align  : 'center',
            editor : false
        }
    ],
    features : {
        dependencies : true,
        eventMenu    : true,
        taskEdit     : {
            editorConfig : {
                title : 'Reservation',
                width : '36em'
            },
            items : {
                generalTab : {
                    title : 'Booking details',
                    items : {
                        resourcesField : {
                            label : 'Workspace'
                        },
                        bookingCategory : {
                            type     : 'combo',
                            name     : 'bookingCategory',
                            label    : 'Booking type',
                            editable : false,
                            items    : ['Meeting', 'Desk', 'Coworking'],
                            weight   : 210
                        },
                        host : {
                            type   : 'text',
                            name   : 'host',
                            label  : 'Host',
                            weight : 220
                        },
                        team : {
                            type   : 'text',
                            name   : 'team',
                            label  : 'Team',
                            weight : 230
                        },
                        attendees : {
                            type   : 'number',
                            name   : 'attendees',
                            label  : 'Attendees',
                            min    : 1,
                            weight : 240
                        },
                        notes : {
                            type   : 'text',
                            name   : 'notes',
                            label  : 'Notes',
                            weight : 250
                        }
                    }
                }
            }
        },
    },
    eventRenderer({ eventRecord, renderData }) {
        renderData.eventColor = bookingColors[eventRecord.get('bookingCategory')] || '#1d5b52';

        return {
            children : [
                {
                    className : 'booking-title',
                    text      : eventRecord.name
                },
                {
                    className : 'booking-meta',
                    text      : `${eventRecord.get('host') || 'TBD'} | ${eventRecord.get('attendees') || 0} seats`
                }
            ]
        };
    }
};

Here, we configure the timeline settings, task editor, and event renderer. The columns array defines the locked grid on the left side of the scheduler. The taskEdit feature customizes the reservation editor popup with the booking-specific fields. The eventRenderer sets the event bar color based on booking category and shows the host name and seat count inside each bar.

Add the utilization panel configuration at the end of the file:

export const utilizationConfig: Partial<ResourceUtilizationConfig> = {
    appendTo    : 'utilization',
    project,
    hideHeaders : true,
    rowHeight   : 44,
    showBarTip  : true,
    showBarText : true,
    features    : {
        nonWorkingTime : false
    },
    columns : [
        {
            type  : 'tree',
            text  : 'Resource / booking',
            field : 'name',
            flex  : 1
        }
    ]
};

This configures the resource utilization widget that we'll display below the scheduler. It shares the same project instance as the scheduler, so both views stay in sync. The tree column lets you expand each resource to see its individual bookings. The showBarTip and showBarText options display the utilization percentage on the bars so you can spot overallocated workspaces at a glance.

Replace the styles in the client/src/style.css file with the following:

@import url('https://fonts.googleapis.com/css2?family=Poppins:wght@400;500;600;700&display=swap');
@import '@bryntum/schedulerpro/fontawesome/css/fontawesome.css';
@import '@bryntum/schedulerpro/fontawesome/css/solid.css';
@import '@bryntum/schedulerpro/schedulerpro.css';
@import '@bryntum/schedulerpro/svalbard-light.css';

* {
    margin: 0;
}

body {
    font-family: 'Poppins', 'Segoe UI', Arial, sans-serif;
}

#app {
    display        : flex;
    flex-direction : column;
    height         : 100vh;
    font-size      : 14px;
}

#scheduler {
    flex : 3;
}

#utilization {
    flex       : 2;
    border-top : 2px solid #e0e0e0;
}

This stylesheet imports the Bryntum Scheduler Pro structural CSS, the Svalbard theme, and gives the page a two-panel layout, with the scheduler above the utilization view. Bryntum provides five themes with light and dark variants.

Replace the code in lient/src/main.ts with the following:

import {
    Combo,
    ResourceUtilization,
    SchedulerPro,
} from '@bryntum/schedulerpro';
import './style.css';
import {
    filterState,
    neighborhoodOptions,
    project,
    schedulerConfig,
    utilizationConfig,
    workspaceTypeOptions
} from './schedulerProConfig';

const scheduler = new SchedulerPro({
    ...schedulerConfig,
    tbar : [
        {
            type         : 'combo',
            ref          : 'workspaceTypeFilter',
            width        : 190,
            editable     : false,
            value        : filterState.workspaceType,
            valueField   : 'value',
            displayField : 'text',
            items        : workspaceTypeOptions,
            placeholder  : 'Workspace type',
            listeners    : {
                change({ value }: { value: string }) {
                    filterState.workspaceType = value || 'all';
                    applyResourceFilters();
                }
            }
        },
        {
            type         : 'combo',
            ref          : 'neighborhoodFilter',
            width        : 205,
            editable     : false,
            value        : filterState.neighborhood,
            valueField   : 'value',
            displayField : 'text',
            items        : neighborhoodOptions,
            placeholder  : 'Neighborhood',
            listeners    : {
                change({ value }: { value: string }) {
                    filterState.neighborhood = value || 'all';
                    applyResourceFilters();
                }
            }
        },
        {
            type     : 'button',
            icon     : 'b-icon-undo',
            text     : 'Reset filters',
            onAction : () => {
                filterState.workspaceType = 'all';
                filterState.neighborhood = 'all';
                (scheduler.widgetMap.workspaceTypeFilter as Combo).value = 'all';
                (scheduler.widgetMap.neighborhoodFilter as Combo).value = 'all';
                applyResourceFilters();
            }
        },
        {
            type     : 'button',
            icon     : 'b-icon-search-plus',
            text     : 'Zoom in',
            onAction : () => scheduler.zoomIn()
        },
        {
            type     : 'button',
            icon     : 'b-icon-search-minus',
            text     : 'Zoom out',
            onAction : () => scheduler.zoomOut()
        }
    ]
});

This instantiates a Scheduler Pro instance, passes in the schedulerConfig, and adds a toolbar with filter dropdowns and zoom buttons. The two combo dropdowns filter the resource grid by workspace type and neighborhood. The reset button clears both filters, and the zoom buttons let you adjust the timeline scale.

Create the partnered utilization view and the filter function below the scheduler:

new ResourceUtilization({
    ...utilizationConfig,
    partner : scheduler
});

function applyResourceFilters(): void {
    const selectedType = filterState.workspaceType;
    const selectedNeighborhood = filterState.neighborhood;

    scheduler.resourceStore.filter({
        filters : (resource: Record<string, unknown>) => {
            const matchesType = selectedType === 'all' || resource.workspaceType === selectedType;
            const matchesNeighborhood = selectedNeighborhood === 'all' || resource.neighborhood === selectedNeighborhood;

            return matchesType && matchesNeighborhood;
        },
        replace : true
    });
}

project.on({
    load() {
        applyResourceFilters();
    }
});

The partner property links the utilization panel to the scheduler so they share the same time axis and scroll position. The applyResourceFilters function runs a combined filter on the resource store whenever a dropdown changes. The project.on('load') listener reapplies the filters after the initial data load completes.

Running the application

Run the client and server concurrently using the following command:

npm run dev

Open http://localhost:5173/. You’ll see the Bryntum Scheduler Pro populated with the example data:

[VIDEO]

You now have a Bryntum Scheduler Pro workspace booking CRUD app powered by MongoDB Atlas. The linked resource utilization view lets you spot overallocated resources.

Next Steps

Now that you have a working Bryntum Scheduler Pro app with MongoDB, explore the additional features you can add on the examples page, such as:

Check out the Bryntum AI feature that lets users interact with scheduling data using natural language through a chat panel.

Both Bryntum and MongoDB have MCP servers to assist with agentic workflows:

The Bryntum MCP server returns API references, configuration options, working source code examples, and migration guides for specific Bryntum versions.
The MongoDB MCP server helps with code generation, database optimization, and data exploration.

Forem: MongoDB Guests

Whistleblower for Database: Set up an internal informant who exposes performance

Whistleblower 1: Performance Informant

High CPU utilization?

Query <5 seconds?

Cache hit ratio <80%?

Whistleblower 2: Cost Informant

Storage >80%:

IOPS > Budget?

Unexpected auto scale?

Whistleblower 3: Reliability Informant

Replication lag >10 Seconds?

Oplog <24Hours?

Connection >80%?

Vacuum Seal Your MongoDB: Cuts Cost Down 50% Today

Online Archive

Table Compact

Resync Secondary:

TTL Index:

Query Optimization:

Flex Cluster:

Conclusion:

My first “local” Vector Search: MongoDB community edition

How did I test this?

How schema anti-patterns in MongoDB can cost you $$$$

Massive boundless array

Timeseries saved time

When multitenant became the villain

$lookup is designed badly

How MongoDB Executes a find() Query: A Complete Lifecycle Guide

The full journey of a find() query (quick overview)

Stage 1: The Client Sends a Query Over the Wire

Stage 2: Authentication Check

Stage 3: Authorization — Can You Read This Collection?

Stage 4: Query Parsing and BSON Validation

Stage 5: The Query Planner Enumerates Candidate Plans

Stage 6: Plan Cache Lookup — The Fast Path

Stage 7: Multi-Plan Trial — The Index Race

What about full collection scans (COLLSCAN)?

Stage 8: Index Scan (IXSCAN) or Collection Scan (COLLSCAN)

If an index is used (IXSCAN)

If no index is used (COLLSCAN)

How to spot a problem

Stage 9: The Storage Layer — WiredTiger Cache and Disk

Checking cache performance

Stage 10: Results Returned to the Client

What if there’s more data?

The Complete Picture — All 10 Stages

Closing notes: Key Takeaways for Production

Introduction to Events using MongoDB

Prerequisites

Step 1: Create the entities

Step 2: Create the events

Step 3: Generate the listeners

Step 4: Generating the rest endpoint

Conclusion

References:

Detective mode: Fixing the MongoDB Aggregation pipeline

What's the Mystery?

The Investigation: Agent Darshan reporting

Evidence #1:

Evidence #2:

Suspects list

Suspect #1:

Suspect #2:

Suspect #3:

Suspect #4:

Recapping all mistakes to fix!

The Solution:

✅ Issue resolved

The Costly Lessons Learned:

FastAPI With LangChain and MongoDB

Prerequisites

Project Architecture

Project Setup

MongoDB Atlas

Docker

LLMs

Ollama installation

Development

The full journey of a `find()` query (quick overview)

Overview of PyMongo’s new async API (`pymongo.async_`)