Database NotesRohit’s interview study guide
/
Chapter 24

MongoDB from Node.js

Using MongoDB from a Node app: the official driver and its connection pool, Mongoose schemas, validation, middleware, populate and lean, and transactions with automatic retries, all run against MongoDB 9.0 with mongodb 7.7 and Mongoose 9.11.

6 topics
Parts marked Advanced are extra depth. Skip them on a first read or a quick revision.

#The official driver

Added

The mongodb package is MongoDB's official Node.js driver, and everything else (Mongoose, Prisma's MongoDB support, most frameworks) is built on it. Its API mirrors mongosh closely: db.collection("products").find(filter) looks almost the same, with two differences that trip people up. Every call is asynchronous and returns a promise, and find() returns a cursor, not an array; nothing is fetched until you iterate it or call toArray().

The other thing to understand is that a MongoClient is not a connection, it's a connection pool plus a monitor. It discovers the deployment (standalone, replica set members, mongos routers), keeps up to maxPoolSize connections (100 by default) per server, routes each operation to the right member, and follows the primary through failovers. The mental model: the client is a taxi company, not a taxi. You create one per process at startup and every request borrows a car from its fleet.

The examples in this chapter run against the sandbox's single-member replica set on 127.0.0.1:27018, with directConnection=true because the member advertises itself as localhost:27017 inside its container.

d01-driver.mjs
import { MongoClient } from "mongodb";

const client = new MongoClient("mongodb://127.0.0.1:27018/?directConnection=true");

async function main() {
  await client.connect();
  const products = client.db("ch22").collection("products");
  await products.deleteMany({});

  const ins = await products.insertMany([
    { _id: 1, name: "ThinkBook 14", category: "Laptops", price: 899, stock: 12 },
    { _id: 2, name: "MacBook Air", category: "Laptops", price: 1299, stock: 5 },
    { _id: 3, name: "Pixel 10", category: "Phones", price: 799, stock: 20 },
    { _id: 4, name: "iPhone 17", category: "Phones", price: 1099, stock: 0 },
  ]);
  console.log("inserted:", ins.insertedCount);

  const inStock = await products
    .find({ stock: { $gt: 0 } }, { projection: { _id: 0, name: 1, price: 1 } })
    .sort({ price: -1 })
    .toArray();
  console.log(inStock);

  const upd = await products.updateOne({ _id: 3 }, { $inc: { stock: -1 } });
  console.log({ matched: upd.matchedCount, modified: upd.modifiedCount });

  console.log(await products.findOne({ _id: 99 }));
}

main()
  .catch(console.error)
  .finally(() => client.close());
Result
inserted: 4
[
  { name: 'MacBook Air', price: 1299 },
  { name: 'ThinkBook 14', price: 899 },
  { name: 'Pixel 10', price: 799 }
]
{ matched: 1, modified: 1 }
null
What’s happening
  1. new MongoClient(uri) only stores the settings. await client.connect() contacts the server, runs the handshake and starts monitoring. (Since driver 4.7 the first operation connects automatically too, but an explicit connect() makes startup failures surface at startup.)
  2. client.db("ch22").collection("products") creates lightweight handles; no network traffic. insertMany sends the four documents in one command and resolves to { insertedCount: 4, insertedIds }.
  3. find(filter, { projection }) builds a cursor. .sort({ price: -1 }) adds to it. Only .toArray() sends the query and collects every batch into an array: the three products with stock > 0, most expensive first, with just name and price. In the driver the projection goes inside an options object, unlike mongosh's second argument.
  4. updateOne resolves to a result object; matchedCount and modifiedCount are how you tell "not found" from "found but already in that state".
  5. findOne for a missing _id resolves to null, not an error. Always handle the null.
  6. client.close() in finally closes the pool so the process can exit. In a web server you'd close it only on shutdown.
AdvancedErrors and ObjectId pitfalls
d02-errors.mjs
import { MongoClient, MongoServerError, ObjectId } from "mongodb";

const client = new MongoClient("mongodb://127.0.0.1:27018/?directConnection=true");
const products = client.db("ch22").collection("products");

try {
  await products.insertOne({ _id: 1, name: "Duplicate" });
} catch (err) {
  if (err instanceof MongoServerError && err.code === 11000) {
    console.log("duplicate:", err.keyValue, "-", err.errmsg);
  } else throw err;
}

// No _id given: the driver generates an ObjectId before sending
const { insertedId } = await products.insertOne({ name: "USB-C Charger", price: 29 });
console.log(insertedId instanceof ObjectId, insertedId.getTimestamp().toISOString().slice(0, 10));

// A string is not an ObjectId: this finds nothing
console.log(await products.findOne({ _id: insertedId.toHexString() }));
console.log((await products.findOne({ _id: new ObjectId(insertedId.toHexString()) })).name);

await client.close();
Result
duplicate: { _id: 1 } - E11000 duplicate key error collection: ch22.products index: _id_ dup key: { _id: 1 }
true 2026-10-16
null
USB-C Charger
What’s happening
  1. Inserting _id: 1 again throws a MongoServerError with code: 11000. err.keyValue tells you which value collided. Check code, never parse the message.
  2. Without an _id, the driver (not the server) creates an ObjectId and adds it to the document before sending, which is why insertedId is known even before the round trip. Its first 4 bytes are a timestamp: getTimestamp() gives today's date.
  3. The classic bug: an id arrives from a URL as a string, "6ad2…", and findOne({ _id: thatString }) returns null, because a string never equals an ObjectId in BSON.
  4. Converting with new ObjectId(hex) finds the document. Validate first with ObjectId.isValid(hex), since the constructor throws on malformed input.
AdvancedProved: one client, or a client per request

The pool only helps if it's shared. One client with maxPoolSize: 10 serves 200 parallel queries; then 50 "requests" each create their own client:

d03-pool.mjs
import { MongoClient } from "mongodb";
const uri = "mongodb://127.0.0.1:27018/?directConnection=true";
const serverConnections = async () => {
  const c = await MongoClient.connect(uri);
  const n = (await c.db("admin").command({ serverStatus: 1 })).connections.current;
  await c.close();
  return n - 1; // minus this checking connection
};

// Good: one client for the whole app, shared by every request
const shared = new MongoClient(uri, { maxPoolSize: 10 });
let created = 0;
shared.on("connectionCreated", () => created++);
const products = shared.db("ch22").collection("products");
await Promise.all(Array.from({ length: 200 }, () => products.find({ price: { $gt: 100 } }).toArray()));
console.log(`one client, 200 parallel queries: ${created} pooled connections, server sees ${await serverConnections()}`);
await shared.close();

// Bad: a new client per request
const perRequest = await Promise.all(Array.from({ length: 50 }, async () => {
  const c = await MongoClient.connect(uri);
  await c.db("ch22").collection("products").findOne({});
  return c;
}));
console.log(`a client per request, 50 requests: server sees ${await serverConnections()} connections`);
await Promise.all(perRequest.map((c) => c.close()));
Result
one client, 200 parallel queries: 10 pooled connections, server sees 13
a client per request, 50 requests: server sees 102 connections
What’s happening
  1. 200 queries start at once on the shared client. The pool opens connections up to maxPoolSize (the connectionCreated event fired 10 times), and the other 190 queries wait in the pool's queue for a free connection rather than opening new ones.
  2. The server counts 13: the 10 pooled connections plus the client's monitoring connections (and anything else connected to this shared sandbox at that moment).
  3. The anti-pattern: each request creates a MongoClient. Every client has its own pool and its own monitoring connections, so 50 requests produced about 100 server connections, each with a full handshake (and in production a TLS handshake and SCRAM authentication) before doing any work.
  4. Under load that pattern exhausts the server's connection limit and makes every request slower. Create the client once at startup, export it, and reuse it. In serverless functions, create it outside the handler so warm invocations reuse it.
AdvancedCursors and bulkWrite
d04-cursor-bulk.mjs
import { MongoClient } from "mongodb";
const client = new MongoClient("mongodb://127.0.0.1:27018/?directConnection=true");
let getMores = 0;
const monitored = new MongoClient("mongodb://127.0.0.1:27018/?directConnection=true", { monitorCommands: true });
monitored.on("commandStarted", (e) => { if (e.commandName === "getMore") getMores++; });

const events = client.db("ch22").collection("events");
await events.drop().catch(() => {});
await events.insertMany(Array.from({ length: 1000 }, (_, i) => ({ n: i, type: i % 3 ? "view" : "click" })));

// Stream a large result instead of loading it all with toArray()
let clicks = 0;
for await (const doc of monitored.db("ch22").collection("events").find({}, { batchSize: 100 })) {
  if (doc.type === "click") clicks++;
}
console.log({ clicks, getMores });

// Many different writes in one round trip
const res = await events.bulkWrite([
  { insertOne: { document: { n: 1000, type: "click" } } },
  { updateMany: { filter: { type: "view", n: { $lt: 10 } }, update: { $set: { type: "seen" } } } },
  { deleteOne: { filter: { n: 0 } } },
  { replaceOne: { filter: { n: 1 }, replacement: { n: 1, type: "replaced" } } },
], { ordered: true });
console.log({ inserted: res.insertedCount, matched: res.matchedCount, modified: res.modifiedCount, deleted: res.deletedCount });
await client.close(); await monitored.close();
Result
{ clicks: 334, getMores: 10 }
{ inserted: 1, matched: 7, modified: 7, deleted: 1 }
What’s happening
  1. for await … of cursor pulls documents batch by batch. The find command returns the first 100; each getMore fetches the next 100. That was 10 getMore commands for 1,000 documents (the last one finds the cursor exhausted), with only one batch in memory at a time.
  2. Counting 334 clicks (every third n, 0 to 999) this way works for a million documents too, where toArray() would hold them all in memory at once.
  3. bulkWrite sends four different operations in one round trip. The updateMany matched 6 views with n < 10 (1, 2, 4, 5, 7, 8), and the replaceOne matched 1: 7 matched and modified in total.
  4. ordered: true (the default) stops at the first error; ordered: false carries on and lets the server run the operations in any order, which is faster for big independent batches.
AdvancedConnection string options you'll meet
  • mongodb+srv://cluster0.example.mongodb.net/ (Atlas) looks up the hosts in DNS SRV records and turns on TLS.
  • replicaSet=rs0: the driver discovers all members from any one of them. directConnection=true talks to exactly one host and skips discovery, as in this sandbox.
  • maxPoolSize (default 100), minPoolSize, maxIdleTimeMS, waitQueueTimeoutMS: pool behaviour.
  • retryWrites=true and retryReads=true are both on by default in current drivers: one automatic retry after a network error or failover.
  • w=majority, readPreference=secondaryPreferred, readConcernLevel=majority: defaults for the client (see Write concern and Read concern and read preference).
  • timeoutMS: one overall time limit per operation, including retries and pool waits (client-side operation timeout in recent drivers).
  • appName=checkout-service: shows up in the server's slow query log and $currentOp, so you can tell which service sent what.

#Mongoose schemas and models

Added

MongoDB doesn't force documents in a collection to share a shape. Most apps want one anyway, so Mongoose adds it in the application: an ODM (object-document mapper) where you declare a schema (fields, types, defaults, validation), compile it into a model bound to a collection, and get back documents: objects with methods like save(), change tracking and virtual properties.

The mental model is a form with labelled boxes. The schema is the printed form; the model is the stack of forms for one collection; a document is one filled-in form. Mongoose checks the form on the way in, casting "799" into the number box and refusing what doesn't fit, before the driver ever sends it.

m01-schema.mjs
import mongoose from "mongoose";

const productSchema = new mongoose.Schema(
  {
    name: { type: String, required: true, trim: true },
    category: { type: String, enum: ["Laptops", "Phones", "Books", "Accessories"], required: true },
    price: { type: Number, required: true, min: 0 },
    stock: { type: Number, default: 0 },
    tags: [String],
  },
  { timestamps: true }
);
productSchema.virtual("inStock").get(function () { return this.stock > 0; });
productSchema.methods.discounted = function (pct) { return Math.round(this.price * (100 - pct)) / 100; };
productSchema.statics.byCategory = function (category) { return this.find({ category }).sort({ price: 1 }); };

const Product = mongoose.model("Product", productSchema);   // collection: "products"

await mongoose.connect("mongodb://127.0.0.1:27018/ch22?directConnection=true");
await Product.deleteMany({});

const p = await Product.create({ name: "  Pixel 10 ", category: "Phones", price: "799", colour: "Obsidian" });
console.log(p.name, typeof p.price, p.stock, p.inStock, p.discounted(10));
console.log(Product.collection.collectionName);

const raw = await mongoose.connection.db.collection("products").findOne({ _id: p._id });
console.log(raw);

await Product.create({ name: "iPhone 17", category: "Phones", price: 1099, stock: 3 });
console.log((await Product.byCategory("Phones")).map((d) => `${d.name} ${d.price}`));
await mongoose.disconnect();
Result
Pixel 10 number 0 false 719.1
products
{
  _id: new ObjectId('6ad24b985a59182a01e6ba03'),
  name: 'Pixel 10',
  category: 'Phones',
  price: 799,
  stock: 0,
  tags: [],
  createdAt: 2026-10-16T16:06:48.048Z,
  updatedAt: 2026-10-16T16:06:48.048Z,
  __v: 0
}
[ 'Pixel 10 799', 'iPhone 17 1099' ]
What’s happening
  1. The schema declares five fields. trim and default transform values, required, enum and min validate them, and { timestamps: true } adds createdAt and updatedAt that Mongoose maintains itself.
  2. mongoose.model("Product", schema) compiles the model and picks the collection name by lower-casing and pluralising the model name: products. Pass a third argument to choose it yourself.
  3. Product.create(...) builds a document and saves it. On the way: the name is trimmed ("Pixel 10"), the string "799" is cast to the number 799 (typeof is number), stock gets its default 0, and colour is silently dropped because it isn't in the schema (the default strict mode).
  4. inStock is a virtual, computed on access and never stored (false, because stock is 0). discounted(10) is an instance method: 799 × 0.9 = 719.1. byCategory is a static on the model.
  5. The raw driver read shows what's really stored: no colour, no inStock, but tags: [], the timestamps and __v: 0, the version key Mongoose uses to detect conflicting array changes.
  6. Mongoose's rules live only in the app. A different program (or mongosh) can still write anything to products; for rules the database enforces, add Schema validation on the collection as well.
AdvancedEdge case: syncIndexes on MongoDB 9.0

Schemas also declare indexes (index: true, unique: true, schema.index({ … })), and Mongoose creates them at startup unless you set autoIndex: false, which is the usual production setting, so index builds happen in deployments instead. Model.syncIndexes() goes further and drops indexes that aren't in the schema. MongoDB 9.0's release notes list a known issue: at feature compatibility version 9.0, listIndexes reports collation: { locale: "simple" } on simple-collation indexes, so ODMs comparing definitions may drop and rebuild unchanged indexes on every start. Mongoose 9.10.4 and 8.24.5 contain the fix (this sandbox has 9.11.2); on older versions, avoid syncIndexes() at startup or preview with diffIndexes().

AdvancedMongoose versions: what changed recently
  • Mongoose 7 removed callbacks entirely (everything is promises) and made strictQuery default to false.
  • Mongoose 8 moved to driver 6 and removed findOneAndRemove and friends.
  • Mongoose 9 (used here, 9.11) removed next() from pre middleware (see Validation and middleware), and 9.11 prints a deprecation warning for the new: true option of findOneAndUpdate: use returnDocument: "after", the driver's own option.
  • Interview and older tutorial code often still shows callbacks, next() in hooks and { new: true }; recognise them as the older style.

#Validation and middleware

Added

Mongoose validation runs in the app before a write is sent: built-in validators (required, min/max, enum, match, minLength) and custom functions. If any fail, the write never reaches MongoDB and you get one ValidationError listing every failing field, ready to turn into a 400 response.

Middleware (hooks) are functions that run before or after an operation: pre("save") to compute a total or hash a password, post("save") to log or publish an event, pre("find") to add a filter to every query. The mental model is airport security: validators are the scanner every bag goes through; middleware are staff who act before or after the flight. Some side doors, like updateOne, skip parts of the check unless you ask for it.

m02-validate.mjs
import mongoose from "mongoose";

const customerSchema = new mongoose.Schema({
  name: { type: String, required: [true, "a customer needs a name"] },
  email: {
    type: String,
    required: true,
    lowercase: true,
    validate: { validator: (v) => /^[^@\s]+@[^@\s]+\.[a-z]+$/.test(v), message: (p) => `${p.value} is not an email address` },
  },
  city: String,
  age: { type: Number, min: [18, "must be an adult, got {VALUE}"] },
});
const Customer = mongoose.model("Customer", customerSchema);
await mongoose.connect("mongodb://127.0.0.1:27018/ch22?directConnection=true");
await Customer.deleteMany({});

try {
  await Customer.create({ email: "ROHIT(at)example", age: 15 });
} catch (err) {
  console.log(err.name);
  console.log(err.message);
  console.log(Object.keys(err.errors));
}

const c = await Customer.create({ name: "Rohit", email: "Rohit@Example.com", age: 30 });
console.log(c.email);

// Update validators are OFF by default for update queries
await Customer.updateOne({ _id: c._id }, { $set: { age: 12 } });
console.log((await Customer.findById(c._id)).age);
try {
  await Customer.updateOne({ _id: c._id }, { $set: { age: 10 } }, { runValidators: true });
} catch (err) {
  console.log(err.message);
}
await mongoose.disconnect();
Result
ValidationError
Customer validation failed: name: a customer needs a name, email: rohit(at)example is not an email address, age: must be an adult, got 15
[ 'name', 'email', 'age' ]
rohit@example.com
12
Validation failed: age: must be an adult, got 10
What’s happening
  1. The first create breaks three rules at once. Mongoose runs all validators and rejects with a single ValidationError whose errors object has one entry per field: name, email, age. Nothing was sent to MongoDB.
  2. The messages come from the schema: a custom required message, a function message that sees the value (already lower-cased by the lowercase setter, so rohit(at)example), and {VALUE} interpolated into the min message.
  3. The valid customer saves, with the email normalised to rohit@example.com by lowercase.
  4. The trap: updateOne with $set: { age: 12 } succeeds and stores 12, because validators run on save()/create(), not on update queries, by default.
  5. With { runValidators: true } the update is validated (against the fields in the update) and rejected: Validation failed: age: must be an adult, got 10. Turn it on globally with mongoose.set("runValidators", true), or the database-side $jsonSchema validator for rules that must hold whatever the client.
AdvancedDocument and query middleware
m03-middleware.mjs
import mongoose from "mongoose";

const orderSchema = new mongoose.Schema({
  customer: String,
  items: [{ product: String, qty: Number, price: Number }],
  total: Number,
  deletedAt: { type: Date, default: null },
});

// Document middleware: runs on save() and create(), with `this` = the document
orderSchema.pre("save", function () {
  this.total = this.items.reduce((sum, i) => sum + i.qty * i.price, 0);
  console.log(`pre save: total = ${this.total}`);
});
orderSchema.post("save", function (doc) {
  console.log(`post save: order ${doc._id.toString().slice(-4)} stored`);
});

// Query middleware: runs on find/findOne/... with `this` = the query
orderSchema.pre(/^find/, function () {
  if (this.getOptions().withDeleted) return;
  this.where({ deletedAt: null });
});

const Order = mongoose.model("Order", orderSchema);
await mongoose.connect("mongodb://127.0.0.1:27018/ch22?directConnection=true");
await Order.deleteMany({});

const o = await Order.create({ customer: "Rohit", items: [{ product: "ThinkBook 14", qty: 1, price: 899 }, { product: "USB-C Charger", qty: 2, price: 29 }] });
await Order.create({ customer: "Maya", items: [{ product: "Pixel 10", qty: 1, price: 799 }], deletedAt: new Date() });

console.log((await Order.find()).map((d) => d.customer));
console.log((await Order.find().setOptions({ withDeleted: true })).map((d) => d.customer));

// updateOne is a query: the save hooks do NOT run, so total goes stale
await Order.updateOne({ _id: o._id }, { $push: { items: { product: "Kindle", qty: 1, price: 159 } } });
const after = await Order.findById(o._id);
console.log(after.items.length, after.total);
await mongoose.disconnect();
Result
pre save: total = 957
post save: order 1bb7 stored
pre save: total = 799
post save: order 1bba stored
[ 'Rohit' ]
[ 'Rohit', 'Maya' ]
3 957
What’s happening
  1. Each create runs the pre("save") hook with this as the document: it computes total from the items (899 + 2 × 29 = 957, then 799) before the insert. post("save") runs after the server confirmed it.
  2. pre(/^find/) is query middleware: it runs for find, findOne, findById and every other method whose name starts with find, with this as the query. It adds deletedAt: null, so soft-deleted Maya disappears from normal queries.
  3. A custom query option (withDeleted) lets the hook be bypassed for admin screens: both orders come back.
  4. updateOne is a query too, so it never runs the save hooks. The Kindle is pushed (3 items) but total stays 957. Logic in save hooks only protects writes that go through save(); for updates, add pre("updateOne") middleware or compute in the update itself.
  5. Note that findById returned the order: the soft-delete filter applied, and Rohit's order isn't deleted.
AdvancedMongoose 9: next() is gone from pre hooks

Older code (Mongoose 8 and earlier) wrote pre hooks with a next callback. In Mongoose 9 the hook's first argument is no longer next:

m04-next.mjs
import mongoose from "mongoose";
const s = new mongoose.Schema({ name: String });
// Mongoose 8 and earlier style: a callback-style hook that calls next()
s.pre("save", function (next) {
  this.name = this.name.trim();
  next();
});
const Thing = mongoose.model("Thing", s);
await mongoose.connect("mongodb://127.0.0.1:27018/ch22?directConnection=true");
try {
  await Thing.create({ name: "  hello " });
} catch (err) {
  console.log(err.name + ": " + err.message);
}
await mongoose.disconnect();
Result
TypeError: next is not a function
What’s happening
  1. The hook declares a parameter called next and calls it at the end, which was the standard pattern for years.
  2. Mongoose 9's type definitions show the new signature: PreMiddlewareFunction = (this, opts?) => void | Promise<void>. The first argument is an options object, not a callback.
  3. Calling it throws TypeError: next is not a function, which rejects the create. The document is never saved.
  4. The fix is to drop next: a synchronous hook just returns, an async hook returns its promise, and to fail you throw an error. That's the style used in the previous example.

#populate

Added

MongoDB's schema advice is to embed what you read together, but some data must live in its own collection and be referenced: customers referenced by many orders, products referenced by order items. populate() is Mongoose's way to follow those references. You store the other document's _id with a ref to its model, and populate("customer") replaces the id with the full document after the query.

The mental model: an order slip with a customer number on it, and a clerk who, before handing you the slips, looks every number up in the customer file and staples the customer's card on. The important detail is how the clerk looks them up: not one trip per slip, but one trip per field for all the slips at once.

m05-populate.mjs
import mongoose from "mongoose";
const { Schema, model } = mongoose;

const Customer = model("PopCustomer", new Schema({ _id: Number, name: String, city: String }), "pop_customers");
const Product = model("PopProduct", new Schema({ _id: Number, name: String, price: Number }), "pop_products");
const Order = model("PopOrder", new Schema({
  _id: Number,
  customer: { type: Number, ref: "PopCustomer" },
  items: [{ product: { type: Number, ref: "PopProduct" }, qty: Number }],
  status: String,
}), "pop_orders");

await mongoose.connect("mongodb://127.0.0.1:27018/ch22?directConnection=true");
await Promise.all([Customer, Product, Order].map((M) => M.deleteMany({})));
await Customer.insertMany([{ _id: 1, name: "Rohit", city: "Toronto" }, { _id: 2, name: "Maya", city: "Mumbai" }]);
await Product.insertMany([{ _id: 1, name: "ThinkBook 14", price: 899 }, { _id: 7, name: "USB-C Charger", price: 29 }, { _id: 3, name: "Pixel 10", price: 799 }]);
await Order.insertMany([
  { _id: 1, customer: 1, status: "shipped", items: [{ product: 1, qty: 1 }, { product: 7, qty: 2 }] },
  { _id: 3, customer: 2, status: "shipped", items: [{ product: 3, qty: 1 }] },
]);

mongoose.set("debug", (coll, method, query) => console.log(`  -> ${coll}.${method}(${JSON.stringify(query)})`));
const orders = await Order.find({ status: "shipped" })
  .populate("customer", "name city")
  .populate("items.product", "name price");
mongoose.set("debug", false);

for (const o of orders) {
  console.log(o._id, o.customer.name, o.items.map((i) => `${i.qty} x ${i.product.name} @ ${i.product.price}`));
}
await mongoose.disconnect();
Result
  -> pop_orders.find({"status":"shipped"})
  -> pop_customers.find({"_id":{"$in":[1,2]}})
  -> pop_products.find({"_id":{"$in":[1,7,3]}})
1 Rohit [ '1 x ThinkBook 14 @ 899', '2 x USB-C Charger @ 29' ]
3 Maya [ '1 x Pixel 10 @ 799' ]
What’s happening
  1. The schemas use the sample shop's numeric ids. customer: { type: Number, ref: "PopCustomer" } says "this number is the _id of a PopCustomer", and the same for each item's product.
  2. mongoose.set("debug", fn) logs every command Mongoose sends, which shows how populate works. First the orders query itself: pop_orders.find({ status: "shipped" }), 2 orders.
  3. Mongoose collects the distinct customer ids from those orders, [1, 2], and runs one find with $in. It does the same for every items.product: one query for [1, 7, 3]. Three queries in total, however many orders there are.
  4. It then replaces each id with the matching document, limited to the selected fields ("name city", "name price"), so o.customer.name and i.product.price just work.
  5. That's how populate avoids the N+1 problem (see The N+1 query problem): it batches by field, not per document. It's still several round trips done by the app, not a server-side join.
AdvancedDangling references and the $lookup equivalent

References aren't foreign keys: nothing stops the referenced document from disappearing.

m06-populate-more.mjs
import mongoose from "mongoose";
const { Schema, model } = mongoose;
const Customer = model("PopCustomer", new Schema({ _id: Number, name: String, city: String }), "pop_customers");
const Order = model("PopOrder", new Schema({ _id: Number, customer: { type: Number, ref: "PopCustomer" }, status: String }, { strict: false }), "pop_orders");
await mongoose.connect("mongodb://127.0.0.1:27018/ch22?directConnection=true");

// A dangling reference: customer 9 doesn't exist
await Order.deleteOne({ _id: 9 });
await Order.create({ _id: 9, customer: 9, status: "paid" });
const o = await Order.findById(9).populate("customer");
console.log(o.customer, o.populated("customer"));

// The same join done by the server, in one round trip, with $lookup
const rows = await Order.aggregate([
  { $match: { status: "shipped" } },
  { $lookup: { from: "pop_customers", localField: "customer", foreignField: "_id", as: "customer" } },
  { $unwind: "$customer" },
  { $project: { _id: 1, name: "$customer.name", city: "$customer.city" } },
]);
console.log(rows);
await mongoose.disconnect();
Result
null 9
[
  { _id: 1, name: 'Rohit', city: 'Toronto' },
  { _id: 3, name: 'Maya', city: 'Mumbai' }
]
What’s happening
  1. Order 9 references customer 9, which doesn't exist. MongoDB accepted it: there's no referential integrity between collections.
  2. populate finds no match and sets o.customer to null. Code that does o.customer.name crashes here, so treat populated fields as possibly null. o.populated("customer") still returns the original id, 9.
  3. The aggregation does the same join on the server: $lookup adds an array of matching customers, $unwind turns it into one object, $project flattens it. One round trip, and the result is plain objects, not Mongoose documents.
  4. $unwind also drops orders with no matching customer (an empty array), which is why order 9 isn't in the result; add preserveNullAndEmptyArrays: true to keep them, as in $unwind.

#lean and performance

Added

A Mongoose query normally hydrates every result: it wraps each raw document in a full Mongoose document with getters, setters, change tracking, virtuals and methods like save(). That's what makes doc.price = 10; await doc.save() work, and it costs CPU and memory for every document. .lean() skips it and returns the plain JavaScript objects the driver decoded. For read-only paths, which is most API reads, that's free speed.

The mental model: a hydrated document is a library book with a borrowing slip, a barcode and a returns record; a lean document is a photocopy. If you only need to read it, the photocopy is quicker to make and lighter to carry.

m07-lean.mjs
import mongoose from "mongoose";
const schema = new mongoose.Schema({ name: String, category: String, price: Number, stock: Number, tags: [String] });
schema.virtual("inStock").get(function () { return this.stock > 0; });
const Item = mongoose.model("LeanItem", schema, "lean_items");
await mongoose.connect("mongodb://127.0.0.1:27018/ch22?directConnection=true");
await Item.deleteMany({});
await Item.insertMany(Array.from({ length: 20000 }, (_, i) => ({ name: `Item ${i}`, category: ["Laptops", "Phones", "Books"][i % 3], price: i % 1000, stock: i % 7, tags: ["a", "b"] })));

async function time(label, fn) {
  let best = Infinity, n;
  for (let run = 0; run < 5; run++) {
    global.gc?.();
    const t = performance.now();
    const docs = await fn();
    best = Math.min(best, performance.now() - t);
    n = docs.length;
  }
  global.gc?.();
  const before = process.memoryUsage().heapUsed;
  const keep = await fn();
  const mb = (process.memoryUsage().heapUsed - before) / 1024 / 1024;
  console.log(`${label.padEnd(22)} ${n} docs  best ${best.toFixed(0)} ms  heap +${mb.toFixed(1)} MB`);
  return keep;
}
const full = await time("find()", () => Item.find());
const lean = await time("find().lean()", () => Item.find().lean());
await time("lean + projection", () => Item.find({}, { name: 1, price: 1 }).lean());

console.log(full[0] instanceof mongoose.Document, typeof full[0].save, full[0].inStock);
console.log(lean[0] instanceof mongoose.Document, typeof lean[0].save, lean[0].inStock);
console.log(Object.keys(lean[0]));
await mongoose.disconnect();
Result
find()                 20000 docs  best 55 ms  heap +72.6 MB
find().lean()          20000 docs  best 27 ms  heap +37.0 MB
lean + projection      20000 docs  best 18 ms  heap +32.3 MB
true function false
false undefined undefined
[ '_id', 'name', 'category', 'price', 'stock', 'tags', '__v' ]
What’s happening
  1. 20,000 small products are loaded three ways, each timed as the best of 5 runs (run with node --expose-gc so memory is measured after a garbage collection).
  2. find() hydrates every result: 55 ms and about 73 MB of heap for 20,000 documents.
  3. find().lean() returns the driver's plain objects: 27 ms and 37 MB, about half the time and half the memory, for exactly the same data from the server.
  4. Adding a projection (name, price) shrinks what the server sends and the driver decodes: 18 ms. Fetching only the fields you need often matters as much as lean().
  5. The trade-off is visible in the last lines: the hydrated document is a mongoose.Document with save() and the inStock virtual (false for stock 0); the lean one is a plain object with no save, no virtuals (undefined) and no getters, just the stored fields including __v. Use lean() for reads you serialise to JSON; keep hydration where you modify and save().
AdvancedPractical rules for lean and performance
  • lean and populate combine: Order.find().populate("customer").lean() returns plain objects with plain populated objects.
  • lean skips more than methods: no defaults applied to missing fields, no getters (e.g. a Decimal128 getter that formats money), no virtuals unless you add the mongoose-lean-virtuals plugin. If a field "disappears" after adding .lean(), this is why.
  • Batch writes: insertMany sends one command; calling save() in a loop sends one round trip per document and runs hooks each time.
  • Big reads: Model.find().cursor() or for await (const doc of Model.find().lean()) keeps memory flat.
  • Counting: estimatedDocumentCount() reads collection metadata instantly; countDocuments(filter) runs a real query.
  • Indexes: set autoIndex: false in production so app restarts don't trigger index builds, and create indexes in migrations or deploy scripts.

#Transactions from Node

Added

Multi-document transactions work the same way from Node as in mongosh (see Multi-document transactions): start a session, run operations with { session }, commit or abort. In application code, use the driver's session.withTransaction(callback): it starts the transaction, runs your callback, commits, and when the server reports a TransientTransactionError (a write conflict, a failover) it runs the whole callback again, and when the commit result is unknown it retries the commit. You write the business logic; the helper handles the retry rules.

The mental model is a recipe card handed to a cook: if the pan catches fire half-way (a conflict), the cook throws everything away and starts the card again from the top. That's why the callback must be safe to run more than once: database work only, no emails or payments inside.

t01-driver-txn.mjs
import { MongoClient } from "mongodb";
const client = new MongoClient("mongodb://127.0.0.1:27018/?directConnection=true");
const db = client.db("ch22");
await db.collection("products").deleteMany({});
await db.collection("orders").deleteMany({});
await db.collection("products").insertMany([{ _id: 3, name: "Pixel 10", price: 799, stock: 20 }, { _id: 2, name: "MacBook Air", price: 1299, stock: 5 }]);

async function placeOrder(orderId, productId, qty) {
  const session = client.startSession();
  try {
    return await session.withTransaction(async () => {
      const products = db.collection("products");
      const orders = db.collection("orders");
      const p = await products.findOneAndUpdate(
        { _id: productId, stock: { $gte: qty } },
        { $inc: { stock: -qty } },
        { session, returnDocument: "after" }
      );
      if (!p) throw new Error(`not enough stock for product ${productId}`);
      await orders.insertOne({ _id: orderId, productId, qty, total: qty * p.price }, { session });
      return p.stock;
    }, { readConcern: { level: "snapshot" }, writeConcern: { w: "majority" } });
  } finally {
    await session.endSession();
  }
}

console.log("stock left:", await placeOrder(101, 3, 2));
try {
  await placeOrder(102, 2, 6);
} catch (err) {
  console.log("failed:", err.message);
}
console.log(await db.collection("orders").find().toArray());
console.log(await db.collection("products").find({}, { projection: { name: 1, stock: 1 } }).toArray());
await client.close();
Result
stock left: 18
failed: not enough stock for product 2
[ { _id: 101, productId: 3, qty: 2, total: 1598 } ]
[
  { _id: 3, name: 'Pixel 10', stock: 18 },
  { _id: 2, name: 'MacBook Air', stock: 5 }
]
What’s happening
  1. client.startSession() creates a session; withTransaction(callback, options) starts a transaction on it with snapshot reads and a majority commit.
  2. Every operation inside passes { session }. The conditional findOneAndUpdate decrements the Pixel's stock only if there's enough (20 → 18) and returns the updated product, then the order is inserted with total: 2 × 799 = 1598.
  3. The callback returns p.stock, and withTransaction resolves to that value after a successful commit: stock left: 18.
  4. The second order wants 6 MacBook Airs; only 5 are in stock, so findOneAndUpdate returns null and the callback throws. An ordinary error (no TransientTransactionError label) makes withTransaction abort and rethrow without retrying.
  5. Afterwards only order 101 exists and the MacBook Air still has 5. endSession() in finally releases the session whatever happened.
AdvancedProved: withTransaction retries a write conflict

Session A holds an uncommitted write on product 3 for 300 ms. Session B's transaction wants the same document:

t02-retry.mjs
import { MongoClient } from "mongodb";
const client = new MongoClient("mongodb://127.0.0.1:27018/?directConnection=true");
const products = client.db("ch22").collection("products");
await products.updateOne({ _id: 3 }, { $set: { stock: 20 } });

// Session A holds a write on product 3 for 300 ms, inside its own transaction
const a = client.startSession();
a.startTransaction();
await products.updateOne({ _id: 3 }, { $inc: { stock: -1 } }, { session: a });
setTimeout(() => a.commitTransaction().then(() => console.log("A committed")), 300);

// Session B wants the same document
const b = client.startSession();
let attempts = 0;
const t = performance.now();
await b.withTransaction(async () => {
  attempts++;
  try {
    await products.updateOne({ _id: 3 }, { $inc: { stock: -1 } }, { session: b });
  } catch (err) {
    if (attempts <= 3 || attempts % 50 === 0) console.log(`attempt ${attempts}: ${err.codeName}, labels ${JSON.stringify(err.errorLabels)}`);
    throw err;
  }
});
console.log(`B committed after ${attempts} attempts, ${Math.round(performance.now() - t)} ms`);
console.log(await products.findOne({ _id: 3 }, { projection: { stock: 1 } }));
await a.endSession(); await b.endSession(); await client.close();
Result
attempt 1: WriteConflict, labels ["TransientTransactionError"]
attempt 2: WriteConflict, labels ["TransientTransactionError"]
attempt 3: WriteConflict, labels ["TransientTransactionError"]
A committed
B committed after 12 attempts, 346 ms
{ _id: 3, stock: 18 }
What’s happening
  1. A's transaction decrements product 3 and keeps its write lock until the timer commits it 300 ms later.
  2. B's first attempt hits the locked document: the server aborts B's transaction with WriteConflict and the TransientTransactionError label. The callback rethrows, as it should.
  3. withTransaction sees the label and runs the callback again. In this driver version (7.7) it waits between attempts with a jittered exponential backoff, starting at 5 ms, growing 1.5× per attempt, capped at 500 ms, as the driver's source comments describe, so 12 attempts fit into the 300 ms.
  4. Once A commits, B's next attempt succeeds and commits. Final stock 20 → 18: both decrements happened exactly once.
  5. Without the helper, you'd write this retry loop yourself, like the mongosh worker in When not to use transactions. The helper gives up after 120 seconds in total.
AdvancedMongoose transactions without passing the session

connection.transaction(fn) is Mongoose's wrapper around withTransaction. With the transactionAsyncLocalStorage option, Mongoose tracks the session through async calls, so you don't pass { session } to each operation:

t03-mongoose-txn.mjs
import mongoose from "mongoose";
mongoose.set("transactionAsyncLocalStorage", true);   // pass the session to every operation automatically

const Product = mongoose.model("TxProduct", new mongoose.Schema({ _id: Number, name: String, stock: Number }), "tx_products");
const Order = mongoose.model("TxOrder", new mongoose.Schema({ _id: Number, productId: Number, qty: Number }), "tx_orders");
await mongoose.connect("mongodb://127.0.0.1:27018/ch22?directConnection=true");
await Product.deleteMany({}); await Order.deleteMany({});
await Product.create({ _id: 8, name: "Noise-cancelling Headphones", stock: 1 });

async function buy(orderId, qty) {
  return mongoose.connection.transaction(async () => {
    await Order.create([{ _id: orderId, productId: 8, qty }]);          // no { session } needed
    const p = await Product.findOneAndUpdate({ _id: 8, stock: { $gte: qty } }, { $inc: { stock: -qty } }, { returnDocument: "after" });
    if (!p) throw new Error("sold out");
    return p.stock;
  });
}

console.log("left:", await buy(1, 1));
try { await buy(2, 1); } catch (err) { console.log("second buyer:", err.message); }
console.log(await Order.find().lean(), await Product.findById(8).lean());
await mongoose.disconnect();
Result
left: 0
second buyer: sold out
[ { _id: 1, productId: 8, qty: 1, __v: 0 } ] { _id: 8, name: 'Noise-cancelling Headphones', stock: 0, __v: 0 }
What’s happening
  1. mongoose.set("transactionAsyncLocalStorage", true) makes Mongoose remember the current transaction's session in Node's AsyncLocalStorage while the callback runs.
  2. The first buyer creates order 1 and takes the last pair of headphones (1 → 0); the callback returns 0.
  3. The second buyer's order insert succeeds inside its transaction, then findOneAndUpdate finds no stock, the callback throws sold out, and the transaction is aborted, taking the order insert with it.
  4. Only order 1 exists. Order.create([...]) takes an array here because that's the form that accepts options; it's needed when you pass { session } by hand.
  5. An earlier version of this script used { new: true }; Mongoose 9.11 printed the new option for findOneAndUpdate() and findOneAndReplace() is deprecated. Use returnDocument: 'after' instead., hence returnDocument: "after".
AdvancedPitfall: a write without the session
Spot the bug

This checkout handler sometimes charges customers twice. Why?

JavaScript
await session.withTransaction(async () => {
  await orders.insertOne({ _id: orderId, total }, { session });
  await payments.charge(customer.card, total);          // calls the payment provider's API
  await products.updateOne({ _id: productId, stock: { $gte: 1 } }, { $inc: { stock: -1 } }, { session });
});
Show answer

The payment call is inside the transaction callback, and withTransaction re-runs the whole callback on a TransientTransactionError.

What’s happening
  1. Under load, the updateOne on a popular product hits a write conflict with another checkout, and the server aborts the transaction with TransientTransactionError.
  2. The database side is clean: the aborted order insert is rolled back.
  3. But payments.charge already ran, and the payment provider isn't part of the MongoDB transaction, so it isn't rolled back.
  4. withTransaction runs the callback again from the top, inserting the order again and charging the card again. With several conflicts, several charges.
  5. Simulated with a stub payments.charge that counts calls, while another checkout held the product for 100 ms: the result was { charges: 6, orders: 1, stock: 3 }. One order, one stock decrement, six charges.
  6. Fix: keep only database work in the callback. Reserve the stock and create the order with status: "pending" in the transaction, charge the card afterwards with an idempotency key, then mark the order paid. Never put side effects in code that may be retried.

SQL and MySQL written fresh; MongoDB built from Rohit’s “mongoDB interview” doc. Every query was run on MySQL 9.7, MongoDB 9.0, Redis 8.10 or PostgreSQL 18.

RohitDownloads · All chapters

Sync progress across devices

Type the same private phrase on your Mac and your phone, and your Learned ticks follow you between them — on every study site.

The phrase never leaves this device: only a fingerprint of it is sent, and the server stores a fingerprint of that. Anyone who knows the phrase could see or change your ticks, so pick something you don’t use elsewhere.

Sync is on in this browser.

Sync ID

This ID must be the same on every device. If another device shows a different one, its phrase is different (capital letters count): tap “Turn off here” on it and type the phrase again exactly.