Database NotesRohit’s interview study guide
Chapter 13

MySQL from Node.js

Talking to MySQL from Node: mysql2 connections and pools, placeholders and prepared statements, transactions, streaming, then Knex, Sequelize, Prisma and Drizzle side by side, migrations, the N+1 problem measured, and connection limits in serverless.

11 topics
Parts marked Advanced are extra depth. Skip them on a first read or a quick revision.

#mysql2: connections and pools

Added

mysql2 is the standard MySQL driver for Node (version 3.24 here). It speaks MySQL's wire protocol over a TCP socket, has a promise API (mysql2/promise), and is what Knex, Sequelize, Drizzle and TypeORM use underneath. It gives you two ways to talk to the server: a single connection (createConnection) and a pool (createPool).

A connection is one socket and one server thread, and it runs one query at a time: send a query, wait for the result, send the next. Opening one costs a TCP handshake, authentication and often a TLS handshake. A pool keeps up to connectionLimit connections open (10 by default) and lends them out. pool.query() borrows a free connection, runs the query, and puts it back; if none is free, the request waits in a queue in your process. The mental model from Connection pooling applies: a taxi rank, where a few cabs serve a long queue because each ride is short.

In a web server you create one pool per process at startup, share it across all requests, and call pool.end() only on shutdown (see Graceful shutdown on the Node site).

pool-basics.mjs
// A new connection per query vs a pool, and how a pool queues work beyond its size.
import mysql from "mysql2/promise";

const config = { host: "127.0.0.1", port: 3307, user: "root", password: "study", database: "ch11_orm" };

// 1. Open, query, close — 200 times.
let t0 = performance.now();
for (let i = 0; i < 200; i++) {
  const conn = await mysql.createConnection(config);
  await conn.query("SELECT name FROM customers WHERE id = ?", [1 + (i % 6)]);
  await conn.end();
}
console.log(`new connection per query: ${((performance.now() - t0) / 200).toFixed(2)} ms per query`);

// 1b. The same with TLS, as you'd use across a network (self-signed sandbox cert, so no verification here).
t0 = performance.now();
for (let i = 0; i < 200; i++) {
  const conn = await mysql.createConnection({ ...config, ssl: { rejectUnauthorized: false } });
  await conn.query("SELECT name FROM customers WHERE id = ?", [1 + (i % 6)]);
  await conn.end();
}
console.log(`new TLS connection each:  ${((performance.now() - t0) / 200).toFixed(2)} ms per query`);

// 2. The same 200 queries through a pool.
const pool = mysql.createPool({ ...config, connectionLimit: 2 });
t0 = performance.now();
for (let i = 0; i < 200; i++) await pool.query("SELECT name FROM customers WHERE id = ?", [1 + (i % 6)]);
console.log(`pooled:                   ${((performance.now() - t0) / 200).toFixed(2)} ms per query`);

// 3. Six slow queries at once on a pool of two connections.
t0 = performance.now();
const at = () => `${Math.round(performance.now() - t0)} ms`;
await Promise.all([1, 2, 3, 4, 5, 6].map(async (n) => {
  const [[row]] = await pool.query("SELECT CONNECTION_ID() AS conn, SLEEP(0.2) AS s");
  console.log(`query ${n} finished at ${at()} on connection ${row.conn}`);
}));

const [[{ open }]] = await pool.query(
  "SELECT COUNT(*) AS open FROM performance_schema.processlist WHERE USER = 'root' AND DB = 'ch11_orm'");
console.log("connections the pool holds open:", open);
await pool.end();

// new connection per query: 0.78 ms per query
// new TLS connection each:  1.85 ms per query
// pooled:                   0.12 ms per query
// query 1 finished at 203 ms on connection 2294
// query 2 finished at 203 ms on connection 2295
// query 3 finished at 405 ms on connection 2294
// query 4 finished at 405 ms on connection 2295
// query 6 finished at 607 ms on connection 2295
// query 5 finished at 607 ms on connection 2294
// connections the pool holds open: 2
What’s happening
  1. Opening a fresh connection for every query costs 0.78 ms per query on localhost, and 1.85 ms with TLS. The query itself is a tiny primary-key lookup; nearly all of that time is the handshake. Across a real network, each round trip of the handshake costs a full network latency, so the gap grows to many milliseconds.
  2. The pool opens its connection(s) once and reuses them: 0.12 ms per query, 6 to 15 times faster, and the server isn't creating and destroying a thread for every request.
  3. Six SLEEP(0.2) queries start at once on a pool with connectionLimit: 2. Two run immediately on connections 2294 and 2295; the other four wait inside Node, in the pool's queue. They finish in pairs at 203, 405 and 607 ms: the pool limits concurrency to its size, and the rest queue rather than fail (waitForConnections: true is the default; queueLimit: 0 means the queue is unbounded).
  4. CONNECTION_ID() shows the same two server connections alternating. Each pool.query() borrows one connection, and returns it when the result arrives, so consecutive queries from the same code can run on different connections. That's fine for independent queries and a disaster for transactions, as Transactions from Node shows.
  5. performance_schema.processlist confirms the server sees exactly 2 connections from this pool. pool.end() closes them; without it, the open sockets keep the Node process alive after the script finishes.
AdvancedPitfall: session state leaks between requests
AdvancedOld vs new: the mysql package vs mysql2
AdvancedWhat a result looks like

query() and execute() resolve to a two-element array, which is why the examples destructure const [rows] = …:

  • For a SELECT, [rows, fields]: rows is an array of plain objects keyed by column name (or arrays with rowsAsArray: true), fields describes the columns' types.
  • For INSERT/UPDATE/DELETE, [result, undefined], where result is a ResultSetHeader with affectedRows, insertId and changedRows. MySQL has no RETURNING, so insertId is how you learn a new row's AUTO_INCREMENT id.
  • For a CALL of a stored procedure, rows is an array with one entry per result set, followed by a ResultSetHeader for the call itself (see Stored procedures).
  • pool.getConnection() gives you a connection to use for several statements (a transaction); you must call conn.release() to return it.

#Prepared statements and placeholders

Added

mysql2 has two ways to send a query with values, and they work differently on the wire:

  • query(sql, values) substitutes the values in the driver: it escapes each one and splices it into the SQL text, then sends one ordinary text query. Safe against SQL injection for values, because escaping is done correctly and consistently, but the server sees literal values.
  • execute(sql, values) uses a real server-side prepared statement (the binary protocol from SQL injection and prepared statements): the first time, the driver sends the SQL with ? placeholders to be prepared, then sends only the values to execute it. mysql2 caches the prepared statement per connection, so repeating the same SQL text skips the prepare step.

The Node site covers the injection side in SQL injection and parameterised queries. Here it's about what each method does and the surprises in between.

placeholders.mjs
// query() vs execute(): what goes over the wire, and what comes back.
import mysql from "mysql2/promise";

const conn = await mysql.createConnection({ host: "127.0.0.1", port: 3307, user: "root", password: "study", database: "ch11_orm" });
const stmtCounters = async () => {
  const [rows] = await conn.query("SHOW SESSION STATUS WHERE Variable_name IN ('Com_stmt_prepare', 'Com_stmt_execute')");
  return rows.map((r) => `${r.Variable_name}=${r.Value}`).join(" ");
};

// 1. query(): the driver escapes values and splices them into the SQL text (client-side).
console.log(mysql.format("SELECT * FROM customers WHERE email = ? AND id > ?", ["o'brien@example.com", 2]));

// 2. execute(): a real server-side prepared statement, prepared once and cached by the driver.
console.log("before:", await stmtCounters());
for (const id of [1, 2, 3]) {
  const [rows] = await conn.execute("SELECT id, name FROM customers WHERE id = ?", [id]);
  console.log(`  execute(${id}) ->`, rows[0].name);
}
console.log("after: ", await stmtCounters());

// 3. Arrays: query() expands them for IN (...); execute() binds exactly one value per ?.
const [inRows] = await conn.query("SELECT id FROM customers WHERE id IN (?)", [[1, 3, 5]]);
console.log("query IN (?) with [1,3,5]:", inRows.map((r) => r.id));
const [exRows] = await conn.execute("SELECT id FROM customers WHERE id IN (?)", [[1, 3, 5]]);
console.log("execute IN (?) with [1,3,5]:", exRows);
const [[bound]] = await conn.execute("SELECT ? AS bound", [[1, 3, 5]]);
console.log("what execute actually bound:", bound);
const ids = [1, 3, 5];
const [okRows] = await conn.execute(`SELECT id FROM customers WHERE id IN (${ids.map(() => "?").join(", ")}) ORDER BY id`, ids);
console.log("execute IN (?, ?, ?):", okRows.map((r) => r.id));

// 4. Types that surprise people.
const [[row]] = await conn.query(
  "SELECT price, price * 2 AS doubled, COUNT(*) OVER () AS n, created_at FROM products JOIN customers ON customers.id = 1 WHERE products.id = 1");
console.log(row, typeof row.price, typeof row.n);

// 5. Several statements in one call are refused unless you opt in.
try {
  await conn.query("SELECT 1; SELECT 2");
} catch (err) {
  console.log("multiple statements:", err.code);
}
await conn.end();

// SELECT * FROM customers WHERE email = 'o\'brien@example.com' AND id > 2
// before: Com_stmt_execute=0 Com_stmt_prepare=0
//   execute(1) -> Rohit
//   execute(2) -> Maya
//   execute(3) -> Liam
// after:  Com_stmt_execute=3 Com_stmt_prepare=1
// query IN (?) with [1,3,5]: [ 3, 5, 1 ]
// execute IN (?) with [1,3,5]: []
// what execute actually bound: { bound: '[1,3,5]' }
// execute IN (?, ?, ?): [ 1, 3, 5 ]
// {
//   price: '899.00',
//   doubled: '1798.00',
//   n: 1,
//   created_at: 2025-01-05T14:00:00.000Z
// } string number
// multiple statements: ER_PARSE_ERROR
What’s happening
  1. mysql.format() shows exactly what query() sends: the quote in o'brien is escaped as \' and the number is inlined. The server receives one plain SQL string and never knows there were placeholders.
  2. The session counters prove execute() is different: three calls with the same SQL produce one Com_stmt_prepare and three Com_stmt_execute. The driver prepared once, cached the statement on this connection (up to maxPreparedStatements, 16,000 by default), and sent only the id each time.
  3. query() expands an array into a list, so IN (?) with [1, 3, 5] becomes IN (1, 3, 5). The rows come back as 3, 5, 1 because there's no ORDER BY: MySQL read them from the email index (which also contains the id), so they're in email order. Without ORDER BY row order is never guaranteed.
  4. execute() binds one value per ?, so the array is sent as the single value '[1,3,5]' — the debug query shows it — and id IN ('[1,3,5]') silently matches nothing. No error, just an empty result. Generate one placeholder per element instead (IN (?, ?, ?)).
  5. Types: DECIMAL comes back as a string ('899.00', '1798.00') so money isn't rounded through a float; decimalNumbers: true changes that if you accept the risk. BIGINT results like COUNT(*) are numbers unless they exceed Number.MAX_SAFE_INTEGER (see supportBigNumbers/bigNumberStrings). DATETIME becomes a JS Date, interpreted in the Node process's time zone: 09:00 in Toronto is 14:00Z. dateStrings: true returns the raw string.
  6. multipleStatements is false by default, so "SELECT 1; SELECT 2" fails with ER_PARSE_ERROR. Keep it off: it's what turns an injection into "and also DROP TABLE".
AdvancedPitfall: execute() with ever-changing SQL
AdvancedNamed placeholders and identifiers

Two more placeholder features, both handled by the driver before anything is sent:

  • namedPlaceholders: true lets you write WHERE id = :id AND status = :status and pass { id: 1, status: "paid" }; the driver converts it to positional ? (with execute) or inlines the escaped values (with query).
  • ?? is an identifier placeholder: query("SELECT ?? FROM ??", [["id", "name"], "customers"]) produces SELECT `id`, `name` FROM `customers`, with backtick quoting. It only works with query(), because server-side prepared statements can't parameterise identifiers. Quoting stops injection, but not someone choosing a column you didn't mean to expose; use an allowlist (see the sort-column example in SQL injection and prepared statements).
Advancedquery() or execute()?
query()execute()
Where values are boundin the driver (escaped into the text)on the server (binary protocol)
Round trips the first time12 (prepare + execute), then 1 from the cache
Arrays for IN (?)expandedbound as one JSON-like string: silently wrong
?? identifiers, multipleStatementssupportednot supported
Result typestext protocol, converted by the driverbinary protocol, typed by the server
Server memorynoneone prepared statement per distinct SQL per connection

Both are safe for values. Many teams default to execute() for hot, repeated queries and query() for dynamic SQL (variable-length IN lists, optional filters), or use a query builder that takes care of both.

#Transactions from Node

Added

A transaction lives on one connection. START TRANSACTION, every statement in it, and the COMMIT or ROLLBACK must all go over the same socket, because that's where the server keeps the transaction's state and locks. With a pool, that means: check out one connection, use only that connection, and always give it back.

The mental model is a shopping trolley. Each pool.query() is grabbing whichever trolley is free, putting one item in, and returning it to the rack. A transaction is one trolley you keep hold of from the entrance to the checkout. Put half your shopping in one trolley and half in another and the checkout (COMMIT) only pays for one of them.

The pattern is the same as the Node site's PostgreSQL version in ACID transactions, written here for mysql2 against the sample schema:

transactions.mjs
// A transaction belongs to ONE connection: check one out, use it for every statement, always release it.
import mysql from "mysql2/promise";

const pool = mysql.createPool({ host: "127.0.0.1", port: 3307, user: "root", password: "study", database: "ch11_orm", connectionLimit: 5 });

async function placeOrder(customerId, productId, quantity) {
  const conn = await pool.getConnection();
  try {
    await conn.beginTransaction();
    const [[product]] = await conn.query(
      "SELECT price, stock FROM products WHERE id = ? FOR UPDATE", [productId]);    // lock the row until COMMIT
    if (!product) throw new Error(`no product ${productId}`);
    if (product.stock < quantity) throw new Error(`only ${product.stock} in stock`);
    const [order] = await conn.query(
      "INSERT INTO orders (customer_id, status, ordered_at) VALUES (?, 'pending', NOW())", [customerId]);
    await conn.query("INSERT INTO order_items VALUES (?, ?, ?, ?)", [order.insertId, productId, quantity, product.price]);
    await conn.query("UPDATE products SET stock = stock - ? WHERE id = ?", [quantity, productId]);
    await conn.commit();
    return order.insertId;
  } catch (err) {
    await conn.rollback();
    throw err;
  } finally {
    conn.release();                                                                 // back to the pool, success or not
  }
}

const count = async () => (await pool.query("SELECT COUNT(*) AS n FROM orders"))[0][0].n;
console.log("orders before:", await count());
console.log("placed order", await placeOrder(6, 3, 2));
try {
  await placeOrder(6, 4, 1);                                                        // iPhone 17: stock 0
} catch (err) {
  console.log("failed:", err.message);
}
try {
  await placeOrder(6, 999, 1);
} catch (err) {
  console.log("failed:", err.message);
}
console.log("orders after:", await count(), "| Pixel stock:", (await pool.query("SELECT stock FROM products WHERE id = 3"))[0][0].stock);
await pool.end();

// orders before: 8
// placed order 9
// failed: only 0 in stock
// failed: no product 999
// orders after: 9 | Pixel stock: 18
What’s happening
  1. pool.getConnection() checks out one connection and keeps it until release(). Every statement of the order goes through conn, never through pool.
  2. beginTransaction() sends START TRANSACTION. SELECT … FOR UPDATE reads the Pixel's price 799.00 and stock 20 and locks that row, so a concurrent order for the same product waits here instead of reading the same stock (see Locking reads: FOR UPDATE and FOR SHARE).
  3. The order insert returns insertId 9 (MySQL has no RETURNING), the item uses it, the stock goes 20 → 18, and commit() makes all three permanent together.
  4. The iPhone order throws only 0 in stock after the transaction started; the catch runs rollback(), so no order row is left behind. The unknown product throws before any write; the rollback is harmless. Orders went from 8 to 9 — only the successful one.
  5. finally { conn.release() } returns the connection on every path. Forget it on an error path and each failure leaks a connection; after connectionLimit failures every getConnection() waits forever and the service hangs. The throw err after rollback matters too: the caller must know the order didn't happen.
Where did B's update go?

Two requests share one pool. Request A sends its transaction through pool.query(); request B, meanwhile, runs one ordinary auto-committed update:

begin-on-pool.mjs
// The classic bug: START TRANSACTION / UPDATE / ROLLBACK sent through pool.query().
import mysql from "mysql2/promise";
import { setTimeout as sleep } from "node:timers/promises";

const pool = mysql.createPool({ host: "127.0.0.1", port: 3307, user: "root", password: "study", database: "ch11_orm", connectionLimit: 3 });
let lastConnection;
pool.pool.on("acquire", (conn) => { lastConnection = conn.threadId; });          // which connection ran the last call?
const q = async (who, sql, params) => {
  await pool.query(sql, params);
  console.log(`${who}: ${sql.padEnd(52)} on connection ${lastConnection}`);
};
const stock = async () => (await pool.query("SELECT stock FROM products WHERE id = 5"))[0][0].stock;

await pool.query("UPDATE products SET stock = 50 WHERE id = 5");

async function requestA() {                       // a "transaction" that changes its mind
  await q("A", "START TRANSACTION");
  await sleep(20);                                // e.g. calling a payment API
  await q("A", "UPDATE products SET stock = stock - 10 WHERE id = 5");
  await sleep(20);
  await q("A", "ROLLBACK");
}
async function requestB() {                       // an unrelated request: one auto-committed update
  await sleep(10);
  await q("B", "UPDATE products SET stock = stock - 1 WHERE id = 5");
  console.log("B: done, stock is now", await stock());
}

await Promise.all([requestA(), requestB()]);
console.log("final stock:", await stock(), "(expected 49: A rolled back, B's update kept)");
await pool.end();

What's the final stock?

Show answer

50. A's ROLLBACK also undid B's update.

Result
A: START TRANSACTION                                    on connection 2298
B: UPDATE products SET stock = stock - 1 WHERE id = 5   on connection 2298
B: done, stock is now 49
A: UPDATE products SET stock = stock - 10 WHERE id = 5  on connection 2298
A: ROLLBACK                                             on connection 2298
final stock: 50 (expected 49: A rolled back, B's update kept)
What’s happening
  1. A's pool.query("START TRANSACTION") borrows connection 2298, opens a transaction on that connection, and returns the connection to the pool — with the transaction still open.
  2. 10 ms later B borrows a free connection. mysql2 hands out the most recently released one: 2298. B's "auto-committed" update runs inside A's open transaction, and B's own check reads 49 on the same connection, so B believes its write is done and returns success.
  3. A's update also lands on 2298, inside the same transaction, and then A's ROLLBACK rolls back everything on that connection since START TRANSACTION: A's change and B's. The stock is back to 50; B's committed-looking write is lost, and nothing logged an error.
  4. Under other timings the statements would land on different connections instead: A's update would auto-commit on its own and the rollback would undo nothing. Either way there is no transaction. On a quiet developer machine the bug can stay hidden, because sequential calls happen to reuse one connection.
  5. The fix is the pattern above: const conn = await pool.getConnection(), every statement on conn, release() in finally. Libraries' transaction helpers (knex.transaction, sequelize.transaction, prisma.$transaction, db.transaction) do exactly that for you.
AdvancedPitfall: slow work inside a transaction
AdvancedRetrying deadlocks

InnoDB detects deadlocks and kills one of the transactions with ER_LOCK_DEADLOCK (errno 1213) (see Deadlocks). The application's job is to retry the whole transaction:

deadlock-retry.mjs
// Two transactions lock the same two rows in opposite order; InnoDB kills one. Retry it.
import mysql from "mysql2/promise";
import { setTimeout as sleep } from "node:timers/promises";

const pool = mysql.createPool({ host: "127.0.0.1", port: 3307, user: "root", password: "study", database: "ch11_orm", connectionLimit: 4 });

async function withTransaction(label, fn, { retries = 3 } = {}) {
  for (let attempt = 1; ; attempt++) {
    const conn = await pool.getConnection();
    try {
      await conn.beginTransaction();
      const result = await fn(conn);
      await conn.commit();
      console.log(`${label}: committed on attempt ${attempt}`);
      return result;
    } catch (err) {
      await conn.rollback();
      const retryable = err.code === "ER_LOCK_DEADLOCK" || err.code === "ER_LOCK_WAIT_TIMEOUT";
      console.log(`${label}: attempt ${attempt} failed with ${err.code} (${err.errno})${retryable && attempt < retries ? ", retrying" : ""}`);
      if (!retryable || attempt >= retries) throw err;
      await sleep(10 * 2 ** attempt + Math.random() * 10);                     // back off, with jitter
    } finally {
      conn.release();
    }
  }
}

const move = (from, to) => async (conn) => {
  await conn.query("UPDATE products SET stock = stock - 1 WHERE id = ?", [from]);
  await sleep(50);                                                             // widen the window
  await conn.query("UPDATE products SET stock = stock + 1 WHERE id = ?", [to]);
};

await Promise.all([
  withTransaction("T1 (7 -> 8)", move(7, 8)),
  withTransaction("T2 (8 -> 7)", move(8, 7)),
]);
const [rows] = await pool.query("SELECT id, stock FROM products WHERE id IN (7, 8) ORDER BY id");
console.log(rows);
await pool.end();

// T2 (8 -> 7): attempt 1 failed with ER_LOCK_DEADLOCK (1213), retrying
// T1 (7 -> 8): committed on attempt 1
// T2 (8 -> 7): committed on attempt 2
// [ { id: 7, stock: 100 }, { id: 8, stock: 8 } ]
What’s happening
  1. T1 locks product 7 and T2 locks product 8, then each sleeps 50 ms and asks for the other's row. Each waits for the other: a cycle.
  2. InnoDB's deadlock detector notices immediately and rolls back one victim (the one that's cheaper to undo), here T2, which gets ER_LOCK_DEADLOCK (1213). T1's blocked update then proceeds and commits.
  3. withTransaction rolls back (InnoDB already has; the call is harmless), backs off for a randomised 20–30 ms so the two don't collide again in lockstep, and runs the whole function again on a fresh connection. T2 commits on attempt 2.
  4. Both moves happened exactly once each, so stocks are back to 100 and 8. Retrying only works if the function is safe to run again from the start — everything it does must be inside the transaction, with no emails sent or HTTP calls made halfway through.
  5. ER_LOCK_WAIT_TIMEOUT (errno 1205, after innodb_lock_wait_timeout, 50 s by default) is also worth one retry, but by default it rolls back only the statement, not the transaction, so roll back yourself before retrying. Locking rows in a consistent order (always lower id first) prevents this particular deadlock entirely.

#Streaming large result sets

Added

By default await pool.query(sql) buffers the entire result: mysql2 reads every row the server sends, builds a JavaScript object for each, and resolves the promise with one big array. For a page of 50 rows that's ideal. For an export of half a million rows, the process holds all of them in memory at once, and the event loop is busy parsing them.

Streaming turns the result into a Node Readable that emits rows as they arrive. With pipeline(), backpressure flows all the way back: when the destination (a file, an HTTP response, a CSV transform) can't keep up, the stream stops reading from the socket, TCP's window fills, and the MySQL server pauses sending. Memory stays roughly constant whatever the row count. The Node site explains the mechanics in Backpressure.

The mental model is filling a bathtub versus drinking from a tap: the buffered query fills the whole tub before you can drink; the stream lets you drink as it flows, and turning the tap down when you can't swallow faster.

stream-rows.mjs
// Reading 500,000 rows: all at once vs streamed row by row.
import mysql from "mysql2/promise";
import { createWriteStream } from "node:fs";
import { Transform } from "node:stream";
import { pipeline } from "node:stream/promises";

const mode = process.argv[2];
const conn = await mysql.createConnection({ host: "127.0.0.1", port: 3307, user: "root", password: "study", database: "ch11_orm" });
console.log(`baseline RSS ${Math.round(process.memoryUsage().rss / 1024 / 1024)} MB`);
const mb = (n) => `${Math.round(n / 1024 / 1024)} MB`;
let peak = 0;
const sample = setInterval(() => { peak = Math.max(peak, process.memoryUsage().rss); }, 5);
const t0 = performance.now();

if (mode === "all") {
  const [rows] = await conn.query("SELECT id, created_at, message FROM big_log");    // the whole result as one array
  let bytes = 0;
  for (const r of rows) bytes += r.message.length;
  console.log(`all at once: ${rows.length} rows, ${bytes} chars`);
} else {
  // conn.connection is the underlying callback-style connection; its query() can return a Readable stream.
  const rows = conn.connection.query("SELECT id, created_at, message FROM big_log").stream({ highWaterMark: 1000 });
  let n = 0;
  const toCsv = new Transform({
    objectMode: true,
    transform(r, _enc, done) { n++; done(null, `${r.id},${r.created_at.toISOString()},${r.message}\n`); },
  });
  await pipeline(rows, toCsv, createWriteStream(new URL("./big_log.csv", import.meta.url)));   // backpressure end to end
  console.log(`streamed:    ${n} rows to a CSV file`);
}
clearInterval(sample);
console.log(`  ${Math.round(performance.now() - t0)} ms, peak RSS ${mb(peak)}`);
await conn.end();

// $ node stream-rows.mjs all
// baseline RSS 61 MB
// all at once: 500000 rows, 81388895 chars
//   170 ms, peak RSS 319 MB
// $ node stream-rows.mjs stream
// baseline RSS 62 MB
// streamed:    500000 rows to a CSV file
//   447 ms, peak RSS 117 MB
What’s happening
  1. big_log has 500,000 rows of about 160 characters (81 MB of message text). The process starts at about 61 MB of resident memory.
  2. Buffered: conn.query() resolves only after all 500,000 row objects exist, and resident memory peaks at 319 MB, about 260 MB just for this result. Ten times the rows would need gigabytes, and a few concurrent exports would take the process down.
  3. Streamed: conn.connection is the callback-style connection underneath the promise wrapper, and its query(…).stream() returns an object-mode Readable. Peak memory is 117 MB, about 55 MB above baseline, and it stays there however many rows there are, because only a buffer's worth (highWaterMark: 1000 rows) is in flight at once.
  4. pipeline(rows, toCsv, file) connects the three streams with backpressure and error handling: if the file write slows down, toCsv stops accepting rows, the row stream stops reading the socket, and MySQL waits. If any stage fails, all three are destroyed and the promise rejects.
  5. The streamed run took longer (447 ms vs 170 ms) because it also formatted and wrote 98 MB of CSV; the buffered run only counted characters. The point isn't speed but bounded memory — and the first rows reach the destination immediately instead of after the whole query finishes.
AdvancedPitfall: slow streams hold a snapshot open
AdvancedLibrary support: streaming in builders and ORMs

#Query builders: Knex

Added

Knex (3.3 here) sits one level above the driver. You still think in tables, joins and SQL, but you build each statement from method calls — knex("orders").join(…).where(…).groupBy(…) — and Knex turns it into SQL for the configured dialect (mysql2, pg, sqlite3, mssql), with every value as a bound parameter. It also gives you transactions, connection pooling (via tarn.js) and a migration runner.

The mental model is SQL with Lego bricks: you can't build anything SQL can't, but you can add or skip bricks conditionally without string concatenation, and the bricks don't care which database they end up in. That's its sweet spot: dynamic queries (filters that may or may not apply) and teams that want to stay close to SQL without an ORM's object model.

knex-demo.mjs
// Knex builds SQL from method calls; you still think in tables, joins and transactions.
import knexFactory from "knex";

const knex = knexFactory({
  client: "mysql2",
  connection: { host: "127.0.0.1", port: 3307, user: "root", password: "study", database: "ch11_orm" },
  pool: { min: 0, max: 5 },
});

// 1. A query built step by step. toSQL() shows exactly what will be sent.
const revenueByCustomer = knex("orders as o")
  .join("customers as c", "c.id", "o.customer_id")
  .join("order_items as oi", "oi.order_id", "o.id")
  .whereNot("o.status", "cancelled")
  .groupBy("c.id", "c.name")
  .select("c.name")
  .sum({ revenue: knex.raw("?? * ??", ["oi.quantity", "oi.unit_price"]) })
  .orderBy("revenue", "desc")
  .limit(3);
console.log(revenueByCustomer.toSQL().toNative());
console.log(await revenueByCustomer);

// 2. Optional filters: build the query conditionally instead of concatenating strings.
async function findProducts({ maxPrice, inStock, search } = {}) {
  const q = knex("products").select("id", "name", "price", "stock").orderBy("price");
  if (maxPrice !== undefined) q.where("price", "<=", maxPrice);
  if (inStock) q.where("stock", ">", 0);
  if (search) q.where("name", "like", `%${search}%`);
  console.log("  ", q.toString());
  return q;
}
console.log((await findProducts({ maxPrice: 900, inStock: true })).map((p) => p.name));
console.log((await findProducts({ search: "book" })).map((p) => p.name));
try {
  await knex("products").whereLike("name", "%book%");                     // Knex's case-sensitive LIKE helper
} catch (err) {
  console.log("whereLike:", err.code, err.sqlMessage);
}

// 3. Inserts: MySQL has no RETURNING, so Knex gives you [insertId].
const ids = await knex("customers").insert({ name: "Ava", email: "ava@example.com", city: "Paris", created_at: knex.fn.now() });
console.log("insert returned:", ids);

// 4. Transactions: everything on trx runs on one connection; throwing rolls back.
try {
  await knex.transaction(async (trx) => {
    const [orderId] = await trx("orders").insert({ customer_id: ids[0], status: "pending", ordered_at: knex.fn.now() });
    await trx("order_items").insert({ order_id: orderId, product_id: 4, quantity: 1, unit_price: 1099 });
    const updated = await trx("products").where({ id: 4 }).andWhere("stock", ">=", 1).decrement("stock", 1);
    if (updated === 0) throw new Error("iPhone 17 is out of stock");
  });
} catch (err) {
  console.log("transaction rolled back:", err.message);
}
console.log("Ava's orders:", (await knex("orders").where({ customer_id: ids[0] }).count({ n: "*" }))[0].n);

await knex("customers").where({ id: ids[0] }).del();
await knex.destroy();
Result
{
  sql: 'select `c`.`name`, sum(`oi`.`quantity` * `oi`.`unit_price`) as `revenue` from `orders` as `o` inner join `customers` as `c` on `c`.`id` = `o`.`customer_id` inner join `order_items` as `oi` on `oi`.`order_id` = `o`.`id` where not `o`.`status` = ? group by `c`.`id`, `c`.`name` order by `revenue` desc limit ?',
  bindings: [ 'cancelled', 3 ]
}
[
  { name: 'Zara', revenue: '1598.00' },
  { name: 'Maya', revenue: '1164.00' },
  { name: 'Rohit', revenue: '1045.00' }
]
   select `id`, `name`, `price`, `stock` from `products` where `price` <= 900 and `stock` > 0 order by `price` asc
[
  'USB-C Charger',
  'SQL Performance Explained',
  'Designing Data-Intensive Applications',
  'Noise-cancelling Headphones',
  'Pixel 10',
  'ThinkBook 14'
]
   select `id`, `name`, `price`, `stock` from `products` where `name` like '%book%' order by `price` asc
[ 'ThinkBook 14', 'MacBook Air' ]
whereLike: ER_COLLATION_CHARSET_MISMATCH COLLATION 'utf8mb3_bin' is not valid for CHARACTER SET 'utf8mb4'
insert returned: [ 7 ]
transaction rolled back: iPhone 17 is out of stock
Ava's orders: 0
What’s happening
  1. The revenue query reads like SQL in method form. toSQL().toNative() shows what's sent: backtick-quoted identifiers, and the status and limit as ? bindings (['cancelled', 3]). knex.raw("?? * ??", […]) is the escape hatch for expressions, with ?? for identifiers. Zara leads with 1598.00 because of the two-Pixel order placed in Transactions from Node; sums of DECIMAL arrive as strings, as with the raw driver.
  2. findProducts adds where clauses only for the filters that were given. toString() (debug only — it inlines values) shows the two different queries. This is the job query builders do best; the string-concatenation alternative is how injection bugs are born.
  3. The book search matches ThinkBook 14 and MacBook Air: the default utf8mb4_0900_ai_ci collation is case-insensitive. Knex's whereLike() helper tries to make LIKE case-sensitive by appending COLLATE utf8_bin, which is a utf8mb3 collation, and MySQL 8+ rejects it on utf8mb4 columns: ER_COLLATION_CHARSET_MISMATCH. A real incompatibility to know about: use where(col, "like", …) and pick a _bin or _cs collation in the schema if you need case-sensitivity.
  4. insert() resolves to [7], the new insertId. On PostgreSQL the same call would use RETURNING; on MySQL, adding .returning("id") only prints .returning() is not supported by mysql and will not have any effect. and still resolves to [id], so you re-select the row if you need its defaults.
  5. knex.transaction(async (trx) => …) checks out one connection, runs BEGIN, passes trx (a query builder bound to that connection), and commits when the function returns or rolls back when it throws. The guarded decrement updated 0 rows because the iPhone has no stock, the function threw, and the order and its item were rolled back: Ava has 0 orders.
AdvancedPitfall: raw fragments are still SQL
AdvancedAdvice: when Knex is the right level

#ORMs: Sequelize

Added

An ORM (object-relational mapper) maps tables to models and rows to objects: you define Customer and Order, declare that a customer hasMany orders, and then write Customer.findAll({ include: [Order] }) instead of a join. The ORM generates the SQL, turns rows into model instances with getters and methods, runs validation, and manages transactions.

The mental model is a translator: you speak objects, the database speaks tables, and the ORM translates in both directions. Translators are wonderful for everyday conversation and dangerous for subtle negotiations — you need to read what they actually said (the SQL log) when performance or correctness matter.

Sequelize (6.37 here) is the oldest major Node ORM, JavaScript-first, with models defined at runtime. These models map onto the site's existing sample tables:

models.mjs
// Sequelize models mapped onto the site's existing sample tables.
import { Sequelize, DataTypes, Op } from "sequelize";

export const sqlLog = [];
export const sequelize = new Sequelize(process.env.DB ?? "ch11_orm", "root", "study", {
  host: "127.0.0.1", port: 3307, dialect: "mysql",
  logging: (sql) => sqlLog.push(sql.replace(/^Executing \(default\): /, "")),
  define: { timestamps: false, underscored: true },           // the sample tables have no createdAt/updatedAt
});

export const Customer = sequelize.define("Customer", {
  name: { type: DataTypes.STRING(50), allowNull: false },
  email: { type: DataTypes.STRING(100), allowNull: false, unique: true, validate: { isEmail: true } },
  city: DataTypes.STRING(50),
  createdAt: { type: DataTypes.DATE, allowNull: false, defaultValue: DataTypes.NOW },
}, { tableName: "customers" });

export const Order = sequelize.define("Order", {
  status: { type: DataTypes.ENUM("pending", "paid", "shipped", "cancelled"), allowNull: false },
  orderedAt: { type: DataTypes.DATE, allowNull: false },
}, { tableName: "orders" });

export const Product = sequelize.define("Product", {
  name: DataTypes.STRING(80),
  price: DataTypes.DECIMAL(10, 2),
  stock: DataTypes.INTEGER,
}, { tableName: "products" });

export const OrderItem = sequelize.define("OrderItem", {
  orderId: { type: DataTypes.INTEGER, primaryKey: true },
  productId: { type: DataTypes.INTEGER, primaryKey: true },
  quantity: DataTypes.INTEGER,
  unitPrice: DataTypes.DECIMAL(10, 2),
}, { tableName: "order_items" });

Customer.hasMany(Order, { foreignKey: "customerId" });
Order.belongsTo(Customer, { foreignKey: "customerId" });
Order.hasMany(OrderItem, { foreignKey: "orderId" });
OrderItem.belongsTo(Product, { foreignKey: "productId" });
export { Op };
What’s happening
  1. new Sequelize(…) creates the connection pool (mysql2 underneath). The logging callback receives every SQL statement Sequelize sends; collecting them in sqlLog is how the examples below show the generated SQL and count queries.
  2. underscored: true maps camelCase attributes to snake_case columns (createdAt → created_at, customerId → customer_id); timestamps: false stops Sequelize from expecting createdAt/updatedAt columns on every table, which these tables don't have.
  3. Each define maps a model to an existing table. Unlisted columns are simply ignored; id is assumed as an auto-increment primary key unless you declare one, as OrderItem does with its composite key.
  4. validate: { isEmail: true } is checked in JavaScript before any SQL; unique: true and allowNull: false are only descriptions here, because the table already exists — MySQL enforces the real constraints.
  5. The associations (hasMany, belongsTo) don't touch the database. They tell Sequelize which foreign keys link the models, which is what makes include, getOrders() and the N+1 query problem possible.
AdvancedVersions: Sequelize 6, 7 and alternatives
AdvancedQuerying with models
sequelize-demo.mjs
import { sequelize, sqlLog, Customer, Order, OrderItem, Product, Op } from "./models.mjs";

// 1. Eager loading with include: one query with JOINs.
const rohit = await Customer.findOne({
  where: { email: "rohit@example.com" },
  include: [{ model: Order, include: [{ model: OrderItem, include: [Product] }] }],
  order: [[Order, "id", "ASC"]],
});
console.log(rohit.name, rohit.Orders.map((o) => `#${o.id} ${o.status}: ` +
  o.OrderItems.map((i) => `${i.quantity}× ${i.Product.name}`).join(", ")));
console.log("SQL:", sqlLog.at(-1));

// 2. Operators and model instances.
const cheap = await Product.findAll({ where: { price: { [Op.lt]: 50 } }, order: [["price", "ASC"]], attributes: ["name", "price"] });
console.log(cheap.map((p) => `${p.name} ${p.price}`));
console.log("SQL:", sqlLog.at(-1));

// 3. Validation runs in JavaScript before any SQL is sent.
try {
  await Customer.create({ name: "Bad", email: "not-an-email" });
} catch (err) {
  console.log(err.name, "-", err.errors.map((e) => e.message).join("; "));
}
// ...while constraints are still enforced by MySQL.
try {
  await Customer.create({ name: "Dup", email: "maya@example.com" });
} catch (err) {
  console.log(err.name, "-", err.parent.code, err.parent.sqlMessage);
}

// 4. Managed transaction: commit on return, rollback on throw.
const t0 = sqlLog.length;
await sequelize.transaction(async (t) => {
  const zara = await Customer.findByPk(6, { transaction: t });
  const order = await Order.create({ customerId: zara.id, status: "pending", orderedAt: new Date() }, { transaction: t });
  await OrderItem.create({ orderId: order.id, productId: 7, quantity: 2, unitPrice: "29.00" }, { transaction: t });
});
console.log(sqlLog.slice(t0).map((s) => s.slice(0, 95)).join("\n"));
await sequelize.close();
Result
Rohit [
  '#1 shipped: 1× ThinkBook 14, 2× USB-C Charger',
  '#2 paid: 1× SQL Performance Explained, 1× Designing Data-Intensive Applications'
]
SQL: SELECT `Customer`.`id`, `Customer`.`name`, `Customer`.`email`, `Customer`.`city`, `Customer`.`created_at` AS `createdAt`, `Orders`.`id` AS `Orders.id`, `Orders`.`status` AS `Orders.status`, `Orders`.`ordered_at` AS `Orders.orderedAt`, `Orders`.`customer_id` AS `Orders.customerId`, `Orders->OrderItems`.`order_id` AS `Orders.OrderItems.orderId`, `Orders->OrderItems`.`product_id` AS `Orders.OrderItems.productId`, `Orders->OrderItems`.`quantity` AS `Orders.OrderItems.quantity`, `Orders->OrderItems`.`unit_price` AS `Orders.OrderItems.unitPrice`, `Orders->OrderItems->Product`.`id` AS `Orders.OrderItems.Product.id`, `Orders->OrderItems->Product`.`name` AS `Orders.OrderItems.Product.name`, `Orders->OrderItems->Product`.`price` AS `Orders.OrderItems.Product.price`, `Orders->OrderItems->Product`.`stock` AS `Orders.OrderItems.Product.stock` FROM `customers` AS `Customer` LEFT OUTER JOIN `orders` AS `Orders` ON `Customer`.`id` = `Orders`.`customer_id` LEFT OUTER JOIN `order_items` AS `Orders->OrderItems` ON `Orders`.`id` = `Orders->OrderItems`.`order_id` LEFT OUTER JOIN `products` AS `Orders->OrderItems->Product` ON `Orders->OrderItems`.`product_id` = `Orders->OrderItems->Product`.`id` WHERE `Customer`.`email` = 'rohit@example.com' ORDER BY `Orders`.`id` ASC;
[
  'USB-C Charger 29.00',
  'SQL Performance Explained 39.00',
  'Designing Data-Intensive Applications 49.00'
]
SQL: SELECT `name`, `price` FROM `products` AS `Product` WHERE `Product`.`price` < 50 ORDER BY `Product`.`price` ASC;
SequelizeValidationError - Validation isEmail on email failed
SequelizeUniqueConstraintError - ER_DUP_ENTRY Duplicate entry 'maya@example.com' for key 'customers.email'
Executing (26d9865f-ff1f-43dd-9bad-e48f8bde1d93): START TRANSACTION;
Executing (26d9865f-ff1f-43dd-9bad-e48f8bde1d93): SELECT `id`, `name`, `email`, `city`, `create
Executing (26d9865f-ff1f-43dd-9bad-e48f8bde1d93): INSERT INTO `orders` (`id`,`status`,`ordered_
Executing (26d9865f-ff1f-43dd-9bad-e48f8bde1d93): INSERT INTO `order_items` (`order_id`,`produc
Executing (26d9865f-ff1f-43dd-9bad-e48f8bde1d93): COMMIT;
What’s happening
  1. A three-level include (customer → orders → items → product) becomes one query with three LEFT OUTER JOINs, and Sequelize folds the flat joined rows back into nested objects: rohit.Orders[0].OrderItems[1].Product.name. The aliases like Orders->OrderItems are how it knows which column belongs where.
  2. That query returns one row per item, repeating the customer's columns on each. With several hasMany includes at the same level, the row count multiplies (orders × addresses × payments), which is when separate: true (a second query per association) is faster than one giant join.
  3. Notice WHERE … = 'rohit@example.com': Sequelize v6 escapes most SELECT values into the SQL text, like mysql2's query(), rather than binding them (inserts use bind parameters). It's still injection-safe for values passed through its API; sequelize.literal() and raw queries are not.
  4. Op.lt builds price < 50. attributes limits the selected columns — a good habit, since models default to SELECT of every defined column.
  5. Two kinds of failure: SequelizeValidationError comes from the model's isEmail rule before any SQL is sent; SequelizeUniqueConstraintError wraps MySQL's own ER_DUP_ENTRY, which you can inspect through err.parent.
  6. sequelize.transaction(async (t) => …) is a managed transaction: one connection, START TRANSACTION, commit on return, rollback on throw. The log shows every statement tagged with the same transaction id. You must pass { transaction: t } to every call; forget one and that statement runs on a different pooled connection outside the transaction — the same bug as in the quiz above. (Sequelize's CLS option can pass it implicitly.)

#Prisma

Added

Prisma takes a schema-first approach. You describe models in a schema.prisma file (or let prisma db pull generate it from an existing database), run prisma generate, and get a fully typed client whose methods and return types match your schema exactly: prisma.customers.findUnique({ where: { email }, include: { orders: true } }) is typed down to each nested field. Prisma Migrate generates SQL migrations from changes to the schema file.

Prisma 7 (7.10.1 here) changed how it runs: the Rust query engine is gone, replaced by a TypeScript client with a WebAssembly query compiler, and the client talks to the database through a driver adapter — for MySQL and MariaDB, @prisma/adapter-mariadb, which uses the mariadb Node driver. The connection URL lives in prisma.config.ts for the CLI, and the generated client is plain TypeScript in your project (provider = "prisma-client" with an output directory).

The mental model: Prisma is a typed API generated from a blueprint. You don't write models in code; you edit the blueprint, regenerate, and the compiler tells you everywhere your code no longer matches.

This chapter's Prisma project introspected the sample schema with prisma db pull, which wrote seven models (excerpt):

schema.prisma
generator client {
  provider = "prisma-client"
  output   = "../generated"
}

datasource db {
  provider = "mysql"
}

model customers {
  id         Int      @id @default(autoincrement())
  name       String   @db.VarChar(50)
  email      String   @unique(map: "email") @db.VarChar(100)
  city       String?  @db.VarChar(50)
  created_at DateTime @db.DateTime(0)
  orders     orders[]
}

model orders {
  id          Int           @id @default(autoincrement())
  customer_id Int
  status      orders_status
  ordered_at  DateTime      @db.DateTime(0)
  order_items order_items[]
  customers   customers     @relation(fields: [customer_id], references: [id], onDelete: NoAction, onUpdate: NoAction, map: "orders_ibfk_1")

  @@index([customer_id], map: "customer_id")
}

enum orders_status {
  pending
  paid
  shipped
  cancelled
}
What’s happening
  1. generator client with provider = "prisma-client" is the Prisma 7 generator: it writes a TypeScript client into ../generated (here client.ts, models/…, enums.ts), which you import like any other module. The older prisma-client-js generator wrote into node_modules/.prisma instead.
  2. datasource db names the database type. In Prisma 7 the URL moved out of the schema into prisma.config.ts (datasource: { url: "mysql://root:study@127.0.0.1:3307/ch11_orm" }), used by CLI commands like db pull and migrate.
  3. db pull mapped each column to a Prisma type with its exact MySQL type (@db.VarChar(50), @db.DateTime(0)), kept the existing index and foreign key names (map: "orders_ibfk_1"), and turned the ENUM column into an enum orders_status.
  4. The relation fields orders orders[] and customers customers don't exist in the database; they're Prisma's view of the foreign key, used for include and nested writes. Introspected names are the table names (customers, order_items); hand-written schemas usually use @@map to keep Customer in code and customers in the database.
AdvancedPitfall: Prisma 7 needs a driver adapter
AdvancedTrade-offs: where Prisma shines and struggles
AdvancedQuerying with the generated client
prisma-demo.ts
import { PrismaMariaDb } from "@prisma/adapter-mariadb";
import { PrismaClient } from "./generated/client.ts";

const adapter = new PrismaMariaDb({ host: "127.0.0.1", port: 3307, user: "root", password: "study", database: "ch11_orm", connectionLimit: 5 });
const prisma = new PrismaClient({ adapter, log: [{ emit: "event", level: "query" }] });
prisma.$on("query", (e) => console.log("  SQL:", e.query.replace(/\s+/g, " "), e.params));

const rohit = await prisma.customers.findUnique({
  where: { email: "rohit@example.com" },
  include: { orders: { include: { order_items: true }, orderBy: { id: "asc" } } },
});
console.log(rohit?.name, rohit?.orders.map((o) => `#${o.id} ${o.status} items=${o.order_items.length}`));
await prisma.$disconnect();
Result
  SQL: SELECT `customers`.`id`, `customers`.`name`, `customers`.`email`, `customers`.`city`, `customers`.`created_at` FROM `customers` WHERE (`customers`.`email` = ? AND 1=1) LIMIT ? OFFSET ? ["rohit@example.com","1","0"]
  SQL: SELECT `orders`.`id`, `orders`.`customer_id`, `orders`.`status`, `orders`.`ordered_at` FROM `orders` WHERE `orders`.`customer_id` = ? ORDER BY `orders`.`id` ASC [1]
  SQL: SELECT `order_items`.`order_id`, `order_items`.`product_id`, `order_items`.`quantity`, `order_items`.`unit_price` FROM `order_items` WHERE `order_items`.`order_id` IN (?,?) [1,2]
Rohit [ '#1 shipped items=2', '#2 paid items=2' ]
What’s happening
  1. The file is TypeScript and runs directly with node prisma-demo.ts on Node 26 (type stripping); the generated client is .ts too. PrismaMariaDb is the driver adapter, holding the connection pool (connectionLimit: 5). The block is marked as not type-checked on this page because it needs the generated client.
  2. log: [{ emit: "event", level: "query" }] plus $on("query") prints every SQL statement with its parameters, which is how you check what Prisma really does.
  3. One findUnique with a two-level include ran three queries, not one join: the customer by email, then that customer's orders, then the items of orders IN (1, 2). Prisma loads each relation level with a separate batched query and stitches the objects together in JavaScript. That's never N+1 — it's one query per level, not per row — but it is more round trips than a join.
  4. All values are bound parameters (? with ["rohit@example.com","1","0"]), including the LIMIT 1 that findUnique adds. The result type is inferred from the include: rohit.orders[0].order_items exists in the type only because it was included.
  5. A join strategy exists behind a preview flag: with previewFeatures = ["relationJoins"] in the generator, relationLoadStrategy: "join" produced a single query with a JSON_ARRAYAGG subquery; without the flag, the option is rejected with Unknown argument `relationLoadStrategy`.

#Drizzle

Added

Drizzle (drizzle-orm 0.45 here) describes tables in TypeScript or JavaScript code and gives you two query APIs: a SQL-shaped builder (db.select().from(orders).innerJoin(…).where(…)) that maps almost one-to-one onto SQL, and a relational API (db.query.customers.findFirst({ with: { orders: true } })) for nested results. It runs on top of your driver (here mysql2's pool) with no engine or code generation, and it's fully typed from the table definitions.

The mental model: Drizzle is SQL with types. If you know the SQL you want, you can write it in Drizzle and the result type follows; there's very little "magic" to second-guess. That's why it became popular for TypeScript backends and edge/serverless runtimes, where a small, dependency-free layer matters.

drizzle-schema.mjs
// Drizzle schema: tables described in code, matching the sample schema's columns.
import { mysqlTable, int, varchar, decimal, datetime, mysqlEnum, primaryKey } from "drizzle-orm/mysql-core";
import { relations } from "drizzle-orm";

export const customers = mysqlTable("customers", {
  id: int("id").autoincrement().primaryKey(),
  name: varchar("name", { length: 50 }).notNull(),
  email: varchar("email", { length: 100 }).notNull().unique(),
  city: varchar("city", { length: 50 }),
  createdAt: datetime("created_at").notNull(),
});

export const orders = mysqlTable("orders", {
  id: int("id").autoincrement().primaryKey(),
  customerId: int("customer_id").notNull().references(() => customers.id),
  status: mysqlEnum("status", ["pending", "paid", "shipped", "cancelled"]).notNull(),
  orderedAt: datetime("ordered_at").notNull(),
});

export const orderItems = mysqlTable("order_items", {
  orderId: int("order_id").notNull().references(() => orders.id),
  productId: int("product_id").notNull(),
  quantity: int("quantity").notNull(),
  unitPrice: decimal("unit_price", { precision: 10, scale: 2 }).notNull(),
}, (t) => [primaryKey({ columns: [t.orderId, t.productId] })]);

export const customersRelations = relations(customers, ({ many }) => ({ orders: many(orders) }));
export const ordersRelations = relations(orders, ({ one, many }) => ({
  customer: one(customers, { fields: [orders.customerId], references: [customers.id] }),
  items: many(orderItems),
}));
export const orderItemsRelations = relations(orderItems, ({ one }) => ({
  order: one(orders, { fields: [orderItems.orderId], references: [orders.id] }),
}));
What’s happening
  1. Each mysqlTable("name", { … }) describes a real table; the property name is what your code uses (customerId) and the string argument is the column (customer_id). The column builders (int, varchar, decimal, mysqlEnum) carry both the SQL type and the TypeScript type: status is typed as the union of the four strings.
  2. .notNull(), .unique(), .references() and primaryKey() describe constraints. At runtime they only matter to drizzle-kit, the migration tool, which compares these definitions with the database and generates DDL.
  3. relations() is separate from the tables and doesn't touch the database. It powers only the relational API (db.query…with), mapping orders.customerId to customers.id for one and many relations.
  4. This is plain .mjs so it runs without a TypeScript step; in a TypeScript project the same file gives fully typed queries with no code generation.
AdvancedTooling: drizzle-kit and Drizzle 1.0
AdvancedBoth query APIs, run
drizzle-demo.mjs
import mysql from "mysql2/promise";
import { drizzle } from "drizzle-orm/mysql2";
import { eq, ne, sql, desc } from "drizzle-orm";
import * as schema from "./drizzle-schema.mjs";
const { customers, orders, orderItems } = schema;

const pool = mysql.createPool({ host: "127.0.0.1", port: 3307, user: "root", password: "study", database: "ch11_orm" });
const db = drizzle(pool, { schema, mode: "default", logger: { logQuery: (q, p) => console.log("  SQL:", q, p) } });

// 1. SQL-shaped query builder: you write the join and the aggregate.
const revenue = await db
  .select({ name: customers.name, revenue: sql`sum(${orderItems.quantity} * ${orderItems.unitPrice})`.mapWith(Number) })
  .from(orders)
  .innerJoin(customers, eq(customers.id, orders.customerId))
  .innerJoin(orderItems, eq(orderItems.orderId, orders.id))
  .where(ne(orders.status, "cancelled"))
  .groupBy(customers.id, customers.name)
  .orderBy(desc(sql`2`))
  .limit(3);
console.log(revenue);

// 2. Relational query API: nested objects, built from the relations() above.
const maya = await db.query.customers.findFirst({
  where: eq(customers.email, "maya@example.com"),
  with: { orders: { with: { items: true } } },
});
console.log(maya.name, maya.orders.map((o) => `#${o.id} ${o.status} (${o.items.length} items)`));

// 3. Insert: MySQL has no RETURNING, so you get the insert id back.
const [result] = await db.insert(customers).values({ name: "Ava", email: "ava@example.com", createdAt: new Date() });
console.log("insertId:", result.insertId);
await db.delete(customers).where(eq(customers.id, result.insertId));
await pool.end();
Result
  SQL: select `customers`.`name`, sum(`order_items`.`quantity` * `order_items`.`unit_price`) from `orders` inner join `customers` on `customers`.`id` = `orders`.`customer_id` inner join `order_items` on `order_items`.`order_id` = `orders`.`id` where `orders`.`status` <> ? group by `customers`.`id`, `customers`.`name` order by 2 desc limit ? [ 'cancelled', 3 ]
[
  { name: 'Zara', revenue: 1656 },
  { name: 'Maya', revenue: 1164 },
  { name: 'Rohit', revenue: 1045 }
]
  SQL: select `customers`.`id`, `customers`.`name`, `customers`.`email`, `customers`.`city`, `customers`.`created_at`, `customers_orders`.`data` as `orders` from `customers` `customers` left join lateral (select coalesce(json_arrayagg(json_array(`customers_orders`.`id`, `customers_orders`.`customer_id`, `customers_orders`.`status`, `customers_orders`.`ordered_at`, `customers_orders_items`.`data`)), json_array()) as `data` from `orders` `customers_orders` left join lateral (select coalesce(json_arrayagg(json_array(`customers_orders_items`.`order_id`, `customers_orders_items`.`product_id`, `customers_orders_items`.`quantity`, `customers_orders_items`.`unit_price`)), json_array()) as `data` from `order_items` `customers_orders_items` where `customers_orders_items`.`order_id` = `customers_orders`.`id`) `customers_orders_items` on true where `customers_orders`.`customer_id` = `customers`.`id`) `customers_orders` on true where `customers`.`email` = ? limit ? [ 'maya@example.com', 1 ]
Maya [ '#3 shipped (2 items)', '#8 shipped (1 items)' ]
  SQL: insert into `customers` (`id`, `name`, `email`, `city`, `created_at`) values (default, ?, ?, default, ?) [ 'Ava', 'ava@example.com', '2026-10-16 16:22:35.973' ]
insertId: 9
  SQL: delete from `customers` where `customers`.`id` = ? [ 9 ]
What’s happening
  1. The builder query is the SQL you'd write by hand, clause for clause. sql`…` is Drizzle's tagged template for expressions: interpolated columns become quoted identifiers and interpolated values become parameters. .mapWith(Number) converts the DECIMAL sum from string to number, which is why it prints 1656 rather than '1656.00' (Zara's total now includes the Sequelize order of two chargers).
  2. The relational query returned Maya with her orders and their items in one SQL statement: a LEFT JOIN LATERAL subquery per level, with JSON_ARRAYAGG building the nested arrays on the server. LATERAL derived tables need MySQL 8.0.14+.
  3. Drizzle, Prisma and Sequelize produce three different shapes for the same nested read: one JSON-aggregating query (Drizzle), one batched query per level (Prisma), one flat join folded in JavaScript (Sequelize). Any of them is fine; looping with a query per row is not.
  4. insert().values() resolves to the mysql2 result header, so the new id is result.insertId (9 here: earlier examples' failed and deleted inserts used up 7 and 8, since AUTO_INCREMENT values are never reused). Columns you didn't pass become default.

#Migrations

Added

A migration is a versioned, ordered change to the database schema — "create customers", "add phone", "add an index" — kept in the repository next to the code that needs it. A migration tool records which migrations have run in a table in the database itself, so every environment (laptop, CI, staging, production) can be brought to the same version by running whatever is pending, in order. Each migration usually has an up (apply) and a down (undo).

The mental model is numbered recipe cards. The database keeps a list of the card numbers it has already followed; deploying means following the next cards in order. You never edit a card that's been followed somewhere — you add a new card.

Tools differ in who writes the cards. Imperative tools (Knex, Sequelize CLI, plain SQL files with Flyway or golang-migrate) have you write each change. Declarative tools (Prisma Migrate, drizzle-kit) compare the schema you describe with the database and generate the change for you, which you then review.

migrate.mjs
// Run Knex migrations programmatically: node migrate.mjs latest | rollback | status
import knexFactory from "knex";

const knex = knexFactory({
  client: "mysql2",
  connection: { host: "127.0.0.1", port: 3307, user: "root", password: "study", database: "ch11_mig" },
  migrations: { directory: new URL("./migrations", import.meta.url).pathname, loadExtensions: [".mjs"] },
});
const cmd = process.argv[2];
try {
  if (cmd === "latest") {
    const [batch, files] = await knex.migrate.latest();
    console.log(`batch ${batch}:`, files.length ? files.map((f) => f.split("/").pop()) : "already up to date");
  } else if (cmd === "rollback") {
    const [batch, files] = await knex.migrate.rollback();
    console.log(`rolled back batch ${batch}:`, files.map((f) => f.split("/").pop()));
  } else {
    const [done, pending] = await knex.migrate.list();
    console.log("applied:", done.map((m) => m.name), "pending:", pending.map((m) => m.file));
  }
  console.log("customers columns:", (await knex("information_schema.COLUMNS")
    .where({ TABLE_SCHEMA: "ch11_mig", TABLE_NAME: "customers" }).orderBy("ORDINAL_POSITION").pluck("COLUMN_NAME")).join(", ") || "(no table)");
} catch (err) {
  console.log("migration failed:", err.code ?? "", err.sqlMessage ?? err.message);
} finally {
  await knex.destroy();
}

// migrations/20261016090000_create_customers.mjs
//   up:   createTable("customers", id, name, email unique, created_at)    down: dropTable("customers")
// migrations/20261016091000_add_phone_and_city.mjs
//   up:   alterTable: add phone VARCHAR(20) NULL, city VARCHAR(50) NULL    down: drop both columns
Result
$ node migrate.mjs status
applied: [] pending: [
  '20261016090000_create_customers.mjs',
  '20261016091000_add_phone_and_city.mjs'
]
customers columns: (no table)
$ node migrate.mjs latest
batch 1: [
  '20261016090000_create_customers.mjs',
  '20261016091000_add_phone_and_city.mjs'
]
customers columns: id, name, email, created_at, phone, city
$ node migrate.mjs latest
batch 1: already up to date
customers columns: id, name, email, created_at, phone, city
mysql> SELECT id, name, batch FROM knex_migrations;
+----+---------------------------------------+-------+
| id | name                                  | batch |
+----+---------------------------------------+-------+
|  1 | 20261016090000_create_customers.mjs   |     1 |
|  2 | 20261016091000_add_phone_and_city.mjs |     1 |
+----+---------------------------------------+-------+
$ node migrate.mjs rollback
rolled back batch 1: [
  '20261016091000_add_phone_and_city.mjs',
  '20261016090000_create_customers.mjs'
]
customers columns: (no table)
$ node migrate.mjs latest
batch 1: [
  '20261016090000_create_customers.mjs',
  '20261016091000_add_phone_and_city.mjs'
]
customers columns: id, name, email, created_at, phone, city
What’s happening
  1. The two migration files (summarised in the comments) live in migrations/, named with a timestamp so they sort in the order they were written. Knex loads .mjs files because of loadExtensions. On an empty database, status lists both as pending.
  2. latest runs every pending migration's up in order and records each in the knex_migrations table with a batch number: both ran together, so both are batch 1. The table now has all six columns.
  3. Running latest again does nothing: the tool compares the files with knex_migrations and finds nothing pending. That's what makes "run migrations on every deploy" safe.
  4. rollback undoes the last batch, running each down in reverse order (drop the columns, then the table), and removes their rows from knex_migrations. Running latest again re-applies them.
  5. In production, rollback is rarely the real recovery plan — down of a DROP COLUMN can't bring the data back. Teams roll forward with a new migration, and deploy schema changes in backward-compatible steps (see Database migrations without downtime on the Node site and Online schema changes).
The half-applied migration

A third migration adds a column and an index, with a typo in the index:

JavaScript
// migrations/20261016092000_loyalty.mjs
export async function up(knex) {
  await knex.schema.alterTable("customers", (t) => {
    t.integer("loyalty_points").notNullable().defaultTo(0);
  });
  await knex.schema.alterTable("customers", (t) => {
    t.index(["loyalty_point"]);                  // typo: the column is loyalty_points
  });
}

Knex runs each migration in a transaction by default. The index fails. What state is the table in, and what happens when you fix the typo and run latest again?

Show answer

The column stays, the migration is still "pending", and the rerun fails on the column.

Result
$ node migrate.mjs latest
migration file "20261016092000_loyalty.mjs" failed
migration failed with error: alter table `customers` add index `customers_loyalty_point_index`(`loyalty_point`) - Key column 'loyalty_point' doesn't exist in table
migration failed: ER_KEY_COLUMN_DOES_NOT_EXITS Key column 'loyalty_point' doesn't exist in table
$ node migrate.mjs status
applied: [
  '20261016090000_create_customers.mjs',
  '20261016091000_add_phone_and_city.mjs'
] pending: [ '20261016092000_loyalty.mjs' ]
customers columns: id, name, email, created_at, phone, city, loyalty_points
$ node migrate.mjs latest        # after fixing the typo
migration file "20261016092000_loyalty.mjs" failed
migration failed with error: alter table `customers` add `loyalty_points` int not null default '0' - Duplicate column name 'loyalty_points'
migration failed: ER_DUP_FIELDNAME Duplicate column name 'loyalty_points'
What’s happening
  1. The first ALTER TABLE adds loyalty_points. In MySQL, DDL statements cause an implicit commit (see Autocommit and implicit commits): the column is committed the moment it's created, whatever transaction Knex opened.
  2. The second ALTER TABLE fails (ER_KEY_COLUMN_DOES_NOT_EXITS — MySQL's own spelling). Knex rolls back its transaction, which undoes nothing that matters, and doesn't record the migration in knex_migrations.
  3. So the database is half-migrated: status says the migration is pending, but the table already has loyalty_points.
  4. After fixing the typo, latest runs the migration from the top, and the first statement now fails with Duplicate column name 'loyalty_points'. The only ways out are a manual ALTER TABLE … DROP COLUMN or editing the migration to skip what exists.
  5. PostgreSQL has transactional DDL, so there the whole migration would roll back cleanly. On MySQL, write one DDL statement per migration, or make each step idempotent (if (!(await knex.schema.hasColumn("customers", "loyalty_points")))), and test migrations against a copy of production before running them there.
AdvancedPractice: running migrations safely
AdvancedDeclarative: Prisma migrate diff

With Prisma you edit the schema and let the tool write the SQL. Adding phone String? @db.VarChar(20) to customers and @@index([status, ordered_at]) to orders in a copy of the introspected schema, then asking for the difference against the live database:

Shell
npx prisma migrate diff --from-config-datasource --to-schema prisma/next.prisma --script
Result
Loaded Prisma config from prisma.config.ts.

-- AlterTable
ALTER TABLE `customers` ADD COLUMN `phone` VARCHAR(20) NULL;

-- DropTable
DROP TABLE `big_log`;

-- CreateIndex
CREATE INDEX `orders_status_ordered_at_idx` ON `orders`(`status`, `ordered_at`);
What’s happening
  1. migrate diff compares two states — here the database behind prisma.config.ts and the edited schema file — and prints the SQL that turns one into the other. prisma migrate dev does the same, saves it as migrations/<timestamp>_<name>/migration.sql and applies it (it also needs a temporary "shadow" database, which it creates on MySQL if the account may).
  2. The two intended changes are there: the nullable column and the composite index, with a generated index name.
  3. And one unintended change: DROP TABLE big_log. That table was created for the streaming example after the schema was introspected, so it isn't in the schema file, and a declarative tool's job is to make the database match the file exactly. Applied blindly, this migration would delete 500,000 rows.
  4. Declarative tools are only as safe as your review: read every generated migration before committing it, keep tables the ORM doesn't own out of its database (or declare them), and be especially wary of renames, which diff tools usually see as drop-plus-add.

#The N+1 query problem

Added

The N+1 problem is running one query to get a list of N things, then one more query for each of them: 1 query for the customers, then N queries for each customer's orders. Each query is fast, so it looks fine in development with five rows. In production with a thousand rows it's a thousand round trips, and the page is slow for reasons no single query explains.

The mental model is shopping with a list but one trip per item: you drive to the shop, buy milk, drive home, check the list, drive back for bread. Every trip is quick; the problem is the trips. The fix is always the same idea — fetch the related rows in bulk: one join, or one WHERE customer_id IN (…) for all of them.

It's most common with ORMs, because lazy loading (customer.getOrders()) hides the query inside what looks like a property access, and with GraphQL resolvers, which resolve each field per object. It also happens in plain SQL code, and inside the database itself (a stored function called per row, as in Stored functions).

This example counts queries on the server, using performance_schema, against a copy of the sample schema with 1,000 customers and 2,990 orders:

n-plus-one.mjs
// "List every customer with their number of orders" — three ways, counted on the server.
import mysql from "mysql2/promise";
import { sequelize, sqlLog, Customer, Order } from "./models.mjs";   // run with DB=ch11_n1

const admin = await mysql.createConnection({ host: "127.0.0.1", port: 3307, user: "root", password: "study" });
const serverCount = async () => Number((await admin.query(
  "SELECT SUM(COUNT_STAR) AS n FROM performance_schema.events_statements_summary_by_digest WHERE SCHEMA_NAME = 'ch11_n1'"))[0][0].n);

async function measure(label, fn) {
  const before = serverCount(), logBefore = sqlLog.length;
  const b = await before;
  const t0 = performance.now();
  const result = await fn();
  const ms = performance.now() - t0;
  console.log(`${label.padEnd(26)} ${String(sqlLog.length - logBefore).padStart(5)} queries logged, ` +
    `${String((await serverCount()) - b).padStart(5)} on the server, ${ms.toFixed(0).padStart(4)} ms ->`,
    result.slice(0, 2), `… (${result.length} customers)`);
}

await Customer.count();                                                       // warm up the pool

await measure("1. lazy loading (N+1)", async () => {
  const customers = await Customer.findAll({ order: [["id", "ASC"]] });       // 1 query
  const out = [];
  for (const c of customers) {
    const orders = await c.getOrders();                                       // +1 query per customer
    out.push(`${c.name}: ${orders.length}`);
  }
  return out;
});

await measure("2. eager include (JOIN)", async () => {
  const customers = await Customer.findAll({ include: [Order], order: [["id", "ASC"]] });
  return customers.map((c) => `${c.name}: ${c.Orders.length}`);
});

await measure("3. batched IN (...)", async () => {
  const customers = await Customer.findAll({ order: [["id", "ASC"]], raw: true });
  const orders = await Order.findAll({ where: { customerId: customers.map((c) => c.id) }, raw: true });
  const byCustomer = Map.groupBy(orders, (o) => o.customerId);
  return customers.map((c) => `${c.name}: ${byCustomer.get(c.id)?.length ?? 0}`);
});

await measure("4. let SQL count it", async () => {
  const [rows] = await sequelize.query(
    "SELECT c.name, COUNT(o.id) AS n FROM customers c LEFT JOIN orders o ON o.customer_id = c.id GROUP BY c.id, c.name ORDER BY c.id");
  return rows.map((r) => `${r.name}: ${r.n}`);
});

console.log("one of the lazy queries:", sqlLog.find((s) => s.includes("`customer_id` = ")));
await admin.end();
await sequelize.close();

// 1. lazy loading (N+1)       1001 queries logged,  1001 on the server,  162 ms -> [ 'Rohit: 2', 'Maya: 2' ] … (1000 customers)
// 2. eager include (JOIN)        1 queries logged,     1 on the server,   12 ms -> [ 'Rohit: 2', 'Maya: 2' ] … (1000 customers)
// 3. batched IN (...)            2 queries logged,     2 on the server,    4 ms -> [ 'Rohit: 2', 'Maya: 2' ] … (1000 customers)
// 4. let SQL count it            1 queries logged,     1 on the server,    2 ms -> [ 'Rohit: 2', 'Maya: 2' ] … (1000 customers)
// one of the lazy queries: SELECT `id`, `status`, `ordered_at` AS `orderedAt`, `customer_id` AS `customerId` FROM `orders` AS `Order` WHERE `Order`.`customer_id` = 1;
What’s happening
  1. performance_schema.events_statements_summary_by_digest counts every statement the server executes, grouped by normalised SQL. Only these examples use the ch11_n1 schema, so the difference before and after each approach is exactly the number of queries it sent — measured by MySQL, not by the ORM's own log. The two counts agree in every row.
  2. Lazy loading: one findAll for 1,000 customers, then c.getOrders() inside the loop sends SELECT … WHERE customer_id = 1, = 2, … : 1,001 queries, 162 ms. Every await is a full round trip, one after another.
  3. Eager include turns it into one LEFT OUTER JOIN query: 12 ms. Batched loading — all customers, then all their orders with one WHERE customer_id IN (…), grouped in JavaScript with Map.groupBy — takes 2 queries and 4 ms; that's what DataLoader does in GraphQL servers, and what Prisma does for include.
  4. If you only need the counts, let the database count: one GROUP BY query, 2 ms, and no order rows transferred at all.
  5. 162 ms is on localhost, where a round trip costs about 0.15 ms. Against a database across a network with a 1 ms round trip, the same 1,001 sequential queries would take over a second — the N+1 cost scales with latency × N, which is why it's invisible in development and painful in production.
AdvancedPractice: how to find N+1 queries
AdvancedThe same problem in Prisma
prisma-n1.ts
// The same "customers with order counts" in Prisma: a loop, include, and same-tick findUnique batching.
import { PrismaMariaDb } from "@prisma/adapter-mariadb";
import { PrismaClient } from "./generated/client.ts";

const adapter = new PrismaMariaDb({ host: "127.0.0.1", port: 3307, user: "root", password: "study", database: "ch11_n1", connectionLimit: 5 });
const prisma = new PrismaClient({ adapter, log: [{ emit: "event", level: "query" }] });
let queries = 0;
const seen: string[] = [];
prisma.$on("query", (e) => { queries++; seen.push(e.query); });

async function measure(label: string, fn: () => Promise<string[]>) {
  queries = 0; seen.length = 0;
  const t0 = performance.now();
  const out = await fn();
  console.log(`${label.padEnd(30)} ${String(queries).padStart(5)} queries, ${(performance.now() - t0).toFixed(0).padStart(4)} ms ->`, out.slice(0, 2), `(${out.length})`);
}
await prisma.customers.count();

await measure("1. loop with await (N+1)", async () => {
  const customers = await prisma.customers.findMany({ orderBy: { id: "asc" } });
  const out: string[] = [];
  for (const c of customers) out.push(`${c.name}: ${(await prisma.orders.findMany({ where: { customer_id: c.id } })).length}`);
  return out;
});
await measure("2. include", async () => {
  const customers = await prisma.customers.findMany({ orderBy: { id: "asc" }, include: { orders: true } });
  return customers.map((c) => `${c.name}: ${c.orders.length}`);
});
console.log("   include ran:", seen.map((q) => q.replace(/SELECT .* FROM/, "SELECT … FROM").slice(0, 90)));
await measure("3. findUnique in the same tick", async () => {
  const customers = await prisma.customers.findMany({ orderBy: { id: "asc" }, select: { id: true, name: true } });
  const orders = await Promise.all(customers.map((c) => prisma.customers.findUnique({ where: { id: c.id } }).orders()));
  return customers.map((c, i) => `${c.name}: ${orders[i]?.length}`);
});
console.log("   batched into:", seen.slice(1).map((q) => q.replace(/SELECT .* FROM/, "SELECT … FROM").slice(0, 90)));
await measure("4. _count", async () => {
  const customers = await prisma.customers.findMany({ orderBy: { id: "asc" }, select: { name: true, _count: { select: { orders: true } } } });
  return customers.map((c) => `${c.name}: ${c._count.orders}`);
});
console.log("   _count ran:", seen.map((q) => q.slice(0, 160)));
await prisma.$disconnect();
Result
1. loop with await (N+1)        1001 queries,  307 ms -> [ 'Rohit: 2', 'Maya: 2' ] (1000)
2. include                         2 queries,   16 ms -> [ 'Rohit: 2', 'Maya: 2' ] (1000)
   include ran: [
  'SELECT … FROM `customers` WHERE 1=1 ORDER BY `customers`.`id` ASC',
  'SELECT … FROM `orders` WHERE `orders`.`customer_id` IN (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,'
]
3. findUnique in the same tick     3 queries,   67 ms -> [ 'Rohit: 2', 'Maya: 2' ] (1000)
   batched into: [
  'SELECT … FROM `customers` WHERE `customers`.`id` IN (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?',
  'SELECT … FROM `orders` WHERE `orders`.`customer_id` IN (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,'
]
4. _count                          1 queries,    3 ms -> [ 'Rohit: 2', 'Maya: 2' ] (1000)
   _count ran: [
  'SELECT `customers`.`id`, `customers`.`name`, `customers`.`email`, `customers`.`city`, `customers`.`created_at`, COALESCE(`aggr_selection_0_orders`.`_aggr_count_orders`, 0) AS `_aggr_count_orders` FROM `customers` LEFT JOIN (SEL'
]
What’s happening
  1. A typed client doesn't prevent N+1: awaiting findMany inside a loop is 1,001 queries, 307 ms.
  2. include is 2 queries: the customers, then all their orders with one IN (…) list of 1,000 placeholders.
  3. Prisma has a built-in dataloader: findUnique calls made in the same tick (here, all started by Promise.all before any is awaited) are batched into one WHERE id IN (…), and the fluent .orders() relation calls are batched too: 1,000 calls become 3 queries. The same calls awaited one by one in a loop would not be batched, because each would be in its own tick.
  4. _count pushes the counting into SQL: one query with a LEFT JOIN on a grouped subquery, 3 ms. When you need aggregates, ask for aggregates.

#Connection limits in serverless

Added

On a long-running server, one Node process holds one pool, and you size it once. Serverless functions (AWS Lambda, Vercel and Netlify functions, Cloud Run with many instances) break that model: the platform starts as many instances as there are concurrent requests, each instance is a separate process with its own pool, and an instance that has finished a request is frozen with its connections still open, waiting for the next one. A traffic spike of 500 concurrent requests means up to 500 instances, each wanting at least one connection, against a database whose max_connections might be 150.

The mental model is a restaurant (the database) with 150 seats, and a tour company (the platform) that sends a separate taxi for every tourist. Each taxi keeps a reserved seat even while its tourist is out sightseeing, and when the 151st taxi arrives, the door is shut.

This simulates 30 concurrent function instances — 30 separate pools in one Node process — against this chapter's separate MySQL server with max_connections = 20:

serverless.mjs
// 30 concurrent "function instances", each with its own module-level pool, against max_connections = 20.
import mysql from "mysql2/promise";
import { setTimeout as sleep } from "node:timers/promises";

const server = { host: "127.0.0.1", port: 3413, user: "web", password: "Web-pass-123", database: "ptdemo" };
const admin = await mysql.createConnection({ host: "127.0.0.1", port: 3413, user: "root", password: "study" });
const threads = async () => (await admin.query("SHOW GLOBAL STATUS LIKE 'Threads_connected'"))[0][0].Value;

function makeInstance(poolOptions) {
  const pool = mysql.createPool({ ...server, ...poolOptions });     // created once per instance, at cold start
  return {
    pool,
    async handler() {                                               // one invocation
      const [[row]] = await pool.query("SELECT COUNT(*) AS n FROM log_parted WHERE id < 1000");
      return row.n;
    },
  };
}

for (const [label, options] of [
  ["default pool (connectionLimit 10)", {}],
  ["connectionLimit 1, idleTimeout 1 s", { connectionLimit: 1, maxIdle: 0, idleTimeout: 1000 }],
]) {
  const instances = Array.from({ length: 30 }, () => makeInstance(options));
  const results = await Promise.allSettled(instances.map((i) => i.handler()));
  const failed = results.filter((r) => r.status === "rejected");
  console.log(`${label}: ${results.length - failed.length} ok, ${failed.length} failed` +
    (failed.length ? ` (${failed[0].reason.code})` : "") + `; server threads right after: ${await threads()}`);
  await sleep(1500);                                                 // instances sit idle ("frozen") between invocations
  console.log(`  1.5 s later, idle instances still hold: ${await threads()} connections`);
  await Promise.all(instances.map((i) => i.pool.end()));
}
await admin.end();

// default pool (connectionLimit 10): 19 ok, 11 failed (ER_CON_COUNT_ERROR); server threads right after: 20
//   1.5 s later, idle instances still hold: 20 connections
// connectionLimit 1, idleTimeout 1 s: 16 ok, 14 failed (ER_CON_COUNT_ERROR); server threads right after: 17
//   1.5 s later, idle instances still hold: 1 connections
What’s happening
  1. Each "instance" creates its pool at module scope, as the usual advice says ("create the pool outside the handler so warm invocations reuse it"). One request per instance means each pool opens one connection.
  2. Thirty concurrent requests need thirty connections. The server allows 20, one of which is the monitoring connection, so 11 requests fail with ER_CON_COUNT_ERROR — users see errors although the database is barely working. Threads_connected sits at the cap, 20.
  3. After the burst, the instances are idle but their pools keep the connections: 20 are still open 1.5 s later. Real frozen Lambdas behave worse than this simulation: a frozen process runs no timers at all, so its pool can't even close idle connections; they stay until the function is reclaimed or the server's wait_timeout (8 hours by default) closes them.
  4. connectionLimit: 1, maxIdle: 0, idleTimeout: 1000 makes each instance hold at most one connection and give it back when idle (here it worked, down to just the monitoring connection, because this process isn't frozen). But the burst still failed — 14 errors this run, the exact number depending on timing — because 30 concurrent instances still need 30 connections at once. Smaller pools reduce waste; they don't change the arithmetic.
  5. The real fixes change the arithmetic: a connection proxy in front of the database that multiplexes many client connections onto a few server connections (Amazon RDS Proxy, ProxySQL, PlanetScale's and Neon's poolers, Cloudflare Hyperdrive), or capping the function's concurrency (Lambda reserved concurrency) below the database's connection budget, or an HTTP-based database API. None of those were run here.
AdvancedAlternative: a connection per invocation
AdvancedOptions: serverless-friendly databases and drivers

SQL and MySQL written fresh; MongoDB built from Rohit’s “mongoDB interview” doc. Every query was run on MySQL 9.7, MongoDB 9.0, Redis 8.10 or PostgreSQL 18.

RohitDownloads · All chapters

Sync progress across devices

Type the same private phrase on your Mac and your phone, and your Learned ticks follow you between them — on every study site.

The phrase never leaves this device: only a fingerprint of it is sent, and the server stores a fingerprint of that. Anyone who knows the phrase could see or change your ticks, so pick something you don’t use elsewhere.

Sync is on in this browser.

Sync ID

This ID must be the same on every device. If another device shows a different one, its phrase is different (capital letters count): tap “Turn off here” on it and type the phrase again exactly.