#How a pipeline works
find() can filter, project and sort, but it can't compute across documents: it can't total an order, count orders per customer or join two collections. For that MongoDB has the aggregation pipeline: an array of stages you pass to db.collection.aggregate([...]). Documents from the collection enter the first stage; whatever that stage outputs becomes the input of the next stage; the output of the last stage is the result you get back, as a cursor.
The mental model is a factory assembly line. Each stage is one station with one job: $match throws away documents you don't want, $group piles documents into bins and replaces each bin with a summary document, $project reshapes each document, $sort reorders the line. A station only ever sees what the station before it passed down. That's the single most useful fact about pipelines: the order of stages is the order of work, and a field that an earlier stage removed or renamed simply doesn't exist further down.
If you know SQL, the stages map closely onto clauses: $match is WHERE (or HAVING when it comes after $group), $group is GROUP BY with its aggregate functions, $project is the SELECT list, $sort/$skip/$limit are ORDER BY/OFFSET/LIMIT, $lookup is a LEFT OUTER JOIN. The difference is that in SQL the clause order you write is rearranged into a fixed logical order, while in a pipeline you write the order yourself, and you can repeat stages ($match, then $group, then $match again).
The examples on this page use the site's sample data as MongoDB collections in a database called ch19: customers, products, employees, and orders, which embed their line items (items: [{ productId, qty, unitPrice }]) instead of keeping them in a separate table. This is the "interesting example" from my notes, rewritten on that data: shipped orders per customer.
db.orders.aggregate([
{ $match: { status: "shipped" } },
{ $group: {
_id: "$customerId",
totalOrders: { $sum: 1 },
firstStatus: { $first: "$status" }
} },
{ $project: { _id: 0, customerId: "$_id", totalOrders: 1, firstStatus: 1 } },
{ $sort: { totalOrders: -1, customerId: 1 } }
])[
{ totalOrders: 2, firstStatus: 'shipped', customerId: 2 },
{ totalOrders: 1, firstStatus: 'shipped', customerId: 1 },
{ totalOrders: 1, firstStatus: 'shipped', customerId: 3 }
]$matchruns first and keeps only orders whosestatusis"shipped": 8 orders go in, 4 come out (_id1, 3, 5 and 8, belonging to customers 1, 2, 3 and 2).$groupsorts those 4 documents into bins by the value of_id: "$customerId". The$in"$customerId"means "the value of the field", not the literal string. Three bins form: customer 1 (1 order), customer 2 (2 orders), customer 3 (1 order).- Each bin becomes one new document.
totalOrders: { $sum: 1 }adds 1 per document in the bin (a count);firstStatus: { $first: "$status" }takes the status of the first document that reached the bin. After the$matchevery status is"shipped", so this field is constant here; it's in my notes to show that a grouped document has no fields except_idand the ones you compute. $projectreshapes each summary:_id: 0drops the group key,customerId: "$_id"copies it into a properly named field, and1keeps the two computed fields. Fields kept with1come first in their existing order, then the new computed field, which is whycustomerIdprints last.$sortorders bytotalOrdersdescending, thencustomerIdascending to break the tie between customers 1 and 3. Without that second key, the order of tied documents isn't guaranteed.
Running the first stages on their own shows what each station hands to the next. This is also the best way to debug a pipeline: cut it off after stage n and look.
db.orders.aggregate([ { $match: { status: "shipped" } }, { $project: { customerId: 1, status: 1 } } ])
db.orders.aggregate([
{ $match: { status: "shipped" } },
{ $group: { _id: "$customerId", totalOrders: { $sum: 1 }, firstStatus: { $first: "$status" } } }
])[
{ _id: 1, customerId: 1, status: 'shipped' },
{ _id: 3, customerId: 2, status: 'shipped' },
{ _id: 5, customerId: 3, status: 'shipped' },
{ _id: 8, customerId: 2, status: 'shipped' }
]
[
{ _id: 3, totalOrders: 1, firstStatus: 'shipped' },
{ _id: 1, totalOrders: 1, firstStatus: 'shipped' },
{ _id: 2, totalOrders: 2, firstStatus: 'shipped' }
]- After
$match, the documents are still whole orders (projected here only to keep the output short). The_idis the order id. - After
$group, they are completely different documents:_idis now the customer id, and the only other fields are the accumulators. The originalstatus,itemsandorderedAtfields are gone unless an accumulator carried them through. - The groups came out in the order 3, 1, 2.
$groupdoesn't sort its output; the order depends on how the server's hash table happened to fill. If the order matters, add a$sortafter it, as the main example does.
aggregate() usually runs on a collection, but db.aggregate() runs a pipeline that doesn't start from a collection. $documents (MongoDB 5.1+) feeds literal documents in, which is handy for trying out an expression without creating test data:
db.aggregate([ { $documents: [ { x: 1 }, { x: 2 } ] }, { $set: { double: { $multiply: ["$x", 2] } } } ])[ { x: 1, double: 2 }, { x: 2, double: 4 } ]$documentsmust be the first stage of adb.aggregate()call; it emits the two literal documents into the pipeline.$set(an alias of$addFields) computesdoublefrom each document'sx:1 → 2,2 → 4.- The same
db.aggregate()form is used for admin pipelines such as$currentOpand$listLocalSessions, which read server state rather than a collection.
aggregate() returns a cursor, just like find(). mongosh prints the first batch for you, but in application code you iterate it or call .toArray(). Results larger than one batch (101 documents in the first batch by default, then up to 16 MB per batch) are fetched lazily as you iterate. Every single document a pipeline returns must fit in the 16 MB BSON limit; $facet is where people usually hit that.
db.orders.aggregate([
{ $group: { _id: "$customerId", totalOrders: { $sum: 1 } } },
{ $match: { status: "shipped" } }
]).toArray()Show answer
[]: an empty array, with no error.
$groupruns first over all 8 orders and outputs 5 documents shaped{ _id: <customerId>, totalOrders: <n> }.- Those documents have no
statusfield:$grouponly outputs_idand the accumulators you asked for. $match: { status: "shipped" }compares a missing field with"shipped", which is false for every document, so nothing survives.- MongoDB doesn't warn you, because a missing field is a perfectly legal thing to match on. The fix is to put the
$matchfirst (it then acts like SQLWHERE) or, if you meant to filter groups, match on a field the group produced ({ totalOrders: { $gte: 2 } }, which acts likeHAVING).