mongodb aggregation framework in action !
TRANSCRIPT
![Page 1: MongoDB Aggregation Framework in action !](https://reader034.vdocuments.us/reader034/viewer/2022042602/55a45dbd1a28ab7b6f8b4642/html5/thumbnails/1.jpg)
MongoDBAggregationFrameworkin action !
MongoDB User Group - Nantesmardi 20 janvier 2015
@sebprunier
![Page 2: MongoDB Aggregation Framework in action !](https://reader034.vdocuments.us/reader034/viewer/2022042602/55a45dbd1a28ab7b6f8b4642/html5/thumbnails/2.jpg)
/me
Sébastien Prunier
Développeur chez SERLI
@sebprunier
http://blog.sebprunier.com
![Page 3: MongoDB Aggregation Framework in action !](https://reader034.vdocuments.us/reader034/viewer/2022042602/55a45dbd1a28ab7b6f8b4642/html5/thumbnails/3.jpg)
Sondage (1/2)
?
![Page 4: MongoDB Aggregation Framework in action !](https://reader034.vdocuments.us/reader034/viewer/2022042602/55a45dbd1a28ab7b6f8b4642/html5/thumbnails/4.jpg)
Sondage (2/2)
?Aggregation Framework
![Page 5: MongoDB Aggregation Framework in action !](https://reader034.vdocuments.us/reader034/viewer/2022042602/55a45dbd1a28ab7b6f8b4642/html5/thumbnails/5.jpg)
MongoDB Aggregation Framework
Opérations simples
Map-Reduce
Aggregation Pipeline
![Page 6: MongoDB Aggregation Framework in action !](https://reader034.vdocuments.us/reader034/viewer/2022042602/55a45dbd1a28ab7b6f8b4642/html5/thumbnails/6.jpg)
Opérations simple
db.collection.count(filter?)
db.collection.distinct(attribute)
db.collection.group(query)
![Page 7: MongoDB Aggregation Framework in action !](https://reader034.vdocuments.us/reader034/viewer/2022042602/55a45dbd1a28ab7b6f8b4642/html5/thumbnails/7.jpg)
Map-Reduce
db.collection.mapreduce( mapFunction, reduceFunction, options)
![Page 8: MongoDB Aggregation Framework in action !](https://reader034.vdocuments.us/reader034/viewer/2022042602/55a45dbd1a28ab7b6f8b4642/html5/thumbnails/8.jpg)
Aggregation Pipeline
db.collection.aggregate( [ stage1, stage2, ..., stageN ])
![Page 9: MongoDB Aggregation Framework in action !](https://reader034.vdocuments.us/reader034/viewer/2022042602/55a45dbd1a28ab7b6f8b4642/html5/thumbnails/9.jpg)
Aggregation Pipeline
json
json
json
json
$match
$unwind
$group
![Page 10: MongoDB Aggregation Framework in action !](https://reader034.vdocuments.us/reader034/viewer/2022042602/55a45dbd1a28ab7b6f8b4642/html5/thumbnails/10.jpg)
Opérateurs vs. SQLSQL Terms, Functions, and Concepts MongoDB Aggregation Operators
WHERE $match
GROUP BY $group
HAVING $match
SELECT $project
ORDER BY $sort
LIMIT $limit
SUM() $sum
COUNT() $sum
join N/A~ $unwind / arrays
![Page 11: MongoDB Aggregation Framework in action !](https://reader034.vdocuments.us/reader034/viewer/2022042602/55a45dbd1a28ab7b6f8b4642/html5/thumbnails/11.jpg)
Dataset
campings
marvel geeks
![Page 12: MongoDB Aggregation Framework in action !](https://reader034.vdocuments.us/reader034/viewer/2022042602/55a45dbd1a28ab7b6f8b4642/html5/thumbnails/12.jpg)
Campings
{"_id" : ObjectId("54bd347880d46d750f7a48c2"),"raking_date" : ISODate("2013-06-06T22:00:00Z"),"publication_date" : ISODate("2013-06-06T22:00:00Z"),"typology" : "CAMPING","ranking" : "2 étoiles","category" : "Tourisme","mention" : "-","commercial_name" : "CAMPING LA BERGERIE","address" : "4231 route de Giens","zip_code" : "83400","city" : "HYÈRES","phone" : "494589175","email" : "[email protected]","website" : "www.camping-de-la-bergerie.com","visit_type" : "-","capacity" : "-","number_of_rooms" : "-","number_of_spaces" : "55","number_of_housing_units" : "-","number_of_chambers" : "-"
}data.gouv.fr
![Page 13: MongoDB Aggregation Framework in action !](https://reader034.vdocuments.us/reader034/viewer/2022042602/55a45dbd1a28ab7b6f8b4642/html5/thumbnails/13.jpg)
Campings
“ Déterminer le nombre de campings pour chaque niveau de classement
(1 étoile, 2 étoiles, etc…) ”
$group
![Page 14: MongoDB Aggregation Framework in action !](https://reader034.vdocuments.us/reader034/viewer/2022042602/55a45dbd1a28ab7b6f8b4642/html5/thumbnails/14.jpg)
Campings
db.campings.aggregate([
{$group : {_id : "$ranking", total : {$sum : 1}}}
])
{ "_id" : "5 étoiles", "total" : 177 }
{ "_id" : "4 étoiles", "total" : 940 }
{ "_id" : "3 étoiles", "total" : 2224 }
{ "_id" : "2 étoiles", "total" : 1681 }
{ "_id" : "1 étoile", "total" : 389 }
![Page 15: MongoDB Aggregation Framework in action !](https://reader034.vdocuments.us/reader034/viewer/2022042602/55a45dbd1a28ab7b6f8b4642/html5/thumbnails/15.jpg)
Campings
“ Top 5 des villes avec le plus de campings ”
$group$sort $limit
$project
![Page 16: MongoDB Aggregation Framework in action !](https://reader034.vdocuments.us/reader034/viewer/2022042602/55a45dbd1a28ab7b6f8b4642/html5/thumbnails/16.jpg)
Campingsdb.campings.aggregate([
{$group : {_id : "$city", total : {$sum : 1}}},
{$sort : {total : -1}},
{$limit : 5},
{$project: {_id: 0, city : "$_id", total: 1}}
])
{ "city" : "ARGELÈS-SUR-MER" , "total" : 29 }
{ "city" : "AGDE", "total" : 23 }
{ "city" : "VIAS", "total" : 20 }
{ "city" : "SAINT-JEAN-DE-MONTS" , "total" : 20 }
{ "city" : "LES MATHES", "total" : 17 }
![Page 17: MongoDB Aggregation Framework in action !](https://reader034.vdocuments.us/reader034/viewer/2022042602/55a45dbd1a28ab7b6f8b4642/html5/thumbnails/17.jpg)
Campings
“ Nombre de villes avec seulement un camping ”
$group$match
$project
![Page 18: MongoDB Aggregation Framework in action !](https://reader034.vdocuments.us/reader034/viewer/2022042602/55a45dbd1a28ab7b6f8b4642/html5/thumbnails/18.jpg)
Campingsdb.campings.aggregate([
{$group : {_id : "$city", total : {$sum : 1}}},
{$match : {total : 1}},
{$group: {_id: null, count: {$sum: 1 }}},
{$project: {_id: 0, count: 1}}
])
{ "count" : 2802 }
![Page 19: MongoDB Aggregation Framework in action !](https://reader034.vdocuments.us/reader034/viewer/2022042602/55a45dbd1a28ab7b6f8b4642/html5/thumbnails/19.jpg)
Marvel
{"_id" : 4,"title" : "Rogue (2004) #5","description" : "...","format" : "Comic","creators" : {
"available" : 6,"items" : [...]
},"characters" : {
"available" : 1,"items" : [
{"name" : "Rogue"
}]
},...
}
developer.marvel.com
![Page 20: MongoDB Aggregation Framework in action !](https://reader034.vdocuments.us/reader034/viewer/2022042602/55a45dbd1a28ab7b6f8b4642/html5/thumbnails/20.jpg)
Marvel
“ Top 5 des personnages apparaissant dans le plus de bandes dessinées ”
$match $project$unwind$group
$sort $limit
![Page 21: MongoDB Aggregation Framework in action !](https://reader034.vdocuments.us/reader034/viewer/2022042602/55a45dbd1a28ab7b6f8b4642/html5/thumbnails/21.jpg)
Marvel
“Deconstructs an array field from the input documents to output a document for each element.”
$unwind
{ "name" : "john doe", "tags" : ["A", "B", "C"]}
{ "name" : "john doe", "tags" : "A"}
{ "name" : "john doe", "tags" : "B"}
{ "name" : "john doe", "tags" : "C"}
$unwind
![Page 22: MongoDB Aggregation Framework in action !](https://reader034.vdocuments.us/reader034/viewer/2022042602/55a45dbd1a28ab7b6f8b4642/html5/thumbnails/22.jpg)
Marveldb.comics.aggregate([
{$match : {"characters.returned" : {$gt : 0}}},
{$project : {title : 1, characters : 1}},
{$unwind : "$characters.items"},
{$group : {_id : "$characters.items.name", total : {$sum : 1}}},
{$sort : {total : -1}},
{$limit : 5}
])
{"_id": "Spider-Man","total": 2413}
{"_id": "X-Men","total": 2320}
{"_id": "Iron Man","total": 1904}
{"_id": "Wolverine","total": 1594}
{"_id": "Captain America" ,"total": 1367}
![Page 23: MongoDB Aggregation Framework in action !](https://reader034.vdocuments.us/reader034/viewer/2022042602/55a45dbd1a28ab7b6f8b4642/html5/thumbnails/23.jpg)
Marvel
“ Créez la collection des personnages à partir de la collections des bandes
dessinées ”
$match $project$unwind $group
$out
![Page 24: MongoDB Aggregation Framework in action !](https://reader034.vdocuments.us/reader034/viewer/2022042602/55a45dbd1a28ab7b6f8b4642/html5/thumbnails/24.jpg)
Marvel
db.comics.aggregate([
{$match : {"characters.returned" : {$gt : 0}}},
{$project : {title : 1, characters : 1}},
{$unwind : "$characters.items"},
{$group : {
_id : "$characters.items.name",
total : {$sum : 1},
comics : {$push : {id : "$_id", title : "$title"}}}
},
{$out : "characters"}
])
![Page 25: MongoDB Aggregation Framework in action !](https://reader034.vdocuments.us/reader034/viewer/2022042602/55a45dbd1a28ab7b6f8b4642/html5/thumbnails/25.jpg)
Marvel
{
"_id": "Frog-Man",
"total": 2,
"comics": [
{
"id": 38126,
"title": "Spider-Man: New York Stories (Trade Paperback)"
},
{
"id": 39753,
"title": "Spider-Island: Avengers (2011) #1"
}
]
}
![Page 26: MongoDB Aggregation Framework in action !](https://reader034.vdocuments.us/reader034/viewer/2022042602/55a45dbd1a28ab7b6f8b4642/html5/thumbnails/26.jpg)
Geeks
{"_id" : ObjectId("54bd3f1a84c2c169160a88d5"),"nom" : "Mark Zuckerberg","ville" : "Palo Alto","likes" : [
"Facebook","Tongues","PHP"
],"imageUrl" : "static/GIT_HASH/img/geek5.jpg","location" : {
"type" : "Point","coordinates" : [
-122.1430195,37.4418834
]}
}
code-story.net
![Page 27: MongoDB Aggregation Framework in action !](https://reader034.vdocuments.us/reader034/viewer/2022042602/55a45dbd1a28ab7b6f8b4642/html5/thumbnails/27.jpg)
Geeks
“ Rechercher ce qui rassemble le plus de geeks autour de Paris ”
$geoNear$project $unwind $group $match
$sort
![Page 28: MongoDB Aggregation Framework in action !](https://reader034.vdocuments.us/reader034/viewer/2022042602/55a45dbd1a28ab7b6f8b4642/html5/thumbnails/28.jpg)
Geeks
db.geeks.aggregate([ { $geoNear: { near: { type: "Point", coordinates: [2.352241, 48.856638] }, distanceField: "distance", spherical: true, maxDistance: 10000 } }, ...])
![Page 29: MongoDB Aggregation Framework in action !](https://reader034.vdocuments.us/reader034/viewer/2022042602/55a45dbd1a28ab7b6f8b4642/html5/thumbnails/29.jpg)
Geeks
{ "_id" : "java", "total" : 8, "friends" : [ { "nom" : "...", "distance" : 3.015741215085397 }, { "nom" : "...", "distance" : 9864.484291991206 }, ... ]}
![Page 30: MongoDB Aggregation Framework in action !](https://reader034.vdocuments.us/reader034/viewer/2022042602/55a45dbd1a28ab7b6f8b4642/html5/thumbnails/30.jpg)
Index
● Index utilisé pour certains opérateurs, en début de pipeline○ $match○ $sort○ $geoNear
● explain○ db.collection.aggregate([...],
{explain: true})
![Page 31: MongoDB Aggregation Framework in action !](https://reader034.vdocuments.us/reader034/viewer/2022042602/55a45dbd1a28ab7b6f8b4642/html5/thumbnails/31.jpg)
Index
● Optimisations
{ $sort: { age : -1 } },
{ $match: { status: 'A' } }
{ $match: { status: 'A' } },
{ $sort: { age : -1 } }
“ Optimizations are subject to change between releases. “
![Page 32: MongoDB Aggregation Framework in action !](https://reader034.vdocuments.us/reader034/viewer/2022042602/55a45dbd1a28ab7b6f8b4642/html5/thumbnails/32.jpg)
Sharding
● Découpage du pipeline en deux parties○ 1ère partie (jusqu’au premier $group ou $sort)
exécutée sur chaque shard (certains shards peuvent être exclus grâce à un $match)
○ 2ème partie exécutée sur le shard primaire à partir des résultats consolidés de la première partie
![Page 33: MongoDB Aggregation Framework in action !](https://reader034.vdocuments.us/reader034/viewer/2022042602/55a45dbd1a28ab7b6f8b4642/html5/thumbnails/33.jpg)
The end
Merci pour votre attention !
● Questions / Réponses● Partage d’expérience