MongoDB Mapreduce Tutorial – Real-time Example & Commands

In the last tutorial we had learned about atomic operations and objectid in MongoDB. Today, we are going to discuss MongoDB mapreduce tutorial, in which we will know how mapreduce will work in MongDB. Along with this, we will learn MongoDB Mapreduce example and commands.

MongoDB Mapreduce

MongoDB Mapreduce is a data processing paradigm for constricting large amount of data into useful aggregated results. Which we can use for processing large number of data.

To understand it in a more better way, let’s take these two MongoDB Mapreduce example:

MongoDB Mapreduce Example

MongoDB Mapreduce Example – 1

Here, map operation is performed to each input document. Map operation emits key-value pairs. For keys that have multiple values, MongoDB applies the reduce phase, which collects and condenses the aggregated data. Finally the result will be stored in collections.

Optionally the output of reduce function may pass through finalize function to process the results of aggregation. All map-reduce functions in MongoDB are JavaScript and run with mongod process.

They take documents of a single collection as input and perform any sorting and limiting before beginning the map stage.

MongoDB Mapreduce Example – 2

Now we will consider another example on a collection named examples that contain documents of the following type:

{
    _id: ObjectId("50a8240b927d5d8b5891743c"),
    cust_id: "a123",
    ord_date: new Date("Jan 04, 2019"),
    status: 'A',
    price: 25,
    items: [ { sku: "m", qty: 5, price: 2.5 },
          { sku: "n", qty: 5, price: 2.5 } ]
}

Define map function to process each input document:

var mapFunction1 = function() {
            emit(this.cust_id, this.price);
          };

Define the corresponding reduce function with two arguments CustId and Prices:

var reduceFunction1 = function(CustId, Prices) {
                 return Array.sum(Prices);
              };

Now perform map-reduce on all documents in the examples collection.

db.examples.mapReduce(
            mapFunction1,
            reduceFunction1,
            { out: "map_reduce_example" }
           )

This operation will give the output collection as map_reduce_example.

How to calculate order and total quantity?

Now we will calculate order and total quantity with average quantity per item for the same example.

Define map function to process each input document:

var mapFunction2 = function() {
           for (var idx = 0; idx < this.items.length; idx++) {
             var key = this.items[idx].sku;
             var value = {
                      count: 1,
                      qty: this.items[idx].qty
                    };
           emit(key, value);
           }
        };

Define the corresponding reduce function with two arguments key and ObjVals:

var reduceFunction2 = function(key, ObjVals) {
          reducedVal = { count: 0, qty: 0 };
          
          for (var idx = 0; idx < ObjVals.length; idx++) {
             reducedVal.count += ObjVals[idx].count;
             reducedVal.qty += ObjVals[idx].qty;
          }
          
          return reducedVal;
          };

Now define a finalize function with two arguments keys and reducedVal.

var finalizeFunction2 = function (keys, reducedVal) {
            reducedVal.avg = reducedVal.qty/reducedVal.count;
            return reducedVal;
          };

After this we will perform map-reduce operation on examples collection.

db.examples.mapReduce( mapFunction2,
           reduceFunction2,
           {
            out: { merge: "map_reduce_example" },
            query: { ord:
                    { $gt: new Date('26/01/2019') }
                  },
            finalize: finalizeFunction2
            }
           )

MongoDB MapReduce Command

Following are some commands, which can we use in map-reduce.

db.collection.example( 
    function() {emit(key, value);}, 
//Define map function
    function(key,values) {return reduceFunction}, { 
//Define reduce function
    out: collection,
    query: document,
    sort: document,
    limit: number
  }
)

Here in the above example it will query the collection, and then map the output documents to emit key-value pairs. We can perform this reduce operation on keys that are having multiple values.

Here, we are going to discuss the MongoDB Mapreduce commands:

1. Map: It is a JavaScript function, which is use to generate key-value pairs.
2. Reduce: A JavaScript function use for grouping all the documents which have the same key.
3. Out: It specifies the location of map-reduce query output.
4. Query: We can use it to specify the selection criteria for selecting documents.
5. Sort: Use for specify sort criteria and optional commands.
6. Limit: It specifies the number of documents to be returned. It is an optional command to be used.

Summary

Hence, we have studied about map-reduce operation in MongoDB with real-time examples and commands. In our next tutorial, we will see RockMongo abd GridFS.

If you have a query, feel free to ask in the comment session.