Tech Is Hard

Credibility = Talent x Years of experience + Proven hardcore accomplishment

Tag Archives: mongoose

Making Mongoose Keep Its Promises on the Server


So far I really like the deferred/promise model. On the client it lets me get data as early as I want to, but defer using the response until it’s convenient; maybe not until inside some event handler. I can also place a call to .fail() for any promise in a convenient place, even before I make a call for data. The Deferred object coordinates calling any registered fail function(s) if anything errors — that’s right, you can specify multiple handlers for the failure — and all the done, then and always functions that have been registered, when appropriate.

To do the same thing on the server, I found jQuery Deffered for nodejs. After requiring it

var Deferred = require( "JQDeferred" );

and with a little function in my application

function promise (query) {
  var dfd = Deferred ();

  query .run (function (err, doc) {
    if (doc) {
      if (doc.length > 0 || Object.keys(doc).length > 0) {
        console .log ("found\n", util .inspect(query));
        dfd .resolve (doc);
      }
      else {
        console .log ("not found or no rows\n", util .inspect(query)); 
        dfd .resolve (null);
      }
    }
    else {
      console .log ("error or no data object\n", util .inspect(query));
      dfd.reject (err ? err : null);
    }
  });
  return dfd .promise ();
}

Works for any Mongoose method that returns a query, like find, findOne. It’ll need to change to fold in the rest of my calls. In one function, I need to get 3 rows to verify something before I take action. Using jQuery Deferred and this promise function, dealing with 3 asynchronous calls with very little code looks like

var CloseAll = function (query, resp) {

  Deferred .when (
    promise (modelA .findOne ({modelAkey:query.body.A})), 
    promise (modelB .findOne ({modelBkey:query.body.B})), 
    promise (modelC .findOne ({modelCkey:query.body.C}))) 
    .fail (function (err)    { resp .send (_.extend (query.body, err), 500); })
    .done (function (dataA, dataB, dataC) {
          now do stuff that require all 3 things

Pretty sweet.

Functions to Create HTTP Callbacks for Mongoose


In setting up the APIs in my express application, there are many functions in which all I’m doing is sending the data passed back from my Mongoose call.  Having some functions that can set up canned mongoose callback functions is handy.  There are different functions to result in the response codes I want, depending on the context — am I getting or saving data?  The helper functions return an appropriate function to pass to Mongoose:

/**
 * Returns a foobar
 */
var getFoobar = function (q, r) {
  Foomodel
    .findById (q.params.id)
      .run (notFoundResponder ("Get foobar " + q.params.id, _.bind(r.send, r)));
};
var notFoundResponder = function (strOp, respond) {
  return function (e, doc) {
    if (doc && Object.keys(doc).length > 1) {
      console .log (strOp + " successful");
      respond (doc, 200);
    }
    else
      respond (strOp + " not found or error: " + util .inspect (doc) + " " + util .inspect (e), 404); 
  };
};
var emptyResponder = function (strOp, respond) {
  return function (e, doc) {
    if (e) 
      respond (strOp + " error: " + util .inspect (e), 500);
    else {
      console .log (strOp + " successful");      
      respond (doc, Object.keys(doc).length > 0 ? 200 : 204);
    }
  };
};
var saveResponder = function (strOp, respond) {
  return function (e, doc) {
    if (doc && Object.keys(doc).length > 1) {
      console .log (strOp + " successful");
      respond (doc, 201);
    }
    else
      respond (strOp + " not saved: " + util .inspect (doc) + " " + util .inspect (e), 500); 
  };
};
var updateResponder = function (strOp, respond) {
  return function (e, doc) {
    if (e)
      respond (strOp + " not updated: " + util .inspect (e), 500);
    else {
      console .log (strOp + " successful");
      respond (doc, 200);
    }
  };
};

Coordinating the Dependent Asynchronous Calls


I think this is better now.  nest() will take an unlimited number of function arguments, coordinating the callback from each to pass the results to the next function.  If there are errors along the way it should callback to the outermost callback.

var nest = function () {
  return _.chain (arguments) .reverse() .reduce (function (next, arg) {
    return function (doc, last) { 
      arg (doc, function (err, doc) {
        if (err)
          last (err);
        else if (next) 
          next (doc, last);
        else 
          last (null, doc);
      }); 
    };
  }, null) .value ();
};

/**
 * outermost function serving HTTP request
 */
var getFooAPI = function (q, r) {
  nest (getFoo, getSomeOther, sumResults)(q.params, 
    (function (err, doc) {
      if (err)
        r .send ("Error in getFooAPI " + util .inspect (err), 500);
      else
        r .send (doc, 201);
    }));
};

var sumResults = function (doc, handler) {
  handler (null, {"after": "doing some stuff"});
};

/**
 * Returns a Foo
 */
var getFoo = function getGame(param, handler) {
  return Foo .findById (param.id) .run (handler);
};

/**
 * Given a Foo document (from the prev find) get me the 
 * other stuff
 */
var getSomeOther = function getGamePicks (Foo, handler) {
  return Other .find ({'something.foo':Foo.key}, {stuff: 1}) .run (handler);
};

As you can see, each function to do something takes an argument that gets passed from its outer call along with a callback function.

Driving the Synchronous Mongoose Updates


This turned out to be a lot harder mentally.  And I still think I could serialize this more instead of the dynamic wrapping it does.

/**
 * Runs synchronous, dependent updates
 */
var syncUpdate = function (updates, respond) {
  return _.chain (updates) .reverse() .reduce (function update(next, arg) {
    return function (last) { 
      arg.model .update (arg.where, arg.update) .run (function (err, doc) {
        if (err)
          last (err);
        else if (next) 
          next (last);
        else 
          last (null, doc);
      });
    };
  }, null) .value ();
};

That reverse/reduce/return takes my updates and makes them inside out. We create callback wrapper code that checks for errors and if there’s another nested function to call. By passing “last”, the final callback function, to each nested invocation, we can jump all the way out on error. You can see that “next” is the accumulated function nest that we pass to reduce, and it gets evaluated inside the callback code. More of a macro type thing. If there’s something to call next, we continue to pass the “last” function along. And when there’s no more “next” to call, we call “last”.
Then I call it like:

  syncUpdate ([ 
   { model: Foo,    
     update: { $set: { field1: 'value1' } },         
     where: { _id: id } },
   { model: Bar,    
     update: { $set: { 'array.$.boolthing': true} }, 
     where: { 'array.foo': id, 'array.boolthing': false } },
   { model: FooBar, 
     update: { $set: { 'field': 'val'} },            
     where: { _id: id } } ]) 
     (
      function (err, doc) {
        if (err)
          r .send ("Error saving pick " + util .inspect (err), 500);
        else
          r .send ("saved pick " + doc, 201);
      });

One interesting difference is using syncUpdate() to return the function I want to call, which I immediately do, with my simple callback function as an argument. I think this is an improvement mechanically, from passing a success message and having the callback code be black box-ish.

I’ve got some ideas. I know of course, this should handle things other than Mongoose updates. It should be any function, and we’re getting pretty used to calling things with a callback function. In a nested situation what I want to do is call the specified function with a callback function that lets me test the result and if appropriate, pass the result on to another specified function. Or in the event of an error, at least for now, jump all de way out mon.

Used to stuff like this in assembler all the time and it’s a lot easier because you get to do anything you want. No rules to the way I allocate and initialize memory structures. In Javascript, however, I have to come up with a syntax that scales infinitely, for the nesting capability, and is flexible in the types of functions it can call.

Data-driving the Asynchronous Mongoose Updates in my Node Code


This was the first experiment to be able to ask for a sequence or set of Mongoose updates.  I wanted to be able to set up dependency and independent updates.  First I dealt with the asynchronous updates.  There is no rollback mechanism or anything, and the operation stops as soon as any of the updates fails, but we don’t get a good response unless they all succeed.  The secret sauce for the asynchronous updates was using Underscore.js’s after() function.

/**
 * Runs parallel, independent updates
 */
var asyncUpdate = function (successMsg, updates, resp) {
  var good = _.after (updates.length, function (doc) { 
    resp (successMsg + " update successful", 200); 
  });
  _.each (updates, function (arg, key, list) {
    arg.model 
      .update (arg.where, arg.update) 
      .run (function (e, doc) {
        if (e)  
          resp (arg.model.modelName + " update error: " + util .inspect (e), 500); 
        else {
          console .log (arg.model.modelName + " update successful");
          good(doc);
        }
      });
  });
};

Then call it with an array of objects of update control information.

 asyncUpdate ('Totally updated', [ 
   { model: Foo,    update: { $set: { field1: 'value1' } },         where: { _id: id } },
   { model: Bar,    update: { $set: { 'array.$.boolthing': true} }, where: { 'array.foo': id, 'array.boolthing': false } },
   { model: FooBar, update: { $set: { 'field': 'val'} },            where: { _id: id } }
 ], _.bind(r.send, r));

Obviously this isn’t very generic and it’s directly responding to HTTP calls in the wrong place, but it’s just an experiment. The net effect of this one is to stop if there’s an error, but to wait until all updates are successful to respond with success.

%d bloggers like this: