Tech Is Hard

Credibility = Talent x Years of experience + Proven hardcore accomplishment

Tag Archives: async

Making Mongoose Keep Its Promises on the Server


So far I really like the deferred/promise model. On the client it lets me get data as early as I want to, but defer using the response until it’s convenient; maybe not until inside some event handler. I can also place a call to .fail() for any promise in a convenient place, even before I make a call for data. The Deferred object coordinates calling any registered fail function(s) if anything errors — that’s right, you can specify multiple handlers for the failure — and all the done, then and always functions that have been registered, when appropriate.

To do the same thing on the server, I found jQuery Deffered for nodejs. After requiring it

var Deferred = require( "JQDeferred" );

and with a little function in my application

function promise (query) {
  var dfd = Deferred ();

  query .run (function (err, doc) {
    if (doc) {
      if (doc.length > 0 || Object.keys(doc).length > 0) {
        console .log ("found\n", util .inspect(query));
        dfd .resolve (doc);
      }
      else {
        console .log ("not found or no rows\n", util .inspect(query)); 
        dfd .resolve (null);
      }
    }
    else {
      console .log ("error or no data object\n", util .inspect(query));
      dfd.reject (err ? err : null);
    }
  });
  return dfd .promise ();
}

Works for any Mongoose method that returns a query, like find, findOne. It’ll need to change to fold in the rest of my calls. In one function, I need to get 3 rows to verify something before I take action. Using jQuery Deferred and this promise function, dealing with 3 asynchronous calls with very little code looks like

var CloseAll = function (query, resp) {

  Deferred .when (
    promise (modelA .findOne ({modelAkey:query.body.A})), 
    promise (modelB .findOne ({modelBkey:query.body.B})), 
    promise (modelC .findOne ({modelCkey:query.body.C}))) 
    .fail (function (err)    { resp .send (_.extend (query.body, err), 500); })
    .done (function (dataA, dataB, dataC) {
          now do stuff that require all 3 things

Pretty sweet.

Promises, promises. A concise pattern for getting and showing my JSON array with jQuery and Underscore


I’m forgetting the async/sync stuff from before in favor of thenables. Deferred and its more polite personality, promise. There are just certain things that I think would be really tough to do with a callback architecture.

Normally, if I were showing a view of something I wouldn’t $.get() it from the server until my DOM is loaded, because I can’t do anything with the response until I’ve got elements to manipulate. But with jQuery’s implementation of AJAX calls, which return a promise, I can set it up so that I get data as soon as the Javascript is loaded and guarantee that I don’t try to move the data to DOM elements until it’s ready, too.

var mydata = $.get (window.location.origin + ":8080/foobar");

$( function () {
  mydata .done (showdata);
});

function showdata (data) {
  $( 'ul#something') .html (
      _.map (data, function (row) {
           return '<li>' +row.name +'</li>';
      }) .join ( '' ) );
}

If data looked like [{“name”:”john”},{“name”:”bob”},{“name”:”fred”},{“name”:”alice”}], then our <ul> now looks like

<ul id="something"><li>john</li><li>bob</li><li>fred</li><li>alice</li></ul>

I think that’s slick. showdata won’t get called until the DOM’s loaded and we have the data, but there’s no waiting before we ask for it from the server. We use map() to output an array of HTML strings (instead of concatenating), then join those and put the output into $(‘ul#something’). There’s only one variable used — which is a very good thing, BTW, lots of variables is a code smell, IMO.

Another nice pattern I use:

Imagine this jQuery GET for some set of choices within a “topic”

var topic_data = $.get (
  window.location.origin + ':8080' + '/topic/' + some_criteria);
{"name":"foobar","choices":["bob","fred","alice"]}

And separate documents for a user’s choice for that topic, which are referenced by the “name” of the topic.  In other words, I had to get that topic to find the key to use for the next GET.  I need to render the topic_data in showTopicChoices regardless of what happens in that GET.

topic_data .done (showTopicChoices, function (topic) {
  $.get (window.location.origin + ':8080' + '/user/topic/' + topic.name) 
    .done (overlayUserChoice);
 });

It’s a nice way to separate the rendering of more general data from other indicators.

Coordinating the Dependent Asynchronous Calls


I think this is better now.  nest() will take an unlimited number of function arguments, coordinating the callback from each to pass the results to the next function.  If there are errors along the way it should callback to the outermost callback.

var nest = function () {
  return _.chain (arguments) .reverse() .reduce (function (next, arg) {
    return function (doc, last) { 
      arg (doc, function (err, doc) {
        if (err)
          last (err);
        else if (next) 
          next (doc, last);
        else 
          last (null, doc);
      }); 
    };
  }, null) .value ();
};

/**
 * outermost function serving HTTP request
 */
var getFooAPI = function (q, r) {
  nest (getFoo, getSomeOther, sumResults)(q.params, 
    (function (err, doc) {
      if (err)
        r .send ("Error in getFooAPI " + util .inspect (err), 500);
      else
        r .send (doc, 201);
    }));
};

var sumResults = function (doc, handler) {
  handler (null, {"after": "doing some stuff"});
};

/**
 * Returns a Foo
 */
var getFoo = function getGame(param, handler) {
  return Foo .findById (param.id) .run (handler);
};

/**
 * Given a Foo document (from the prev find) get me the 
 * other stuff
 */
var getSomeOther = function getGamePicks (Foo, handler) {
  return Other .find ({'something.foo':Foo.key}, {stuff: 1}) .run (handler);
};

As you can see, each function to do something takes an argument that gets passed from its outer call along with a callback function.

Data-driving the Asynchronous Mongoose Updates in my Node Code


This was the first experiment to be able to ask for a sequence or set of Mongoose updates.  I wanted to be able to set up dependency and independent updates.  First I dealt with the asynchronous updates.  There is no rollback mechanism or anything, and the operation stops as soon as any of the updates fails, but we don’t get a good response unless they all succeed.  The secret sauce for the asynchronous updates was using Underscore.js’s after() function.

/**
 * Runs parallel, independent updates
 */
var asyncUpdate = function (successMsg, updates, resp) {
  var good = _.after (updates.length, function (doc) { 
    resp (successMsg + " update successful", 200); 
  });
  _.each (updates, function (arg, key, list) {
    arg.model 
      .update (arg.where, arg.update) 
      .run (function (e, doc) {
        if (e)  
          resp (arg.model.modelName + " update error: " + util .inspect (e), 500); 
        else {
          console .log (arg.model.modelName + " update successful");
          good(doc);
        }
      });
  });
};

Then call it with an array of objects of update control information.

 asyncUpdate ('Totally updated', [ 
   { model: Foo,    update: { $set: { field1: 'value1' } },         where: { _id: id } },
   { model: Bar,    update: { $set: { 'array.$.boolthing': true} }, where: { 'array.foo': id, 'array.boolthing': false } },
   { model: FooBar, update: { $set: { 'field': 'val'} },            where: { _id: id } }
 ], _.bind(r.send, r));

Obviously this isn’t very generic and it’s directly responding to HTTP calls in the wrong place, but it’s just an experiment. The net effect of this one is to stop if there’s an error, but to wait until all updates are successful to respond with success.

%d bloggers like this: