How to Easily and Consistently Round Up with Integer Division

I’ve encountered this in different languages for different purposes.  The first time was some assembler code that calculated how many total bytes it would take to hold an arbitrary number of bits.  The problem is it’s usually coded the way we think about the problem: divide, then if there’s a remainder, add 1 to the quotient.

A quick, easy way without any special checking, is to add the (divisor-1) to the dividend before doing the division.  In the bits to bytes example, I changed the code to do it this way

howManyBytes = (howManyBits + 7)/8

Instead of this way

howManyBytes = howManyBits/8
if (howManyBytes*8 < howManyBits)
  howManyBytes ++

You’re guaranteed to get the rounded up number without any extra work.

World’s Most Efficient Implementation of Discrete Timers in PHP

Well it may not be THE most efficient, but it’s pretty close. I’ve seen a lot of timer code in my time, and it always looks too elaborate for what it needs to do. After all, if I’m using timers then I’m probably very concerned about how long things take — I don’t want to add much overhead to track it. I came up with this code during an important project to allow me to record how long certain functionality was taking.

I wanted something that would be as simple and lightweight as possible. In order to maintain discrete timers that can’t interfere with each other, many timer implementations require instantiating a new instance of some class. Instead, I opted for a static array in my timer function, with the index of the array as a unique timer “name”; this serves the same purpose. With this function, I can print out how long the overall script has been running, at any time. I can create a new timer (which also starts it running), print how long since that timer’s been running and optionally keep the timer’s value or reset it each time I access its current value.

We need an empty static array to hold timers. The first step is to get the current system time. By passing true to microtime(), we get a float representing the number of seconds for the current time. When no timer name (the index in our static array) is passed in, we just need to subtract the script start time from current and return that.

If there is a timer name passed, that’s when things get creative. We have to get the time value from the last call with this timer, or null if the timer hasn’t been used before. If we’re initializing the timer, set it to the current system time and return 0.0 (to force the type) for its value. If the timer exists and we’re resetting (which is the default) it, set it to the current system time.

Finally we return the difference between now and the last call.

I should note that when using elapsed() for the script’s overall execution time, the value is returned in seconds, otherwise the return value is milliseconds (prevents having extremely small numeric values).

/**
* Return an elapsed milliseconds since the timer was last reset.
*
* If a timer "name" is not specified, then returns the elapsed time since the
* current script started execution. This script timer can not be "reset".
* THIS DEFAULT SCRIPT TIMER RETURNS IN SECONDS
*
* Always "resets" the specified timer unless you specify false for the reset
* parameter. Of course, on the first call for a particular timer, it will always
* reset to the time of the call.
*
* examples:
* elapsed(__FUNCTION__); // using __FUNCTION__ or __METHOD__ makes it easy to be unique
* ...
* ...
* echo "It took " . elapsed(__FUNCTION)/1000 . " seconds."
*
* @param string $sTname Name of the timer
* @param boolean $bReset Do you want to reset this timer to 0?
* @return float Elapsed time since timer was last reset
*/
function elapsed($sTname = null, $bReset = true) {

    static $fTimers = array(); // To hold "now" from previous call
    $fNow = microtime(true); // Get "now" in seconds as a float

    if (is_null($sTname))
        return ($fNow - $_SERVER['REQUEST_TIME']);

    $fThen = isset($fTimers[$sTname]) ? $fTimers[$sTname] : null; // Copy over the start time, so we can update to "now"

    if (is_null($fThen) || $bReset) {
        $fTimers[$sTname] = $fNow;
        if (is_null($fThen))
            return 0.0;
    }
    return 1000 * ($fNow - $fThen);
}

printf (
    "Since script started %f Create 2 new timers 'fred' %f and 'alice' %f", 
    elapsed(), elapsed('fred'), elapsed('alice'));

for ($i = 0; $i <100; $i++) {
    printf (
        "Since script started %f 'fred' gets reset %f, but 'alice' doesn't %f", 
        elapsed(), elapsed('fred'), elapsed('alice', false));
}

Since script started 0.391466 Create 2 new timers ‘fred’ 0.000000 and ‘alice’ 0.000000
Since script started 0.391521 ‘fred’ gets reset 0.041962, but ‘alice’ doesn’t 0.037909
Since script started 0.391545 ‘fred’ gets reset 0.021935, but ‘alice’ doesn’t 0.056982
Since script started 0.391563 ‘fred’ gets reset 0.019073, but ‘alice’ doesn’t 0.075102
Since script started 0.391578 ‘fred’ gets reset 0.015020, but ‘alice’ doesn’t 0.090122
Since script started 0.391594 ‘fred’ gets reset 0.015020, but ‘alice’ doesn’t 0.104904
Since script started 0.391609 ‘fred’ gets reset 0.015974, but ‘alice’ doesn’t 0.119925
Since script started 0.391624 ‘fred’ gets reset 0.015020, but ‘alice’ doesn’t 0.134945
Since script started 0.391639 ‘fred’ gets reset 0.015020, but ‘alice’ doesn’t 0.149965
Since script started 0.391654 ‘fred’ gets reset 0.015020, but ‘alice’ doesn’t 0.164986
Since script started 0.391669 ‘fred’ gets reset 0.014782, but ‘alice’ doesn’t 0.180006
Since script started 0.391684 ‘fred’ gets reset 0.015020, but ‘alice’ doesn’t 0.195980
Since script started 0.391699 ‘fred’ gets reset 0.015020, but ‘alice’ doesn’t 0.211000
…

I think it’s convenient to use a pattern like

function foobar () {
    elapsed (__METHOD__);
    // lots of code
    $timeused = elapsed (__METHOD__);
}

Making Mongoose Keep Its Promises on the Server

So far I really like the deferred/promise model. On the client it lets me get data as early as I want to, but defer using the response until it’s convenient; maybe not until inside some event handler. I can also place a call to .fail() for any promise in a convenient place, even before I make a call for data. The Deferred object coordinates calling any registered fail function(s) if anything errors — that’s right, you can specify multiple handlers for the failure — and all the done, then and always functions that have been registered, when appropriate.

To do the same thing on the server, I found jQuery Deffered for nodejs. After requiring it

var Deferred = require( "JQDeferred" );

and with a little function in my application

function promise (query) {
  var dfd = Deferred ();

  query .run (function (err, doc) {
    if (doc) {
      if (doc.length > 0 || Object.keys(doc).length > 0) {
        console .log ("found\n", util .inspect(query));
        dfd .resolve (doc);
      }
      else {
        console .log ("not found or no rows\n", util .inspect(query)); 
        dfd .resolve (null);
      }
    }
    else {
      console .log ("error or no data object\n", util .inspect(query));
      dfd.reject (err ? err : null);
    }
  });
  return dfd .promise ();
}

Works for any Mongoose method that returns a query, like find, findOne. It’ll need to change to fold in the rest of my calls. In one function, I need to get 3 rows to verify something before I take action. Using jQuery Deferred and this promise function, dealing with 3 asynchronous calls with very little code looks like

var CloseAll = function (query, resp) {

  Deferred .when (
    promise (modelA .findOne ({modelAkey:query.body.A})), 
    promise (modelB .findOne ({modelBkey:query.body.B})), 
    promise (modelC .findOne ({modelCkey:query.body.C}))) 
    .fail (function (err)    { resp .send (_.extend (query.body, err), 500); })
    .done (function (dataA, dataB, dataC) {
          now do stuff that require all 3 things

Pretty sweet.

Promises, promises. A concise pattern for getting and showing my JSON array with jQuery and Underscore

I’m forgetting the async/sync stuff from before in favor of thenables. Deferred and its more polite personality, promise. There are just certain things that I think would be really tough to do with a callback architecture.

Normally, if I were showing a view of something I wouldn’t $.get() it from the server until my DOM is loaded, because I can’t do anything with the response until I’ve got elements to manipulate. But with jQuery’s implementation of AJAX calls, which return a promise, I can set it up so that I get data as soon as the Javascript is loaded and guarantee that I don’t try to move the data to DOM elements until it’s ready, too.

var mydata = $.get (window.location.origin + ":8080/foobar");

$( function () {
  mydata .done (showdata);
});

function showdata (data) {
  $( 'ul#something') .html (
      _.map (data, function (row) {
           return '<li>' +row.name +'</li>';
      }) .join ( '' ) );
}

If data looked like [{“name”:”john”},{“name”:”bob”},{“name”:”fred”},{“name”:”alice”}], then our <ul> now looks like

<ul id="something"><li>john</li><li>bob</li><li>fred</li><li>alice</li></ul>

I think that’s slick. showdata won’t get called until the DOM’s loaded and we have the data, but there’s no waiting before we ask for it from the server. We use map() to output an array of HTML strings (instead of concatenating), then join those and put the output into $(‘ul#something’). There’s only one variable used — which is a very good thing, BTW, lots of variables is a code smell, IMO.

Another nice pattern I use:

Imagine this jQuery GET for some set of choices within a “topic”

var topic_data = $.get (
  window.location.origin + ':8080' + '/topic/' + some_criteria);

{"name":"foobar","choices":["bob","fred","alice"]}

And separate documents for a user’s choice for that topic, which are referenced by the “name” of the topic.  In other words, I had to get that topic to find the key to use for the next GET.  I need to render the topic_data in showTopicChoices regardless of what happens in that GET.

topic_data .done (showTopicChoices, function (topic) {
  $.get (window.location.origin + ':8080' + '/user/topic/' + topic.name) 
    .done (overlayUserChoice);
 });

It’s a nice way to separate the rendering of more general data from other indicators.

Powerful jQuery Shorthand to Move Data Into DOM

I realize this is basic, though not obvious, jQuery stuff, but it’s powerful in the amount of work that can be done.  One could devise a fairly elaborate convention to make rendering driven largely by the markup.  Assume some JSON object like

var myJson = { "name": "Grant", "age": "ancient", "position": "seated" };

and elements like

 
<p myns:field="name"></p>
<p myns:field="age"></p>
<p myns:field="position"></p>

With this jQuery call I can set the text of all the elements without any knowledge of the model’s contents.

    $('[myns\\:field]') .text(function(idx, txt) {
      return myJson [$(this) .attr ('myns:field')];
    });

There’s a lot of potential; I’ll have to see if it’s worth it under varied scenarios.

Reducing an Array of Objects to a Hash Using a Property as Key

I don’t know if this is obvious, but it’s a pretty cool, quick way to take an array of objects and get them keyed by some field in the object.  Assume an array of objects, foo.  Using Underscore.js’s reduce() function, we’ll create three different hashes, each using a different field in foo’s objects as the key.

var foo = [
 {name: 'John', phone: '555-1234', social: '299-99-0123'},
 {name: 'Sally', phone: '555-5465', social: '399-99-0123'},
 {name: 'Fred', phone: '555-4356', social: '499-99-0123'},
 {name: 'Lisa', phone: '555-2318', social: '299-99-1234'},
 {name: 'Ambrose', phone: '555-9023', social: '299-99-4352'}
 ];

console .log ( _.reduce (foo, function (reduced, item) {
 reduced[item.name] = item;
 return reduced;
}, {}));
console .log ( _.reduce (foo, function (reduced, item) {
 reduced[item.phone] = item;
 return reduced;
}, {}));
console .log ( _.reduce (foo, function (reduced, item) {
 reduced[item.social] = item;
 return reduced;
}, {}));

The log looks like:

Object

1. Ambrose: Object
   1. name: “Ambrose”
   2. phone: “555-9023”
   3. social: “299-99-4352”
2. Fred: Object
   1. name: “Fred”
   2. phone: “555-4356”
   3. social: “499-99-0123”
3. John: Object
   1. name: “John”
   2. phone: “555-1234”
   3. social: “299-99-0123”
4. Lisa: Object
   1. name: “Lisa”
   2. phone: “555-2318”
   3. social: “299-99-1234”
5. Sally: Object
   1. name: “Sally”
   2. phone: “555-5465”
   3. social: “399-99-0123”

Object

1. 555-1234: Object
2. 555-2318: Object
3. 555-4356: Object
4. 555-5465: Object
5. 555-9023: Object

Object

1. 299-99-0123: Object
2. 299-99-1234: Object
3. 299-99-4352: Object
4. 399-99-0123: Object
5. 499-99-0123: Object

Functions to Create HTTP Callbacks for Mongoose

In setting up the APIs in my express application, there are many functions in which all I’m doing is sending the data passed back from my Mongoose call.  Having some functions that can set up canned mongoose callback functions is handy.  There are different functions to result in the response codes I want, depending on the context — am I getting or saving data?  The helper functions return an appropriate function to pass to Mongoose:

/**
 * Returns a foobar
 */
var getFoobar = function (q, r) {
  Foomodel
    .findById (q.params.id)
      .run (notFoundResponder ("Get foobar " + q.params.id, _.bind(r.send, r)));
};
var notFoundResponder = function (strOp, respond) {
  return function (e, doc) {
    if (doc && Object.keys(doc).length > 1) {
      console .log (strOp + " successful");
      respond (doc, 200);
    }
    else
      respond (strOp + " not found or error: " + util .inspect (doc) + " " + util .inspect (e), 404); 
  };
};
var emptyResponder = function (strOp, respond) {
  return function (e, doc) {
    if (e) 
      respond (strOp + " error: " + util .inspect (e), 500);
    else {
      console .log (strOp + " successful");      
      respond (doc, Object.keys(doc).length > 0 ? 200 : 204);
    }
  };
};
var saveResponder = function (strOp, respond) {
  return function (e, doc) {
    if (doc && Object.keys(doc).length > 1) {
      console .log (strOp + " successful");
      respond (doc, 201);
    }
    else
      respond (strOp + " not saved: " + util .inspect (doc) + " " + util .inspect (e), 500); 
  };
};
var updateResponder = function (strOp, respond) {
  return function (e, doc) {
    if (e)
      respond (strOp + " not updated: " + util .inspect (e), 500);
    else {
      console .log (strOp + " successful");
      respond (doc, 200);
    }
  };
};

Coordinating the Dependent Asynchronous Calls

I think this is better now.  nest() will take an unlimited number of function arguments, coordinating the callback from each to pass the results to the next function.  If there are errors along the way it should callback to the outermost callback.

var nest = function () {
  return _.chain (arguments) .reverse() .reduce (function (next, arg) {
    return function (doc, last) { 
      arg (doc, function (err, doc) {
        if (err)
          last (err);
        else if (next) 
          next (doc, last);
        else 
          last (null, doc);
      }); 
    };
  }, null) .value ();
};

/**
 * outermost function serving HTTP request
 */
var getFooAPI = function (q, r) {
  nest (getFoo, getSomeOther, sumResults)(q.params, 
    (function (err, doc) {
      if (err)
        r .send ("Error in getFooAPI " + util .inspect (err), 500);
      else
        r .send (doc, 201);
    }));
};

var sumResults = function (doc, handler) {
  handler (null, {"after": "doing some stuff"});
};

/**
 * Returns a Foo
 */
var getFoo = function getGame(param, handler) {
  return Foo .findById (param.id) .run (handler);
};

/**
 * Given a Foo document (from the prev find) get me the 
 * other stuff
 */
var getSomeOther = function getGamePicks (Foo, handler) {
  return Other .find ({'something.foo':Foo.key}, {stuff: 1}) .run (handler);
};

As you can see, each function to do something takes an argument that gets passed from its outer call along with a callback function.

Driving the Synchronous Mongoose Updates

This turned out to be a lot harder mentally.  And I still think I could serialize this more instead of the dynamic wrapping it does.

/**
 * Runs synchronous, dependent updates
 */
var syncUpdate = function (updates, respond) {
  return _.chain (updates) .reverse() .reduce (function update(next, arg) {
    return function (last) { 
      arg.model .update (arg.where, arg.update) .run (function (err, doc) {
        if (err)
          last (err);
        else if (next) 
          next (last);
        else 
          last (null, doc);
      });
    };
  }, null) .value ();
};

That reverse/reduce/return takes my updates and makes them inside out. We create callback wrapper code that checks for errors and if there’s another nested function to call. By passing “last”, the final callback function, to each nested invocation, we can jump all the way out on error. You can see that “next” is the accumulated function nest that we pass to reduce, and it gets evaluated inside the callback code. More of a macro type thing. If there’s something to call next, we continue to pass the “last” function along. And when there’s no more “next” to call, we call “last”.
Then I call it like:

  syncUpdate ([ 
   { model: Foo,    
     update: { $set: { field1: 'value1' } },         
     where: { _id: id } },
   { model: Bar,    
     update: { $set: { 'array.$.boolthing': true} }, 
     where: { 'array.foo': id, 'array.boolthing': false } },
   { model: FooBar, 
     update: { $set: { 'field': 'val'} },            
     where: { _id: id } } ]) 
     (
      function (err, doc) {
        if (err)
          r .send ("Error saving pick " + util .inspect (err), 500);
        else
          r .send ("saved pick " + doc, 201);
      });

One interesting difference is using syncUpdate() to return the function I want to call, which I immediately do, with my simple callback function as an argument. I think this is an improvement mechanically, from passing a success message and having the callback code be black box-ish.

I’ve got some ideas. I know of course, this should handle things other than Mongoose updates. It should be any function, and we’re getting pretty used to calling things with a callback function. In a nested situation what I want to do is call the specified function with a callback function that lets me test the result and if appropriate, pass the result on to another specified function. Or in the event of an error, at least for now, jump all de way out mon.

Used to stuff like this in assembler all the time and it’s a lot easier because you get to do anything you want. No rules to the way I allocate and initialize memory structures. In Javascript, however, I have to come up with a syntax that scales infinitely, for the nesting capability, and is flexible in the types of functions it can call.

Data-driving the Asynchronous Mongoose Updates in my Node Code

This was the first experiment to be able to ask for a sequence or set of Mongoose updates.  I wanted to be able to set up dependency and independent updates.  First I dealt with the asynchronous updates.  There is no rollback mechanism or anything, and the operation stops as soon as any of the updates fails, but we don’t get a good response unless they all succeed.  The secret sauce for the asynchronous updates was using Underscore.js’s after() function.

/**
 * Runs parallel, independent updates
 */
var asyncUpdate = function (successMsg, updates, resp) {
  var good = _.after (updates.length, function (doc) { 
    resp (successMsg + " update successful", 200); 
  });
  _.each (updates, function (arg, key, list) {
    arg.model 
      .update (arg.where, arg.update) 
      .run (function (e, doc) {
        if (e)  
          resp (arg.model.modelName + " update error: " + util .inspect (e), 500); 
        else {
          console .log (arg.model.modelName + " update successful");
          good(doc);
        }
      });
  });
};

Then call it with an array of objects of update control information.

 asyncUpdate ('Totally updated', [ 
   { model: Foo,    update: { $set: { field1: 'value1' } },         where: { _id: id } },
   { model: Bar,    update: { $set: { 'array.$.boolthing': true} }, where: { 'array.foo': id, 'array.boolthing': false } },
   { model: FooBar, update: { $set: { 'field': 'val'} },            where: { _id: id } }
 ], _.bind(r.send, r));

Obviously this isn’t very generic and it’s directly responding to HTTP calls in the wrong place, but it’s just an experiment. The net effect of this one is to stop if there’s an error, but to wait until all updates are successful to respond with success.