Tech Is Hard

Credibility = Talent x Years of experience + Proven hardcore accomplishment

Reducing an Array of Objects to a Hash Using a Property as Key


I don’t know if this is obvious, but it’s a pretty cool, quick way to take an array of objects and get them keyed by some field in the object.  Assume an array of objects, foo.  Using Underscore.js’s reduce() function, we’ll create three different hashes, each using a different field in foo’s objects as the key.

var foo = [
 {name: 'John', phone: '555-1234', social: '299-99-0123'},
 {name: 'Sally', phone: '555-5465', social: '399-99-0123'},
 {name: 'Fred', phone: '555-4356', social: '499-99-0123'},
 {name: 'Lisa', phone: '555-2318', social: '299-99-1234'},
 {name: 'Ambrose', phone: '555-9023', social: '299-99-4352'}
 ];

console .log ( _.reduce (foo, function (reduced, item) {
 reduced[item.name] = item;
 return reduced;
}, {}));
console .log ( _.reduce (foo, function (reduced, item) {
 reduced[item.phone] = item;
 return reduced;
}, {}));
console .log ( _.reduce (foo, function (reduced, item) {
 reduced[item.social] = item;
 return reduced;
}, {}));

The log looks like:

Object
  1. Ambrose: Object
    1. name: “Ambrose”
    2. phone: “555-9023”
    3. social: “299-99-4352”
  2. Fred: Object
    1. name: “Fred”
    2. phone: “555-4356”
    3. social: “499-99-0123”
  3. John: Object
    1. name: “John”
    2. phone: “555-1234”
    3. social: “299-99-0123”
  4. Lisa: Object
    1. name: “Lisa”
    2. phone: “555-2318”
    3. social: “299-99-1234”
  5. Sally: Object
    1. name: “Sally”
    2. phone: “555-5465”
    3. social: “399-99-0123”
Object
  1. 555-1234: Object
  2. 555-2318: Object
  3. 555-4356: Object
  4. 555-5465: Object
  5. 555-9023: Object
Object
  1. 299-99-0123: Object
  2. 299-99-1234: Object
  3. 299-99-4352: Object
  4. 399-99-0123: Object
  5. 499-99-0123: Object
Advertisements

Functions to Create HTTP Callbacks for Mongoose


In setting up the APIs in my express application, there are many functions in which all I’m doing is sending the data passed back from my Mongoose call.  Having some functions that can set up canned mongoose callback functions is handy.  There are different functions to result in the response codes I want, depending on the context — am I getting or saving data?  The helper functions return an appropriate function to pass to Mongoose:

/**
 * Returns a foobar
 */
var getFoobar = function (q, r) {
  Foomodel
    .findById (q.params.id)
      .run (notFoundResponder ("Get foobar " + q.params.id, _.bind(r.send, r)));
};
var notFoundResponder = function (strOp, respond) {
  return function (e, doc) {
    if (doc && Object.keys(doc).length > 1) {
      console .log (strOp + " successful");
      respond (doc, 200);
    }
    else
      respond (strOp + " not found or error: " + util .inspect (doc) + " " + util .inspect (e), 404); 
  };
};
var emptyResponder = function (strOp, respond) {
  return function (e, doc) {
    if (e) 
      respond (strOp + " error: " + util .inspect (e), 500);
    else {
      console .log (strOp + " successful");      
      respond (doc, Object.keys(doc).length > 0 ? 200 : 204);
    }
  };
};
var saveResponder = function (strOp, respond) {
  return function (e, doc) {
    if (doc && Object.keys(doc).length > 1) {
      console .log (strOp + " successful");
      respond (doc, 201);
    }
    else
      respond (strOp + " not saved: " + util .inspect (doc) + " " + util .inspect (e), 500); 
  };
};
var updateResponder = function (strOp, respond) {
  return function (e, doc) {
    if (e)
      respond (strOp + " not updated: " + util .inspect (e), 500);
    else {
      console .log (strOp + " successful");
      respond (doc, 200);
    }
  };
};

Coordinating the Dependent Asynchronous Calls


I think this is better now.  nest() will take an unlimited number of function arguments, coordinating the callback from each to pass the results to the next function.  If there are errors along the way it should callback to the outermost callback.

var nest = function () {
  return _.chain (arguments) .reverse() .reduce (function (next, arg) {
    return function (doc, last) { 
      arg (doc, function (err, doc) {
        if (err)
          last (err);
        else if (next) 
          next (doc, last);
        else 
          last (null, doc);
      }); 
    };
  }, null) .value ();
};

/**
 * outermost function serving HTTP request
 */
var getFooAPI = function (q, r) {
  nest (getFoo, getSomeOther, sumResults)(q.params, 
    (function (err, doc) {
      if (err)
        r .send ("Error in getFooAPI " + util .inspect (err), 500);
      else
        r .send (doc, 201);
    }));
};

var sumResults = function (doc, handler) {
  handler (null, {"after": "doing some stuff"});
};

/**
 * Returns a Foo
 */
var getFoo = function getGame(param, handler) {
  return Foo .findById (param.id) .run (handler);
};

/**
 * Given a Foo document (from the prev find) get me the 
 * other stuff
 */
var getSomeOther = function getGamePicks (Foo, handler) {
  return Other .find ({'something.foo':Foo.key}, {stuff: 1}) .run (handler);
};

As you can see, each function to do something takes an argument that gets passed from its outer call along with a callback function.

Driving the Synchronous Mongoose Updates


This turned out to be a lot harder mentally.  And I still think I could serialize this more instead of the dynamic wrapping it does.

/**
 * Runs synchronous, dependent updates
 */
var syncUpdate = function (updates, respond) {
  return _.chain (updates) .reverse() .reduce (function update(next, arg) {
    return function (last) { 
      arg.model .update (arg.where, arg.update) .run (function (err, doc) {
        if (err)
          last (err);
        else if (next) 
          next (last);
        else 
          last (null, doc);
      });
    };
  }, null) .value ();
};

That reverse/reduce/return takes my updates and makes them inside out. We create callback wrapper code that checks for errors and if there’s another nested function to call. By passing “last”, the final callback function, to each nested invocation, we can jump all the way out on error. You can see that “next” is the accumulated function nest that we pass to reduce, and it gets evaluated inside the callback code. More of a macro type thing. If there’s something to call next, we continue to pass the “last” function along. And when there’s no more “next” to call, we call “last”.
Then I call it like:

  syncUpdate ([ 
   { model: Foo,    
     update: { $set: { field1: 'value1' } },         
     where: { _id: id } },
   { model: Bar,    
     update: { $set: { 'array.$.boolthing': true} }, 
     where: { 'array.foo': id, 'array.boolthing': false } },
   { model: FooBar, 
     update: { $set: { 'field': 'val'} },            
     where: { _id: id } } ]) 
     (
      function (err, doc) {
        if (err)
          r .send ("Error saving pick " + util .inspect (err), 500);
        else
          r .send ("saved pick " + doc, 201);
      });

One interesting difference is using syncUpdate() to return the function I want to call, which I immediately do, with my simple callback function as an argument. I think this is an improvement mechanically, from passing a success message and having the callback code be black box-ish.

I’ve got some ideas. I know of course, this should handle things other than Mongoose updates. It should be any function, and we’re getting pretty used to calling things with a callback function. In a nested situation what I want to do is call the specified function with a callback function that lets me test the result and if appropriate, pass the result on to another specified function. Or in the event of an error, at least for now, jump all de way out mon.

Used to stuff like this in assembler all the time and it’s a lot easier because you get to do anything you want. No rules to the way I allocate and initialize memory structures. In Javascript, however, I have to come up with a syntax that scales infinitely, for the nesting capability, and is flexible in the types of functions it can call.

Data-driving the Asynchronous Mongoose Updates in my Node Code


This was the first experiment to be able to ask for a sequence or set of Mongoose updates.  I wanted to be able to set up dependency and independent updates.  First I dealt with the asynchronous updates.  There is no rollback mechanism or anything, and the operation stops as soon as any of the updates fails, but we don’t get a good response unless they all succeed.  The secret sauce for the asynchronous updates was using Underscore.js’s after() function.

/**
 * Runs parallel, independent updates
 */
var asyncUpdate = function (successMsg, updates, resp) {
  var good = _.after (updates.length, function (doc) { 
    resp (successMsg + " update successful", 200); 
  });
  _.each (updates, function (arg, key, list) {
    arg.model 
      .update (arg.where, arg.update) 
      .run (function (e, doc) {
        if (e)  
          resp (arg.model.modelName + " update error: " + util .inspect (e), 500); 
        else {
          console .log (arg.model.modelName + " update successful");
          good(doc);
        }
      });
  });
};

Then call it with an array of objects of update control information.

 asyncUpdate ('Totally updated', [ 
   { model: Foo,    update: { $set: { field1: 'value1' } },         where: { _id: id } },
   { model: Bar,    update: { $set: { 'array.$.boolthing': true} }, where: { 'array.foo': id, 'array.boolthing': false } },
   { model: FooBar, update: { $set: { 'field': 'val'} },            where: { _id: id } }
 ], _.bind(r.send, r));

Obviously this isn’t very generic and it’s directly responding to HTTP calls in the wrong place, but it’s just an experiment. The net effect of this one is to stop if there’s an error, but to wait until all updates are successful to respond with success.

Software Developers in the Past were Much Better than Today (as a whole)


Can it be argued with?  It seems statistical truth.  Back in the 70s, 80s and part of the 90s, way before everyone and their brother was a “Web developer”, there were few enough people doing the work that the percentage who were very good — with various language, scientific, mathematical and electronics backgrounds — was MUCH, MUCH higher than it could possibly be when MANY, MANY more people are doing it.

Let me put it another way: if the NFL decided to kick start a few thousand more football teams, do you think the quality of play would be the same as it is today?  Wouldn’t there be a lot of people playing that currently don’t have the skill?  There can only be so much supply for a highly-skilled profession.  People can’t just  choose to be good software engineers any more than they can go out and simply train hard enough to be a talented professional football player.

So the number of really good software people has gone from maybe 1 in 10 to 1 in probably 12,000, if you crunch the numbers.

An outcome of the same internet explosion that has outstretched the top quality supply is that everyone can spout off.  It seems too, the amount of self-promotion is inversely proportional to meaningful understanding or communication.  So there’s often this large, single, near-unanimous, overwhelming, AND HORRIBLY WRONG opinion or impression of a concept and how to implement it.

I have to say that it’s possible to be self taught and as good as the educated/highly-experienced.  But it’s very, very unlikely.  The catch is, the self-taught  don’t know enough to know how little they know in comparison.  That is also logical.  When one’s knowledge is limited, he has no way of seeing it.

I’m sorry.

Look What I Done


Why does it bother me to see people broadcast their successes, however trivial, so they can get a “go, you!” from others who either already should have voiced their support and shared in the joy, or people who have no business in it?  It’s the more so, because, as I understand, these wall whisperers have families and close friends whom they share time and interests with.  Is it not enough praise that you come home and share your promotion with a family?  It should be on the news ticker, too?  I share with my daughter and my cat, and well, I guess I don’t know any better.  But I’m sure as hell not going to stop the presses because my lab results came back negative.  Oops.  Just did.

Code Differently


Whenever I look at existing code, an automatic reaction is “what would another way be and what would it look like?  Is there some appeal to that?”  This can often bring new ways to look at the overall and reduce the work.  But I always have to make sure the appeal isn’t just because it’s different.  Boy George already used that.

Don’t Expect Me to Know What You’re Tweeting, Cuz I Don’t Read Your Wall


I saw something about a study that showed heavy Facebook users were more likely to feel unhappy with their own life/unfulfilled/envious, and it got me to thinking.

I used to say that TV, especially once there were more than 5 channels to choose from, was the beginning of the end.  I once read that “with television, people spend more time watching other — mostly fictional — people live, instead of living themselves”.  As much as I have it on myself, I try not to sit and watch unless it’s a movie I haven’t seen.  (Off topic sort of, but just yesterday I heard a psychologist say, “children have to know mommy and daddy need their time and they should just watch TV or something” — how automatic and horrendous.)

It’s what a lot of people do with Facebook, and to less extent maybe Twitter and blogs.  The latter two might play into the phenomenom I see as unique to the Web media, of needing an audience to quench the aforementioned envy.  Most people have probably already heard of Facebook addiction and the like, but I think there’s something more subtle at work, too.

Facebook creates the perfect storm for wasting time observing and comparing.  I know many of the people, thus am under similar constraints of success — same school, same profession, same geo, etc — but of course there’s going to be people posting stuff that makes my life look less glamorous.  (And like chat rooms, is any of it pumped up?)

So we read this and become envious, we spend more time reading it and we do less for ourselves, so we achieve less, etc.

In my opinion, the reality show genre is finely tuned to this.  When we watch actors, although part of us is in the fantasy of the script/plot, we don’t expect ourselves to be “them” – this is a movie, they are actors.  But when the participants become everyman who can wait in line to audition without previous qualification, without having earned it…  Show participants aren’t actors, so the audience identifies much more (“I could/should…”), but the odds are still hugely against them just because of numbers, so we become more immersed in an observer, wannabe role.

I don’t know what to make of it, but I’m just sayin.  How come I have to leave messages on cell phones so much?  How come people want to use texts, where they don’t *have* to respond?  Why do people now want to broadcast on their “wall” and expect the world to know, instead of communicating directly with the individuals who need to know?  Seems like a time saver, huh?  Well now I also have to monitor the daily flow from all the people who broadcast to me to pick out what might be information.  I’m worse off.

So I might save time when planning my own event because I can just “post” it.  BUT I have to check everyone’s posts to see what I might be interested in.  Instead doesn’t it save everyone time when each of us spends more communicating, so that everyone doesn’t have to listen in and decide for themselves whether it’s relevant to them?  Is there an ego factor at work here?  Where we want to imagine the world wants to listen to our thoughts and plans?

Do we need an audience because everyone else is famous, because we have such an immediate media, and round and round it goes

The Terminator Scenario


Machines that can build more of themselves are here.  It is a revolutionary concept to have something that, to some degree, can output itself.  We saw that in code years ago.  But this is stuff you can hold in your hand.

In answer to the detractors who ask “does the world need more plastic crap?”, I wonder if in the long run, such a system would mean less overall manufactured plastics.  Sort of a just-in-time inventory like concept.  Wouldn’t there be less mass-produced plastic?  And the supply chain for raw materials could become more streamlined, resulting in more efficiencies.

Ponderable, at least.

%d bloggers like this: