Release the Kraken.js – Part I

Image credit: Yarnington 2011

Of course when you’re a kid, you can be friends with anybody. […] You like Cherry Soda? I like Cherry Soda! We’ll be best friends!

Jerry Seinfeld

We learned from Rene Zellweger that sometimes you can have people at “Hello”. In case of myself and the project Kraken.js, it had me at “Node.js/express.js/Dust.js/jQuery/Require.js/Bootstrap”. OK, not fit for a memorable movie quote but fairly accurate – that list of libraries and platforms was what we at JazzHub eventually arrived at as our new stack to write cloud development tools for BlueMix. Imagine my delight when all these checkboxes were ticked in a presentation describing PayPal’s experience in moving from Java to Node for their Lean UX needs, and announcing project Kraken. You can say I had my ‘Cherry Soda’ moment.

Of course, blessed with ADD as I am, I made a Flash-like scan of the website, never spending time to actually, you know, study it. Part of it is our ingrained fear and loathing of frameworks in general, and also our desire to start clean and very slowly add building blocks as we need them. This approach is also advocated by Christian Heilmann in his article ‘The Vanilla Web Diet’ in the Smashing Magazine Book 4.

Two things changed my mind. First, as part of NodeDay I had a pleasure to meet and talk to Bill Scott, Senior Director of UX Engineering, Jeff Harrell, Director of Engineering Architecture, and Richard Ragan, Principal Engineer and ‘the Dust.js guy’, all from PayPal, and now feel retroactively bad for not giving the fruit of their labor more attention. Second, I had a chat with a friend from another IBM group also working on a Node.js project, and he said “If we were to start again today, I would definitely give Kraken.js a go – I wish we had it back then, because we had to essentially re-implement most of it”. Strike two – a blog post is in order.

So now with a combination of guilt, intra-company encouragement and natural affinity of our technologies, I started making inventory of what Kraken has to offer. At this point, hilarity ensued: the Getting Started page requires installation of Yeoman, and then the Kraken generator. Problem: there is no straightforward way to install Yeoman on Windows. I have noticed during a recent NodeDay that when it comes to Node, Macs outnumber Windows machines 99 to 1, but considering that Kraken is targeting enterprise developers switching to Node, chances are many of them are running Windows on their work machines. A Yeoman for Windows would definitely help spread the good word.

Update: it turns out I was working on the outdated information – after posting the article, I tried again and managed to install Yeoman on a Windows 7 machine.

Using Yeoman is of course convenient to bootstrap your apps – you create a directory, type ‘yo kraken’, answer couple of questions and the scaffolding of your app is set up for you. At this point our team lead Curtis said ‘I would be fine with a zip file and no need to install Yeoman’. He would have been right if using Yeoman was a one-time affair, but it turns out that Kraken Yeoman generator is a gift that keeps on giving. You can return to it to incrementally build up your app – you can add new pages, models, controllers, locales etc. It is a major boost to productivity – I can see why they went through the trouble.

Anyway, switched to my MacBook Pro, I installed Yeoman, then Kraken generator, then typed ‘yo Kraken’ and my app was all there. Once you start analyzing the app structure, you see the Kraken overhead – there are more directories, more files, and Kraken has decided on how things should be structured and tries to stick to it. On the surface of it, it is less clear for the beginner, but I am already past the pure express ‘Hello, World’ apps. I am interested in how to write real world production apps for the enterprise, and Kraken delivers. It addresses the following enterprise-y needs:

  1. Structure – app is clearly structured – models, controllers, templates, client and server side – it is all well labelled and easy to figure out
  2. i18n – Kraken is serious about enterprise needs, so internationalization is baked in right from the start
  3. Errors – it provides translated files for common HTTP errors (404, 500, 503) as a pattern
  4. Templates – Kraken uses Dust, which fits our needs like a glove, and the templates are located in the public folder so that they can be served to both Node.js and the browser, allowing template sharing. Of course, templates are pre-compiled into JS files for performance reasons.
  5. Builds – Kraken uses Grunt to automate all the boring tasks of preparing your app for deployment, and also incorporates Mocha tests as a standard part. Having a consistent approach to testing (and the ability to incrementally add tests for new pages) will improve quality of your apps in the long run.
Kraken.js HelloWorld app directory structure

One thing that betrays that PayPal deals with financial transactions is how serious they are about security. Thanks to its Lusca module, Kraken bakes protection against the most prevalent types of attacks right into your app. All enterprise developers know this must be done before going into production, so why tacking it on in a panicky scramble at the last minute when you can bake it into the code from the start.

Of course, the same goes for i18n – Makara module handles translations. At this point I paused looking at the familiar file format from the Java world – *.properties files. My gut reaction would be to make all the supporting files in JSON format (and to Kraken authors’ credit, they did do so for all the configuration files). However, after thinking about the normal translation process, I realized that NLS files are passed to the translation team that is not necessarily very technical. Properties files are brain-dead – key/value pairs with comments. This makes them easy to author – you don’t want to receive JSON NLS files with missing quotes, commas or curly braces.

Makara is designed to work with Dust.js templates, and Kraken provides a Dust helper to inline locale-based substitutions into the template directly (for static cases), as well as an alternative for more dynamic situations.

As for the main file itself, it did give me pause because the familiar express app was nowhere to be found (I absent-mindedly typed ‘node app.js’, not realizing the app is really in index.js):

'use strict';

var kraken = require('kraken-js'),
    app = {};

app.configure = function configure(nconf, next) {
    // Async method run on startup.

app.requestStart = function requestStart(server) {
    // Run before most express middleware has been registered.

app.requestBeforeRoute = function requestBeforeRoute(server) {
    // Run before any routes have been added.

app.requestAfterRoute = function requestAfterRoute(server) {
    // Run after all routes have been added.

if (require.main === module) {
    kraken.create(app).listen(function (err) {
        if (err) {

module.exports = app;

This is where the ‘framework vs library’ debate can be legitimate – in a true library, I would require express, dust etc. in addition to kraken, start the express server and somehow hook up kraken to it. I am sure express server is somewhere under the hood, but I didn’t spend enough time to find it (Lenny Markus is adamant this is possible) . This makes me wonder how I would approach using cluster with kraken – forking workers to use all the machine cores. Another use case – how would I use here – piggy-backs on the express server and reuses the port. I will report my findings in the second instalment.

I would be remiss not to include Kappa here – it proxies NPM repository allowing control over what is pulled in via NPM and provides for intra-enterprise sharing of internal modules. Every enterprise will want to reuse code but not necessarily immediately make it Open Source on GitHub. Kappa fits the bill, if only for providing an option – I have not tested it and don’t know if it would work in our particular case. If anything, it does not suffer from NIH – it essentially wraps a well regarded npm-delegate module.

Let’s not forget builds: as part of scaffolding, Yeoman creates a Gruntfile.js configured to build your app and do an amazing amount of work you will get for free:

  1. Run JSHint to verify your JS files
  2. Build Require.js modules
  3. Run less compiler on the css files
  4. Build locale-specific versions of your dust templates based on i18n properties
  5. Compile dust templates into *.js files for efficient use on the client
  6. Run Mocha tests

I cannot stress this enough – after the initial enthusiasm with Node.js, you are back to doing all these boring but necessary tasks, and to have them all hooked up in the new app allows the teams to scale while retaining quality standards and consistency. Remember, as a resident Node.js influencer you will deal with many a well intentioned but inexperienced Java, PHP or .NET developer – having some structural reinforcement of good practices is a good insurance against everything unravelling into chaos. There is a tension here – you want to build a modern system based on Node.js micro-services, but don’t want each micro service to be written in a different way.

Kraken is well documented, and there are plenty examples to understand how to do most of the usual real world tasks. For an enterprise developer ready to jump into Node.js, the decision tree would be like this:

  1. If you prefer frameworks and like opinionated software for the sake of consistency and repeatability, use Kraken in its entirety
  2. If you are starting with Node.js, learn to write express.js apps first – you want to understand how things work before adding abstractions on top of them. Then, if you are OK with losing some control to Kraken, go to (1). Otherwise, consider continuing with your express app but cherry-picking Kraken modules as you go. To be fair, if Kraken is a framework at all (a very lightweight one as frameworks go), it falls into the class of harvested frameworks – something useful extracted out of the process of building applications. This is the good kind.
  3. If you ‘almost’ like Kraken but some details bother you or don’t quite work in your particular situation, consider participating in the Open Source project – helping Kraken improve is much cheaper than writing your own framework from scratch.

A friend of mine once told me (seeing me dragging my guitar for a band rehearsal) that she “must sit down one afternoon and learn how to play the guitar already”. I wisely opted out of such a mix of arrogance and cluelessness here and called this ‘Part I’ of the Kraken.js article. I am sure I will be able to get a better idea of its relative strengths and weaknesses after a prolonged use. For now, I wanted to share my first impressions before I lose the innocence and lack of bias.

Meanwhile, in the words of “the most interesting man in the world”: I don’t always use frameworks for Node.js development, but when I do, I prefer Kraken.js.

© Dejan Glozic, 2014

Node.js on the Road – San Francisco


This post is a first of sorts – as with my review of RESS: The Essentials that opened my Reviews category, it will be the first in the Events. In this context, plural is actually justified because it covers not one but two events: Node.js on the Road on Feb 27th in San Francisco’s hosted by New Relic and NodeDay on Feb 28th in San Jose hosted by PayPal. Both are now available – you can read the NodeDay post here.

I ended up at ‘Node.js on the Road’ by a stroke of luck. Ever since the days when EclipseCon was being hosted in Santa Clara Convention Center, my preferred travel plan was to fly to SFO, then drive down 101 South for the event. The 11:45am Air Canada YYZ-SFO flight is just perfect for this, and it gives me a free afternoon and evening in San Francisco. Just when I was trying to figure out what to do with the available time, along came a tweet about ‘Node.js on the Road’ to be held 6-9pm PT in the New Relic’s downtown SF location on Feb 27th. Seemed like a perfect way to spend the evening before heading South.

Looking for parking on a work day in downtown San Francisco is not for the faint of heart. I ended up parking in an the underground garage of the nearby building. I arrived a bit ahead of time but it turned out that 6pm was ‘when the food was about to arrive, and also the people’, not the actual start of the event. Just as well, because my jet-lagged mind welcomed some time to pipe down and also solved my dinner problem.

Node.js on the Road Agenda
Node.js, food and beer - a perfect combination
Node.js, food and beer – a perfect combination

The even was opened by Bryan Cantrill, SVP Engineering of Joyent, the corporate steward of Node.js. It was a welcome introduction, explaining the format of the evening which was a bit vague to me. He also said something counter-intuitive for many people skeptical of ‘JavaScript on the server’: “Node.js is the most debuggable dynamic environment”. Veterans of Java and JVM would love to disagree, but everybody knows that huge VM dumps created when a real-world production Java program gives up the ghost are a nightmare to analyze, so it is not hard to imagine where he is coming from.

Bryan Cantrill, SVP Engineering, Joyent

First on the list of presenters was TJ Fontaine, the newly minted Node.js Core team lead, replacing Isaac Schlueter who left to become the CEO of the new NPM startup (short for ‘Node Package Manager’, a registry largely responsible for the thriving Node.js ecosystem). His talk was centered around the road ahead – what is left to finish in Node before it can be declared v1.0, how companies and individuals using it can contribute and inform the process. There are multiple ways to make Node better at this point: reporting bugs as we are hitting them in production, contributing source for either enhancements or bug fixes, helping with the documentation (Node currently has a bit of a documentation problem). There are also multiple ways to ensure the community continues to thrive – Stack Overflow, IRCs, mailing lists, meetups. Not all of these options are equally appealing to all (TJ is not a fan of mailing lists or IRCs) – YMMV.

TJ also spent some time to announce what is coming in Node 0.12 which is ‘very immanent’. It was largely a rehash of what was already mentioned in a related blog post. Some are expected (performance improvements), some are too detailed for my current knowledge of Node.js (TLS enhancements, buffers, streams) although I am sure there are developers which are expecting them with anticipation. And then some are controversial. Take execSync – after all the effort to explain that Node.js is asynchronous to the core, being able to run blocking execution seems like a compromise with the devil (to TJ’s defense, he recommended extreme caution when using this function for all the easily understandable reasons).

TJ Fontaine, Node.js Core team lead
TJ Fontaine, Node.js Core Team Lead

Next on the agenda was Dav Glass, Node.js architect from Yahoo!. In my chat with the presenters prior to the event, I learned that the idea is to bring different presenters at each stop to tell the story of their own use of Node.js in real world projects. Yahoo! seems to be turning into a massive Node.js shop, and Dav illustrated it with the ‘We love Node’ slide. Yahoo! runs Node both in production and internally, using a huge Continuous Integration (CI) system called Manhattan. At the moment, some of the boxes in Yahoo! pages are served by Node (others use Java or PHP). As expected, transition to Node in running systems is often a gradual transition, not a big switch.

Dav’s war stories? In his experience, it is rarely Node’s problem – most of the time issues end up being caused by downstream modules deep in the bowels of the running system. Again, since the system needs to keep going, Node needs to be inserted into the giant old infrastructure. Yahoo! is using Node largely unmodified – the only minor changes being done around security and authentication.

Another common theme is the use of NPM to serve internal modules – not all the Node modules are open sourced, requiring a way to serve them internally. One of the internal code quality requirement is that you are not allowed to share a library unless it has 80% test code coverage.

Dav demonstrated some impressive throughput numbers, from processes serving > 1000 rps (requests per second), to some extreme examples of hot rod modules hitting insane figures of 25000-36000 rps.

Dave Glass, Node.js architect, Yahoo
Dav Glass, Node.js Architect, Yahoo! in front of ‘We Love Node!’ slide.

Jeff Harrell is a new hire who came to PayPal 2 years ago as a new Director of UI Engineering to ‘disrupt the PayPal system’. His story has already been somewhat familiar to me due to the write up on the Project Kraken. Jeff added new observations to this story, explaining that the goal of their wholesale move from Java to Node was every bit as driven by developer performance as app performance. In a what is turning into a familiar pattern, their topology has Node front end hooked up to the legacy infrastructure that is either difficult to change, or is not a good fit for Node. PayPal is currently running 20+ Node apps, written by 200 developers (both new hires and Java converts).

According to Jeff, converting Java developers was an interesting process. He observed stages not very different from Kubler-Ross stages of grief, from denial, anger and bargaining to acceptance and finally enthusiasm for Node at the end of the curve. Of course, not everything was perfect: PayPal, being a payment processing company, is a heavy user of SSL, and in Node v0.10, SSL cipher time left much to be desired. This was just a point in time problem, however: in v0.11 and the upcoming v0.12 SSL performance is much improved, and will continue to be a high priority.

Jeff Harrell, Director of UI Engineering, PayPal
Jeff Harrell, Director of UI Engineering, PayPal

The last part of the event was a Q/A panel with all the participants. A lot of questions were for Jeff Harrell – it seems that audience uses PayPal as a good model of a traditional Java shop moving to Node with a clear head, with a desire to improve productivity and make Lean UX real and practical, whereas they have a harder time identifying with the more complex Yahoo!’s systems (a lot of participants work for startups that are years away from reaching Yahoo! scale). Some participants expressed the familiar concern about type safety (or lack thereof) in JavaScript. According to panelists, smaller modules in Node tend to make type safety less of an issue. In fact, Node.js inherits from Unix philosophy – it forces you to think small and combine (and write tests – smaller components are more testable).

As for real-world robustness, Yahoo! has provided Open Source launchers (they are on GitHub) which run one process per core, and restart process if there are problems.

Some participants wanted to know how to get Node introduced into a skeptical organization. Panelists agreed that doing it instead of asking for permission is the way to go (and also making it provable – you can argue technology all day long, but when business people realize you can do it faster, they like the outcome).

Yahoo! is definitely not in that phase right now – Yahoo! has thousands of engineers that all know JavaScript, so convincing them is an easier task. With such a brain power, it is easy to go crazy with inter-process communication and try to squeeze the last ounce of performance using binary protocols, but the recommendation is to start with HTTP and see if anything else is needed.

Q/A panel - TJ, Jeff and Dave
Q/A panel – TJ, Jeff and Dav

All in all, it was an enjoyable and not over-long evening, and a perfect prequel to the NodeDay on Friday. Read about NodeDay in my next post.

Nowhere to park on Spear street
Nowhere to park on Spear street

© Dejan Glozic, 2014

On the LinkedIn’s Dusty Trail


There is an old New Yorker cartoon (from 1993) where a dog working on the computer tells another dog: “On the Internet, nobody knows you are a dog”. That’s how I occasionally feel about the GitHub projects – they could be solid, multi-contributor powerhouses churning out release after release, and there could be great ideas stalled when the single contributor got distracted by something new and shiny. The thing is that you need to inspect all the project artifacts carefully to tell which is which – single contributors can mount a powerful and amplified presence on GitHub (until they leave, that is).

This issue comes with the territory in Open Source (considering how much you pay for all the hard work of others), but nowhere is it more acute than in LinkedIn’s choice of templating library in 2011. In an often quoted blog post, LinkedIn engineering team embarked on a quest to standardise around one templating library that can be run both on the server and on the client, and also check a number of other boxes they put forward. Out of several contenders they finally settled on Dust.js. This library uses curly braces for pattern substition, which puts it in a hipster-friendly category alongside Mustache and Handlebars. But here is the rub: while the library ticks most of LinkedIn’s self-imposed requirements, its community support leaves much to be desired. The sole contributor seemed unresponsive.

Now, if it was me, I would move on, but LinkedIn’s team decided that they will not be deterred by this. In fact, it looks like they kind of liked the fact that they can evolve the library as they learn by doing. The problem was that committer rights work by bootstrapping – only the original committer can accept LinkedIn changes, which apparently didn’t happen with sufficient snap. Long story short, behold the ‘LinkedIn Fork of Dust.js’, or ‘dustjs-linkedin’ as it is known to NPM.

I followed this story with mild amusement and shaking my head until the end of 2013, when PayPal saw the Node.js light. As part of their Node.js conversion, they picked LinkedIn’s fork of Dust.js for their templating needs. This reminded me of how penguins jump into water – they all wait until one of them jumps, then they all follow in a quick succession. Channeling my own inner penguin, I decided the water was fine and started playing with dustjs-linkedin myself.

This is not my first foray into the world of Node.js, but in my first attempt I used Jade, which is just too DRY for my taste. Being a long time Eclipse user, I just could not revert to a command line, so I resorted to a collection of Web Development tools, then added Nodeclipse, mostly for the project creation wizard and the launcher. Eclipse is very important to me because it answers one of the key issues plaguing Node.js developers that go beyond ‘Hello, World’ – how do I control and structure all the JavaScript files (incidentally one of the hard questions that Zef Hemel posed in his blog post on blank canvas projects).

Then again, Nodeclipse is not perfect, and dustjs-linkedin is not one of the rendering engines they cover in the project wizard. I had to create an Express project configured for Jade, turn around and delete Jade from the project, and use NPM to install dustjs-linkedin locally (i.e. in the project tree under ‘node_modules’), like so:


After working with Nodeclipse for a while, and not being able to use their Node launcher (I had to configure my own external tool launcher), I am now questioning its value but at least it got the initial structure set up for me. Now that I have a good handle of the overall structure, I could create new Node projects myself, so general purpose Web tooling with HTML, JSON and JavaScript editors and helpers may be all you need (of course, you need to also install Node.js and NPM but you would need to do it in all scenarios).

Hooking up nodejs-linkedin also requires consolidate.js in a way that is a bit puzzling to me but it seems to work well, so I didn’t question it (the author is TJ of Express fame, and exploring the code I noticed that nodejs-linkedin is actually listed as one of the recognized engines). The change required to pull in dustjs-linkedin and consolidate, declare dust as a new engine and map it as the view engine for the app:

var express = require('express')
, routes = require('./routes')
, dust = require('dustjs-linkedin')
, cons = require('consolidate')
, user = require('./routes/user')
, http = require('http')
, path = require('path');

var app = express();

// all environments
app.set('port', process.env.PORT || 3000);
app.set('views', __dirname + '/views');
app.engine('dust', cons.dust);
app.set('view engine', 'dust');

That was pretty painless so far. We have configured our views to be in the /views directory, so Dust files placed there can be directly found by Express. Since Express is an MVC framework (although much lighter than what you are normally used to in the JEE world), the C part is handled by routers, placed in the /routes directory. Our small example will have just a landing page rendered by /views/index.dust and the controller will be /routes/index.js. However, to add a bit of interest, I will toss in block partials, showing how to create a base template, then override the ‘payload’ in the child template.

We will start by defining the base template in ‘layout.dust’:

<!DOCTYPE html>
    <link rel='stylesheet' href='/stylesheets/style.css' />
	This is the base content.

We can now include this template in ‘index.dust’ and define the content section:

This is loaded from a partial.
Another paragraph.

We now need to define index.js controller in /routes because the controller invokes the view for rendering:

* GET home page.
exports.index = function(req, res) {
           { title: '30% Turtleneck, 70% Hoodie' });

In the code above we are sending the result of rendering the view using Dust to the response. We specify the collection of key/value pairs that will be used by Dust for variable substitution. The only part left is to hook up our controller to the site path (our root) in app.js:

app.get('/', routes.index);

And that is it. Running our Node.js app using Nodeclipse launcher will make it start listening on the localhost port 3000:


So far so good, and it is probably best to stop while we are ahead. We have a Node.js project in Eclipse configured to use Express and LinkedIn’s fork of Dust.js for templating, and everything is working. In my next installment, I will dive into Dust.js in earnest. This is going to be fun!

© Dejan Glozic, 2014

Rocket Science vs System Integration

By certified su from The Atherton Tablelands, Queensland , Australia (Integration) [CC-BY-2.0 ( or CC-BY-2.0 (], via Wikimedia Commons

Note: In case you missed it, there is still a chance to get a free eBook of RESS Essentials from Packt Publishing. All you need to do is check the book out at Packt Publishing web site, drop a line about the book back at the comments section of my article with your email address and enter a draw for one of the three free e-books. And now, back to regular programming.

When I was in the army, my mother told me that she kept noticing soldiers on the streets. I mean, they didn’t suddenly appear, they were always there, but she only started noticing them because of me. Once I returned from my regular 12 month service, soldiers around her receded into oblivion again.

This is a neat trick that you can try yourself. Have you noticed that you start spotting other cars like your own on the road as soon as you buy it, even though you didn’t care less about them before (unless you really, really wanted it, in which case you were spotting it as an object of desire, anticipating the purchase)? If you care to learn another useless phrase, the scholarly term for this phenomenon is perceptual vigilance.

When we were building web apps in the 00’s, a number of technologies were considered a ‘solved problem’. You get yourself an Apache server, install it on a Linux box, strap on Tomcat for servlets (all right, you can use PHP instead) and add MySQL for storage. Of course, there were commercial versions that offered, well, ‘bigger’ but the basic principles were the same. Nobody was excited about serving pages, or storing data, or doing anything commoditized that way. The action was in solving the actual end-user problem, and the sooner you can assemble a system out of the off-the-shelf components and get to the problem solving part, the better.

Well, once you need to serve millions upon millions of users, strike the ‘solved’ part. You start hitting the limits of your components, and you need to start paying attention to the things you stopped noticing years ago. That’s what is now happening across the software development world. The ‘solved’ problems are being solved again, from the very bottom. Web servers, data storage, caching, page templates, message queues – everything is being revisited and reopened. It is actually fascinating to watch – like a total gutting and rebuilding of a house whose owners grew tired of the basement flooding every time it rains. It is as if millions of developers suddenly noticed the components that they were taking for granted all these years and said ‘let’s rebuild everything from scratch’.

When you look at a Node.js ‘Hello, World’ server example, it is fascinating in its simplicity until you consider how much stuff you need to add to get to the level of functionality that is actually needed in production. You are really looking at several dozens of JS modules pulled in by NPM. Using the house paradigm again, these modules are replacing all the stuff you had in your house before you reduced it to a hole in the ground. And in this frenzy of solving the old problems in the new world I see two kinds of approaches – those of idealists and pragmatists.

There is a feeding frenzy on GitHub right now to be the famous author of a great new library or framework. To stay in the Node.js neighborhood, the basement is taken by @ryah, most of the walls are solidly put by @tjholowaychuk (although there are some new walls being added), and so on. Idealists fear that if they do not hurry, the house will be rebuilt and there will be no glory left in the component business – that they will need to go back to be pragmatists, assemblers of parts.

For some people that is almost an insult. Consider the following comment by Bhaskar Ghosh, LinkedIn’s senior director of data infrastructure engineering in a recent article by Gigaom:

Ghosh, well, he shot down the idea of relying too heavily on commercial technologies or existing open source projects almost as soon as he suggests it’s a possibility. “We think very carefully about where we should do rocket science,” he told me, before quickly adding, “[but] you don’t want to become a systems integration shop.”

Of course, Bhaskar is exaggerating to prove the point – LinkedIn is both a heavy provider and a consumer of Open Source projects. Consider this (incomplete) list for example – they are not exactly shy using off the shelf code where it makes sense. Twitter is no stranger to Open Source as well – they share the use of Netty with LinkedIn and many other (albeit heavilly modified) open source projects. Nevertheless, Twitter shares the desire to give back to the Open Source world – my team together with many others are using their awesome Bootstrap toolkit, and their modifications are shared with the community. You can read it all at the Twitter Open Source page, run by our good friend @cra.

Recently, PayPal has switched a lot of their Java production servers to Node.js and documented the process. The keyboards are still hot from the effort and they are already open sourcing many of the libraries they wrote for the effort. They even joined forces with LinkedIn on the fork of the Dust.js templating library that the owner abandoned (kudos to Richard Ragan from PayPal from writing a great tutorial).

There are many other examples that paint a picture of a continuum. Idealists see this as a great time to have their name attached to a popular Open Source project and cement their place in the source code Hall of Fame – it’s 1997 all over again, baby!. On the other hand, startups have no problem being pragmatists – it is very hard to build something from scratch even with off the shelf Open Source software – who has time to write new libraries when existing ones are good enough to drive your startup into the ground 10 times over.

In the space between the two extremes, companies with something to lose and with skin in the game stride the cautious middle road – use what works, modify what ‘almost’ works, write their own when no existing software fits the bill, fully aware of the immediate and ongoing cost. And the grateful among them give back to the Open Source community as a way of saying ‘thanks’ for getting a boost when they themselves where but a fledgling startup.

© Dejan Glozic, 2014

Of Mice and Men


The best laid schemes of mice and men
Often go awry.

Robert Burns, 1785.

Dear readers of this blog, you may have noticed a lapse in the rhythm that I faithfully followed from the very beginning (a new post every Tuesday, more recently switching to Monday). You may have inferred that I am ‘recharging’, away or simply taking a holiday break.

Far from it. I had a plan to look back at the blog and the topics I have covered over the last six months. For example, looking back at the post on databases, I have second thoughts about using SQL database in a way that is a better fit for something like MongoDB. Our practical experience is that schema changes in a rapid evolution environment are a real drag and being able to evolve the data on demand in a schema-less database would be really beneficial to us.

Other posts are holding up. We are as averse to the Extreme AJAX as ever, but we are now seeing use cases where some client-side structure (possibly using Backbone.js) would be beneficial, particularly when using Web Sockets to push updates to the client. We are still allergic to frameworks and their aggressive takeover of your life and control over code. And we are still unhappy with the horrible hacks required to implement client side templating. One day when Web Components are supported by all the modern browsers, we will remember this time as an ugly intermezzo when we had to resort to a lot of JavaScript in lieu of first class Web composition that Web Components provide (you can bookmark this claim and come to laugh at me if Web Components do not deliver on their promise).

As I said, I planned to cover all that, and also lined up a holiday project. We had recently implemented nice dashboards for our needs in Jazz Platform and Jazz Hub, and I was curious how hard it would be to re-implement the server side using Node.js and MongoDB (instead of JEE and SQL DB). The client side is already fine, using jQuery, Bootstrap and Require.js, and does not need to change. I also wanted to see how hard it would be to configure Express.js to use LinkedIn fork of the Dust.js templating library. PayPal team had a lot of fun recently moving their production applications to exactly that stack, and I was inspired by Bill Scott’s account of that effort.

The plan was to write the blog post on Sunday 22nd, and play with Node.js over the next two weeks. On Saturday 21st, eastern US and Canada were hit by a nasty ice storm. The storm deposited huge amounts of ice on the electrical wires and trees in midtown Toronto where I live. Ice is very heavy and after a while, huge mature trees that are otherwise great to have around started giving up, losing limbs and knocking down power lines in the process. By Sunday 22nd 3am we lost power, along with about a million people in Toronto at the peak.

Losing power in a low rise means no heat, no Internet or cable, no cooking and no hot water (add ‘no water’ for a high rise). We conserved our phones, following the progress of Toronto Hydro power restoration using the #darkTO hashtag on Twitter (everything else was useless). Toronto Hydro outage map crashed under load, resulting in Toronto Hydro posting the map via Twitter picture updates (kudos for Twitter robust infrastructure, which is more than I can say about Toronto Hydro servers). After a while, we drained our phones. I charged my phone from a fully charged MacBook Pro (an operation I was able to repeat twice before MacBook Pro lost its juice). We had four laptops to use as door stops/phone chargers. I could read some eBooks on a fully charged iPad, but somehow was not in the mood. Dinner was cold sandwiches by the candle light. Not as romantic in a cold room.

By Sunday night, the temperature in the apartment dropped to 19.5C (that’s 67F for my American friends). We slept fully clothed. On Monday morning we packed up and went to IBM building in Markham that had power to shower, get some core temperature back, eat a warm meal and charge all the devices. We also used the opportunity to book a hotel room in Toronto downtown (no big trees to knock down power lines – yey for the big soul-less downtown buildings). When we went back home to pack, the room temperature dropped to 18C. The temperature outside dropped to bitterly cold -10C, and going to -14C over night.

Over Monday night, the power was restored. We returned on Tuesday morning, only to find Internet and cable inoperative. Estimated time for repair – 22 hours. In addition, our building is somewhat old and its hot water furnace does not like going from 0 to full blast quickly, resulting in a temperature that was 2-3 degrees less than usual. It was a matter of time until I succumbed to common cold.

Internet and cable was restored on Wednesday, 4 days after the outage started. Over the last couple of days the winter outside let up a bit, allowing the ice on the trees to melt and furnace to bring the temperature to the normal levels. My cold is on the downswing, enough for me to write this blog post. I will still need to wait for the cold-induced watery eyes to return to normal before taking the planned photos for my 2014 Internet profiles.

Why am I writing all this? Just to show you that the real takeaway message for 2013 is not horrible first world problems we grapple with daily (should I use Node.js or not, should I use MongoDB or Couch DB or just stay with RDBs), but that most of us are privileged to live with the trappings of civilization allowing us to not worry about warm water, heat, food and clean clothing. On Wednesday when my daughter was seriously unhappy that internet was not back yet, I felt a bit like ‘the most ungrateful generation’ in the Louis CK show (“Everything is amazing and nobody is happy”). As I am writing this, there are still my fellow Torontonians without power. Their houses are probably icicles by now. My heart goes to them and hope they get power as soon as possible.

As for myself, I can tell you that the fact that when you write Node.js code you may find yourself in a callback hell didn’t mean much when I was sitting in a cold room, lit by candle light and frantically refreshing the #darkTO thread, while my battery was slowly draining. A lesson in humility and a reason to count your blessings delivered in the time of the year we normally see the re-runs of It’s a Wonderful Life on TV (if you still watch TV, that is).

Therefore, all of you reading this, have a great 2014! If you are in a warm room, have clean clothes and a warm meal, and can read this (i.e. your wifi is operational), your life is amazing and you are better off than many of your fellow human beings. Now go back to the most pressing topics of your lives, like this one:

© Dejan Glozic, 2013

Sitting on the Node.js Fence


I am a long standing fan of Garrison Keillor and his Prairie Home Companion. I fondly remember a Saturday in which he delivered his widely popular News From Lake Wobegon, and that contained the following conundrum: “Is ambivalence a bad thing? Well, yes and no”. It also accurately explains my feeling towards Node.js and the exploding Node community.

As I am writing this, I can hear the great late Joe Strummer singing Should I Stay or Should I Go in my head (“If I stay it will be trouble, if I go it will be double”). It is really hard to ignore the passion that Node.js has garnered from its enthusiastic supporters, and if the list of people you are following on Twitter is full of JavaScript lovers, Node.js becomes Kim Kardashian of your Twitter feed (as in, every third tweet is about it).

In case you were somehow preoccupied by, say, living life to the fullest to notice or care about Node.js, it is a server-side framework written in JavaScript sitting on a bedrock of C++, compiled and executed by Google’s Chrome V8 engine. A huge cottage industry sprung around it, and whatever you may need for your Web development, chances are there is a module that does it somewhere on GitHub. Two years ago, in a widely re-tweeted event, Node.js overtook Ruby on Rails as the most watched GitHub repository.

Recently, in a marked trend of maturation from exploration and small side projects, some serious companies started deploying Node into production. I heard everything about how this US Thanksgiving most traffic to was from mobile devices, handled by Node.js servers.

Then I heard from Nick Hart how PayPal switched from JEE to Node.js. A few months ago I stumbled upon an article about Node.js taking over the enterprise (like it or not, to quote the author). I am sure there are more heartwarming stories traded at a recent #NodeSummit. Still, not to be carried away, apart from mobile traffic, Node.js only serves a tiny panel for the web site, so more of a ‘foot in the door’ than a total rewrite for the desktop. Even earlier, LinkedIn engineering team blogged about their use of Node.js for LinkedIn Mobile.

It is very hard to cut through the noise and get to the core of this surge in popularity. When you dig deeper, you find multiple reasons, not all of them technical. That’s why discussions about Node.js sometimes sound to me like the Abott and Costello’s ‘Who’s on First’ routine. You may be discussing technology while other participants may be discussing hiring perks, or skill profile of their employees, or context switching. So lets count the ways why Node.js is popular and note how only one of them is actually technical:

  1. Node.js is written from the ground up to be asynchronous. It is optimized to handle problems where there is a lot of waiting for I/O. Processing tasks not waiting for I/O while others are queued up can increase throughput without adding processing and memory overhead of the traditional ‘one blocking thread per request’ model (or even worse, ‘one process per request’). The sweet spot is when all you need to do is lightly process the result of I/O and pass it along. If you need to spend more time to do some procedural number-crunching, you need to spawn a ‘worker’ child process, at which point you are re-implementing threading.
  2. Node.js is using JavaScript, allowing front-end developers already familiar with it to extend their reach into the server. Most new companies (i.e. not the Enterprise) have a skill pool skewed towards JavaScript developers (compared to the Enterprise world where Java, C/C++, C# etc. and similar are in the majority).
  3. New developers love Node.js and JavaScript and are drawn to it like moths to a candelabra, so having Node.js projects is a significant attraction when competing for resources in a developer-starved new IT economy.

Notice how only the first point is actually engineering-related. This is so prevalent that it can fit into a tweet, like so:

Then there is the issue of storage. There have been many libraries made available for Node.js to connect to SQL databases, but that would be like showing up at an Arcade Fire concert in a suit. Oh, wait, that actually happened:

Never mind then. Back to Node.js: why not going all the way and use something like MongoDB, storing JSON documents and using JavaScript for DB queries? So now not only do front end developers not need to chase down server-side Java guys to change the served HTML or JSON output, they don’t have to chase DBAs to change the DB models. My point is that once you add Node.js, it is highly likely you will revamp your entire stack and architecture, and that is a huge undertaking even without deadlines to spice up your life with a wallop of stress (making you park in a wrong parking spot in your building’s garage, as I did recently).

Now let’s try to apply all this to a talented team in a big corporation:

  1. The team is well versed in both Java (on the server) and JavaScript (on the client), and have no need to chase down Java developers to ‘add a link between pages’ as in the PayPal’s case. The team also knows how to write decent SQL queries and handle the model side of things.
  2. The team uses IDEs chock-full of useful tools, debuggers, incremental builders and other tooling that make for a great workflow, particularly when the going gets tough.
  3. The team needs to deliver something new on a tight schedule and is concerned about adding to the uncertainty of ‘what to build’ by also figuring out ‘how to build it’.
  4. There is a fear that things that the team has grown to expect as gravity (debugging, logging etc.) is still missing or immature.
  5. While there is a big deal made about ‘context switching’, the team does it naturally – they go from Java to JavaScript without missing a beat.
  6. There is a whole slew of complicated licensing reasons why some Open Source libraries cannot be used (a problem startups rarely face).

Mind you, this is a team of very bright developers that eat JavaScript for breakfast and feel very comfortable around jQuery, Require.js, Bootstrap etc. When we overlay the reasons to use Node.js now, the only one that still remains is handling huge number of requests without paying the RAM/CPU tax of blocking I/O, one thread per request.

As if things were not murky enough, Servlets 3.1 spec supported by JEE 7.0 now comes with asynchronous processing of requests (using NIO) and also protocol upgrade (from HTTP/S to WebSockets). These two things together mean that the team above has the option of using non-blocking I/O from the comfort of their Java environment. They also mean that they can write non-blocking I/O code that pushes data to the client using WebSockets. Both things are currently the key technical reasons to jump ship from Java to Node. I am sure Oracle added both things feeling the heat from the Node.js, but now that they are here, they may be just enough reason to not make the transition yet, considering the cost and overhead.

Update: after posting the article, I remembered that Twitter has used Netty for a while now to get the benefit of asynchronous I/O coupled by performance of JVM. Nevertheless, the preceding paragraph is for JEE developers who can now easily start playing with async I/O without changing stacks. Move along, nothing to see here.

Then there is also the ‘Facebook effect’. Social scientists have noticed the emergence of a new kind of depression caused by feeling bad about your life compared to carefully curated projections of other people’s lives on Facebook. I am yet to see a posting like “my life sucks and I did nothing interesting today” or “I think my life is passing me by” (I subsequently learned that Sam Roberts did say that in Brother Down, but he is just pretending to be depressed, so he does not count). Is it possible that I am only hearing about Node.js success stories, while failed Node.js projects are quietly shelved never to be spoken of again?

Well, not everybody is suffering in silence. We all heard about the famous Walmart memory leak that is now thankfully plugged. Or, since I already mentioned MongoDB, how about going knee-deep into BSON to recover your data after your MongoDB had a hardware failure. Or a brouhaha about the MongoDB 100GB scalability warning? Or a sane article by Felix Geisendörfer on when to use, and more importantly, when NOT to use Node.js (as Node.js core alumnus, he should know better than many). These are all stories of the front wave of adopters and the inevitable rough edges that will be filed down over time. The question is simply – should we be the one to do that or somebody can do the filing for us?

In case I sound like I am justifying reasons why we are not using Node.js yet, the situation is quite the opposite. I completely love the all-JavaScript stack and play with it constantly in my pet projects. In a hilarious twist, I am acting like a teenage Justin Bieber fan and the (younger) team lead of the aforementioned team needs to bring me down with a cold dose of reality. I don’t blame him – I like the Node.js-based stack in principle, while he would have to burn the midnight oil making it work at the enterprise scale, and debugging hard problems that are inevitable with anything non-trivial. Leaving aside my bad case of FOMO (Fear Of Missing Out), he will only be persuaded with a problem where Node.js is clearly superior, not just another way of doing the same thing.

Ultimately, it boils down to whether you are trying to solve a unique problem or just play with a new and shiny technology. There is a class of problems that Node.js is perfectly suitable for. Then there are problems where it is not a good match. Leaving your HR and skills reasons aside, the proof is in the pudding (or as a colleague of mine would say, ‘the devil is in the pudding’). There has to be a unique problem where the existing stack is struggling, and where Node.js is clearly superior, to tip the scales.

While I personally think that Node.js is a big part of an exciting future, I have no choice but to agree with Zef Hemel that it is wise to pick your battles. I have to agree with Zef that if we were to rebuild something for the third time and know exactly what we are building, Node.js would be a great way to make that project fun. However, since we are in the ‘white space’ territory, choosing tried and true (albeit somewhat boring) building blocks and focusing the uncertainty on what we want to build is a good tradeoff.

And that’s where we are now – with me sitting on the proverbial fence, on a constant lookout for that special and unique problem that will finally open the Node.js floodgates for us. When that happens, you will be the first to know.

© Dejan Glozic, 2013