The Genius of Bootstrap (OK, and Foundation)

Credit: Carlos Paes, 2005, Wikimedia Commons
Credit: Carlos Paes, 2005, Wikimedia Commons

This week we spent a lot of time sifting through the available options for the client side Web component model. We were doing it in the context of figuring out what to use for the next generation of Bluemix, so we were really trying to think hard and strategic. It is a strange time to do this. Web Components are so close you can touch them (on Chrome at least), but the days you can code against the entire standard and not bat an eyelash are still further into the future than we would have liked (the same can be said for ES6 – the future is going to be great, just wait a little longer).

You must believe

In its core, the Web is based on linked documents. That didn’t change all these years not matter how much exciting interactive stuff we managed to cram on top. In fact, when people fond of the founding principles cry ‘don’t break the Web’, they mostly rail on approaches that create black holes in the Web universe – domains where rules of the Web such as the ability to crawl the DOM, follow the links and browser history stop applying.

By and large, Web document is consumed as a whole by the browsers. There is no native HTML component model, at least not in the way similar to CSS and JavaScript. It is possible to include any number of modular CSS files, and any number of individual JavaScript libraries (not that it is particularly healthy for your performance). Not so for your markup – in fact browsers are positively hostile to content coming from other places (I don’t blame them because security).

In that climate, any component model so far was mounted on top of a library or framework. Before you can use jQuery widgets, you need jQuery to provide the plug-in component model. All the solutions to date were necessarily two-part: first you buy into a particular buffet table aka proprietary component model, then you can fill up your plate from the said buffet. This is nerve-racking – you must pick the particular model that you think will work for you and stay with your project long enough (and be maintained in the future). Rolling a complete set of building blocks on your own is very expensive, but so is being locked into a wrong library or framework.

Client side only

Another problem that most of the usual offerings share is that they are unapologetically client side. What it means is that a typical component will provide some dummy content such as ‘Please wait…’ if it shows content, or nothing if it is a ‘building block’ widget of some kind. Only after JavaScript loads will it spring to life, which means show anything useful. Widgets that are shown on user input (calendar picker being the quintessential example) suffer no ill consequences from this approach, but if you put client-side-only widgets on the main page, your SEO, performance and user experience will suffer.

Whether this is of importance to you depends on where you stand on the ‘server vs client side’ religious war. Twitter made it very clear that loading JavaScript, making an XHR request back to the mother ship for data, and then rendering the data on the client is not working for them in their seminal 2012 blog post. I am against it as well, as we were bitten hard with bad initial performance of large JavaScript SPAs. YMMV.

Hidden DOM

Web Components as a standard bring another thing to the table: hidden DOM. When you add a custom component to your page, the buck stops at the component boundary – parent styles will not leak into the component, and DOM queries will not include elements inside the custom component. This yields vital encapsulation currently possible only using iframes, with all the nastiness they bring to the table. However, it also makes it hard to style and provide initial state of the components while rendering the page on the server.

In theory, Node.js may allow us to run JavaScript on the server and construct the initial content (again, a theory, I am not sure it is actually possible without ugly hacks). Even if possible, it would not work for other server stacks. Essentially Web Components want you to just drop the component in your markup, set a few properties and let it do its stuff, which in most cases means ‘nothing’ until JavaScript for the component loads.

Model transfiguration

One of the perennial problems of starting your rendering on the server and resuming on the client is model transfer. You had to do some work on the server to curate data required to render the component’s initial state. It would be a waste to discard this data and let the JavaScript for the component go through the same process again when loaded. There are two different approaches to this:

  1. Model embedding – during server side rendering, nuggets of data are embedded in the markup using HTML5 data-* properties. Client side JavaScript uses these nuggets to reconstruct the model without the need to make network requests.
  2. Model bootstrapping – this approach is used by some MV* frameworks (e.g. BackboneJS). You can construct your component’s model, use it to render on the server, then inline the model as text in HTML to be eval-ed on the client. The result is the same – model is ready and does not need to be synced with the server, necessitating a network request.

Enter Bootstrap

Our experience with proprietary web components was mostly with Dojo/Dijit, since IBM made a sizeable investment in this open source library and a ton of products were written using it. It has all the characteristics of a walled garden – before you sample from its buffet (Dijit), you need to buy into the widget component model that Dojo Core provides. Once you do it, you cannot mix and match with Prototype, YUI, or jQuery UI. This is not an exclusive fault of Dojo – all JavaScript component models are like this.

Remember when I told you how Twitter wanted to be able to send something from the server ready for the browser to consume? When we first discovered Bootstrap, we were smitten by its approach. Were were looking for a proprietary widget system to which we had to sell our souls but failed to find it (in fact, the Bootstrap creator Mark Otto expressed open distaste for components that require extensive JavaScript).

Consider:

  1. There is no hidden DOM. There is just plain HTML that is styled by Bootstrap CSS.
  2. This HTML can arrive from the server, or can be dynamically created by JavaScript – no distinction.
  3. Behaviour is added via jQuery plug-ins.
  4. Plug-ins look for Bootstrap components in the DOM and attach event listeners, and start the dynamic behaviour (e.g. Carousel).
  5. The data needed by JavaScript is extracted from ‘data-*’ properties in HTML, and can be programmatically modified once JavaScript loads (model embedding, remember?).

Considering Twitter’s blog post on server side rendering, it is no wonder Bootstrap is incredibly easy to put to use in such a context. You don’t pass a list of entries to the ‘menu’ component, only to be turned into a menu when JavaScript loads. Your menu is simply an ‘ul’ element, with menu items being ‘li’ elements that are just styled to look like a menu. Thanks to CSS3, a lot of animation and special effects are provided natively by the browser, without the need for custom JavaScript to slow down your page. As a result, Bootstrap is really mostly CSS with a sprinkling of JavaScript for behavior (no surprise because it grew out of Twitter’s style guide document).

<div class="dropdown">
   <button class="btn btn-default dropdown-toggle" type="button" id="dropdownMenu1" data-toggle="dropdown" aria-expanded="true">
      Dropdown
      <span class="caret"></span>
   </button>
   <ul class="dropdown-menu" role="menu" aria-labelledby="dropdownMenu1">
      <li role="presentation"><a role="menuitem" tabindex="-1" href="#">Action</a></li>
      <li role="presentation"><a role="menuitem" tabindex="-1" href="#">Another action</a></li>
      <li role="presentation"><a role="menuitem" tabindex="-1" href="#">Something else here</a></li>
      <li role="presentation"><a role="menuitem" tabindex="-1" href="#">Separated link</a></li>
   </ul>
</div>

How important this is for your use case depends on the components. Building block components such as menus, nav bars, tabs, containers, carousels etc. really benefit from server-side construction because they can be immediately rendered by the browser, making your page feel very snappy and immediately useful. The rest of the page can be progressively enhanced as JavaScript arrives and client-side-only components are added to the mix.

If server side is not important to you, Web Components custom element approach seems more elegant:

<fancy-dropdown></fancy-dropdown>

The rest of the markup visible in Bootstrap example is all in the hidden DOM. Neat, except if you want something rendered on the server as well.

Truth to be told, it seems to be possible to create Web Components that act similarly to Bootstrap components. In fact, there is a demo showing a selection of Bootstrap components re-imagined as custom elements. I don’t know how real or ‘correct’ this is, just adding it to the mix for completeness. What is not clear is whether this is merely possible or actually encouraged for all custom element creators.

Haters gonna hate

Bootstrap is currently in its third major version and has been immensely popular, but for somewhat different reasons than I listed here. It comes with a very usable, fresh and modern looking theme that many developers use as-is, never bothering to customize. As a result, there are many cookie-cutter web sites out there, particularly if put together by individuals rather than brand-sensitive corporations and startups.

This has created a massive wave of hate from designers. In the pre-Bootstrap days, developers normally could not design if their life depended on it, putting designers on the critical path for every single UI. Now, most internal, prototype and throwaway apps and sites can look ‘good enough’, freeing up designers to focus on big, long running projects and clients looking to impart their own ‘design language’ on their properties.

I would claim that while Bootstrap as-is may not be suitable for a real professional product, Bootstrap approach is something that should not be thrown away with the bathwater. I know that ‘theming Bootstrap’ sounds like ‘Cuba Libre without the rum’ (note for teetotalers – it’s just Coke). If a toolkit is mostly CSS, and you replace it, what is left? Well, what is left are class names, documentation, jQuery plug-ins and the general approach. A small team of designers and developers can create a unique product or company theme, and the army of developers can continue to use all of Bootstrap documentation without any change.

I know many a company designer is tempted to ‘start fresh’ and build a custom system, but it a much bigger job than it looks like, and is not much different from just theming Bootstrap, with the difference being that you are now on the hook to provide JavaScript for behavior and extensively document it. You can create themes that transform Bootstrap beyond recognition, demonstrated in the Bootstrap Expo. And it is a massive challenge to match the open source network effect (599 contributors, 10,495 commits).

Devil’s Advocate

In the past, there were complains that Bootstrap is bloated (which can be addressed to a degree by cherry-picking Less/Sass files and building a custom CSS), not accessible (this is getting better over time), and has too many accessor rules (no change here). Another complaint is that when a component doesn’t quite do what is desired, modifications eventually cost more than if the component was written from scratch.

I have no problem buying any and all of these complaints, but still claim that the approach is more important than the actual design system. In fact, I put Zurb’s Foundation in the title to indicate a competitor that uses an identical approach (styling HTML with jQuery for behaviour). I could use either (in fact, I have a growing appreciation for Foundation’s clean and understated look that is less immediately recognizable compared to Bootstrap). And the community numbers are nothing to sneeze at (603 contributors, 7,919 commits).

So your point is…

My point is that before thinking about reusable Web components for your project, settle on a design system, be it customized Bootstrap, Foundation or your own. This will ensure a design language fit for your product, and will leave a lot of options open for the actual implementation of user interfaces. Only then should you think of client-side-only components, and you should only use them for building blocks that you can afford to load lazily.

© Dejan Glozic, 2014

Rocket Science vs System Integration

By certified su from The Atherton Tablelands, Queensland , Australia (Integration) [CC-BY-2.0 (http://creativecommons.org/licenses/by/2.0) or CC-BY-2.0 (http://creativecommons.org/licenses/by/2.0)], via Wikimedia Commons

Note: In case you missed it, there is still a chance to get a free eBook of RESS Essentials from Packt Publishing. All you need to do is check the book out at Packt Publishing web site, drop a line about the book back at the comments section of my article with your email address and enter a draw for one of the three free e-books. And now, back to regular programming.

When I was in the army, my mother told me that she kept noticing soldiers on the streets. I mean, they didn’t suddenly appear, they were always there, but she only started noticing them because of me. Once I returned from my regular 12 month service, soldiers around her receded into oblivion again.

This is a neat trick that you can try yourself. Have you noticed that you start spotting other cars like your own on the road as soon as you buy it, even though you didn’t care less about them before (unless you really, really wanted it, in which case you were spotting it as an object of desire, anticipating the purchase)? If you care to learn another useless phrase, the scholarly term for this phenomenon is perceptual vigilance.

When we were building web apps in the 00’s, a number of technologies were considered a ‘solved problem’. You get yourself an Apache server, install it on a Linux box, strap on Tomcat for servlets (all right, you can use PHP instead) and add MySQL for storage. Of course, there were commercial versions that offered, well, ‘bigger’ but the basic principles were the same. Nobody was excited about serving pages, or storing data, or doing anything commoditized that way. The action was in solving the actual end-user problem, and the sooner you can assemble a system out of the off-the-shelf components and get to the problem solving part, the better.

Well, once you need to serve millions upon millions of users, strike the ‘solved’ part. You start hitting the limits of your components, and you need to start paying attention to the things you stopped noticing years ago. That’s what is now happening across the software development world. The ‘solved’ problems are being solved again, from the very bottom. Web servers, data storage, caching, page templates, message queues – everything is being revisited and reopened. It is actually fascinating to watch – like a total gutting and rebuilding of a house whose owners grew tired of the basement flooding every time it rains. It is as if millions of developers suddenly noticed the components that they were taking for granted all these years and said ‘let’s rebuild everything from scratch’.

When you look at a Node.js ‘Hello, World’ server example, it is fascinating in its simplicity until you consider how much stuff you need to add to get to the level of functionality that is actually needed in production. You are really looking at several dozens of JS modules pulled in by NPM. Using the house paradigm again, these modules are replacing all the stuff you had in your house before you reduced it to a hole in the ground. And in this frenzy of solving the old problems in the new world I see two kinds of approaches – those of idealists and pragmatists.

There is a feeding frenzy on GitHub right now to be the famous author of a great new library or framework. To stay in the Node.js neighborhood, the basement is taken by @ryah, most of the walls are solidly put by @tjholowaychuk (although there are some new walls being added), and so on. Idealists fear that if they do not hurry, the house will be rebuilt and there will be no glory left in the component business – that they will need to go back to be pragmatists, assemblers of parts.

For some people that is almost an insult. Consider the following comment by Bhaskar Ghosh, LinkedIn’s senior director of data infrastructure engineering in a recent article by Gigaom:

Ghosh, well, he shot down the idea of relying too heavily on commercial technologies or existing open source projects almost as soon as he suggests it’s a possibility. “We think very carefully about where we should do rocket science,” he told me, before quickly adding, “[but] you don’t want to become a systems integration shop.”

Of course, Bhaskar is exaggerating to prove the point – LinkedIn is both a heavy provider and a consumer of Open Source projects. Consider this (incomplete) list for example – they are not exactly shy using off the shelf code where it makes sense. Twitter is no stranger to Open Source as well – they share the use of Netty with LinkedIn and many other (albeit heavilly modified) open source projects. Nevertheless, Twitter shares the desire to give back to the Open Source world – my team together with many others are using their awesome Bootstrap toolkit, and their modifications are shared with the community. You can read it all at the Twitter Open Source page, run by our good friend @cra.

Recently, PayPal has switched a lot of their Java production servers to Node.js and documented the process. The keyboards are still hot from the effort and they are already open sourcing many of the libraries they wrote for the effort. They even joined forces with LinkedIn on the fork of the Dust.js templating library that the owner abandoned (kudos to Richard Ragan from PayPal from writing a great tutorial).

There are many other examples that paint a picture of a continuum. Idealists see this as a great time to have their name attached to a popular Open Source project and cement their place in the source code Hall of Fame – it’s 1997 all over again, baby!. On the other hand, startups have no problem being pragmatists – it is very hard to build something from scratch even with off the shelf Open Source software – who has time to write new libraries when existing ones are good enough to drive your startup into the ground 10 times over.

In the space between the two extremes, companies with something to lose and with skin in the game stride the cautious middle road – use what works, modify what ‘almost’ works, write their own when no existing software fits the bill, fully aware of the immediate and ongoing cost. And the grateful among them give back to the Open Source community as a way of saying ‘thanks’ for getting a boost when they themselves where but a fledgling startup.

© Dejan Glozic, 2014

Sitting on the Node.js Fence

Sat_on_the_Fence_-_JM_Staniforth

I am a long standing fan of Garrison Keillor and his Prairie Home Companion. I fondly remember a Saturday in which he delivered his widely popular News From Lake Wobegon, and that contained the following conundrum: “Is ambivalence a bad thing? Well, yes and no”. It also accurately explains my feeling towards Node.js and the exploding Node community.

As I am writing this, I can hear the great late Joe Strummer singing Should I Stay or Should I Go in my head (“If I stay it will be trouble, if I go it will be double”). It is really hard to ignore the passion that Node.js has garnered from its enthusiastic supporters, and if the list of people you are following on Twitter is full of JavaScript lovers, Node.js becomes Kim Kardashian of your Twitter feed (as in, every third tweet is about it).

In case you were somehow preoccupied by, say, living life to the fullest to notice or care about Node.js, it is a server-side framework written in JavaScript sitting on a bedrock of C++, compiled and executed by Google’s Chrome V8 engine. A huge cottage industry sprung around it, and whatever you may need for your Web development, chances are there is a module that does it somewhere on GitHub. Two years ago, in a widely re-tweeted event, Node.js overtook Ruby on Rails as the most watched GitHub repository.

Recently, in a marked trend of maturation from exploration and small side projects, some serious companies started deploying Node into production. I heard everything about how this US Thanksgiving most traffic to Walmart.com was from mobile devices, handled by Node.js servers.

Then I heard from Nick Hart how PayPal switched from JEE to Node.js. A few months ago I stumbled upon an article about Node.js taking over the enterprise (like it or not, to quote the author). I am sure there are more heartwarming stories traded at a recent #NodeSummit. Still, not to be carried away, apart from mobile traffic, Node.js only serves a tiny panel for the Walmart.com web site, so more of a ‘foot in the door’ than a total rewrite for the desktop. Even earlier, LinkedIn engineering team blogged about their use of Node.js for LinkedIn Mobile.

It is very hard to cut through the noise and get to the core of this surge in popularity. When you dig deeper, you find multiple reasons, not all of them technical. That’s why discussions about Node.js sometimes sound to me like the Abott and Costello’s ‘Who’s on First’ routine. You may be discussing technology while other participants may be discussing hiring perks, or skill profile of their employees, or context switching. So lets count the ways why Node.js is popular and note how only one of them is actually technical:

  1. Node.js is written from the ground up to be asynchronous. It is optimized to handle problems where there is a lot of waiting for I/O. Processing tasks not waiting for I/O while others are queued up can increase throughput without adding processing and memory overhead of the traditional ‘one blocking thread per request’ model (or even worse, ‘one process per request’). The sweet spot is when all you need to do is lightly process the result of I/O and pass it along. If you need to spend more time to do some procedural number-crunching, you need to spawn a ‘worker’ child process, at which point you are re-implementing threading.
  2. Node.js is using JavaScript, allowing front-end developers already familiar with it to extend their reach into the server. Most new companies (i.e. not the Enterprise) have a skill pool skewed towards JavaScript developers (compared to the Enterprise world where Java, C/C++, C# etc. and similar are in the majority).
  3. New developers love Node.js and JavaScript and are drawn to it like moths to a candelabra, so having Node.js projects is a significant attraction when competing for resources in a developer-starved new IT economy.

Notice how only the first point is actually engineering-related. This is so prevalent that it can fit into a tweet, like so:

Then there is the issue of storage. There have been many libraries made available for Node.js to connect to SQL databases, but that would be like showing up at an Arcade Fire concert in a suit. Oh, wait, that actually happened:

Never mind then. Back to Node.js: why not going all the way and use something like MongoDB, storing JSON documents and using JavaScript for DB queries? So now not only do front end developers not need to chase down server-side Java guys to change the served HTML or JSON output, they don’t have to chase DBAs to change the DB models. My point is that once you add Node.js, it is highly likely you will revamp your entire stack and architecture, and that is a huge undertaking even without deadlines to spice up your life with a wallop of stress (making you park in a wrong parking spot in your building’s garage, as I did recently).

Now let’s try to apply all this to a talented team in a big corporation:

  1. The team is well versed in both Java (on the server) and JavaScript (on the client), and have no need to chase down Java developers to ‘add a link between pages’ as in the PayPal’s case. The team also knows how to write decent SQL queries and handle the model side of things.
  2. The team uses IDEs chock-full of useful tools, debuggers, incremental builders and other tooling that make for a great workflow, particularly when the going gets tough.
  3. The team needs to deliver something new on a tight schedule and is concerned about adding to the uncertainty of ‘what to build’ by also figuring out ‘how to build it’.
  4. There is a fear that things that the team has grown to expect as gravity (debugging, logging etc.) is still missing or immature.
  5. While there is a big deal made about ‘context switching’, the team does it naturally – they go from Java to JavaScript without missing a beat.
  6. There is a whole slew of complicated licensing reasons why some Open Source libraries cannot be used (a problem startups rarely face).

Mind you, this is a team of very bright developers that eat JavaScript for breakfast and feel very comfortable around jQuery, Require.js, Bootstrap etc. When we overlay the reasons to use Node.js now, the only one that still remains is handling huge number of requests without paying the RAM/CPU tax of blocking I/O, one thread per request.

As if things were not murky enough, Servlets 3.1 spec supported by JEE 7.0 now comes with asynchronous processing of requests (using NIO) and also protocol upgrade (from HTTP/S to WebSockets). These two things together mean that the team above has the option of using non-blocking I/O from the comfort of their Java environment. They also mean that they can write non-blocking I/O code that pushes data to the client using WebSockets. Both things are currently the key technical reasons to jump ship from Java to Node. I am sure Oracle added both things feeling the heat from the Node.js, but now that they are here, they may be just enough reason to not make the transition yet, considering the cost and overhead.

Update: after posting the article, I remembered that Twitter has used Netty for a while now to get the benefit of asynchronous I/O coupled by performance of JVM. Nevertheless, the preceding paragraph is for JEE developers who can now easily start playing with async I/O without changing stacks. Move along, nothing to see here.

Then there is also the ‘Facebook effect’. Social scientists have noticed the emergence of a new kind of depression caused by feeling bad about your life compared to carefully curated projections of other people’s lives on Facebook. I am yet to see a posting like “my life sucks and I did nothing interesting today” or “I think my life is passing me by” (I subsequently learned that Sam Roberts did say that in Brother Down, but he is just pretending to be depressed, so he does not count). Is it possible that I am only hearing about Node.js success stories, while failed Node.js projects are quietly shelved never to be spoken of again?

Well, not everybody is suffering in silence. We all heard about the famous Walmart memory leak that is now thankfully plugged. Or, since I already mentioned MongoDB, how about going knee-deep into BSON to recover your data after your MongoDB had a hardware failure. Or a brouhaha about the MongoDB 100GB scalability warning? Or a sane article by Felix Geisendörfer on when to use, and more importantly, when NOT to use Node.js (as Node.js core alumnus, he should know better than many). These are all stories of the front wave of adopters and the inevitable rough edges that will be filed down over time. The question is simply – should we be the one to do that or somebody can do the filing for us?

In case I sound like I am justifying reasons why we are not using Node.js yet, the situation is quite the opposite. I completely love the all-JavaScript stack and play with it constantly in my pet projects. In a hilarious twist, I am acting like a teenage Justin Bieber fan and the (younger) team lead of the aforementioned team needs to bring me down with a cold dose of reality. I don’t blame him – I like the Node.js-based stack in principle, while he would have to burn the midnight oil making it work at the enterprise scale, and debugging hard problems that are inevitable with anything non-trivial. Leaving aside my bad case of FOMO (Fear Of Missing Out), he will only be persuaded with a problem where Node.js is clearly superior, not just another way of doing the same thing.

Ultimately, it boils down to whether you are trying to solve a unique problem or just play with a new and shiny technology. There is a class of problems that Node.js is perfectly suitable for. Then there are problems where it is not a good match. Leaving your HR and skills reasons aside, the proof is in the pudding (or as a colleague of mine would say, ‘the devil is in the pudding’). There has to be a unique problem where the existing stack is struggling, and where Node.js is clearly superior, to tip the scales.

While I personally think that Node.js is a big part of an exciting future, I have no choice but to agree with Zef Hemel that it is wise to pick your battles. I have to agree with Zef that if we were to rebuild something for the third time and know exactly what we are building, Node.js would be a great way to make that project fun. However, since we are in the ‘white space’ territory, choosing tried and true (albeit somewhat boring) building blocks and focusing the uncertainty on what we want to build is a good tradeoff.

And that’s where we are now – with me sitting on the proverbial fence, on a constant lookout for that special and unique problem that will finally open the Node.js floodgates for us. When that happens, you will be the first to know.

© Dejan Glozic, 2013