Should We Fight or Embrace the DOM?

Dom Tower, Utrecht, 2013, Anitha Mani (Wikimedia Commons)
Dom Tower, Utrecht, 2013, Anitha Mani (Wikimedia Commons)

Now, my story is not as interesting as it is long.

Abe Simpson

One of the privileges (or curses) of experience is that you amass a growing number of cautionary tales with which you can bore your younger audience to death. On the other hand, knowing the history of slavery came in handy for Captain Pickard to recognize that what Federation was trying to do by studying Data is create an army of Datas, not necessarily benefit the humanity. So experience can come in handy once in a while.

So let’s see if my past experience can inform a topic de jour.

AWT, Swing and SWT

The first Java windowing system (AWT) was based on whatever the underlying OS had to offer. The original decision was to ensure the same Java program runs anywhere, necessitating a ‘least common denominator’ approach. This translated to UIs that sucked equally on all platforms, not exactly something to get excited about. Nevertheless, they embraced the OS, inheriting both the shortcomings and receiving the automatic improvements.

The subsequent Swing library took a radically different approach, essentially taking on the responsibility of rendering everything. It was ‘fighting the OS’, or at least side-stepping it by creating and controlling its own reality. In the process, it also became responsible for keeping up with the OS. The Eclipse project learned that fighting the OS is trench warfare that is never really ‘won’. Using an alternative system (SWT) that accepted the windowing system of the underlying OS turned out to be a good strategic decision, both in terms of the elusive ‘look and feel’, and for riding the OS version waves as they sweep in.

The 80/20 of custom widgets

When I was working on the Eclipse project, I had my own moment of ‘sidestepping’ the OS by implementing Eclipse Forms. Since browsers were not ready yet, I wrote a rudimentary engine that gave me text reflows, hyperlinks and images. This widget was very useful when mixed with other normal OS widgets inside Eclipse UI. As you could predict, I got the basic behavior fairly quickly (the ’80’ part). Then I spent couple of years (including help from younger colleagues), doing the ‘last mile’ (the ’20’) – keyboard, accessibility, BIDI. It was never ‘finished’ and it was never quite as good as the ‘real’ browser, and not nearly as powerful.

One of the elements of that particular custom widget was managing the layout of its components. In essence the container was managing a collection of components but the layout of the components was delegated to the layout manager that could be set on the container. This is an important characteristic that will come in handy later in the article. I remember the layout class as one of the trickiest and hardest to get done right and fully debug. After it was ‘sort of’ working correctly, everybody dreaded touching it, and consequently forgot how it worked.

DOM is awesome

I gave you this Abe Simson moment of reflection to set the stage for a battle that is raging today between people who want to work with the browser’s DOM, and people who think it is the root of all evil and should be worked around. As is often these days, both points of view came across my twitter feed from different directions.

In the ’embrace the DOM’ corner, we have the Web Components crowd, who are thinking that DOM is just fine. In fact, they want us to expand it to turn it into a universal component model (instead of buying into the ‘bolt on’ component models of widget libraries). I cannot wait for it: I always hated the barrier of entry for Web libraries. In order to start reusing components today, you first need to buy into the bolt-on component model (not unlike needing to buy another set top box in order to start enjoying programming from a new content provider).

‘Embracing the DOM’ means a lot of things, and in a widely retweeted article by Reto Schläpfer about React.js, he argued that the current MV* client side framework treat DOM as the view, managing data event flow ‘outside the DOM’. Reto highlights the example of React.js library as an alternative, where the DOM that already manages the layout of your view can be pressed into double-duty of serving as the ‘nervous system’.

This is not entirely new, and has been used successfully elsewhere. I wrote previously on DOM event bubbling used in Bootstrap that we used successfully in our own code. Our realization that with it we didn’t feel the need for MVC is now echoed by React.js. In both cases, layout and application events (as opposed to data events) are fused – layout hierarchy is used as a scaffolding for the event paths to flow, using the build-in DOM behavior.

For completeness, not all people would go as far as claim that React.js obviates the need for client-side MVC – for example, Backbone.js has been shown to play nicely with React.js.

DOM is awful

In the other corner are those that believe that DOM (or at least the layout part of it) is broken beyond repair and should be sidestepped. My micro-service camarade de guerre Adrian Rossouw seems to be quite smitten with the Famo.us framework. This being Adrian, he approached this is the usual comprehensive way, collecting all the relevant articles using wayfinder.co (I am becoming increasingly spoiled/addicted to this way of capturing Internet wisdom on a particular topic).

Studying Famo.us is an archetype of a red herring – while its goal is to allow you build beautiful apps using JavaScript, transformations and animation, the element relevant to this discussion is that they sidestep DOM as the layout engine. You create trees and use transforms, which Famo.us uses to manage the DOM as an implementation detail, mostly as a flat list of nodes. Now recall my Abe Simpson story about SWT containers and components – doesn’t it ring similar to you? A flat list of components and a layout manager on top of it controlling the layout as manifestation of a strategy pattern.

Here is what Famo.us has to say about they approach to DOM for layouts and events:

If you inspect a website running Famo.us, you’ll notice the DOM is very flat: most elements are siblings of one another. Inspect any other website, and you’ll see the DOM is highly nested. Famo.us takes a radically different approach to HTML from a conventional website. We keep the structure of HTML in JavaScript, and to us, HTML is more like a list of things to draw to the screen than the source of truth of a website.

Developers are used to nesting HTML elements because that’s the way to get relative positioning, event bubbling, and semantic structure. However, there is a cost to each of these: relative positioning causes slow page reflows on animating content; event bubbling is expensive when event propagation is not carefully managed; and semantic structure is not well separated from visual rendering in HTML.

They are not the only one with ‘the DOM is broken’ message. Steven Wittens in his Shadow DOM blog post argues a similar position:

Unfortunately HTML is crufty, CSS is annoying and the DOM’s unwieldy. Hence we now have libraries like React. It creates its own virtual DOM just to be able to manipulate the real one—the Agile Bureaucracy design pattern.

The more we can avoid the DOM, the better. But why? And can we fix it?

……

CSS should be limited to style and typography. We can define a real layout system next to it rather than on top of it. The two can combine in something that still includes semantic HTML fragments, but wraps layout as a first class citizen. We shouldn’t be afraid to embrace a modular web page made of isolated sections, connected by reference instead of hierarchy.

Beware what you are signing up for

I would have liked to have a verdict for you by the end of the article, but I don’t. I feel the pain of both camps, and can see the merits of both approaches. I am sure the ‘sidestep the DOM’ camp can make their libraries work today, and demonstrate how they are successfully addressing the problems plaguing the DOM in the current browser implementations.

But based on my prior experience with the sidestepping approach, I call for caution. I will also draw on my experience as a father of two. When a young couple goes through the first pregnancy, they focus on the first 9 months, culminating in the delivery. This focus is so sharp and short-sighted that many of the couples are genuinely bewildered when the hospital hands them their baby and kicks them to the hospital entrance, with newly purchased car seat safely secured in the back. It only dawns on them at that point that baby is forever – that the bundle of joy is now their responsibility for life.

With that metaphor in mind, I worry about taking over the DOM’s responsibility for layout. Not necessarily for what it means today, but couple of years down the road when both the standards and the browser implementations inevitably evolve. Will it turn into a trench warfare that cannot be won, a war of attrition that drains resources and results in abandoned libraries and frameworks?

Maybe I can figure that one out after a nap.

© Dejan Glozic, 2014

Advertisements

Design for the T-Rex Vision

T-rex_by_Lilyu

Note: In case you missed it, there is currently a chance to get a free eBook of RESS Essentials from Packt Publishing. All you need to do is check the book out at Packt Publishing web site, drop a line about the book back at the comments section of my article with your email address and enter a draw for one of the three free books. And now, back to regular programming.

As a proud owner of a rocking home theater system, I think I played Jurassic Park about a gazillion times (sorry, neighbors). There I got some essential life lessons, for example that T-Rex’s vision is based on movement – it does not see you if you don’t move. While I am still waiting to put that knowledge into actual use, it got me thinking recently about design, and the need to update and change the visuals of our Web applications.

Our microwave recently died, and we had to buy a new one. Once installed, the difference was large enough to make me very conscious about its presence in the kitchen. But a funny thing happen – after a few days of me getting used to it, I stopped seeing it. I mean, I did see it, otherwise where did I warm up my cereals in the morning? But I don’t see it – it just blends into the overall kitchen-ness around me. There’s that T-Rex vision in practice.

Recently an article by Addison Duvall – Is Cool Design Really Uncool caught my attention because it posits that new design trends get old much too soon thanks to the quick spreading via the Internet. This puts pressure on the designers to constantly chase the next trend, making them the perennial followers. Addison offers an alternative (the ‘true blue’ approach) in which a designer ‘sticks to his/her guns’ as a personal mark or style, but concedes that only a select few can actually be trend setters instead of followers. Probably due to them turning into purveyors of a ‘niche’ product and their annoying need to eat and clothe themselves, thus nudging them back into the follower crowd.

According to Mashable, Apple reported that 74% of their devices are already running iOS 7. This means that 74% of iPhone and iPad users look at flat, Retina-optimized and non-skeuomorphic designs every day (yours truly included). When I showed the examples of iOS 7 around the office when I updated to it, I got a mixed reaction, with some lovers of the old school Apple design truly shocked, and in a bad way. Not myself though – as a true Steven Spielberg disciple, I knew that it will be just a matter of time until this becomes ‘the new normal’ (whether there will be tears on my cheeks and the realization that ‘I love Big Brother’ is still moot). My vision is based on movement, and after a while, iOS 7 stopped moving and I don’t see it any more. You know what I did see? When iPad Retina arrived with iOS 6 installed and I looked at the horrible, horrible 3D everywhere (yes, I missed iPad Air by a month – on the upside, it is company-provided, so the price is right). I could not update to iOS 7 soon enough.

I guess we established that for many people (most people in fact), the only way around T-Rex vision is movement. That’s why we refresh the designs ever so often. In the words of Don Draper and his ‘Greek mentor’, the most exciting word in advertizing is ‘new’. Not ‘better’, ‘nicer’, ‘more powerful’, just ‘new’ – different, not the same old thing you have been starting at forever (well, for a year anyway, which seems like eternity in hindsight). The designs do not spoil like a (Greek) yogurt in your fridge, they just become old to us. This is not very different from my teenage daughter staring at a full closet and declaring ‘I have nothing to wear’. It is not factually true but in her teenage T-Rex eyes, the closet is empty.

OK then, so we need to keep refreshing the look of our Web apps. What does that mean in practice? Nothing good, it turns out. You would think that in a true spirit of separation of style and semantics, all you need to do is update a few CSS files and off you go.

Not so fast. You will almost certainly hit the following bummers along the way:

  1. Look is not only ‘skin deep’. In the olden days when we used Dojo/Dijit, custom widgets were made out of DOM elements (DIVs, SPANs) and then styled. This was when these widgets needed to run on IE6 (shudder). This means that updating the look means changing the DOM structure of the widgets, and the accompanied JavaScript.
  2. If the widget is reused, there is high likelihood that upstream clients have reached into the DOM and made references to these DOM nodes. Why? Because they can (there is no ‘private’ visibility in JavaScript). I really, really hate that and cannot wait for shadow DOM to become a reality. Until then, try updating one of those reused widgets and wait for the horrified upstream shrieks. Every time we moved to the new MINOR version of Dojo/Dijit, there was blood everywhere. This is no way to live.
  3. Aha, you say, but the newer way is so much better. Look at Bootstrap – it is mostly CSS, with only 8kB of minified gzipped JavaScript. Yes, I agree – the new world we live in is nicer, with native HTML elements directly styleable. Nevertheless, as we now use Boostrap 3.0, we wonder what would happen if we modify the vanilla CSS and then try to move to 4.0 when it arrives – how much work will that be? And please don’t start plugging Zurb Foundation now (as in “ooh, it has more subtle styles, it is easier to skin than Bootstrap”). I know it is nice, but in my experience, nothing is ever easy – it would just be difficult in a somewhat different way.
  4. You cannot move to the new look only partially. That’s the wonder of the new app-based, Cloud-based world that never ceases to amaze me. Yes, you can build a system out of loosely coupled services running in the Cloud, and it will work great as long as these services are giving you JSON or some other data. But if you try to assemble a composite UI, the browser is where the metaphorical rubber meets the road – woe unto you if your components are not all moved to the new look in lockstep – you will have a case of the Gryphon UI on your hands.

I wish I had some reassuring words for you, but I don’t. This is a hard problem, and it is depressing that the federated world of the Cloudy future didn’t change that complexity even a bit. In the end, everything ends up in a single browser window, and needs to match. Unless all your apps adopt something like Bootstrap, don’t change the default styles and move to the each new version diligently, you will suffer the periodic pain of re-skinning so that the T-Rexes that use your UIs can see it again (for a short while). It also helps to have sufficient control over all the moving parts so that coordinated moves can actually be made and followed through, in the way LinkedIn moved from several stacks and server side rendering to shared Dust.js templates on the client. Taking advantage of HTML5, CSS3 and shadow DOM will lessen the pain in the future by increasing separation between style and semantics, but it will never entirely diminish it.

If you think this is limited to the desktop, think again. Yes, I know that in theory you need to be visually consistent within the confines of your mobile app only, which is a much smaller world, but you also need to not look out of place after the major mobile OS update. Some apps on my iPhone caught up, some still look awkward. Facebook app was flat for a while already. Twitter is mostly content anyway, so they didn’t need to do much. Recently Dropbox app refreshed its look and is now all airy with lightweight icons and hair lines everywhere, while slate.com app is still chunky with 3D gradients (I guess they didn’t budget for frequent app refreshes). Oh, well – I guess that’s the price of doing pixel business – you need to budget for the new virtual clothes regularly.

Oh, look – my WordPress updated the Dashboard – looks so pretty, so – new!

© Dejan Glozic, 2013

Beam My Model Up, Scotty!

star-trek-transporter

I know, I know. Scotty is from the original series, and the picture above is from TNG. Unlike Leonard from The Big Bang Theory, I prefer TNG over the original, and also Picard over Kirk. Please refrain from hate mail.

Much of both real and virtual ink has been spilled over Star Trek transporter technology and quirks. Who can forget the episode when Scotty has preserved himself as a transporter pattern in an endless loop only to be found and freed 70 years later by the TNG crew (see, there’s your Scotty-TNG link). But this article is about honouring all the hours I spent on various Star Trek instalments by applying the transporter principles to Web development.

As the Web development collective mind matures, a consensus is forming that spaghetti are great on your dining table but bad for your code. MVC design pattern on the server is just accepted as gravity, and several JavaScript frameworks are claiming it is necessary on the client as well. But is it always?

Here is case in point. Our team recently rebuilt a dashboard – a relatively complex peace of interactive technology. This is our third attempt at it (third time’s the charm, right)? Armed with all the experience of the previous two versions, plus our general desire to pull back from the extreme Ajax, we were confident this would be it. Why build a dashboard? Yes, I know, everybody and his uncle has one, and there are perfectly good all singing, all dancing commercial ones to be had. Here is a counter-question: why do you have a kitchen in your house when you can eat at perfectly good restaurants? Everybody has a dashboard because it is much more convenient to have your own (Google Analytics has one, WordPress I am writing this on has one). They can be simple because they don’t need to cater to all the possible scenarios, and much more tightly integrated. And not to mention cheaper (like eating at home vs. eating out). But I digress.

In the previous version, we sent a lot of JavaScript to the client, and after transporting and parsing it, JavaScript turned around and used XHR to fetch the model from the server as JSON. Then we wired up the model and the view and every time users made a change, we would update the model to keep it in sync. Backbone.js or Angular.js would have come in handy for this, only they didn’t exist when we wrote that code. Plus we hate frameworks.

In this version we wanted the dashboard page to arrive mostly assembled from the server. That part was clear. But we wanted to avoid the MV* complexity and performance consequences if possible. Plus it is really hard to even apply the usual data binding with dashboards because you cannot just wire the template with the model and let the framework do its magic. In a dashboard, the template itself is editable – users can drag and drop widgets around, delete them and add new ones. Essentially both the template and the data are built up from the model. This is not your grandmother’s MVC, that’s for sure.

Then we remembered Star Trek transporters and figured – why don’t we disperse the model and embed the model shards into the DOM as we are building the initial HTML on the server? We were waiting to hit a brick wall at some point, but it didn’t happen – this is an actually viable option. Here is why this approach (I will call it M/V) works for us like a charm:

  1. Obviously it is easy to do on the server – when we are building the initial response, embedding model shards into the HTML is trivial using HTML5 custom data properties (those attributes that start with ‘data-‘).
  2. The model arrives ready to use on the client – no need for an extra request to fetch the model after the fact. OK, Backbone.js has a way of bootstrapping models so that they arrive as payload with the initial response. However, that approach qualifies as CrazyHacks™ at best, and you save nothing payload-wise – the sum of the shards equals one whole model.
  3. When the part of the DOM needs to change due to the user interaction, it is easy to keep the associated ‘data-*’ properties in sync.
  4. When moving entire DOM branches (say, when dragging widgets between columns or changing layouts), model shards stay with their DOM elements – this is a real winner in our use case.
  5. We use DOM custom event bubbling to notify about the change – one listener at the top of the DOM branch is all that is needed to keep track of the sharded model’s dirty state. This helps us to stay lean because there is no need for JavaScript pub/sub implementations that increase size. It is laughably trivial to fire and listen to these events using jQuery, and they are fast, being implemented natively by the browser. Bootstrap uses the same approach.
  6. When the time comes to save, we traverse the DOM, collect the shards and assemble them again into a transient JavaScript object, then send it as JSON to the server via XHR. We use PUT/POST when we have the entire model, but more often the not, the view only renders a fraction of it (lazy loading), so we use PATCH instead.

This approach bought us some unreal savings in performance and size. Right now, we brought down total JavaScript from ~245kB to ~70KB minified gzipped (this includes jQuery, Require.js and Bootstrap). The best part is that this JavaScript loads asynchronously, after the initial content has already been displayed. The page is not only objectively fast, it is subjectively even faster because the initial content (what Twitter team calls ‘time to first tweet’) is sent in the first response.

As it is often the case in life, this approach may not work for everybody. A pattern where true MVC really shines is when you have multiple views listening to the same model. Changing the model updates all the views simultaneously, without the need for the pub/sub hell. But if you have exactly one view and one model, using M/V becomes a real option.

I can hear the collective gasp of architects in the audience. There are many things you can say about M/V, but architecturally pure it is not. But here is the problem. Speed and page size should be a valid architectural concern too. In fact, if you want your Web app to be usable on mobile devices, it is practically the only concern that matters. Nobody is going to wait your architecturally pure pig of a page to load while waiting for coffee – life is too short for 1MB+ web pages, pure or not.

Say No to layer cake, embrace small, live long and prosper!

© Dejan Glozic, 2013