Components Are (Still) Hard on the Web

matryoshka_dolls_3671820040_2

Here’s Johnny! I know, I know. It’s been a while I posted something. In my defence, I was busy gathering real world experience – the kind that allows me to forget the imposter syndrome for a while. We have been busy shipping a complex microservice system, allowing me to test a number of my blog posts in real life. Most are holding up pretty well, thank you very much. However, one thing that continues to bother me is that writing reusable components is still maddeningly hard on the Web in 2016.

Framing the problem

First, let’s define the problem. In a Web app of a sufficient complexity, there will be a number of components that you would want to reuse. You already know we are a large Node.js shop – reusing modules via NPM is second nature to us. It would be so nice to be able to npm install a Web component and just put it in your app. In fact, we do exactly that with React components. Alas, it is much harder once you leave the React world.

First of all, let’s define what ‘reusing a Web component’ would mean:

  1. You can put a component written by somebody else somewhere on your page.
  2. The component will work well and will not do something nasty.
  3. The component will not look out of place in your particular design.
  4. You will know what is happening inside the component, and will be able to react to its lifecycle.

First pick a buffet

Component (or widget) reuse was for the longest time a staple of desktop UI development. You bought into the component model by using the particular widget toolkit. That is not a big deal on Windows or MacOs – you have no choice if you want to make native applications. The same applies to native mobile development. However, on the Web there is no single component model. Beyond components that are intrinsic part of HTML, in order to create custom components you need to buy into one of the popular Web frameworks first. You need to pick the proverbial buffet first before you can sample from it.

In 2016, the key battle is currently between abstracting out and embracing the Web Platform. You can abstract the platform (HTML, CSS, DOM) using JavaScript. This is the approach used by React (and my team by extension). Alternatively, you can embrace the platform and use HTML as your base (what Web Components, Polymer and perhaps Angular 2 propose). You cannot mix and match – you need to pick your approach first.

OK, I lied. You CAN mix and match, but it becomes awkward and heavy. React abstracts out HTML but if you use a custom component instead of the built-in HTML, React will work fine. All the React traits (diff-ing the two incremental iterations of virtual DOM, then applying the difference to the actual DOM) works for custom components as well. Therefore, it is fine to slip in a Web Component into a React app.

The opposite does not really work well – consuming a React component in Angular or Polymer is awkward and not worth it really. Not that the original direction is worth it necessarily – you need to load Web Components JavaScript AND React JavaScript.

Don’t eat the poison mushroom

One of the ways people loaded components in their pages in the past was by using good old iframes. Think what you will about them, but you could really lock the components down that way. If you load a component into your own DOM, you need to really trust it. Single origin policy and CORS are supposed to help you prevent a component leaking data from your page to the mother ship. Nevertheless, particularly when it comes to more complex components, it pays to know what they are doing, go through the source code etc. This is where open source really helps – don’t load a black box component into your DOM.

The shoes don’t match the belt

One of the most complex problems to deal with when consuming a Web component of any type is the design. When you are working in a native SDK, the look and feel of the component is defined by the underlying toolkit. All iOS components have the ‘right’ look and feel out of the box when you consume them. However, Web apps have their own themes that create combinatorial explosion. A reusable component needs to do one of the following things:

  1. Be configurable and themeable so that you can either set a few parameters to better blend it into your style guide, or provide entire template to really dial it in
  2. Be generic and inoffensive enough to be equidistant from any parent theme
  3. Be instantly recognizable (think youtube player) in a way that makes it OK that it has its own look and feel.

A very complex reusable component with a number of elements can be very hard to dial in visually by consumers. In corporations, this may reduce the number of themes you want to support. A large component may take it upon itself to support 2-3 supported and widely used design style guides. Then all you need to do is provide a single parameter (style guide name) to make the component use the right styles across the board.

What is going on inside?

Adding a component into your page is not only a matter of placement and visual style. Virtually all reusable components are interactive. A component can be self-contained (for example, all activity in a youtube player is confined to its bounding box), or expected to interact with the parent. If the component must interact with the parent, you need to consider the abstraction chain. Consider the simple countdown timer as a reusable component. Here is how the abstraction chain works:

timer-abstraction

The timer itself uses two low-level components – ‘Start’ and ‘Stop’ buttons. Inside the timer, the code will add click listeners for both buttons. The listeners will add semantic meaning to the buttons by doing things according to their role – starting and stopping the timer.

Finally, when this component is consumed by your page, only one listener is available – ‘onTimerCountdown()’. Users will interact with the timer, and when the timer counts down to 0, the listener you registered will be notified. You should be able to expect events at the right semantic level from all reusable components, from the simplest calendars to large complex components.

If a component can be made part of a larger document, two things you will care the most is serialization and dirty state. When users interact with the component and make a modification, you want to be told that the component is changed. This should trigger the dirty state of the parent. When the user clicks ‘Save’, you should be able to serialize the component and store this state in the larger document. Inversely, on bootstrap you should be able to pass the serialized state to the component to initialize itself.

Note that the actual technology used does not matter here – even the components embedded using iframes can use window.postMessage to send events up to the parent (and accept messages from the parent). While components living in your DOM will resize automatically, iframe-ed components will need to also send resizing events via window.postMessage to allow the parent to set the new size of the iframe.

The long tail

More complex reusable components don’t only have client-side presence. They have a need to call back to the server and fetch the data they need. You can configure such a component in two ways:

  1. You can fetch the data it requires for the component. In that case, the component is fully dependent on the container and it is container’s responsibility to perform all the XHR calls to fetch the data and pass it to the component. This approach may be best for pages that want full control of the network calls. As an added bonus, you can fit such a component into a data flow such as Flux, where some of the data may be coming from Web Socket driven server-side push, not just XHR request.
  2. You can proxy the requests that the component is performing. This approach is also acceptable because it allows the proxy to control which third-party servers are going to be whitelisted.
  3. You can configure CORS so that the component can make direct calls on its own. This needs to be done carefully to avoid the component siphoning data from the page to servers you don’t approve.

On all of these cases you may still want to be told about the events inside the web component using the component events as discussed above.

Frameworks are just the beginning

So there you go – all the problems you need to wrestle with when trying to reuse components in a larger project. Chances are the component is written in the ‘wrong’ framework, but trying to make the component load in your page is only the beginning. Fitting the component into the page visually, figuring out what is happening in it events-wise, and feeding it data from the server is the real battle. Unless you are trying to load a calendar widget, this is where you will spend most of your time.

© Dejan Glozic, 2016

Advertisements

The Genius of Bootstrap (OK, and Foundation)

Credit: Carlos Paes, 2005, Wikimedia Commons
Credit: Carlos Paes, 2005, Wikimedia Commons

This week we spent a lot of time sifting through the available options for the client side Web component model. We were doing it in the context of figuring out what to use for the next generation of Bluemix, so we were really trying to think hard and strategic. It is a strange time to do this. Web Components are so close you can touch them (on Chrome at least), but the days you can code against the entire standard and not bat an eyelash are still further into the future than we would have liked (the same can be said for ES6 – the future is going to be great, just wait a little longer).

You must believe

In its core, the Web is based on linked documents. That didn’t change all these years not matter how much exciting interactive stuff we managed to cram on top. In fact, when people fond of the founding principles cry ‘don’t break the Web’, they mostly rail on approaches that create black holes in the Web universe – domains where rules of the Web such as the ability to crawl the DOM, follow the links and browser history stop applying.

By and large, Web document is consumed as a whole by the browsers. There is no native HTML component model, at least not in the way similar to CSS and JavaScript. It is possible to include any number of modular CSS files, and any number of individual JavaScript libraries (not that it is particularly healthy for your performance). Not so for your markup – in fact browsers are positively hostile to content coming from other places (I don’t blame them because security).

In that climate, any component model so far was mounted on top of a library or framework. Before you can use jQuery widgets, you need jQuery to provide the plug-in component model. All the solutions to date were necessarily two-part: first you buy into a particular buffet table aka proprietary component model, then you can fill up your plate from the said buffet. This is nerve-racking – you must pick the particular model that you think will work for you and stay with your project long enough (and be maintained in the future). Rolling a complete set of building blocks on your own is very expensive, but so is being locked into a wrong library or framework.

Client side only

Another problem that most of the usual offerings share is that they are unapologetically client side. What it means is that a typical component will provide some dummy content such as ‘Please wait…’ if it shows content, or nothing if it is a ‘building block’ widget of some kind. Only after JavaScript loads will it spring to life, which means show anything useful. Widgets that are shown on user input (calendar picker being the quintessential example) suffer no ill consequences from this approach, but if you put client-side-only widgets on the main page, your SEO, performance and user experience will suffer.

Whether this is of importance to you depends on where you stand on the ‘server vs client side’ religious war. Twitter made it very clear that loading JavaScript, making an XHR request back to the mother ship for data, and then rendering the data on the client is not working for them in their seminal 2012 blog post. I am against it as well, as we were bitten hard with bad initial performance of large JavaScript SPAs. YMMV.

Hidden DOM

Web Components as a standard bring another thing to the table: hidden DOM. When you add a custom component to your page, the buck stops at the component boundary – parent styles will not leak into the component, and DOM queries will not include elements inside the custom component. This yields vital encapsulation currently possible only using iframes, with all the nastiness they bring to the table. However, it also makes it hard to style and provide initial state of the components while rendering the page on the server.

In theory, Node.js may allow us to run JavaScript on the server and construct the initial content (again, a theory, I am not sure it is actually possible without ugly hacks). Even if possible, it would not work for other server stacks. Essentially Web Components want you to just drop the component in your markup, set a few properties and let it do its stuff, which in most cases means ‘nothing’ until JavaScript for the component loads.

Model transfiguration

One of the perennial problems of starting your rendering on the server and resuming on the client is model transfer. You had to do some work on the server to curate data required to render the component’s initial state. It would be a waste to discard this data and let the JavaScript for the component go through the same process again when loaded. There are two different approaches to this:

  1. Model embedding – during server side rendering, nuggets of data are embedded in the markup using HTML5 data-* properties. Client side JavaScript uses these nuggets to reconstruct the model without the need to make network requests.
  2. Model bootstrapping – this approach is used by some MV* frameworks (e.g. BackboneJS). You can construct your component’s model, use it to render on the server, then inline the model as text in HTML to be eval-ed on the client. The result is the same – model is ready and does not need to be synced with the server, necessitating a network request.

Enter Bootstrap

Our experience with proprietary web components was mostly with Dojo/Dijit, since IBM made a sizeable investment in this open source library and a ton of products were written using it. It has all the characteristics of a walled garden – before you sample from its buffet (Dijit), you need to buy into the widget component model that Dojo Core provides. Once you do it, you cannot mix and match with Prototype, YUI, or jQuery UI. This is not an exclusive fault of Dojo – all JavaScript component models are like this.

Remember when I told you how Twitter wanted to be able to send something from the server ready for the browser to consume? When we first discovered Bootstrap, we were smitten by its approach. Were were looking for a proprietary widget system to which we had to sell our souls but failed to find it (in fact, the Bootstrap creator Mark Otto expressed open distaste for components that require extensive JavaScript).

Consider:

  1. There is no hidden DOM. There is just plain HTML that is styled by Bootstrap CSS.
  2. This HTML can arrive from the server, or can be dynamically created by JavaScript – no distinction.
  3. Behaviour is added via jQuery plug-ins.
  4. Plug-ins look for Bootstrap components in the DOM and attach event listeners, and start the dynamic behaviour (e.g. Carousel).
  5. The data needed by JavaScript is extracted from ‘data-*’ properties in HTML, and can be programmatically modified once JavaScript loads (model embedding, remember?).

Considering Twitter’s blog post on server side rendering, it is no wonder Bootstrap is incredibly easy to put to use in such a context. You don’t pass a list of entries to the ‘menu’ component, only to be turned into a menu when JavaScript loads. Your menu is simply an ‘ul’ element, with menu items being ‘li’ elements that are just styled to look like a menu. Thanks to CSS3, a lot of animation and special effects are provided natively by the browser, without the need for custom JavaScript to slow down your page. As a result, Bootstrap is really mostly CSS with a sprinkling of JavaScript for behavior (no surprise because it grew out of Twitter’s style guide document).

<div class="dropdown">
   <button class="btn btn-default dropdown-toggle" type="button" id="dropdownMenu1" data-toggle="dropdown" aria-expanded="true">
      Dropdown
      <span class="caret"></span>
   </button>
   <ul class="dropdown-menu" role="menu" aria-labelledby="dropdownMenu1">
      <li role="presentation"><a role="menuitem" tabindex="-1" href="#">Action</a></li>
      <li role="presentation"><a role="menuitem" tabindex="-1" href="#">Another action</a></li>
      <li role="presentation"><a role="menuitem" tabindex="-1" href="#">Something else here</a></li>
      <li role="presentation"><a role="menuitem" tabindex="-1" href="#">Separated link</a></li>
   </ul>
</div>

How important this is for your use case depends on the components. Building block components such as menus, nav bars, tabs, containers, carousels etc. really benefit from server-side construction because they can be immediately rendered by the browser, making your page feel very snappy and immediately useful. The rest of the page can be progressively enhanced as JavaScript arrives and client-side-only components are added to the mix.

If server side is not important to you, Web Components custom element approach seems more elegant:

<fancy-dropdown></fancy-dropdown>

The rest of the markup visible in Bootstrap example is all in the hidden DOM. Neat, except if you want something rendered on the server as well.

Truth to be told, it seems to be possible to create Web Components that act similarly to Bootstrap components. In fact, there is a demo showing a selection of Bootstrap components re-imagined as custom elements. I don’t know how real or ‘correct’ this is, just adding it to the mix for completeness. What is not clear is whether this is merely possible or actually encouraged for all custom element creators.

Haters gonna hate

Bootstrap is currently in its third major version and has been immensely popular, but for somewhat different reasons than I listed here. It comes with a very usable, fresh and modern looking theme that many developers use as-is, never bothering to customize. As a result, there are many cookie-cutter web sites out there, particularly if put together by individuals rather than brand-sensitive corporations and startups.

This has created a massive wave of hate from designers. In the pre-Bootstrap days, developers normally could not design if their life depended on it, putting designers on the critical path for every single UI. Now, most internal, prototype and throwaway apps and sites can look ‘good enough’, freeing up designers to focus on big, long running projects and clients looking to impart their own ‘design language’ on their properties.

I would claim that while Bootstrap as-is may not be suitable for a real professional product, Bootstrap approach is something that should not be thrown away with the bathwater. I know that ‘theming Bootstrap’ sounds like ‘Cuba Libre without the rum’ (note for teetotalers – it’s just Coke). If a toolkit is mostly CSS, and you replace it, what is left? Well, what is left are class names, documentation, jQuery plug-ins and the general approach. A small team of designers and developers can create a unique product or company theme, and the army of developers can continue to use all of Bootstrap documentation without any change.

I know many a company designer is tempted to ‘start fresh’ and build a custom system, but it a much bigger job than it looks like, and is not much different from just theming Bootstrap, with the difference being that you are now on the hook to provide JavaScript for behavior and extensively document it. You can create themes that transform Bootstrap beyond recognition, demonstrated in the Bootstrap Expo. And it is a massive challenge to match the open source network effect (599 contributors, 10,495 commits).

Devil’s Advocate

In the past, there were complains that Bootstrap is bloated (which can be addressed to a degree by cherry-picking Less/Sass files and building a custom CSS), not accessible (this is getting better over time), and has too many accessor rules (no change here). Another complaint is that when a component doesn’t quite do what is desired, modifications eventually cost more than if the component was written from scratch.

I have no problem buying any and all of these complaints, but still claim that the approach is more important than the actual design system. In fact, I put Zurb’s Foundation in the title to indicate a competitor that uses an identical approach (styling HTML with jQuery for behaviour). I could use either (in fact, I have a growing appreciation for Foundation’s clean and understated look that is less immediately recognizable compared to Bootstrap). And the community numbers are nothing to sneeze at (603 contributors, 7,919 commits).

So your point is…

My point is that before thinking about reusable Web components for your project, settle on a design system, be it customized Bootstrap, Foundation or your own. This will ensure a design language fit for your product, and will leave a lot of options open for the actual implementation of user interfaces. Only then should you think of client-side-only components, and you should only use them for building blocks that you can afford to load lazily.

© Dejan Glozic, 2014