Benchmarking Falcor.js

In the past few weeks I was figuring out how to solve a problem of a chatty communication between client and server considering that we are close to the release and I have already got few ideas in mind to improve the product I’m working on right now.

Thinking to possible solutions I thought to search online few different approaches that could help me out to solve the problem, at the beginning I was thinking to refactor the REST APIs in order to create them closer to what the UI needs (backend for frontend pattern) but then I remembered that I had bookmarked few projects that could help me out to implement this pattern without reinventing the wheel.
So on friday night I started to investigate Falcor.js, a library made by Netflix that was trying to solve exactly my same problem and they honestly solve the issue in a really smart way.

Let’s imagine you have a client that needs to call several REST end points in order to aggregate the data for a specific view, independently from how large is the amount of data to retrieve you have to bear in mind few other drawbacks like:

  • latency
  • no caching on specific data because they are real time data or tight to a specific user
  • amount of data to display in a view (maybe without a paging API available to split them in several views)
  • pre-flight calls for CORS end points
  • internet connection speed (mobility vs home vs office)
  • content negotiation

All of these, and probably many others, could be causes of a bad user experience and quite often we postpone to address these problems after the release of our online products.
So if we can minimise the impact in somehow we could provide a better user experience and therefore our products could be faster, more interesting and raise a good success with our users also when they have poor connection signal on their favourite device.

Here is where Falcor.js comes in support, in fact with Falcor we can minimise the amount of calls to specific end points because this library is leveraging the idea of a unique data model that could be interrogate by our clients asynchronously via Falcor APIs optimising the amount of queries to it under the hood.
The query system allows not only to fetch data from Falcor model but also to get only the data you need to use in a specific view.

services-diagram
From Falcor.js website

Looking at the image above you can spotted immediately the possible problem that Falcor solves brilliantly with a unique model.
In fact the Router is aggregating data from different end points and therefore the client can request exactly what needs in a nice and simple way.

Let’s try to explain how the system works first:

falcor-end-to-end
From Falcor.js website

The client will create a connection to a JSON model using the HTTPDataSource provided by Falcor.js client library that will allow to start the connection between the client and the backend data model.
On the backend a Falcor router is created and inside this instance there will be the description of the different queries available and what are the data to return as router system.
Doing this each client will download only the data that effectively needs to render the page and not an element more (sometimes reducing drastically the amount of data to load).
Also each client won’t need to interrogate ad hoc services for retrieving ad hoc data created for it but will just query the same data model created for the entire application only querying different end points.
As you can see from the diagram above this part sits between the client and your backend system creating a middleware that could be use only for specific end points or for the entire application.

Another interesting characteristic of Falcor is how it can optimise the query to the data model, in fact activating the batch mode, Falcor will gather all the queries to a specific route in a tick of your application performing all of them and possibly optimising to a unique roundtrip the requests instead of multiple roundtrips!

Last but not least Falcor allows you to query the APIs implementing a paging mechanism when you are iterating elements inside lists. For instance if you have an array of elements to display in your view but the APIs provided by the backend team don’t include any paging parameter, Falcor helps you via the query system, retrieving only a certain amount of elements via the paging mechanism.

So after watching the few videos available on Falcor and reading all the documentation in the website I started to experiment directly on the chatty issue I’ve got in my project.
I can’t really share the code I’ve used for my spike mainly because I’m using the product end points but I can share with you some benchmarks and thoughts on that for now.

Currently the catalogue I’m working with is composed by 5 calls to 5 different end points in order to display THE catalogue inside the view.
There is also a timer where every few minutes these 5 calls are performed again in order to retrieve new data that possibly could refresh the products available for the user inside the page.
Obviously behind the scene the application is doing several other calls in order to synchronise its status and performing some checks, but for now let’s focus only on the catalogue part.
I extracted the 5 calls in a simple HTML page replicating the current situation in the product and gathering some metrics to understand which was the starting point and how much Falcor would be able to improve the situation.
These are the benchmark I’ve retrieved for 5 XHR calls to different end points when the data are cached and when they aren’t:

NO CACHED XHR (after 5 calls):
average time on 10 tests ~678ms
average kb on 10 tests ~111kb

CACHED XHR (after 5 calls):
average time on 10 tests ~126ms
average kb on 10 tests ~3.1 kb

Then what I did was implementing a Node.js gateway where using the Falcor Router and the Falcor model I was able to query only the data I needed (and not downloading the entire JSON with information not needed for that specific view), I’ve optimised the query to the Falcor model via batching requests and these are the results with Falcor in place in front of the product APIs:

NO CACHED WITH FALCOR (Falcor optimise the 5 calls to only 2 calls):
average time on 10 tests ~32ms
average kb on 10 tests ~5 kb

CACHED WITH FALCOR (Falcor optimise the 5 calls to only 2 calls):
average time on 10 tests ~10ms
average kb on 10 tests ~406 bytes

This is really a good performance boost considering the amount of data we downloaded by default that are now optimised via Falcor queries; several roundtrips saved because of the batching requests optimisation natively available in Falcor APIs and the nice asynchronous implementation to fetch data from the unique model.

I’ve to admit that I was really surprised and I believe sooner than later I’ll introduce Falcor.js in production because it can really simplify the pipeline of work and provide great benefits to your applications in particular when you are targeting different low end devices like in my project.
Another think that would be good to invest time on(maybe another weekend :P) is GraphQL, the main “competitor”, maybe I’ll be able to do a 1 to 1 comparison based on the same problem and see which is the best library for a specific problem.

Meanwhile if you want to start playing with it I recommend Falcor website and I encourage you to keep an eye on this blog because I’d like to share in another post more technical information about Falcor that will allow you to understand how easy is working with it.

 

How to Dockerize your Node application

Today I’d love to share how to wrap a Node microservice or monolithic application inside a Docker container.
I assume that you’ve already installed Docker, boot2docker and Node in your machine.
If you don’t, please check the official Docker page and Node page and after picking your operating system follow the instructions.

I’ve created a simple Node application with Hapi.js that once called returns the classic “hello world” message as response, obviously you can have a way more complex application as well, my main purpose here is talking about how to setup Docker with your Node.js application.

In order to wrap your application inside a Docker container you need to create a Dockerfile, this file is basically defining the setup for the environment where your application will run.
Checking the Docker containers currently available on Docker hub, you can see for Node the official container page that will allow you to pick the right Node version for your application.

Screen Shot 2016-04-03 at 17.01.57

As you can see there are many different Node containers available, you can also use other solution like using ubuntu, fedora or centOS as base OS where your node application is going to run but I preferred to use the official one because my server side application doesn’t require any particular configuration.
For this example we are going to use the 5-onbuild that basically is doing the “dirty job” for us, obviously this is a sample application but you can customise your container as you prefer.
What the onbuild version is doing is basically:

  • creating a directory of the application inside the container
  • copying package.json in that folder
  • running “npm install” command
  • running “npm start” command

It’s fundamental to define inside the package.json all the dependencies and the start script otherwise the application is not going to work inside the container.

Inside our Dockerfile we are going to write:

FROM node:5-onbuild
EXPOSE 8080

So basically we are inheriting all the steps described above regarding the onbuild Dockerfile plus we are exposing the port 8080 that is the one used by our Node application:

const server = new Hapi.Server();
server.connection({ port: 8080 });

It’s a best practice for any Dockerfile starting always with a FROM command and reusing the images that are already created by the community on Docker hub so remember when you want to try something different check always what’s available on Docker hub.

Good so now let’s package our container and try if it works correctly.
In order to build the container we’ll need to run the following command:

docker build -t <username>/<applicationName> .

Basically here we are saying to docker to dockerize the entire folder application (dot at the end of the command)  and the container will be called <userName>/<applicationName>
This is another Docker best practice, potentially you can call your container as you prefer but a pattern is adding first your Docker username then slash (“/”) then your application name:

docker build -t lucamezzalira/docker-hapi .

Not let’s try if it works:

docker run -p 49160:8080 -d lucamezzalira/docker-hapi

With this command we are running our container, therefore our Node application, mapping the port 8080 of the container to the port 49160 of the host.
You can check easily if the application is working correctly just typing docker ps in your console, you should see something like that:

Screen Shot 2016-04-03 at 17.29.48

So now, because I’m working on Mac, I need to retrieve the boot2docker IP and check if in the port 49160 I’m able to see my hello world application up and running.
In order to do that I’ll run the command:

boot2docker ip

That should return the external IP where my application  is running, you can also see which is the container IP using the command:

docker inspect CONTAINER ID (for example: docker inspect afb5810152f6)

The container ID is easily retrievable via docker ps (first column in the picture above).
This command will return a JSON file with a lot of information related to the container.

So now that I’ve the IP where my application is running and the related port  I can type inside my bowser the address and see the result!

Screen Shot 2016-04-03 at 17.36.00

Obviously you can also map that IP to etc/hosts file adding something more meaningful like boot2docker or whatever name your think is more appropriate!

If you want to download from Docker hub the container I’ve prepared for this post just type:

docker pull lucamezzalira/docker-hapi

This is a very basic introduction to Docker world and Node, I’m working on a sample microservices application that will involve few interesting concepts and pattern, so keep an eye on this blog 😉