Reactive Programming with RxJS

In the past 6 months I spent quite few time trying to understand Reactive Programming and how it could help me on my daily job.
So I’d like to share in this post a quick example made with RxJS just to show you how Reactive Programming could help when you are handling asynchronous data streams.

If you are not familiar at all with these concepts I’d suggest to watch first my presentation on Communicating Sequential Process and Reactive Programming with RxJS (free registration) or check the slides below.

For this example I thought to create a basic bingo system that I think is a good asynchronous application example that fits perfectly with the Reactive Programming culture.
I won’t introduce in this blog post concepts like hot and cold observables, iterator pattern or observer pattern mainly because all these theoretical information are present in the webinar and the slides previously mentioned. 

You can clone the project repository directly from my git account.

Let’s start talking a little bit about the engine, basically a bingo system is composed by an engine where the numbers are called every few seconds and shared with the users in order to validate in which ticket bought by the user the number called is present.
For this purpose working with observables will facilitate the communication and the information flow between the engine and the ticket objects.
In BallsCallSystem class, after setting up the object creating few constant that we’re going to use inside the core engine, we’re going to implement the core functionality of the engine:

let stream = Rx.Observable
              .interval(INTERVAL)
              .map((value) => {return numbersToCall[value]})
              .take(TOTAL_CALLS);

These few lines of code are expressing the following intents:

  1. we create an observable (Rx.Observable)
  2. that every few milliseconds (interval method)
  3. iterate trough the interval values (incremental value from 0 to N) and return a value retrieved from the array numbersToCall (function described inside the map method)
  4. and after a certain amount of iteration we need to close the observable because the game is ended (take method) so all the observer will stop to execute their code

If we compare with an imperative programming implementation made with CSP (communicating sequential processes) I’ll end up having something similar to this one:

this.int = setInterval(this.sendData.bind(this), 3000);
[....]
sendData(){
   var val = this.numbersToCall[this.counter];
   console.log("ball called", val);
 
   csp.putAsync(this.channel, val);
   this.counter++;
 
   if(this.counter > this.numbersToCall.length){
      clearInterval(this.int);
      this.channel.close();
      console.log("GAME OVER");
   }
 }

As you can see I needed to express each single action I wanted to do in order to obtain the core functionality of my bingo system.
These 2 implementations are both solving exactly the same problem but as you can see the reactive implementation is way less verbose and easy to read than the imperative one where I’ve control of anything is happening inside the algorithm but at the same time I don’t really have a specific reason to do it.

Moving ahead with the reactive example, when we create an observable that streams data we always need an observer to retrieve these data.
So now let’s jump to Ticket class and see how we can validate against a ticket the numbers called by the engine

First of all we pass the observable via injection to a Ticket object:

let t2 = new Ticket("t2", engine.ballStream);

Then, inside the Ticket class we subscribe to the observable and we handle the different cases inside the stream (when we receive data, when an error occurs and when the stream will be terminated):

obs.subscribe(this.onData.bind(this), this.onError.bind(this), this.onComplete.bind(this));
onData(value){
    console.log("number called", value, this.tid);
    if(this.nums.indexOf(value) >= 0){
       this.totalNumsCalled.push(value);
       console.log(value + " is present in ticket " + this.tid);
    } 
 }
 
 onError(err){
    console.log("stream error:", err);
 }
 
 onComplete(){
    console.log("total numbers called in " + this.tid + ": " + this.totalNumsCalled.length);
    console.log(this.totalNumsCalled);
 }

Also here you can notice the simplicity of an implementation, for instance if we are working with React it will be very easy to handle the state of an hypothetical Ticket component and create a resilient and well structured view where each stream state is handled correctly.

An interesting benefit provided by reactive programming is for sure the simplicity and the modularity at the same time how our implementations are working.
I would really recommend to spend sometime watching the webinar in order to get the first approach to Reactive Programming and to understand better the purpose of the example described briefly above.

Hapi.js and MongoDB

During the Fullstack conference I saw a small project made with Hapi.js during a talk, so I decided to invest some time working with Hapi.js in order to investigate how easy it was create a Node.js application with this framework.

I’ve to admit, this is a framework really well done, with a plugin system that give you a lot of flexibility when you are creating your server side applications and with a decent community that provides a lot of useful information and plugins in order to speed up the projects development.

When I started to read the only book available on this framework I was impressed about the simplicity, the consideration behind the framework but more important I was impressed where Hapi.js was used for the first time.
The first enterprise app made with this framework was released during Black Friday on Walmart ecommerce. The results were amazing!
In fact one of the main contributor of this open source framework is Walmart labs, that means a big organisation with real problems to solve; definitely a good starting point!

Express vs Hapi.js

If you are asking why not express, I can reply with few arguments:

  • express is a super light and general purpose framework that works perfectly for small – medium size application.
  • hapi.js was built on top of express at the beginning but then they move away in order to create something more solid and with more built in functionalities, a framework should speed up your productivity and not giving you a structure to follow.
  • express is code base instead hapi.js is configuration base (with a lot of flexibility of course)
  • express uses middleware, hapi.js uses plugins
  • hapi.js is built with testing and security in mind!

Hapi.js

Let’s start saying working with this framework is incredibly easy when you understand the few concepts you need to know in order to create a Node project.

I created a sample project where I’ve integrated a mongo database, exposing few end points in order to add a new document inside a mongo collection, update a specific document, retrieve all documents available inside the database and  retrieving all the details of a selected document.

Inside the git repo you can find also the front end code (books.html in the project root) in Vanilla Javascript, mainly because if you are passionate about React or Angular or any other front end library, you’ll be able to understand the integration without any particular framework knowledge.

What I’m going to describe now will be how I’ve structured the server side code with Hapi.js.

In order to create a server in Hapi.js you just need few lines of code:

let server = new Hapi.Server();
server.connection();
server.start((err) => console.log('Server started at:', server.info.uri));

As you can see in the example (src/index.js) I’ve created the server in the first few lines after the require statements and I started the server (server.start) after the registration of the mongoDB plugin, but one step per time.

After creating the server object, I’ve defined my routes with server.route method.
The route method will allow you to set just 1 route with an object or several routes creating an array of objects.
Each route should contain the method parameter where you’ll define the method to reach the path, you can also set a wildcard (*) so any method will be accepted in order to retrieve that path.
Obviously then you have to set the route path, bear in mind you have to start always with slash (/) in order to define correctly the path.
The path accepts also variables inside curly brackets as you can see in the last route of my example: path: ‘/bookdetails/{id}’.

Last but not least you need to define what’s going to happen when a client is requesting that particular path specifying the handler property.
Handler expects a function with 2 parameters: request and reply.

This is a basic route implementation:

{
   method: 'GET',
   path: '/allbooks',
   handler: (request, reply) => { ... }
}

When you structure a real application, and not an example like this one, you can wrap the handler property inside the config property.
Config accepts an object that will become your controller for that route.
So as you can see it’s really up to you pick up the right design solution for your project, it could be inline because it’s a small project or a PoC rather than an external module because you have a large project where you want to structure properly your code in a MVC fashion way (we’ll see that in next blog post ;-)).
In my example I created the config property also because you can then use an awesome library called JOI in order to validate the data received from the client application.
Validate data with JOI is really simple:

validate: {
   payload: {
      title: Joi.string().required(),
      author: Joi.string().required(),
      pages: Joi.number().required(),
      category: Joi.string().required()
   }
}

In my example for instance I checked if I receive the correct amount of arguments (required()) and in the right format (string() or number()).

MongoDB plugin

Now that we have understood how to create a simple server with Hapi.js let’s go in deep on the Hapi.js plugin system, the most important part of this framework.
You can find several plugins created by the community, and on the official website you can find also a tutorial that explains step by step how to create a custom plugin for hapi.js.

In my example I used the hapi-mongodb plugin that allows me to connect a mongo database with my node.js application.
If you are more familiar with mongoose you can always use the mongoose plugin for Hapi.js.
One important thing to bear in mind of an Hapi.js plugin is that when it’s registered will be accessible from any handler method via request.server.plugins, so it’s injected automatically from the framework in order to facilitate the development flow.
So the first thing to do in order to use our mongodb plugin on our application is register it:

server.register({
   register: MongoDB,
   options: DBConfig.opts
}, (err) => {
   if (err) {
      console.error(err);
      throw err;
   }

   server.start((err) => console.log('Server started at:', server.info.uri));
});

As you can see I need just to specify which plugin I want to use in the register method and its configuration.
This is an example of the configuration you need to specify in order to connect your MongoDB instance with the application:

module.exports = {
   opts: {
      "url": "mongodb://username:password@id.mongolab.com:port/collection-name",       
      "settings": {          
         "db": {             
            "native_parser": false         
         }
      }    
   }
}

In my case the configuration is an external object where I specified the mongo database URL and the settings.
If you want a quick and free solution to use mongoDB on the cloud I can suggest mongolab, when you register you’ll have 500mb of data for free per account, so for testing purpose is really the perfect cloud service!
Last but not least, when the plugin registration happened I can start my server.

When I need to use your plugin inside any handler function I’ll be able to retrieve my plugin in this way:

var db = request.server.plugins['hapi-mongodb'].db;

In my sample application, I was able to create few cases: add a new document (addbook route), retrieve all the books (allbooks route) and the details of a specific book (bookdetails route).

Screen Shot 2015-12-04 at 23.44.38

If you want to update a record in mongo, remember to use update method over insert method, because, if correctly handled, update method will check inside your database if there are any other occurrences and if there is one it will update that occurrence otherwise it will create a new document inside the mongo collection.
Below an extract of this technique, where you specify in the first object the key for searching an item, then the object to replace with and last object you need to add is an object with upsert set to true (by default is false) that will allow you to create the new document if it doesn’t exist in your collection:

db.collection('books').updateOne({"title": request.payload.title}, dbDoc, {upsert: true}, (err, result) => {
    if(err) return reply(Boom.internal('Internal MongoDB error', err));
    return reply(result);
});

SAMPLE PROJECT GITHUB REPOSITORY

Resources

If you are interested to go more in deep about Hapi.js, I’d suggest to take a look to the official website or to the book currently available.
An interesting news is that there are other few books that will be published soon regarding Hapi.js:

that usually means Hapi js is getting adopt from several companies and developers and definitely it’s a good sign for the health of the framework.

Wrap up

In this post I shared with you a quick introduction to Hapi.js framework and his peculiarities.
If you’ve enjoyed please let me know what you would interested on so I’ll be able to prepare other posts with the topics you prefer.
Probably the next one will be on the different template systems (handlebars, react…) or about universal application (or isomorphic application as you prefer to call them) or a test drive of few plugins to use in Hapi.js web applications.

Anyway I’ll wait for your input as well 😀

2015 the birth of London JS community

We are very close to the end of 2015 and it’s time give a look back to the first 11 months of this year understanding what I’ve accomplished and trying to get some resolutions for the new year as well!

2015 for my professional life  was a year of changes: I moved to a new job, I learnt new programming paradigms (reactive and functional programming), I met a lot of incredible and talented  people and many many other things.

Probably the biggest changes (or old loves?) are the creation of London JavaScript community and my returning as speaker on technical talks after a couple of years of inactivity.

Last May I started a new adventure creating a new JavaScript community in London; I spent a month to understand how and why I could do that properly.
I had the possibility in the past to be staff member of the largest Actionscript Italian community and it was really a great experience.
I was young, with a lot of passion, not much experience and the community was the perfect place for growing properly, meeting new friends, learning from the different people, trying to solve technical puzzle everyday and having fun why not!

Starting again this experience in a new country it was a real challenge for me, but this time, with way more experience than the first time, clear ideas and a touch of madness.

In 2013 I had the opportunity to spend few months in Silicon Valley and I went to several meetup events and conferences from San Francisco to San Jose.
What I was really impressed of that environment was for sure the vibe I was able to breathe in any of these events.
People from all over the world that help each others, facilitating the connections between individuals, creating opportunities and sharing knowledge.
I was astonished about this way of doing community and when I moved from Italy to London I spent the first 18 months going to different events trying to retrieve the same experience and vibe.

When I started the London Javascript community my main goal was definitely recreating that vibe in this great city where the best developers all over the world are working in interesting projects with a lots of challenges to solve.
That’s why I decided to do that, filling a gap present in London communities where there were many and strong vertical framework communities (like React and Angular one), but not a strong and general JavaScript community.

After 8 months I’ve to admit I’m very happy about the results that this community was able to generate:

  • ~1400 members
  • 7 live events
  • 1 code lab for half day
  • 2 webinars
  • an average of 60 people per event
  • Listed as O’Reilly Community partner, Google Community and Skillsmatter Community
  • Community Partner of great events like: FullStack conference and Dot.JS conference
  • more than 1900 tweets & retweets with more than 500 followers

I had also the opportunity to meet great speakers, amazing developers and passionate people.

What this community is giving me back is really invaluable and really hard to find in similar activities too, I’m really grateful of any single moment spent organising any event.

What about the resolutions for 2016?

Recently I started to gather the interest from several companies that are asking me to organise meetup events in their offices and that means all the hard work done is paying this community back!

I contacted few speakers from California and next year I’m organising a couple of webinars with speakers from Silicon Valley and directly from Google office in Mountain View.

I’m already in contact with few great speakers ready to share with the community tips and tricks on different topics like webpack, jspm, angular, webrtc, react and ES6! Keep an eye on the community page in order to discover more about these events.

Last but not least, I’m working right now on the official community website that will be a SPA website with socials integration and the possibility to subscribe to a technical newsletter where I’m going to share the best articles, tutorials and events on the web.

Obviously these are already in the roadmap but I’m really open to listen what the JavaScript community is looking for! So don’t waste this opportunity and share your ideas on how to grow and make more special OUR community!

ES2015 Destructuring assignment: by value or reference?

This week I’ve organised a meetup on ES2015 in my community, where the speaker presented his favourite features of the language.

Right after the talk I had a chance to talk with my best friend that was asking if destructuring assigns the values copying the value to the new variable or instead by reference if you work with Object or Array.
Because I hadn’t a change before to work with this new ES2015 feature I did a quick example just to get an answer to this question.

It looks like destructuring feature works by reference and it’s not copying the value.
That means anytime you’re going to change a value inside a variable that contains an Object or Array assigned via destructuring, also the original Object or Array will be affected as you can see in this simple example: destructuring example ES2015

destructuring ES2015

So when you work with destructuring bear in mind to pay a lot of attention when you change a value inside your destructured Objects and Arrays!

Leading Lean Software Development: book review

It’s a long time that I don’t share thoughts regarding a book that I’ve read, but this time it’s totally worth the effort to write down this blog post.
I used to read between 15 to 20 books per year during my commuting time and I really enjoy the different points of views of the authors and it’s fascinating how the same topic can be treated in so many ways.

Leading Lean Software Development

The book that caught my attention is called Leading Lean Software Development from Addison Wesley.
I read many books on Lean and Agile topics but this one, I’ve to say, it’s one of the best for people like me that works in software development with these methodologies.
I strongly believe that there isn’t a framework that could rule them all, so I love learn and try different approaches like Scrum, Kanban, Kaizen, Grows and many others outside there and catch the essence in order to use the best technique when I need it.

This book does exactly that, it’s not only emphasising the importance of Lean methodologies but it’s mixing Lean methodologies with software engineering practices and give many example how these techniques helped to succeed not only in the software industry but also in the “real world”.

The most stunning example is regarding how the Empire State Building was created in less than a year and how it’s possible to have the same efficiency in the software development!

There so many take aways inside Leading Lean Software Development book that it’s hard to share all of them, but I think it’s a great collection of my believes in the Lean and software development world.
For the first time I could say that I’m represented by a book!
It’s a MUST read book in my opinion and the most interesting part is that there is another book written by the same authors way more practical that I can’t wait to read: Implementing Lean Software Development.

I hope you’ll find the time to read them and if you want to share other books with great value just leave a comment

Haxe-watchify: automatic build tool for Haxe and OpenFL projects

I’ve started recently working in a new company very focused on cross platform projects with Haxe.
In my commuting time I worked on an automatic build tool for Haxe and OpenFL projects.
The tool is called haxe-watchify and with a sample JSON file or directly through the command line, you’ll be able to setup how to continuously build your project in background during your development flow.
Haxe-watchify has got interesting features in particular for the Haxe target like the possibility to setup the completion server instead the traditional compiler to speed up the building of your projects.
In fact the completion server implements a cache system to build faster your projects, in this case haxe-watchify takes care for you to start the server and communicate with it.

Currently I’ve published the tool on npm registry so in order to install it just type in your CLI:

npm install haxe-watchify -g

I wrote an extensive documentation on how to use the tool on the readme file on the project repository otherwise you can check the –help command directly on your terminal window.
I tried for now only on Mac OS X so if you find any bug in any other platforms please let me know

I’ve already thought few possible implementations to add in the next releases like a pre and post build in order to launch your tests or run static analysis tool or assets optimisation and then move to the build.
Anyway I’m very keen to learn more about your current projects workflow and how haxe-watchify could help you to improve your situation.

if you want to share any comment please do feel free to share adding a comment to this post or via email

The power of feedback loops

Working in software development requires that we find the right techniques to achieve quality and customer satisfaction. Learning about development and flow techniques from experienced peers can be helpful. But if the team has a lack of senior peers or insufficient commitment, we may have to find alternative routes.

While studying how to improve my methods, I watched an interesting video wherein Noel Ford explained how useful feedback loops are in software development. Working from that premise, I started collecting and applying the best practices that could most help developers deliver software that meets customer expectations while also improving code quality.

First of all, let’s define a feedback loop. According to Wikipedia, “Feedback occurs when outputs of a system are ‘fed back’ as inputs as part of a chain of cause and effect that forms a circuit or loop.”

Any Agile or Lean framework incorporates this concept, but I think the closest implementation of feedback loop is defined inside kaizen with the PDCA cycle.

PDCA cycle

PDCA stands for “plan-do-check-act” and in kaizen is a technique used for continuous improvement of a company’s standards. The team tries first to analyze a problem, retrieve the metrics in the current situation, define a strategy for improvement toward achieving the intended result, and finally retrieve the metrics of the process to assess whether improvement really occurred so the team can adapt strategy to obtain better results and create a new standard.

This is a continuous process that should be carried out several times to aim for the best possible results.

Source: http://upload.wikimedia.org/wikipedia/commons/a/a8/PDCA_Process.png

Feedback loops in software development

As software creation follows an empirical process, we need to create several interim checkpoints to determine whether what we’re producing is really what our customers are looking for. Often during development we don’t realize how much time (and money) we waste simply because we don’t gather good specifications or show the progress of our work to the customer. Just because the waste is to a certain degree hidden doesn’t mean it is without cost; we often invest more time and expense than we realize, yet with a few shrewd tactics could actually prevent such loss and achieve what our customers really expect.

We can add several types of feedback loops during development to help accomplish that. The key rule of any feedback loop technique is “shorten, and often”!

Let’s go through the different types of feedback loops we can apply:

TDD/BDD/Unit Test

Test-driven development, behavior-driven development, or simple unit testing is the first feedback loop that should be established on any project.

It’s helpful for the healthiness of your project not only because helps you to better define the requirements, design better APIs, and obviously check the sanity of what you’ve written or refactored but also because testing gives you feedback on your work in seconds, and you can have this feedback every time you save a code file or when you push the code in your version control system.

Static analysis

Often we avoid proper static analysis in the belief that we lack the time to define what we need to analyze, but it doesn’t really take a huge effort to establish a good strategy, and the potential benefits are important.

Static analysis can help you immediately recognize areas in your software that could be simplified or improved without deep-diving inside the code. It supplies a powerful feedback loop that in seconds or minutes can provide metrics to analyze during the whole development life cycle.

One of my favorite metrics techniques is cyclomatic complexity. Applying this metric during the whole life of your software will help you understand the healthiness of your project and what areas need attention.

Code coverage is another useful metric, as it can suggest the extent to which your system is covered by tests. Unfortunately, though, it doesn’t give you feedback on the test quality.

Pair programming

Introduced by XP, pair programming is a powerful loop technique that provides peer feedback in seconds while letting you consolidate your knowledge and explain problems or solutions to someone else. Developers often skip this option when the business department deems it too expensive to have two people working on the same piece of code, but research demonstrates it’s not as costly as one might think. In fact, several studies show that while pair programming lowers developer productivity by a mere 10 to 15 percent, it increases the robustness of your software, consolidates and improves the skills of developers, and, most important, helps detect bugs and architecture mistakes more than any other technique. If executed in the right way, it can be a great investment for both your company and your software.

Code review

Code review definitely improves project quality and normally should be performed every time a developer in the team makes a pull request directly inside the version control system. Unfortunately, they are often skipped or, worse, ignored because we trust our peers too much or don’t have time. But if applied properly this technique can improve the code quality and architecture of your software to an amazing degree.

Usually it takes days, not seconds or minutes, so bear in mind that with this technique your developers don’t have the kind of frequent feedback afforded by the previously described techniques.

Agile Modeling

Several books describe and discuss this topic; I suggest scheduling one or more sessions at the beginning of any project to architect enough of your software that you can start development and review the design and implementations iteration by iteration (possibly with every sprint). You’ll see a constant improvement in your code without a huge up-front effort, meaning less need to accommodate changes for future business requests and more likelihood that your architecture and design decisions will faithfully reflect the project specifications. Usually these feedback loops are the only ones that maintain overall code structure and architecture sanity as viewed with an “eagle eye.”

Daily stand-up

This ceremony, used not only by Scrum teams but in many Kanban implementations, primarily provides an opportunity to synchronize team member efforts as well as a starting point for discussing potential improvements. As the name suggests, it’s a daily activity whose value hinges on not only on the technical aspects but also team communication and the big picture behind the project. These aspects are fundamental, because often developers are so focused on details they miss the perspective that could keep them from spending unnecessary work or time on secondary features (remember the Pareto principle).

Retrospective

An Agile retrospective, if well organized, is the most effective ceremony inside the Scrum framework to produce feedback that really helps improve projects and promote healthy teams. I facilitated several retrospectives and I’ve always gained a lot from them; if you can create a good atmosphere, defining properly what you want to achieve, you may reap surprisingly good results. For this purpose the retrospective is organized at the end of each iteration (usually every two weeks), but I really suggest more often if the team is recently defined or has more than one new member, to help newcomers feel part of the team and acquire its mind-set and to permit review of the resulting team performance as a whole.

Conclusion

Feedback loops help on a daily basis with project improvement and delivery success; more feedback loops mean greater control of software quality and better customer satisfaction, exactly the goals you are trying to achieve every day. Don’t be afraid to try new techniques to improve your team and your company. Fail fast, retrieve metrics, and learn from them; the worst that can happen is that you really improve!