Photo by Josh Riemer on Unsplash

A way of connecting microservices in NodeJS

Introduction

Microservices architecture is the current trend in terms of software development. But what is really the microservices architecture? The concept consists in breaking down the application in small key functions that we call services that suffice specific application needs. Email or User management are examples of common application services. The services are modular and can be deployed and developed without affecting each other. With microservices architecture you are able to choose the technologies that best suit your needs, for example you could use NodeJS to create the services of your application but have one specific service running on python for ML models.

New trends bring new challenges one of this architecture is to setup the communication between services. One of the options is to use a common REST API calls but the synchronous nature of HTTP requests increase application latency and system fault tolerance can be poor. Another option is to use a message broker where Service A can communicate with Service B by publishing a message in the message broker and System B is listening(subscribing) for messages. It is important to say that this also bring its own challenges for example how to implement rollback strategies.

Rabbitmq is a message broker that implements the advance message queuing protocol (AMQP). Rabbitmq is a good choice to decouple services that offers both reliability and scalability to a microservices architecture. Besides that, it is available in multiple platforms and major programming languages.

Amqplib is one of the client libraries for NodeJS, it is highly configurable offering great flexibility on how do you setup your application. Normally there is a relation between complexity and flexibility and this is the case in amqplib. To properly setup amqplib for a microservices environment it requires some degree of expertise particularly in understanding what needs to be configured. Rascal is a config-drive wrapper for amqplib that simplifies the configuration process of a nodejs rabbitmq client. According to the documentation what Rascal brings to the amqplib client library is:

-Config driven vhosts, exchanges, queues, bindings, producers and consumers

-Cluster connection support

-Transparent content parsing

-Transparent encryption / decryption

-Automatic reconnection and resubscription

-Advanced error handling including delayed, limited retries

-Redelivery protection

-Channel pooling

-Flow control

-Publication timeouts

-Safe defaults

-Promise and callback support

-TDD support

Example

Let’s setup an example to show how Rascal can help to configure services communication through RabbitMQ. Suppose that you have a system where after an order is placed you sent an email to the customer informing that the transaction was successfully completed. So, in this system you have 3 services: order, email and a log service that logs every message.

Let’s divide our strategy in 4 main point:

1. Order Service publishes to Services exchange with routing key Service.order.email;

2. Services exchange distributes the message according to the bindings. Log queue will receive any message that goes to Service exchange (assuming that routing keys are respected) and Email Service will receive the messages with a destination equal to email.

3. Email service subscribes the email queue and acknowledges (or not) the receiving of the message.

4. If the message is rejected by email service it will go to a dead-letter exchange with a routing key of DeadLetter.email. The message is then routed to the email dead letter queue.

How to configure rascal to implement our strategy?

(assuming that you have node and rabbitmq server installed)

Rascal as a config wrapper of amqplib needs a json config file where you specify your strategy configurations. From the publisher side (order service) you should declare the exchanges that are being used as well as the queues and bindings.

The exchanges and queues will be durable (meaning will not be deleted after broker restart) and email queue will have a default dead letter exchange and dead letter routing key. So, when we nack an email message those will automatically be routed to that exchange and with that routing key. Then we declare the bindings. Email queue will receive any message with destination as email. Log queue will receive any message from the Services exchange with a routing key containing “Service.”. For example imagine that you have a user service that needs to send an email when a user is registered, you would send a message to the Services exchange with a routing key of “Service.user.email”, that message would, also, be routed to email and log queue. It is important for an easier binding configuration that you choose wisely the name convention of the routing keys. Finally we configure the publication, basically we create an email_publication that publishes to services exchange with the routing key “Service.order.email”.

In terms of application, you need to call the broker import the configs, and to publish just call broker.publish(‘your publication’,data_message).

Regarding our subscriber (email service) we need to configure the subscription in the config file.

So here we are subscribing every message that goes to the email_queue while limiting the amount of not acknowledge messages in our application to 5.

In terms of the application we need to create the subscriber.

We start by creating the broker with our config file and subscribe the email subscription. Then we start listening for events. Whenever a message arrives a “message” event fire. In this email service in order to identify the service that sent the message, to choose the body to use, I am using the sender routing key. The message is acknowledge whenever everything goes fine, otherwise it is nack with ackOrNack(err, { strategy: ‘republish’, immediateNack: true }). This allows for the error to be put in the header of the message by republishing followed by an immediate nack. The error events that we listen in the publisher and subscriber are emitted by amqplib and we should listen and deal with in our application.

One important point that I have not considered is redeliveries. Basically, you could have a message that crashes your app before you acknowledge the message. If you are automatically restarting the application this would turn into a loop of the message being indefinitely delivered. To counter that Rascal has the redeliveries concept where you keep count of the number of times the message was delivered. As the count is stored at application level you would need to create a strategy for that. Rascal gives direct support for clustered applications (where you store the count in the master) and there is also a rascal-redis package for that (rascal-redis-counter).

You can check full code here.

Please let your comment down. Suggestions (and corrections) are always welcomed.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store