Now, imagine going to a restaurant and standing in line to order food but realizing you can only get your hands on the food after the previous person receives his food. Would you ever want to go there again? In contrast, imagine the same restaurant introducing a mechanism like a device that can help you order food and it gets delivered to their system which is a few meters away from where you are sitting. Here, you are the consumer, the restaurant is the publisher/producer, and the device you are holding acts as the message broker which is connected to their system. With message brokers in place, you get the luxury of ordering without waiting for any other process by any other person, and you are notified via the same device once your order is ready!

A message broker is an intermediary computer program part of the computer network that is used by software applications to communicate by exchanging formally-defined messages between a sender and a receiver. It is an architectural pattern for message validation, message transformation, and routing. Essentially, it mediates communication among applications, minimizing the mutual awareness that applications should have of each other in order to be able to exchange messages, effectively implementing decoupling.

Message brokers act as a bridge between different message queues and enable distributed processing of messages across multiple cluster nodes. They provide scalability, reliability, transparency, automated management capabilities, and extendable security features, making them highly versatile platforms for building mission-critical applications.

In general, message brokers are beneficial when message senders and receivers are spread across platforms or written in other programming languages. In short, message brokers help integrate applications and automate message transmission.

Why use message brokers?

They use asynchronous communications among services so that the sending app doesn’t have to wait for the receiving app’s response. This feature also helps in scaling the system. Using a message broker allows for increased control over inter-service communications, ensuring data is sent securely, reliably, and efficiently between components. Message brokers can play a similar role in integrating multi-cloud environments, enabling communication between workloads and runtimes on different platforms. They also work well with serverless computing, in which individual cloud-hosted services run on an as-needed basis. Without the use of a system like a broker, you would end up utilizing resources like time, and systems massively as the business requirements vary and scale.

How do message brokers work?

Now that we saw why use a message broker, we will see how it works. In a synchronous process, the source submits a message to the end consumer app, waits for it to complete the task, and accepts the response before continuing. The source app cannot take further action until a response is received.

A message broker, however, uses asynchronous processing, where there is no waiting for the source app or the consumer app to take action. It doesn’t require waiting for real-time responses. The result is greater multitasking ability because one activity doesn’t need to wait for another to conclude.

Fundamental Components of a Message broker

Here are some fundamental components of a message broker:

• A producer is an app responsible for transmitting messages. It’s linked to the message broker, also known as publishers in the publish/subscribe model.
• A consumer is a service that receives messages waiting in the message broker. They’re known as subscribers in the publish-subscribe pattern.
• A queue or topic is a folder in a system. Message brokers use them to store messages.
• An exchanger is an object that resides on queues and directs the message broker to form a group where consumers or producers can create or listen to send or receive messages.

Models of a Message broker

As we saw earlier, the primary function of a message broker is to route messages among apps. It’s analogous to a mediator or intermediary adjudicating between two parties if communication has broken down. The models of a message broker are mainly broken into 2;

1) Point-to-Point Model

A typical one-to-one relationship. Each message in the queue is forwarded to and consumed by a single consumer. P2P messaging is useful when a message needs to be acted upon only once. Senders and receivers in these systems require assurance that each payment will happen only once, and P2P ensures this. Payroll and Financial transactions are all examples of P2P processing.

2) Publish/Subscribe model

Publish/Subscribe messaging, also known as the pub/sub model, allows a producer to send messages to a topic. In this approach, the producer is known as a publisher, while the consumer is referred to as a subscriber. Different publishers can write on the same topic, and different subscribers can receive messages from one or more publishers(as shown in the image below). This architecture enables easy interest-driven delivery depending on the topics that apps subscribe to. It can also facilitate adopting an event-driven architecture-based system with fewer dependencies among apps. Examples include real-time streaming apps.

Challenges of a message broker

1. Takes a long time to set up things – Message brokers require many setup and configuration options and aren’t as easy to implement.
2. Increased system complexity – Integrating a message broker into your system adds a new component to the overall infrastructure. This requires addressing additional factors such as supporting the network between parts or security risks.
3. Debugging can be a headache – Since there are many components involved and decouples, debugging in case of any issues can be time-consuming.

Examples of Message Brokers popularly used:

1. RabbitMQ is an open-source message broker software that provides advanced and complex routing configurations.
2. Amazon MQ is a cloud-based message broker and a part of Amazon Web Services (AWS). It provisions and maintains a message broker for businesses and reduces their routine tasks.
3. Apache Kafka is a distributed messaging system that originally tracked website activity and is popularly used for real-time data storage.
4. Amazon SNS is a push notification from AWS. SNS uses a publish-subscribe messaging model to transmit individual messages.
5. Amazon Simple Queue Service (SQS) is a fully controlled queuing solution from AWS, much like Amazon SNS. SQS can automatically resize to fit the load of data. The message consumption process is different for SQS than SNS. SNS uses a push technique to offer messages to consumers without retaining them. SQS, on the other hand, uses a pull approach, requiring consumers to manually grab messages from SQS queues.

You read about what message brokers are and what their primary purpose is. Perhaps, time to implement or use one? Let us know how you use it in case you do or any interesting use-cases it can be used for!