First, what is PubSub?
Publish-Subscribe, also known as pub/sub or PubSub, is a messaging pattern that facilitates event-driven architecture. It aims to decouple applications into a publisher and a subscriber/consumer rather than linking them directly as a point-to-point pattern. In PubSub, the publisher sends a message to an intermediary service that makes it available to any number of subscribers. A subscriber is any client or service that needs to consume the message. The publisher and subscriber do not need to know about each other.
Examples of familiar PubSub implementation include the government emergency alerts like the Covid Alert Level change, tsunami warnings, earthquakes etc., that are pushed out to our smartphones.
Implementing PubSub in the context of integration
Looking at the example of emergency alerts, it would be impractical for our devices to poll for updates from the government’s service non-stop. That would consume time and computing resources, and that, of course, counters the efficiencies that the PubSub pattern is trying to deliver.
This is where message brokers come in. Message brokers are intermediate programs that help provide message forwarding and routing from the publisher to subscriber(s). To achieve a successful PubSub pattern, it’s imperative to select a message broker that fits your needs. (Check out our previous blog about choosing a smart message broker).
What are the pros and cons of SubPub?
The good news
- It’s a loosely coupled solution: The message publisher is unaware that the message consumer exists, making for modularised and robust software components, and increasing maintainability.
- You can have unlimited subscribers: The publisher can share a single message with multiple destination systems via the message broker.
- It’s resilient: When an outage occurs on the destination system, it doesn’t disrupt the message publication but creates a backlog of messages to be reprocessed. Then, the piled-up messages can be reprocessed seamlessly once the system is up and running again.
Some considerations
- It can introduce complexity: Because the interactions between systems are now asynchronous, it can increase the complexity in testing. You will need to observe the result on the subscribing process, rather than wait for the publishing process to complete the process.
- There’s no immediate feedback to the publisher of the message. Instead, it needs to go through a different channel (e.g. a call-back message from the destination system reverting to the source system).
- Latency: There’s no denying that adding another layer of data transfer increases the time it takes to transfer a message from source to destination. To be noted, though – as modern message brokers are highly efficient, the time added is usually minuscule.
Here’s a real-world PubSub example
This is an example of successfully implementing the PubSub pattern using a message broker for one of our clients who is a leading infrastructure construction, roadworks and aggregate supplier in New Zealand, Australia, and Fiji, employing over 8000 staff. The message broker of choice is Azure Service Bus:
- Contract work orders are published to client’s API Management endpoint./li>
- These work orders need to be posted to three different internal systems
- 1. Subscriber 1: Sends to work order system – Loc8
- 2. Subscriber 2: Sends to ERP system – Navision
- 3. Subscriber 3: Sends to finance system – JDE
- To implement the solution, we set up one single service bus topic and three subscriptions, one for each system.
- Then we have three Boomi processes that pick up the messages from the subscriptions and map the work order to the destination systems in the correct format.
- One of the advantages of this setup is that if JDE becomes unavailable (through an outage, for example), the work orders will build up on the service bus subscriptions and be resubmitted once the outage is resolved.
Summary
The PubSub pattern allows you to design more robust integration systems with resilience, scalability, and flexibility. First, however, it’s essential to select the message broker that suits your needs and then design a supportable solution.
Co-authored by Harris Kristanto & Simon Clendon
