A recent project we undertook presented us with an interesting challenge: how to synchronise two systems with very different databases behind them. Considering the project’s needs we decided that Azure Service Bus – a cloud-based messaging service – would be a good fit.
In our solution a change in system A would use Azure Service Bus to send a message to system B. The message type would specify what the message signified, and the data inside the message would be used to bring system B in line. All that would be needed was a way to receive messages in either system, and a simple piece of reusable code that could easily send these messages.
Below I’ve given an introduction to messaging in Azure Service Bus, and an outline of our solution. The code samples I’ve given use a thin wrapper around the Azure SDK – you can find help in getting started in this Microsoft Tutorial.
Topic and Subscriptions
The main difficulty with Azure Service Bus comes down to managing where messages are sent. To understand this you need to get your head around topics and subscriptions.
- Topics can be thought of as a channel on which you will send and receive messages.
- Subscriptions are pipes that connect to a topic.
Messages are queued up by subscription, and reading messages from your subscription will not affect other subscriptions.
Returning to our project, here are two examples of how we could configure the addressing and delivery of messages between systems A and B:
Example 1
Both A and B would use the same topic name, but would each have a different subscription name.
This would mean that any message sent by either system would be received by both, and that both could use the queue however they needed: because each system would be using different subscriptions, each could read messages without changing the queue for the other system.
Example 2
If we don’t want a system to receive the messages that it has sent itself, we could use two different topics. Doing this means that subscriptions no longer matter (as subscriptions are per topic, as mentioned above).
System A would listen for messages on ‘TopicA’ and send messages to ‘TopicB’
System B would listen for messages on ‘TopicB’ and send messages to ‘TopicA’
Now let’s look at some sample code.
Receive message
Below is some code that will hook into a topic, accept messages and pass them on to the MessageProcessorFactory. Take note of the options available when you receive a message to signal what the bus should do: I’ve included comments in the code below for Complete, Abandon and Deadletter.
IBusProvider provider = new AzureNamespaceBusProvider(this); // where ‘this’ implements IOptions<AzureQueueConfiguration>
IBusTopic topic = provider.GetTopic(“YourTopicName”, createIfNecessary: true);
IBusSubscriptionClient client = topic.GetSubscriptionClient(“YourSubscriptionName”, createIfNecessary: true);
client.Start(m =>
{
Console.WriteLine(“Got a message, ” + m.Label + “, ” + m.MessageId);
try
{
var message = m.ReadDataAs<MessageBase>();
_messageProcessorFactory.Process(message);
m.Complete(); // say we processed OK.. otherwise, azure will retry again in a bit
}
catch(Exception ex)
{
m.Abandon(); // in the case of general unknown failure – will cause the message to be tried again
// could also m.DeadLetter() – Remove it forever
}
});
To implement IOptions<AzureQueueConfiguration> you need to have your service bus access key. Implement the below property on your class that you pass into the AzureNamespaceBusProvider.
public AzureQueueConfiguration Value
{
get
{
return new AzureQueueConfiguration
{
ConnectionString = “YourAccessKey”
};
}
}
Depending on the environment we were deploying to, we placed this code in either a Windows service or an Azure web job. It is worth adding some logging so you’ve got something to go on if you need to troubleshoot. Also, having a console worker that executes this code can make debugging issues much easier.
Send message
The code to send a message is nice and compact.
var provider = new AzureNamespaceBusProvider(this); // where ‘this’ implements IOptions<AzureQueueConfiguration>
var topic = provider.GetTopic(“YourTopicName”, createIfNecessary: true);
topic.Send(new OutboundBusMessage { Label = “CreateJobMessage”, Data = message });
That’s it.
Conclusion
Using Azure Service Bus isn’t going to be the best option for every project – for example in another recent case we’ve gone for the more straightforward use of web hooks – but it does work well for slightly more complex problems. Returning to the system we set up, for the last six months it’s been exchanging about 10 messages a day as we keep two systems in sync with each other during a phased upgrade from one to the other. It’s saved us a lot of time here as we haven’t needed to continue working directly with the legacy system’s database. As the old system is decommissioned we’re expecting the messaging system to hit a peak daily load somewhere in triple figures, shortly after which its work will be done.
Whether and how you use Azure Service Bus messaging will depend on the scale and needs of your system, but for suitable projects its topic and subscription approach gives it a deal of flexibility. I’d personally recommend that you plan a strong naming convention for your topics and subscriptions, though, as without it your configs may become a mess, and issues with messages going to the wrong place or being lost in the ether are especially hard to resolve. Plan ahead and that pain can be avoided.
Featured image: Philip McErlean/Flickr, Creative Commons
