In this series of articles I am going to discuss how to configure and use RabbitMQ with the .NET client libraries. In this first article let’s talk about what message queueing is and then more specifically what RabbitMQ is.
What is Message Queueing
Message queuing gives you a mechanism to allow an application to asynchronously send a message to a receiver. This means that the sender and receiver do not need to interact with the message at the same time. A message is sent to a queue where it is stored until the receiver retrieves the message.
Message queues can be inter-process where the queue resides in memory on a single server or for integrating systems across multiple servers. This can be done by using in-memory queues but it is also common to use durable queues in which the messages are persisted to disk, meaning that messages are not lost should any system or server go offline for any period of time.
Message queuing systems come in many forms—both as commercial proprietry products and as open-source products. An example of a commercial solution is IBM MQ. Examples of open-source message queuing systems include RabbitMQ, JBoss Messaging , and Apache ActiveMQ.
The open-source message queuing systems are free to download and use but it is also common for companies to provide commercial support agreements (which you can pay for that allow your organization to reach out for technical support should you run into any production issues). RabbitMQ is no different in this respect and Pivotal Software, Inc., the company that owns RabbitMQ, provides a purchasable support agreement. Having a support agreement is normally a prerequisite for large organizations adopting an open-source platform.
Common Message Queuing Traits
A messaging broker is installed onto a server or a set of servers, and a series of name queues and exchanges are defined where they are registered with the message broker. An application can then send a message to the queue and any number of receiving systems can receive the message and act on its contents.
Queuing systems such as RabbitMQ will typically have the following features available to them:
- Durability: Messages can either be kept in memory or persisted to disk. If they are persisted to disk, then the message will be preserved should the server containing the queues crash. If this happens, once the server comes back up, the receiving applications can pick up where they left off.
- Security: A message broker should be able to let you set up different security permissions for different users. Some applications may have permission to write but not read from the queue, or another application could just have permissions to read from the queue.
- Time to Live: A message may have a time to live property set on it. This will mean a message can be set to expire after a certain timespan.
- Filtering and Routing: A messaging system might have the ability to filter the messages to decide to which queue a message should be delivered.
- Batching: Batching allows messages to be saved up and then delivered all at the same time as part of a larger batch; this is typically used when a destination server is not always on-line.
- Receipt Acknowledgement: A message publisher may require an acknowledgement from the message broker that the message has been successfully delivered.
Message Queuing Protocols
Many commercial message queuing systems are based on proprietary protocols. These protocols are kept closed so that the vendor can restrict what operating systems (OSes) and development platforms can interact with the message broker. This closed nature means that, if any customer of the messaging broker provider wanted it to work with another platform, they might have to pay a high consultancy fee to get the support they need.
With the advent of open-source messaging systems, a number of open queuing protocol standards have been developed. The two most predominant standards are:
- Advanced Message Queuing Protocol (AMQP): This is a feature-rich message queuing protocol and it is the protocol used by RabbitMQ.
- Streaming Text-Oriented Messaging Protocol (STOMP): Stomp is a simple text-based messaging protocol.
RabbitMQ is an open-source messaging system that allows you to integrate applications together by using messages and queues. RabbitMQ implements the AMQP. The underlying RabbitMQ server is written in the Erlang programming language which was originally designed for the telecoms industry by Ericsson. Erlang supports distributed, fault-tolerant applications and is, therefore, an ideal language to use to build a message queuing system.
While RabbitMQ is built with Erlang, it also supports many other development platforms via client libraries. Because of Erlang’s heritage of working with telecoms networks, it is the ideal platform for handling messages for enterprise applications.
The RabbitMQ server is a message broker that acts as the message coordinator for the applications that you want to integrate together. This means that you can give your systems a common platform for sending and receiving messages.
Rabbit MQ contains many features to make your systems integration as painless as possible. These include:
- Reliability: With RabbitMQ being built on top of Erlang, the message broker is already built on top of solid, high-performance, reliable, and durable foundations. Messages can be persisted to disk to guard from lost messages in the event that a server is restarted, and you can send message delivery acknowledgements to a sender so they can be sure that the message has been received and stored.
- Routing: RabbitMQ works by passing your messages through exchanges before they are stored in a queue. There are different exchange types which let you perform routing, but you can also work with more complex routing scenarios by binding exchanges together.
- Clustering and High Availability: To increase the reliability and availability of RabbitMQ, you can cluster several servers together on a local network which forms a single logical message broker. Queues can also be mirrored across multiple servers in a cluster so that, in the advent of a server failure, you will not lose any messages.
- Management Web User Interface (UI): RabbitMQ comes with a browser-based UI that lets you manage users and their permissions, exchanges, queues, and more. This tool is an excellent window into your RabbitMQ servers.
- Command-Line Interface: In addition to the Management UI, there is a command-line tool called “rabbitmqctrl” and “rabbitmqadmin.” These command-line tools offer the same level of administration as the web UI, with the added advantage that you can incorporate RabbitMQ administration in scripts.
A major benefit of using RabbitMQ is the fact that it is a cross-platform system. You can run Erlang and the RabbitMQ server on various platforms including Windows Servers, Linux and Unix systems such as (Debian/Ubuntu, Fedora, and Solaris), Mac OS X, and on various cloud platforms such as Amazon EC2 and Microsoft Azure. Rabbit MQ also has client libraries that support many different programming environments such as Microsoft .NET, Java, Erlang, Ruby, Python, PHP, Perl, C/C++, Node.JS, Lisp, and more.
This cross-platform nature of RabbitMQ means you could have clients written in different programming languages easily sending and receiving messages from a RabbitMQ server hosted in any of the supported environments.
RabbitMQ is an open-source messaging solution but, as a company, you have the option of taking out a paid support plan (which is a requirement for many large organizations).
RabbitMQ is built to the AMQP Version 0-9-1. AMQP is a networking protocol that enables client applications to communicate with messaging middleware broker servers. In our case, the client will be our .NET application and the server is the RabbitMQ server or cluster of servers. Let’s take a closer look at the AMQP protocol.
In the next article we will go into more detail with the AMQP messaging standard that RabbitMQ uses.