Introduction

RabbitMQ is an open-source message broker that allows applications to communicate with each other by sending messages between them. Deploying RabbitMQ in a clustered configuration can enhance its performance, reliability, and redundancy. In this article, we will discuss how to set up a RabbitMQ cluster with three nodes using HashiCorp Nomad, a tool that simplifies the scheduling and management of microservices.

Prerequisites

Before we dive into the deployment, ensure you have the following requirements met:

1. Nomad: Make sure Nomad is installed and configured on your system. You can follow the official installation guide.
2. Docker: Since we will be using Docker containers for RabbitMQ, install Docker and ensure it is running. Refer to the Docker installation guide.
3. RabbitMQ: Familiarity with RabbitMQ concepts and configs is helpful but not strictly necessary.

Step 1: Create the RabbitMQ Docker Image

We’ll use the official RabbitMQ image along with the management plugin to easily monitor and manage our cluster.

docker pull rabbitmq:management

Step 2: Define the Nomad Job Specification

With Nomad, you define jobs in a HCL (HashiCorp Configuration Language) file. Below is an example of a Nomad job specification to deploy a RabbitMQ cluster with three nodes.

Create a file named rabbitmq.nomad:

job "rabbitmq-cluster" {
  datacenter = "dc1"
  type = "service"

  group "rabbitmq" {
    count = 3

    network {
      port "client" {}
      port "management" {}
    }

    service {
      name = "rabbitmq"
      port = "client"

      tag = "v1"
      check {
        type     = "tcp"
        port     = "client"
        interval = "10s"
        timeout  = "2s"
      }
    }

    task "rabbitmq" {
      driver = "docker"

      config {
        image = "rabbitmq:management"
        ports = ["client", "management"]

        args = [
          "rabbitmq-server",
          "--cluster-name", "rabbit@$(NODE_NAME)",  # dynamic name for each node
          "--loopback-users", "none"  # disable loopback users for external access
        ]
      }

      env {
        RABBITMQ_ERLANG_COOKIE = "secret_cookie"
      }

      resources {
        cpu    = 500 # 500 MHz
        memory = 512 # 512MB
      }

      lifecycle {
        hook = "prestart"
        command = "bash"
        args = ["-c", "sleep 3"] # to allow previous tasks to start properly
      }

      network {
        mbits = 10
        port "client" {
          static = 5672
        }
        port "management" {
          static = 15672
        }
      }
    }
  }
}

Explanation:

• Job Definition: We defined a Nomad job with the name rabbitmq-cluster containing a group for RabbitMQ nodes.
• Count: The count is set to 3, which means three instances of the RabbitMQ task will be created.
• Network Configuration: We are defining two ports: one for the RabbitMQ client (5672) and one for the management interface (15672).
• Service Checks: Basic health checks are included to monitor RabbitMQ’s health.
• Environment Variables: The RABBITMQ_ERLANG_COOKIE is a required variable for cluster communication, ensure it’s secure and consistent across nodes.

Step 3: Deploying the Job

Deploy the job by executing the following command:

nomad job run rabbitmq.nomad

You can check the status of your job by using:

nomad job status rabbitmq-cluster

Step 4: Verify Cluster Status

After starting the job, you can verify that all nodes are up and running successfully. To do this, you can access the RabbitMQ management interface by navigating to http://<YOUR-NOMAD-MASTER-IP>:15672 in a web browser.

Log in using the default credentials:

• Username: guest
• Password: guest

From the UI, you should see all three nodes listed in the cluster status.

Conclusion

In this article, we successfully deployed a three-node RabbitMQ cluster using HashiCorp Nomad. This deployment method leverages Nomad’s powerful features for orchestration, making it easier to manage application workloads across various environments.

Further Reading

• RabbitMQ Official Documentation
• HashiCorp Nomad Documentation
• RabbitMQ Clustering

By utilizing open-source tools like RabbitMQ and Nomad, you can efficiently manage and scale your messaging infrastructure to fit your application needs. Happy coding!