Kedify | RabbitMQ

Use events from RabbitMQ to trigger autoscaling with Kedify and KEDA

RabbitMQ is an open-source message broker that allows applications to communicate by sending messages through queues

RabbitMQ’s support for multiple messaging protocols, coupled with its flexible message queuing and routing capabilities, makes it highly adaptable for various use cases. With KEDA, RabbitMQ setups can automatically scale application components to match workload demands, providing efficient resource utilization and responsiveness in environments from e-commerce to IoT and media processing. This integration enhances the ability to handle high loads and large numbers of concurrent connections, ensuring that applications remain scalable and resilient under varied operational conditions.

Featured Use Cases

Scenario:

An online retail platform experiences variable order volumes throughout the day, with spikes during promotional events and holiday sales.

RabbitMQ Scaler Usage:

Order messages are continuously pushed to a RabbitMQ queue from the website as customers place orders, which then need to be processed by the backend systems to fulfill orders.

KEDA Usage:

KEDA scales the number of order processing workers based on the RabbitMQ queue length, ensuring that order handling capacity matches demand peaks without delays.

Get Started

                    apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: order-processing-scaledobject
  namespace: default
spec:
  scaleTargetRef:
    name: order-processing-deployment
  pollingInterval: 10
  minReplicaCount: 1
  maxReplicaCount: 100
  triggers:
  - type: rabbitmq
    metadata:
      queueName: order-queue
      host: rabbit.cluster
      mode: QueueLength
      queueLength: '5'

Scenario:

A smart city project collects data from various IoT sensors deployed throughout the city to monitor traffic and environmental conditions.

RabbitMQ Scaler Usage:

Sensor data is sent to RabbitMQ queues from thousands of devices, which must be aggregated and processed to update dashboards and trigger alerts.

KEDA Usage:

KEDA dynamically adjusts the number of data aggregation services based on the incoming message rate in RabbitMQ, ensuring timely processing and responsiveness to changes in city conditions.

Get Started

                    apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: iot-data-aggregation-scaledobject
  namespace: default
spec:
  scaleTargetRef:
    name: data-aggregation-service
  minReplicaCount: 3
  maxReplicaCount: 50
  triggers:
  - type: rabbitmq
    metadata:
      queueName: iot-data-queue
      host: rabbit.cluster
       mode: MessageRate
      queueLength: '20'

Scenario:

A video streaming platform requires efficient scaling to handle bursts of video upload activities by users during major events or promotions.

RabbitMQ Scaler Usage:

Videos uploaded by users are queued in RabbitMQ for transcoding before they can be streamed to viewers.

KEDA Usage:

KEDA triggers transcoding jobs as the queue length increases, ensuring that videos are processed quickly and the user experience remains consistent even under load.

Get Started

                    apiVersion: keda.sh/v1alpha1
kind: ScaledJob
metadata:
  name: video-transcoding-job
  namespace: default
spec:
  jobTargetRef:
    template:
      spec:
        containers:
        - name: video-transcoder
          image: video-transcoder:latest
        restartPolicy: OnFailure
  triggers:
  - type: rabbitmq
    metadata:
      queueName: video-transcoding-queue
      host: rabbit.cluster
      mode: QueueLength
      queueLength: '10'

Scenario:

A financial services company needs to generate end-of-day reports based on transactions recorded throughout the day.

RabbitMQ Scaler Usage:

Transaction data is collected in a RabbitMQ queue and needs to be batch processed to generate reports after market close.

KEDA Usage:

KEDA schedules report generation jobs to run during low activity periods, scaling resources to handle varying amounts of transaction data efficiently.

Get Started

                    apiVersion: keda.sh/v1alpha1
kind: ScaledJob
metadata:
  name: batch-report-generation-job
  namespace: default
spec:
  jobTargetRef:
    template:
      spec:
        containers:
        - name: report-generator
          image: financial-report-generator:latest
        restartPolicy: Never
  triggers:
  - type: rabbitmq
    metadata:
      queueName: report-queue
      host: rabbit.cluster
      mode: QueueLength
      queueLength: '50'