Kedify | AWS SQS

Use events from AWS SQS to trigger autoscaling with Kedify and KEDA

Amazon Web Services Simple Queue Service is a fully managed message queuing service that allows you to decouple and scale microservices, distributed systems, and serverless applications.

It enables asynchronous communication and is designed to be highly scalable and simple to use in a broad range of scenarios. The integration with KEDA enhances this capability by enabling fine-grained, event-driven autoscaling that responds dynamically to the fluctuating demands on message processing, ensuring optimal resource utilization and performance without manual intervention.

Featured Use Cases

Scenario:

E-commerce platforms need to dynamically scale services that process customer orders during high traffic times like sales or promotions.

AWS SQS Scaler Usage:

Automatically scale based on the number of messages in the SQS queue, which represent incoming orders waiting to be processed.

KEDA Usage:

KEDA scales order processing backends by monitoring the SQS queue length, ensuring that each order is processed quickly even during peak times.

Get Started

                    apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: order-processing-scaledobject
  namespace: default
spec:
  scaleTargetRef:
    name: order-processor
  minReplicaCount: 1
  maxReplicaCount: 100
  triggers:
  - type: aws-sqs-queue
    metadata:
      queueURL: https://sqs.eu-west-1.amazonaws.com/account_id/order-queue
      queueLength: "10"
      awsRegion: "eu-west-1"

Scenario:

Handling real-time data for applications like social media analytics, where events must be processed as they come in.

AWS SQS Scaler Usage:

Scale services based on the volume of events that are queued in SQS, requiring rapid processing to provide timely insights.

KEDA Usage:

Utilizes KEDA to scale analytical services dynamically, managing large bursts of data and reducing latency in event processing.

Get Started

                    apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: event-processing-scaledobject
  namespace: default
spec:
  scaleTargetRef:
    name: event-processor
  minReplicaCount: 2
  maxReplicaCount: 50
  triggers:
  - type: aws-sqs-queue
    metadata:
      queueURL: https://sqs.eu-west-1.amazonaws.com/account_id/event-queue
      queueLength: "20"
      awsRegion: "eu-west-1"

Scenario:

Microservices architectures require decoupled services where components communicate through messages, often necessitating dynamic scaling based on workload.

AWS SQS Scaler Usage:

Efficiently scales microservices based on the load characterized by message queues in SQS, supporting decoupled architecture and improving system resilience.

KEDA Usage:

KEDA adjusts the number of service instances in real-time by monitoring the length of SQS queues, ensuring optimal performance and fault tolerance.

Get Started

                    apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: microservice-scaler
  namespace: default
spec:
  scaleTargetRef:
    name: microservice-handler
  minReplicaCount: 1
  maxReplicaCount: 30
  triggers:
  - type: aws-sqs-queue
    metadata:
      queueURL: https://sqs.eu-west-1.amazonaws.com/account_id/microservice-queue
      queueLength: "5"
      awsRegion: "eu-west-1"

Scenario:

Retail operations require robust systems to manage stock levels efficiently, particularly when dealing with multiple suppliers and fluctuating demand.

AWS SQS Scaler Usage:

Automates inventory adjustments and reordering processes by scaling services that monitor and react to messages related to stock changes.

KEDA Usage:

By tracking messages in SQS related to inventory levels, KEDA ensures that inventory management systems can dynamically adjust to changes, preventing stockouts or overstocking.

Get Started

                    apiVersion: keda.sh/v1alpha1
kind: ScaledJob
metadata:
  name: inventory-management-job
  namespace: default
spec:
  jobTargetRef:
   template:
      spec:
        containers:
        - name: inventory-handler
          image: inventory-management-system:latest
          command:
          - 'python'
          - '-u'
          - 'manage_inventory.py'
  maxReplicaCount: 10
  triggers:
  - type: aws-sqs-queue
    metadata:
      queueURL: https://sqs.eu-west-1.amazonaws.com/account_id/inventory-queue
      queueLength: "15"
      awsRegion: "eu-west-1"