Envoy HTTP Scaler

Kedify Envoy HTTP Scaler ensures that your service scales based on incoming HTTP requests using a custom Envoy proxy. To use this scaler an existing Envoy proxy is required in your environment and needs to be configured to send metrics to Kedify Scaler.

Details

The kedify-envoy-http scaler is designed specifically for ScaledObject resources to enable scaling based on incoming HTTP traffic using custom Envoy proxy. Unlike kedify-http, it does not support scaling to zero, as Envoy cannot hold traffic during scale to zero. The scaler monitors traffic using the custom Envoy proxy and routes traffic accordingly.

By using this scaler, users can define specific metrics such as request rate or concurrency to determine the scaling needs of the application, ensuring optimal performance and resource utilization.

Trigger Specification

This specification describes the kedify-envoy-http trigger that scales workloads based on incoming HTTP traffic using custom Envoy proxy.

Here is an example of trigger configuration using the Kedify Envoy HTTP scaler:

triggers:
  - type: kedify-envoy-http
    metadata:
      scalingMetric: requestRate # or concurrency
      targetValue: '10'
      granularity: '1s'
      window: '1m0s'
      externalProxyMetricKey: 'my_app_com' # <-- this should match a [envoy_cluster_name]

Parameter list:

scalingMetric - The metric used for scaling, which can be either requestRate or concurrency.
targetValue - The target value for the scaling metric. When the incoming traffic meets or exceeds this value, KEDA will scale out the deployment. (Default: 100)
granularity - The granularity at which the request rate is measured. For example, “1s” means one second. (Only for requestRate, Default: 1s)
window - The window over which the request rate is averaged. For example, “1m0s” means one minute. (Only for requestRate, Default: 1m)
externalProxyMetricKey - Matching external metric name, used for aggregating metrics from Envoy proxy. (eg. concrete cluster_name for Envoy)

Example ScaledObject with Kedify Envoy HTTP trigger

Here is a full example of a scaled object definition using the Kedify Envoy HTTP trigger:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: app-envoy
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: app-1
  cooldownPeriod: 5
  minReplicaCount: 1
  maxReplicaCount: 10
  triggers:
    - type: kedify-envoy-http
      metadata:
        scalingMetric: requestRate
        targetValue: '10'
        granularity: '1s'
        window: '1m0s'
        externalProxyMetricKey: 'my_app_com'

Configuring exising Envoy proxy

Kedify Envoy HTTP Scaler uses Envoy to route traffic and get metrics for applications to improve reliability and performance. This setup prevents situations where the interceptor may become a bottleneck. Standard reverse proxies such as Envoy, nginx, or HAProxy are better equipped to handle such conditions.

To route application traffic through a custom Envoy and enable it to flush metrics to KEDA for scaling, the following configuration snippet is needed within your Envoy fleet. This configuration ensures that metrics are pushed to the interceptor every second, complete with all necessary labels and values for HTTP-based scaling.

stats_flush_interval: 1s
stats_sinks:
  name: keda_http_add_on
  typed_config:
    '@type': type.googleapis.com/envoy.config.metrics.v3.MetricsServiceConfig
    transport_api_version: V3
    report_counters_as_deltas: true
    emit_tags_as_labels: true
    grpc_service:
      google_grpc:
        target_uri: keda-add-ons-http-interceptor-kedify-proxy-metric-sink.keda.svc.cluster.local:9901 # <-- use proper cluster domain
        stat_prefix: kedify_

This utilizes the stats_sink extension, implementing the V3 gRPC MetricsService. For each scaled application where these Envoy metrics should be aggregated with the internal interceptor metrics, you should configure externalProxyMetricKey in the trigger metadata:

triggers:
  - type: kedify-envoy-http
    metadata:
      scalingMetric: requestRate
      targetValue: '10'
      granularity: '1s'
      window: '1m0s'
      externalProxyMetricKey: 'my_app_com'

Here, my_app_com refers to the upstream cluster_name from the Envoy configuration used to route the traffic to the desired application. Depending on your Envoy config, the section may look similar to this:

static_resources:
  listeners:
    - filter_chains:
        - filters:
            - name: envoy.filters.network.http_connection_manager
              typed_config:
                '@type': type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                route_config:
                  virtual_hosts:
                    - domains: ['www.my-app.com']
                      routes:
                        - match:
                            prefix: '/'
                          route:
                            cluster: my_app_com # <-- cluster_name from virtual_host
                            auto_host_rewrite: false
  clusters:
    - name: my_app_com # <-- matching cluster_name upstream

The two Envoy metrics that are ingested and processed are:

cluster.upstream_rq_total - for the request rate scaling metric
cluster.upstream_rq_active - for the concurrency scaling metric