HTTP Scaling with Istio VirtualServices

Note

This example was run on k3d running on Linux.

This guide demonstrates how to scale applications within an Istio service mesh using VirtualService resources to manage HTTP traffic. You’ll deploy a sample application, configure the necessary Istio resources (Gateway, VirtualService), deploy a KEDA HTTPScaledObject, and observe how Kedify automatically manages traffic routing for efficient load-based scaling including scale-to-zero when there’s no demand.

Architecture Overview

For applications managed by Istio and exposed via VirtualService, Kedify integrates with the service mesh using its autowiring feature. When using the kedify-http scaler, the traffic flow is adjusted to include Kedify’s proxy:

Istio Gateway -> VirtualService -> kedify-proxy -> Service -> Deployment

The kedify-proxy intercepts traffic routed by the VirtualService, collects metrics based on hosts defined in the HTTPScaledObject, and enables scaling decisions. When traffic increases, Kedify scales your application up; when traffic decreases, it scales down—even to zero if configured. Kedify automatically modifies the VirtualService destination host to point to the kedify-proxy service.

Prerequisites

• A running Kubernetes cluster with Istio installed.

Tip

If you want to just replicate the tutorial, start a local cluster using k3d:

# Create k3d cluster
k3d cluster create --port "9080:80@loadbalancer" --k3s-arg "--disable=traefik@server:*"

# Install Istio
kubectl create ns istio-system

helm repo add istio https://istio-release.storage.googleapis.com/charts
helm repo update

helm install istio-base istio/base -n istio-system --wait
helm install istiod istio/istiod -n istio-system --wait
helm install istio-ingressgateway istio/gateway -n istio-system --wait

• The kubectl command line utility installed and accessible.
• Install hey to send load to a web application.

Step 1: Prepare Istio Sidecar Injection

Enable Istio sidecar injection for the namespaces where your application will run:

kubectl create ns keda
kubectl patch namespace keda -p '{"metadata": {"labels": {"istio-injection": "enabled"}}}'
kubectl patch namespace default -p '{"metadata": {"labels": {"istio-injection": "enabled"}}}'

Step 2: Install KEDA

Install KEDA and its HTTP add-on in the keda namespace:

helm repo add kedacore https://kedacore.github.io/charts
helm install keda kedacore/keda --namespace keda
helm install http-add-on kedacore/keda-add-ons-http --namespace keda

Verify that KEDA and its components are running:

kubectl get pods -n keda

You should see output similar to:

NAME                                                    READY   STATUS    RESTARTS   AGE
keda-add-ons-http-controller-manager-79db447c7d-7r9xb   3/3     Running   0          39s
keda-add-ons-http-external-scaler-dfb9f7bcc-dxqft       2/2     Running   0          39s
keda-add-ons-http-external-scaler-dfb9f7bcc-fh76m       2/2     Running   0          39s
keda-add-ons-http-external-scaler-dfb9f7bcc-phcw9       2/2     Running   0          39s
keda-add-ons-http-interceptor-6dd68bbc87-5jz6v          2/2     Running   0          38s
keda-add-ons-http-interceptor-6dd68bbc87-5ng2w          2/2     Running   0          39s
keda-add-ons-http-interceptor-6dd68bbc87-pnp47          2/2     Running   0          38s
keda-admission-webhooks-554fc8d77f-n7mml                2/2     Running   0          39s
keda-operator-dd878ddf6-7ww29                           2/2     Running   0          39s
keda-operator-metrics-apiserver-86cc9c6fff-t4prd        2/2     Running   0          39s

Step 3: Deploy Sample Application

For this example, we’ll use podinfo as our sample application:

helm upgrade --install --wait podinfo --namespace default oci://ghcr.io/stefanprodan/charts/podinfo

Create the Istio Gateway and initial VirtualService:

istio.yaml

apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: app
  namespace: default
spec:
  selector:
    istio: ingressgateway
  servers:
    - hosts:
        - 'www.podinfo.com'
      port:
        name: http
        number: 80
        protocol: HTTP
---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: podinfo
  namespace: default
spec:
  hosts:
    - 'www.podinfo.com'
  gateways:
    - app
  http:
    - match:
        - uri:
            prefix: /
      route:
        - destination:
            host: podinfo
            port:
              number: 9898

• Gateway: Defines the entry point for traffic into the Istio service mesh.
• VirtualService: Configures Istio’s routing. It attaches to the specified gateway, matches requests for the host www.podinfo.com, and routes traffic to the podinfo service. Kedify will automatically update the destination.host field to point to the kedify-proxy when autowiring is active.

Apply the configuration:

kubectl apply -f istio.yaml

Step 4: Configure Autoscaling

Create an HTTPScaledObject to define the scaling behavior:

httpscaledobject.yaml

kind: HTTPScaledObject
apiVersion: http.keda.sh/v1alpha1
metadata:
  name: podinfo
  namespace: default
spec:
  hosts:
    - www.podinfo.com
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: podinfo
    service: podinfo
    port: 9898
  replicas:
    min: 0
    max: 10
  scalingMetric:
    requestRate:
      targetValue: 1
  scaledownPeriod: 5

• hosts (www.podinfo.com): The hostname defined in the VirtualService to monitor for traffic.
• scaleTargetRef.service (podinfo): The Kubernetes Service associated with the application deployment.
• scaleTargetRef.port (9898): The port on the service to monitor.
• scalingMetric.requestRate.targetValue (1): Target request rate. KEDA scales out when the rate per replica exceeds this value.

Apply the HTTPScaledObject:

kubectl apply -f httpscaledobject.yaml

You should see the HTTPScaledObject appear in the Kedify Dashboard.

Step 5: Configure Traffic Routing

Update the VirtualService to route traffic through Kedify’s proxy:

virtualservice_through_keda.yaml

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: podinfo
  namespace: default
spec:
  hosts:
    - 'www.podinfo.com'
  gateways:
    - app
  http:
    - match:
        - uri:
            prefix: /
      route:
        - destination:
            host: keda-add-ons-http-interceptor-proxy.keda.svc.cluster.local
            port:
              number: 8080

Apply the updated VirtualService:

kubectl apply -f virtualservice_through_keda.yaml

Step 6: Test Autoscaling

First, let’s verify that the application is accessible through the Gateway:

GATEWAY_IP=$(kubectl get svc -nistio-system istio-ingressgateway -o json | jq --raw-output '.status.loadBalancer.ingress[0].ip')
curl -H "host: www.podinfo.com" "http://$GATEWAY_IP"

If everything is correctly configured, you should receive a successful HTTP response:

{
  "hostname": "podinfo-5965fc9856-4tfpg",
  "version": "6.6.3",
  "revision": "b0c487c6b217bed8e6a53fca25f6ee1a7dd573e3",
  "color": "#34577c",
  "logo": "https://raw.githubusercontent.com/stefanprodan/podinfo/gh-pages/cuddle_clap.gif",
  "message": "greetings from podinfo v6.6.3",
  "goos": "linux",
  "goarch": "amd64",
  "runtime": "go1.22.3",
  "num_goroutine": "8",
  "num_cpu": "16"
}

Now, let’s simulate higher load using hey:

hey -n 10000 -c 150 -host "www.podinfo.com" "http://$GATEWAY_IP"

After sending the load, you’ll see a response time histogram in the terminal:

Response time histogram:
  0.301 [1]     |
  0.310 [2746]  |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  0.319 [3499]  |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  0.327 [2694]  |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  0.336 [683]   |■■■■■■■■
  0.345 [99]    |■
  0.354 [20]    |
  0.362 [1]     |
  0.371 [23]    |
  0.380 [37]    |
  0.389 [80]    |■

In the Kedify Dashboard, you can also observe the traffic load and resulting scaling.

Network Policies

If you’re using NetworkPolicies in your cluster, you’ll need to configure them to allow traffic between Kedify’s proxy and your application. Here are example policies:

For Kedify interceptor egress:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-interceptor
  namespace: keda
spec:
  podSelector:
    matchLabels:
      app.kubernetes.io/component: interceptor
      app.kubernetes.io/instance: http-add-on
      app.kubernetes.io/part-of: keda-add-ons-http
  egress:
    - to:
        - namespaceSelector:
            matchLabels:
              istio-injection: enabled
          podSelector:
            matchLabels:
              app.kubernetes.io/name: podinfo
      ports:
        - port: 9898
  policyTypes:
    - Egress

For application ingress:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-from-keda
  namespace: default
spec:
  podSelector:
    matchLabels:
      app.kubernetes.io/name: podinfo
  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              istio-injection: enabled
      ports:
        - protocol: TCP
          port: 9898
  policyTypes:
    - Ingress

Next Steps

Explore the complete HTTP Scaler documentation for more advanced configurations and details about its architecture and features, including autowiring for various ingress types.