by Zbynek Roubalik

July 15, 2026

When HPA Is Not Enough: Sharding Autoscaling in Large Kubernetes Clusters

Kubernetes Horizontal Pod Autoscaler (HPA) is a strong default. It turns CPU, memory, and custom metrics into reliable replica decisions, and KEDA builds on that model to scale workloads from event-driven signals.

The limitation appears when one large cluster runs and scales thousands of services. More workloads create more metric queries and reconciliations, but they do not automatically add capacity to the cluster’s shared HPA controller. HPAs continue to compete for the same controller workers and shared metric paths.

The Kedify Pod Autoscaler (KPA) makes that decision plane shardable. Each KPA controller owns a bounded set of namespaces and is paired with Multitenant KEDA serving those workloads. Platform teams can add autoscaling capacity and isolate noisy workload groups without replacing the ScaledObject API application teams already use.

The Shared HPA Control Plane

HPA runs as a controller in the Kubernetes control plane. It periodically evaluates each HPA, reads the required metrics, calculates a desired replica count, and updates the workload’s scale subresource.

Its operating parameters can be tuned. Controller concurrency, synchronization intervals, and API client limits can move the scalability boundary. In self-managed Kubernetes, platform teams may be able to adjust those settings directly.

That option is often limited on managed Kubernetes. Providers typically operate the control plane and may not expose all kube-controller-manager settings. Even where tuning is available, it remains a cluster-wide compromise: increasing concurrency or shortening the evaluation interval also increases pressure on the API server and metric backends.

More importantly, tuning does not divide the work into independent queues. A slow external metric source, a burst of autoscaler updates, or a large tenant still competes with unrelated workloads using the same controller.

Shard the Decision, Keep the ScaledObject

KPA moves horizontal replica decisions out of the single cluster-level HPA controller and into independently deployable controller instances.

It is deliberately not another scaler configuration API. The ScaledObject remains the source of truth for triggers, replica limits, fallback behavior, and advanced horizontal scaling behavior. Kedify KEDA translates that configuration into either an HPA or a KPA.

An application selects KPA with one annotation:

metadata:
  annotations:
    autoscaling.kedify.io/class: kpa

The generated KPA keeps the familiar Kubernetes autoscaling/v2 spec and status shape. Existing settings under advanced.horizontalPodAutoscalerConfig, including scaling policies and stabilization behavior, continue to apply.

The ownership boundary remains clear. KEDA evaluates triggers and handles activation, cooldown, fallback, and scaling between zero and one replica. KPA owns the selected workload’s horizontal decisions between one and its maximum replica count, updating the workload through the standard Kubernetes scale subresource.

Application teams continue to define how a workload should scale through ScaledObject. Platform teams decide where its scaling control loop should run.

A Tenant-Local Metrics Path

Each KPA controller watches the same non-overlapping namespace set as its paired instance of Multitenant KEDA operator. Together, they form one autoscaling shard.

For KEDA-provided external metrics, KPA connects directly to the paired KEDA operator through mTLS-protected gRPC interface. The request stays inside the shard instead of passing through a cluster-wide external metrics adapter.

Resource metrics follow the Kubernetes-native path. CPU, memory, and container resource metrics continue to come from metrics.k8s.io, just as they do with HPA.

Isolation Becomes Capacity

Sharding adds autoscaling capacity because each KPA instance has its own reconciliation workers, Kubernetes client limits, metric connection, and queue. As the platform grows, another namespace group can be moved into its own shard rather than sending every new autoscaler through one global decision loop.

The same boundary provides operational isolation.

A slow metric backend consumes the workers and connection capacity of its shard instead of occupying the cluster-wide HPA queue. A controller rollout or configuration change can be applied to one workload group without restarting every horizontal scaling loop in the cluster.

Shards can follow boundaries the platform already operates: customers, business units, environments, latency tiers, or groups of namespaces. A tenant does not need to represent a separate company. It can be any set of workloads that should scale and fail independently.

KPA also allows an individual ScaledObject to request a different evaluation interval within limits set by the platform. A burst-sensitive API can evaluate more frequently than a background processor without changing the cadence for every autoscaler in the cluster.

That flexibility is especially useful on managed Kubernetes, where the native HPA controller’s global settings may be unavailable or controlled by the provider. Instead of relying on a control-plane-wide setting, the platform operates the KPA shard and its evaluation policy directly.

KPA does not remove every shared dependency. Shards may still use the same Kubernetes API server, network, or Metrics Server. What it isolates is the horizontal autoscaling decision work: its queues, workers, external metric connections, rollout lifecycle, and controller configuration.

Adopt KPA Workload by Workload

KPA does not require a flag day. HPA and KPA can operate side by side for different ScaledObjects in the same cluster.

HPA remains the compatibility path and a good default for workloads that fit comfortably within the shared controller’s operating envelope. Platform teams can begin with namespaces where autoscaler count, control-loop latency, metric isolation, or tuning flexibility has the clearest value.

The transition does not require application teams to learn a new scaling model. They keep using ScaledObject, while the platform changes which controller owns the horizontal decision.

That is the core difference KPA introduces: compute can remain consolidated in one large Kubernetes cluster, while autoscaling decisions no longer have to remain centralized.

Make autoscaling boundaries match tenant boundaries.

See how Kedify Pod Autoscaler can scale the decision plane behind your largest Kubernetes clusters.

Get Started

Continue the Architecture

Read the Multitenant KEDA architecture for the operator and metrics-routing foundation, then explore the Kedify Pod Autoscaler documentation for configuration and deployment details.

Built by the core maintainers of KEDA for platform teams that need large clusters without a large autoscaling blast radius.