Node Scaling

The Node Scaler adds and removes nodes based on demand. It supports two modes. CRD: NodeScalingPolicy (cluster-scoped)

Modes

NodePool mode (default): adjusts a single GKE managed node pool’s size. Heterogeneous mode: provisions individual GCE VMs with per-pod machine type selection — similar to CAST AI or Karpenter. The bin-packer selects the cheapest machine type that fits pending pods from a configurable list of allowed series.

# Heterogeneous mode example
apiVersion: ops-ai.io/v1alpha1
kind: NodeScalingPolicy
metadata:
  name: default
spec:
  scalingMode: Heterogeneous
  minNodes: 1
  maxNodes: 50
  scaleUpThreshold: 0.8
  scaleDownThreshold: 0.4
  cooldownPeriod: 2m
  pendingPodThreshold: 2
  heterogeneousConfig:
    allowedMachineSeries: [c2d, t2d, n2d, e2]
    seriesPriority: [c2d, t2d, n2d, e2]
    minCPU: 2
    minMemoryGiB: 15
    packingHeadroom:
      cpu: 0.10
      memory: 0.05

Scale-up triggers

The scaler provisions new nodes when:

Pending pods exist (not scheduled to any node)
Cluster CPU or memory utilization exceeds the scale-up threshold (default 80%)
The pending pod count has been stable for 3+ evaluation cycles (~3–6 min)

Pods requesting GPUs or other extended resources are excluded — the heterogeneous scaler only provisions standard CPU/memory VMs.

Scale-down triggers

The scaler removes nodes when:

Average utilization drops below the scale-down threshold (default 40%)
No pods are in Pending phase anywhere in the cluster
The target node is older than 10 minutes (grace period for metrics to stabilize)
The node is not the sole provider of a taint profile (workload isolation)
No rebalance is actively executing

Scale-down is proportional: 50% of excess nodes when near-empty, 33% moderate, 20% near threshold.

For the full scale-down guard list and the interaction between continuous scale-down and scheduled maintenance consolidation, see Node Scale-Down Behavior.

Machine type selection

In heterogeneous mode, the bin-packer:

Collects all pending pods
Groups them by taint profile + PV zone
For each group, selects the cheapest machine type from the allowed series that fits the pods
Respects packingHeadroom (default: 10% CPU, 5% memory reserved for burst)

Series priority order determines preference: [c2d, t2d, n2d, e2] by default.

Key behaviors

PV-aware zone provisioning — nodes are created in the same zone as bound PersistentVolumes.
Taint/label propagation — pod nodeSelectors and tolerations are cloned to provisioned VMs.
10-minute grace period — new nodes are protected from scale-down until metrics stabilize.
Proportional scale-down — removes 20–50% of excess nodes per cycle (not one at a time).

Key knobs

What	Field	Default	Effect of changing
Scale-up threshold	`scaleUpThreshold`	0.8	Lower = scale up sooner (more headroom, higher cost)
Scale-down threshold	`scaleDownThreshold`	0.4	Higher = scale down sooner (less waste, less buffer)
Cooldown period	`cooldownPeriod`	2m	Shorter = faster response, more oscillation risk
Allowed machine series	`heterogeneousConfig.allowedMachineSeries`	`[c2d, t2d, n2d, e2]`	Add/remove series to control node types
Packing headroom	`heterogeneousConfig.packingHeadroom`	cpu 0.10, mem 0.05	Higher = more burst room, fewer pods per node

Dashboard

The Scaling Activity page shows scale events. The main dashboard card shows current scaling state (Idle, ScalingUp, ScalingDown, Cooldown).

Get Started

Platform

Operations

Migrating from CAST AI

Modes

Scale-up triggers

Scale-down triggers

Machine type selection

Key behaviors

Key knobs

Dashboard

​Modes

​Scale-up triggers

​Scale-down triggers

​Machine type selection

​Key behaviors

​Key knobs

​Dashboard

Modes

Scale-up triggers

Scale-down triggers

Machine type selection

Key behaviors

Key knobs

Dashboard