Your cloud bill is mostly for compute you never used. Not metaphorically. Measurably.
CNCF looked at production applications and found that pre-optimization CPU utilization sat at 12% for more than 97% of them. Nearly 90% of the compute being paid for goes unused, and the finding held steady across application sizes and across cloud providers.
Datadog's telemetry from millions of production containers tells the same story from a different angle. 83% of container costs are idle. 54% comes from cluster-level idle, meaning you're running more nodes than the workload needs. The other 29% is workload-level, where individual containers sit on far more CPU and memory than they touch. Containers are roughly a third of EC2 spend, so this is not a rounding error.
Flexera's annual State of the Cloud survey has been reporting something like 30% cloud waste for years, with 82% of enterprises admitting they have a waste problem. Everyone knows. Nobody fixes it. That gap is the interesting part.
Nobody is being careless
The easy read is that engineering teams are sloppy with resources. The data says otherwise.
When CNCF asked what drives overspending, over-provisioning came back as the top answer at 70%, well ahead of lack of visibility (40%) or resource sprawl (43%). Over-provisioning is not an accident. It's a decision people make on purpose, repeatedly, because the alternative is worse.
Here's the actual reasoning behind it. You have a p99 latency target. Your runtime occasionally does something unpredictable, a garbage collection pause, a memory spike, a slow request blocking a worker. If you size your instances to average load, those moments blow through your SLA. So you size for the bad moments instead, and the bad moments are maybe 2% of the time. The other 98%, you're paying for headroom that exists purely to absorb variance.
That's not waste from incompetence. That's waste as insurance.
What you're actually insuring against
The variance has sources, and some runtimes produce more of it than others.
Figma wrote about this in 2018, before they moved their multiplayer server off Node. Their problem wasn't average throughput. It was that a single slow operation would lock up a worker and block syncing for every document assigned to it. Evan Wallace put it plainly: throwing more hardware at the problem wouldn't have solved it, because one slow operation still locks the worker for all files on it.
They also wanted to isolate each document in its own process, which would have contained the blast radius. They couldn't. As Wallace wrote, the memory overhead of the JavaScript VM would have been too high to run a process per document. So the architecture they wanted was priced out by the runtime's memory footprint, and they ended up manually routing "crazy documents" to a separate pool of heavy workers. A human, identifying problem files, by hand, as an ongoing operational chore.
Discord hit the variance problem from a different direction. Their read states service in Go was taking 10 to 50 millisecond garbage collection pauses every two minutes. Invisible to most users. Very visible to anyone in a thousand servers. They had already tuned the GC extensively and couldn't tune further without changing languages. That's Go, which has a concurrent low-latency collector. V8's collector is less forgiving, with major collections running 10 to 100 milliseconds or more depending on heap size.
The Platformatic team makes the point that V8's default strategy optimizes for throughput rather than latency, and that tuning heap flags is damage mitigation rather than a fix. You can move the pauses around. You can't remove them. Node's own documentation on using GC traces is essentially an admission that GC behavior is a production diagnostic concern you're expected to monitor.
So the p99 spike is real, the tuning ceiling is real, and the response is headroom. That headroom is the 12%.
The compounding version
Serverless makes the same tradeoff more expensive because you pay for startup.
Independent Lambda benchmarks put Node cold starts around 150 to 200 milliseconds of init duration, against roughly 50 for Go and single-digit to low-double-digit milliseconds for Rust. tecracer measured Go at about 3.4x faster than Node on init at 128MB, and found that bumping to 1024MB barely moved it, since you're still on one vCPU either way.
The fix everyone reaches for is provisioned concurrency, which is to say: pay to keep instances warm so users don't feel the startup. That's the same move as over-provisioning a cluster, just billed under a different line item. You're buying capacity you don't use so that the capacity you do use behaves predictably.
At that point the pattern is visible across three environments. Kubernetes clusters over-provisioned to absorb GC variance. Container requests padded above real usage. Serverless functions pre-warmed to hide init cost. Same instinct every time, priced separately.
What the migration numbers actually mean
The rewrite case studies get read as performance stories. They're mostly cost stories.
Grab rewrote their Counter Service from Go to Rust and reported 70% infrastructure savings at similar performance characteristics. Worth sitting with that one, because Go is already compiled and already has a good GC. Most of that 70% is not "we got faster." It's "we needed less machine to do the same work, and we no longer had to leave room for the collector."
Cloudflare's Pingora replaced NGINX with Rust and reported roughly 70% less CPU and 67% less memory for the same traffic, along with connection reuse going from 87.1% to 99.92% for one major customer. They also reported zero crashes from service code across a few hundred trillion requests, which matters for capacity planning in a way that's hard to put on a chart. Systems that don't fail unpredictably need less spare capacity.
Figma's rewrite let them run a process per document, which is the architecture they wanted in 2018 and couldn't afford. The order-of-magnitude serialization improvement got the headlines. The architectural unlock is the more interesting result.
The uncomfortable part
None of this says Node is bad. Node is extremely good at the thing it was designed for, which is handling many concurrent connections without thread-per-request overhead, and that design decision was correct enough to reshape server-side development for fifteen years.
The costs showed up later, in a billing environment nobody was designing for in 2009. Nobody at JSConf was thinking about per-millisecond Lambda pricing or Kubernetes requests and limits. The tradeoff Node made was reasonable, and then the industry changed how it charges for compute, and the tradeoff got a price tag attached.
What's genuinely strange about the 12% number is that it's stable. It doesn't vary much by company size or cloud provider, which suggests it isn't a maturity problem that teams grow out of. It's closer to a structural floor: the amount of headroom you need to buy when your runtime's behavior isn't fully predictable, and you've already tuned everything you can tune.
The interesting question isn't how to squeeze utilization from 12% to 20% with better autoscaling. Teams have been running that play for a decade and the number hasn't moved. The interesting question is what would have to change underneath for the headroom to stop being necessary at all.