Forem: Dimitrios Kechagias

Cloud VM benchmarks 2026: performance / price

Dimitrios Kechagias — Fri, 27 Feb 2026 14:05:25 +0000

Time for the (not exactly) yearly cloud compute VM comparison. I started testing back in October 2025, but the benchmarking scope was increased, not just due to more VM families tested (44), but also due to testing the instances over more regions to attain a possible range of performance, as in many cases not all instances are created equal. I will not spoil much if I tell you that there is one new CPU that dominates the top-end results more clearly than any previous year.

Quick Overview

Like last time, this is all about generic CPU performance and especially what you can actually get per $ spent on compute VM instances. Due to the focus on CPU workloads, burstable instances are not included. Single-thread performance is evaluated separately, as there are always workloads that cannot be further parallelized. For multi-thread, each instance type is tested in a 2vCPU configuration which is usually the minimum unit you can order (it corresponds to a single core for SMT-enabled systems, like all Intel and most AMD). The more threads your workload can utilize, the more multiples of that unit you can order.

The comparison should help you maximize performance or price depending on your requirements, by either using the optimal VM types of your provider, or perhaps by launching on a different provider.

If you don't need all the details, you can use the TOC below to jump to what's relevant to you.

Table Of Contents:

What's new
The contenders (2026 edition)
- Amazon Web Services (AWS)
- Google Cloud Platform (GCP)
- Microsoft Azure
- Oracle Cloud Infrastructure (OCI)
- Akamai (Linode)
- DigitalOcean
- Hetzner
Test setup & Benchmarking methodology
Results
- Single-thread Performance
- Multi-thread Performance & Scalability
- Performance / Price (On Demand / Pay As You Go)
- Performance / Price (1-Year reserved)
- Performance / Price (3-Year reserved)
- Performance / Price (Spot / Preemptible VMs)
Conclusions
- Overview
- General Tips
- Caveats for AMD vs Intel vs ARM
- Recommendations per use-case
- Summary per cloud provider

What's new

I kept the same 7 providers as last year (which was down from my max 10 providers from the 2023 comparison), but expanded to 44 VM types tested.

Other changes:

New CPUs: AMD EPYC Turin (whose performance I had explored separately) and Intel Granite Rapids are available on the x86 front, while several new ARM solutions are tested: Google Axion (also explored separately last year), Azure Cobalt 100 and Ampere AmpereOne M.
More testing: Some extra benchmarks added. More testing across regions. In the past I only focused on that for small providers, but the big-three have also shown inconsistency, so the main performance and performance/price numbers will show a range.

The contenders (2026 edition)

As mentioned, I will focus on 2x vCPU instances, as that's the minimum scalable unit for a meaningful comparison (and generally minimum for several VM types), given that most AMD and Intel instances use Hyper-Threading (HT) / Simultaneous Multi Threading (SMT). So, for those systems a vCPU is a Hyper-Thread, or half a core, with the 2x vCPU instance giving you a full core with 2 threads. This will become clear in the scalability section.

I am skipping some very old instance types that are obviously uncompetitive. I am still trying to configure at 2GB/vCPU of RAM (which is variably considered as "compute-optimized", or "general-purpose") and 30GB SSD (not high-IOPS) boot disk for the price comparison to make sense (exceptions will be noted).

The pay-as-you-go/on-demand prices refer to the lower cost region in the US (or Europe). For providers with variable pricing, cheapest regions are almost always in the US. Unlike last year, I will not include the 100% sustained discounts for GCP, as they are not technically on-demand so I may have been unfair to other providers.

For providers that offer 1 year and 3 year committed/reserved discounted prices, the no-downpayment price was listed with that option. The prices were valid for January 2026 - please check for current prices before making final decisions.

As a guide, here is an overview of the various generations of AMD, Intel and ARM CPUs from older (top) to newer (bottom), roughly grouped horizontally in per-core performance tiers, based on this and the previous comparison results:

This should immediately give you an idea of roughly what performance tier to expect based on the CPU type alone, with the important note that for SMT-enabled instances you get a single core for every 2x vCPUs.

A general tip is that you should avoid old CPU generations, as due to their lower efficiency (higher running costs) the cloud providers will actually charge you more for less performance. I will even not include types that were already too old to provide good value last year, to focus on the more relevant products.

Amazon Web Services (AWS)

Instance Type	CPU type	RAM/SSD	Price $/Month	1Y Res. $/Month	3Y Res. $/Month	Spot $/Month
C5a.large (R)	AMD Rome	4/30	64.45	41.09	29.41	29.08
C5.large (S)	Intel Skylake	4/30	68.10	45.47	31.60	28.02
C6a.large (M)	AMD Milan	4/30	63.83	43.04	29.45	28.20
C6i.large (I)	Intel Ice Lake	4/30	70.66	47.55	32.45	29.02
C6g.large (G2)	AWS Graviton2	4/30	55.03	36.64	26.86	26.61
C7a.large (G)	AMD Genoa	4/30	84.82	56.92	38.69	32.07
C7i.large (SR)	Intel Sapphire Rapids	4/30	74.07	49.81	33.95	24.62
C7g.large (G3)	AWS Graviton3	4/30	58.46	40.65	28.97	29.31
C8a.large (T)	AMD Turin	4/30	88.94	64.94	44.19	31.82
C8i.large (GR)	Intel Granite Rapids	4/30	77.65	51.84	35.43	28.74
C8g.large (G4)	AWS Graviton4	4/30	66.22	44.62	30.50	27.93

Amazon Web Services (AWS) pretty much originated the whole "cloud provider" business - even though smaller connected VM providers predated it significantly (e.g. Linode comes to mind) - and still dominates the market. The AWS platform offers extensive services, but, of course, we are only looking at their Elastic Cloud (EC2) VM offerings for this comparison.

There are 2 new CPUs introduced since last year. Intel's Granite Rapids makes an appearance, while the AMD EPYC Turin-powered C8a follows the previous C7a in having SMT disabled (providing a full core per vCPU). I don't want to spoil much, but if you take the fastest CPU by a margin, and disable SMT, expect some impressive "per-2vCPU" results...

With EC2 instances you generally know what you are getting (instance type corresponds to specific CPU), although there's a multitude of ways to pay/reserve/prepay/etc which makes pricing very complicated, and pricing further varies by region (I used the lowest cost US regions). In the 1Y/3Y reserved prices listed, there is no prepayment included - you can lower them a bit further if you do prepay. The spot prices vary even more, both by region and are updated often (especially for newly introduced types), so you'd want to keep track of them.

Google Cloud Platform (GCP)

Instance Type	CPU type	RAM/SSD	Price $/Month	1Y Res. $/Month	3Y Res. $/Month	Spot $/Month
n2-2* (I)	Intel Ice Lake	4/30	63.45	40.19	29.65	22.15
n2d-2* (M)	AMD Milan	4/30	55.46	35.22	26.06	13.10
c2d-2 (M)	AMD Milan	4/30	68.28	43.76	31.82	15.87
t2d-2 (M)	AMD Milan	8/30	63.68	40.86	29.76	12.14
c3-4/2** (SR)	Intel Sapphire Rapids	4/30	63.69	40.72	29.54	11.09
c3d-4/2** (G)	AMD Genoa	4/30	56.32	36.08	26.23	9.90
n4d-2 (T)	AMD Turin	4/30	53.77	34.46	25.08	22.47
c4a-2 (AX)	Google Axion (Arm)	4/30	56.90	38.09	26.49	19.74
c4d-2 (T)	AMD Turin	3/30	57.57	36.86	26.79	23.40
n4-2 (E)	Intel Emerald Rapids	4/30	57.47	36.80	26.74	19.50
c4-2 (E)	Intel Emerald Rapids	4/30	63.69	40.72	29.54	27.20
c4-lssd-4/2** (GR)	Intel Granite Rapids	8/30+375GB SSD	103.75	65.45	47.57	43.70

^{* min_cpu_platform needs to be set to get tested CPU.}
^{** Extrapolated 2x vCPU instance - type requires 4x vCPU minimum size.}

The GCP Platform (GCP) follows AWS quite closely, providing mostly equivalent services, but lags in market share (3rd place, after Microsoft Azure). We are looking at the Google Compute Engine (GCE) VM offerings, which is one of the most interesting in respect to configurability and range of different instance types. However, this variety makes it harder to choose the right one for the task, which is exactly what prompted me to start benchmarking all the available types. To add extra confusion, some types may come with an older (slower) CPU if you don't set min_cpu_platform to the latest available for the type - so you need the extra configuration to get a faster machine for the same price.

This year, we have the addition of the AMD EPYC Turin (c4d and n4d), they are not yet in all regions/zones, but availability is expanding. We also had the introduction of two Intel-based 4th gen instances (n4 and c4). They both feature Emerald Rapids, however the latter can be configured with a local SSD, in which case they come with the newer Intel Granite Rapids. Until GCP allows setting min_cpu_platform to Granite Rapids (they are thinking about it AFAIK), you have to pay for the extra SSD to get the performance. Last year I covered separately the introduction of the Google Axion - powered c4a ARM type, but it is on a full VM comparison for the first time.

At this point, I should mention that the reason I did more extensive testing this year across different regions is the disappointing performance of Emerald Rapids in practice, compared to its showing on my original benchmarks. It seems that as it started to get used, exhibited a performance variance that looks consistent with boost behavior + node contention (i.e. more sensitive to noisy neighbors). I suspect this is why GCP offers the option to turn boost clock off in Emerald Rapids instances for "consistent performance".

GCP prices vary per region and feature some strange patterns. For example when you reserve, t2d instances which give you a full AMD EPYC core per vCPU and n2d instances which give you a Simultaneous Multi-Thread (i.e. HALF a core) per vCPU have the same price per vCPU, but n2d is cheaper on demand and gets a 20% discount for sustained monthly use.

Note that c3, c3d and c4-lssd types have a 4x vCPU minimum. This breaks the price comparison, so I am extrapolating to a 2x vCPU price (half the cost of CPU/RAM + full cost of 30GB SSD). GCP gives you the option to disable cores (you select "visible" cores), so while you have to pay for 4x vCPU minimum, you can still run benchmarks on a 2x vCPU instance for a fair comparison.

Microsoft Azure

Instance Type	CPU type	RAM/SSD	Price $/Month	1Y Res. $/Month	3Y Res. $/Month	Spot $/Month
D2as_v5 (M)	AMD Milan	8/32	65.18	39.40	26.26	15.16
D2ls_v5 (I)	Intel Ice Lake	4/32	64.45	38.98	25.98	17.37
D2pls_v5 (A)	Ampere Altra	4/32	52.04	31.65	21.26	11.57
D2pls_v6 (CO)	Azure Cobalt 100	4/32	47.66	29.07	19.59	11.18
D2ls_v6 (E)	Intel Emerald Rapids	4/32	67.59	42.82	27.82	20.04
D2als_v6 (G)	AMD Genoa	4/32	61.09	37.07	24.71	13.36

Azure is the #2 overall Cloud provider and, as expected, it's the best choice for most Microsoft/Windows-based solutions. That said, it does offer many types of Linux VMs, with quite similar abilities as AWS/GCP. The various types are not easy to use as on AWS/GCP though, for some reason even enterprise accounts start with zero quota on many types, so I had to request quota increases to even test tiny instances.

The v6 instances are new for the comparison, featuring AMD EPYC Genoa, Intel Emerald Rapids and Azure's own Cobalt 100 ARM CPU.

The Azure pricing is at least as complex as AWS/GCP, plus the pricing tool seems worse. They also lag behind the other two major providers in CPU releases - Turin and Granite Rapids are still in closed preview at the time of writing this.

Oracle Cloud Infrastructure (OCI)

Instance Type	CPU type	RAM/SSD	Price $/Month	Spot $/Month
Standard.E6 (T)	AMD Turin	4/30	29.00	15.13
Standard.A1 (A)	Ampere Altra	4/30	20.24	10.75
Standard.A2 (AO)	Ampere AmpereOne	4/30	17.32	17.32
Standard.A4* (AM)	Ampere AmpereOne M	4/30	19.22	10.24

^{* Limited availability currently.}

Oracle Cloud Infrastructure (OCI) was the biggest surprise in my 2023 comparison test. It was a pleasant surprise, not only does Oracle offer by far the most generous free tier (credits for the A1 type ARM VM credits equivalent to sustained 4x vCPU, 24GB RAM, 200GB disk for free, forever), their paid ARM instances were the best value across all providers - especially for on-demand. The free resources are enough for quite a few hobby projects - they would cost you well over $100/month in the big-3 providers.

Note that registration is a bit draconian to avoid abuse, make sure you are not on a VPN and also don't use oracle anywhere in the email address you use for registration. You start with a "free" account, which gives you access to a limited selection of services and apart from the free-tier eligible A1 VMs, you'll struggle to build any other types with the free credit you get at the start.

Upgrading to a regular paid account (which still gives you the free tier credits), you get a selection of VMs. New this year are the AMD EPYC Turin Standard.E6 VMs and the next generation ARM Standard.A4 type powered by the AmpereOne M CPU. If you recall from last year, the AmpereOne A2 instances were slower in quite a few tasks than the older Altra A1. Ampere really needed a step forward, and AmpereOne M (A4) finally delivers meaningful gains in this year’s dataset. I had trouble building older-gen AMD instances, so in the end I did not include them. I also could only build Standard.A4 in one region (Ashburn), even though I tried in Phoenix which Oracle had in the availability list, to no avail.

Oracle Cloud's prices are the same across all regions, which is nice. They do not offer any reserved discounts, but do offer a 50% discount for preemptible (spot) instances. One complication is that their prices are per "Oracle CPU" (OCPU). This seemed to make sense originally, as it corresponded to physical cores - the A1 instances had 1 OCPU per core, so 1 OCPU = 1 vCPU, while SMT x86 had 1 OCPU = 2 vCPU (threads). But then, possibly thinking that their users are getting comfortable with it, they threw a wrench by making 1 OCPU for newer (still non-SMT) ARM types A2 and A4 be equal to 2 vCPU / 2 full Cores. I can't think of a reason for this other than to confuse their customers.

Akamai (Linode)

Instance Type	CPU type	RAM/SSD	Price $/Month
Linode 4GB* (M)	AMD Milan	4/80	24.00
G7 4x2 (M)	AMD Milan	4/80	43.00
G8 4x2 (T)	AMD Turin	4/40	45.00

^{* Shared core.}

Linode, the venerable cloud provider (predating AWS by several years), has now been part of Akamai for a few years.

From the previous years we saw that their shared core types ("Linodes") are the best bang for buck, but it depends on what CPU you are assigned on creation. It seems that currently the most common configuration features an AMD EPYC Milan. I tried to build quite a few and that's what you usually get (if you manage to build an ancient Intel or AMD Rome, try again), I did not see any newer CPUs pop up. The latest EPYC Turin though is available as a dedicated CPU instance. They now mark dedicated instances with their generation, so a G8 should always be the same CPU. As always, the dedicated instances come with SMT, so you are normally getting a core per 2 vCPUs, while the shared instances are virtual cores, so twice the vCPUs gives you twice the multi-thread performance - the caveat is that performance per thread varies depending on how busy the node that holds your VM is.

It is a bit of an annoyance that without testing your VM after creation you can't be sure of what performance to expect, unless you go for the more expensive dedicated VMs, but otherwise, Akamai/Linode is still easy to set up and maintain and has fixed, simple pricing across regions.

DigitalOcean

Instance Type	CPU type	RAM/SSD	Price $/Month
Basic 2/4* (B)	Intel Broadwell	4/80	24.00
PremInt 2/4* (C)	Intel Cascade L	4/120	32.00
PremAMD 2/4* (R)	AMD Rome	4/80	28.00

^{* Shared core.}

DigitalOcean was close to the top of the perf/value charts a few years ago, providing the best value with their shared CPU Basic "droplets". I am actually using DigitalOcean droplets to help out by hosting a free weather service called 7Timer, so feel free to use my affiliate link to sign up and get $200 free - you will help with the free project's hosting costs if you end up using the service beyond the free period. Apart from value, I chose them for the simplicity of setup, deployment, snapshots, backups.

However, they seem to have stopped upgrading their fleet for quite a while now, so you end up with some very old CPUs. If you don't mind the low per-thread performance, they are still not a bad value, given the low prices. I like their simple, region-independent and stable pricing structure, but I wish they would upgrade their shared core data centers.

Hetzner

Instance Type	CPU type	RAM/SSD	Price $/Month
CCX13 (M)	AMD Milan	8/80	17.27
CAX11 (A*)	Ampere Altra	4/40	5.46
CPX22 (G*)	AMD Genoa	4/80	8.63
CX23 (R*)	AMD Rome	4/40	4.31
CX23 (S*)	Intel Skylake	4/40	4.31

^{* Limited/Eu-only availability.}
^{** Shared core.}

Hetzner is a quite old German data center operator and web host, with a very budget-friendly public cloud offering. They are often recommended as a reliable extra-low-budget solution, and I've had much better luck with them than other similar providers.

On the surface, their prices seem to be just a fraction of those of the larger providers, so I did extended benchmark runs over days to make sure there is no significant oversubscribing - except perhaps the cheapest variant (CX23). Only the CCX13 claims dedicated cores. Ironically, those dedicated instances vary significantly in performance depending on which data center you create them in. In the end, the CPX22 (AMD) and CAX11 (ARM) shared core instances are the most stable in performance across instances and regions.

Note that the cheap shared-core types are not widely available, not found in the US regions and they even show no availability at times in the European regions. And while I included a CX23 with EPYC Rome, you will normally get a slower Skylake. I will not include the shared instances in the price/performance charts this time around, as I am thinking that the limited availability does not make them equal contenders.

Test setup & Benchmarking methodology

In order to have much more test runs, I streamlined the test suite into a docker image which you can run yourself. Almost all instances were on 64bit Debian 13, although I had to use Ubuntu 24.04 on a couple, and Oracle's ARM were only compatible with Oracle Linux. To run the entire suite on a system with docker you would do:

docker run -it --rm dkechag/cloud-bench

The suite comprises of:

Benchmark::DKbench

As every year, the main weight is on my my own benchmark suite, which you can now also run on its own docker image. It has both proven very good at approximating real-world performance differences in the type of workloads we use at SpareRoom, and is also good at comparing single and multi-threaded performance (with scaling to hundreds of threads if needed). To run DKbench by itself on a system with docker:

docker run -it --rm dkechag/dkbench

I created multiple instances in different regions and recorded min and max of all runs (both single-thread and dual-thread).

Geekbench 5

I have kept Geekbench, both because it can help you compare results from previous years and because Geekbench 6 seems to be much worse - especially in multi-threaded testing (I'd go as far to say it looks broken to me).

I simply kept the best of 2 runs, you can browse the results here. There's an Arm version too at https://cdn.geekbench.com/Geekbench-5.4.0-LinuxARMPreview.tar.gz.

Phoronix (openbenchmarking.org)

Apart from being popular, Phoronix benchmarks can help benchmark some specific things (e.g. AVX512 extensions) and also results are openly available.

I ran the following benchmarks:

7-zip

phoronix-test-suite benchmark compress-7zip

Very common application and very common benchmark - average compression/decompression scores are recorded.

nginx 3.0 100 connections

phoronix-test-suite benchmark nginx

Select option 3.

Openssl (RSA 4096bit)

phoronix-test-suite benchmark openssl

Select option 1. This benchmark uses SSE/AVX up to AVX512, which might be important for some people. Older CPUs that lack the latest extensions are at a disadvantage.

FFmpeg / libx264

Blender's Big Buck Bunny video was transcoded to an H264 mp4 via FFmpeg, both in single and dual-thread mode.

Results

The raw results can be accessed on this spreadsheet (or here for the full Geekbench results).

In the graphs that follow, the y-axis lists the names of the instances, with the CPU type in parenthesis:

(GR) = Intel Granite Rapids
(E)  = Intel Emerald Rapids
(SR) = Intel Sapphire Rapids
(I)  = Intel Ice Lake/Cooper Lake
(C)  = Intel Cascade Lake
(S)  = Intel Skylake
(B)  = Intel Broadwell
(T)  = AMD Turin
(G)  = AMD Genoa
(M)  = AMD Milan
(R)  = AMD Rome
(G4) = Amazon Graviton4
(G3) = Amazon Graviton3
(G2) = Amazon Graviton2
(CO) = Azure Cobalt 100
(AM) = Ampere AmpereOne M
(AO) = Ampere AmpereOne
(A)  = Ampere Altra

Single-thread Performance

Single-thread performance can be crucial for many workloads. If you have highly parallelizable tasks you can add more vCPUs to your deployment, but there are many common types of tasks where that is not always a solution. For example, a web server can be scaled to service any number of requests in parallel, however the vCPU's thread speed determines the minimum response time of each request.

DKbench Single-Threaded Performance

We start with the latest DKbench, running the 19 default benchmarks (Perl & C/XS) which cover a variety of common server workloads. I tried to build 2-3 instances at different times across at least 3 regions (if the provider allowed), to get a min/max range of performance. Here are the results for single thread:

I think it's the first time in my series of comparisons where a CPU had this clear of a performance lead. AMD's EPYC Turin is simply a tier above anything else. AWS has the fastest setup with that CPU, while GCP’s more expensive C4d seems to vary a lot in performance when their own, cheaper N4d gave more consistent results. Overall, if you are looking for maximum performance per thread, EPYC Turin seems to be the answer if your cloud provider has it.

In the 2024 comparison Intel Emerald Rapids did quite well, but it turns out that is only on non-busy nodes, where the cpu allows for a generous boost - at least for GCP. This is reflected as the range you see on the graph. The new Granite Rapids seems to fix this, providing a bit higher, but mainly more stable performance. So, a solid step forward from Intel, it's just that Turin is really impressive.

As we are waiting for AWS to release Graviton5 publicly, GCP’s Axion is the leader for ARM solutions, impressively offering EPYC Genoa-level performance per thread. I tested Azure's own Cobalt 100 for the first time - it sits between Graviton3 and Graviton4 performance. Ampere's new AmpereOne M finally offers some tangible improvement over the aging Altra, but only matches AWS's older Graviton3.

Lastly, among the lower-cost providers, DigitalOcean has lagged behind in performance, signaling that their fleet is due for an upgrade. Both Akamai and Hetzner offer some fast Milan instances, although for both providers you are not guaranteed what performance level you are going to get when creating an instance - there is the variation shown in the chart. It's not oversubscribing, the performance is stable, it's just that groups of servers are setup differently.

Multi-thread Performance & Scalability

Scalability

DKbench runs the benchmark suite single-threaded and multi-threaded (2 threads in this comparison as we use 2x vCPU instances) and calculates a scalability percentage. The benchmark obviously uses highly parallelizable workloads (if that's not what you are running, you'd have to rely more on the single-thread benchmarking). In the following graph 100% scalability means that if you run 2 parallel threads, they will both run at 100% speed compared to how they would run in isolation. For systems where each vCPU is 1 core (e.g. all ARM systems), or for "shared" CPU systems where each vCPU is a thread among a shared pool, you should expect scalability near 100% - what is running on one vCPU should not affect the other when it comes to CPU-only workloads.

Most Intel/AMD systems though give you a single core that has 2x threads (Hyper-Threads / HT in Intel lingo - or Simultaneous Multi Threads / SMT if you prefer) as a 2x vCPU unit. Those will give you scalability well below 100%. A 50% scalability would mean you have the equivalent of just 1x vCPU, which would be very disappointing. Hence, the farther up you are from 50%, the more performance your 2x vCPUs give you over running on a single vCPU.

As expected, the ARM and shared CPUs are near 100%, i.e. you are getting twice the multithreaded performance going from 1x to 2x vCPUs. You also get that from three x86 types: AWS's Genoa C7a and Turin C8a alongside GCP's older Milan t2d.

From the rest we note that, traditionally, AMD does SMT better than Intel, although the latter has improved from the dismal Ice Lake days when it barely managed over 50%.

Bizarrely, the Akamai AMD Turin give an unusually high (given SMT) scalability of 71.9%. I have verified the result several times, and I can't figure out what their setup is - the single-threaded performance at the same time is very low compared to every other Turin.

DKbench Multi-Threaded Performance

From the single-thread performance and scalability results we can guess how running DKbench multithreaded will turn out, but in any case here it is:

Give the clearly fastest instance two full cores instead of threads and you get the Turin-powered AWS C8a completely dominating the chart. Interestingly, the Google Axion seems at least as good here as the leader from the previous comparison, the Genoa C7a - with Graviton4 very close and Cobalt 100 trailing not far behind.

The SMT-enabled Turin instances follow, with the Top-10 completing with the venerable Milan in a non-SMT Tau instance. Long time followers of these comparisons may remember this was the top of the chart in the 2023 edition.

At the bottom, as expected we have very old Intel Broadwell/Skylake not-as-old Ice Lake and AMD Rome.

Geekbench 5 Score

The old Geekbench 5 is provided for comparison reasons (and I don't trust Geekbench 6):

Both for single and multi-core, the results are very close to what we get with DKbench. Which is a good thing, as both suites try a range of benchmarks to get a balanced generic CPU score.

7zip

Moving on to some popular specific benchmarks - starting with 7zip which is sensitive to memory latency and cache:

While Turin still leads overall, Axion and Graviton4 are impressive and actually even beat it in the decompress part of the benchmark. In fact, Cobalt 100 is the top performer for decompression, but overall the ARM solutions show great performance.

NGINX web server

Results from the 100 connections benchmark:

Another Turin showcase, with the non-SMT AWS C8a in particular almost doubling the score of the second and tripling the score of the C7a. Granite Rapids is also making a great showing.

It's the first time I am running this popular benchmark, and I am a bit puzzled about some of the Milan types coming last.

FFmpeg H264 Video Compression

Another first for this comparison is video compression using FFmpeg and libx264. Results for both single and dual-thread mode:

Once more, EPYC Turin comes first. If we look at single-thread performance only Granite Rapids comes somewhat close. When using 2 full cores Axion can pull ahead of all SMT (i.e. single core) instances except Turin.

OpenSSL RSA4096 (AVX512)

Lastly, in case you have software that can be accelerated by AVX512, I am including an OpenSSL RSA4096 benchmark. They are Intel's extensions so they are on all their CPUs since Skylake, whereas Genoa was the first AMD CPU to implement them. Older AMD CPUs and ARM architectures will be at a disadvantage in this benchmark:

Like in our previous comparison, AMD outperforms Intel at their own game. It's quite a margin for Turin and even Genoa is ahead of anything Intel. Intel does not seem to be prioritising vector performance, as even the latest Granite Rapids does not bring much improvement over the aging Ice Lake.

As expected, ARM and older AMD CPUs that don't support AVX512 are slower than Intel Skylake and newer.

Performance / Price (On Demand / Pay As You Go)

One factor that is often even more important than performance itself is the performance-to-price ratio.

I will start with the "on-demand" price quoted by every provider. While I listed monthly costs on the tables, these prices are actually charged per minute or hour, so there's no need to reserve for a full month.

DKbench single-thread performance/price

The first chart is for single-thread performance/price. I will have to separate Hetzner's shared instances because they are not available in the US and sometimes run out even in Europe (esp. CX23), so I feel they are not exact competition - CCX13 though is available and is included.

Hetzner and Oracle top the list like last year. However, thanks to the incredible performance of Turin, Oracle pretty much matches Hetzner's dedicated instance in performance to cost. They are followed by Linode and also GCP's n4d. The latter, again thanks to the leading single-thread performance of AMD's latest CPU even manages to bring better value than DigitalOcean, which is then followed by in-house ARM solutions like Google Axion and Azure Cobalt 100.

AWS is definitely the worst value on-demand. Their Turin is the best they can do, while their previous gen and older CPUs are the worst values on the table. Unlike the previous comparison, even Azure seems to do better in value.

At this point I think we should see the limited availability Hetzner VMs in comparison to the best value dedicated:

The inexpensive shared-cpu types offer unbeatable value - if you manage to get them. The top one overall (Rome CX23) is actually the hardest to provision, as the CX23 type usually gives you a slow Skylake.

DKbench multi-thread performance/price

Moving on to 2x threads for evaluating multi-threaded performance:

All the non-SMT VMs get a bump here, hence Oracle's ARM take the lead with the new AmpereOne M, with Hetzner and shared core Linode following closely. The second tier consists of Google Axion and Azure Cobalt 100, as well as DigitalOcean droplets. AWS's non-SMT Turin is not that far behind this time, although their older gen 5/6 x86 are again at the very bottom of the chart.

The Hetzner shared-core instances get the bump as well, they provide superb on-demand value compared to the competition:

Performance / Price (1-Year reserved)

The three largest (and most expensive) providers offer significant 1-year reservation discounts. To get the maximum discount you have to lock into a specific VM type, which is why it is extra important to know what you are getting out of each. Also, for AWS you can actually automatically apply the 1 year prices to most on-demand instances by using third party services like DoIT's Flexsave (included in their free tier!), so this segment may still be relevant even if you don't want to reserve.

DKbench single-thread performance/price (1Y)

The first chart is again for single-thread performance/price.

The 1-year discount is enough for GCP’s Turin to match Oracle near the top of the value ranking. On Azure you get some good value running Cobalt 100 or Genoa. If you are on AWS your best bet are the latest C8 family.

DKbench multi-thread performance/price (1Y)

Moving on to evaluating multi-threaded performance using 2x vCPUs:

OCI ARM instances are still at the top, joined by Azure Cobalt 100 with Axion almost keeping up. This is the first instance where AWS can offer similar value, thanks to the C8a with the fast Turin offering twice the physical cores, making up for the higher price.

Performance / Price (3-Year reserved)

DKbench single-thread performance/price (3Y)

Finally, for very long term commitments, AWS, GCP and Azure provide 3-year reserved discounts:

GCP with its Turin instances finally comes just ahead of Oracle and even Hetzner's dedicated VM. Azure also provide good value with their Cobalt 100 and Turin types. It should be noted that even if AWS lags behind the other, at a 3 year commitment it still offers better value than the "classic" value providers Akamai and DigitalOcean.

Multi-thread performance/price (3Y)

Switching to multi-thread, the number of physical cores per vCPU makes the difference:

I didn’t expect this, but Azure Cobalt 100 tops the chart! It is followed by GCP and OCI ARM solutions, but AWS's and GCP's Turin are not far behind.

Performance / Price (Spot / Preemptible VMs)

The large providers (AWS, GCP, Azure, OCI) offer their spare VM capacity at an - often heavy - discount, with the understanding that these instances can be reclaimed at any time when needed by other customers. This "spot" or "preemptible" VM instance pricing is by far the most cost-effective way to add compute to your cloud. Obviously, it is not applicable to all use cases, but if you have a fault-tolerant workload or can gracefully interrupt your processing and rebuild your server to continue, this might be for you.

AWS and OCI will give you a 2-minute warning before your instance is terminated. Azure and GCP will give you 30 seconds, which should still be enough for many use cases (e.g. web servers, batch processing etc).

The discount for Oracle's instances is fixed at 50%, but varies wildly for the other providers per region and can change often, so you have to be on top of it to adjust you instance types accordingly.

For a longer discussion on spot instances see 2023's spot performance/price comparison. Then you can come back to this year's results below.

DKbench single-thread performance/price (Spot)

Applying the lowest January 2026 US spot prices we get:

We get that Oracle's Turin will always be top value, as it has a fixed spot price. From the big 3, GCP and Azure offer the deepest discounts (Genoa and Cobalt 100 types), the former getting top place here. If you compare to the 3 year reservation chart, you are getting about twice the performance per dollar. AWS is much less generous, if you are on their cloud, Turin is once more your best bet. But even with AWS you are getting better value if you are using spot instances than other low cost providers.

Multi-thread performance/price (Spot)

The multi-thread chart:

Azure's Cobalt 100 tops the chart with OCI's AmpereOne M following. Interestingly, in third place and best for the GCP cloud, is the aging t2d Milan which was noted as a great value in previous years. AWS once more has Turin saving the day by just making it into the top 10. You can get great value with all providers that offer spot instances, but you do have to monitor prices.

Conclusions

As always, I provide all the data so you can draw your own conclusions. If you have highly specialized workloads, you may want to rely less on my benchmarks. However, for most users doing general computing, web services, etc. I'd say you are getting a good idea about what to expect from each VM type. In any case, I'll share my own conclusions, some reasonably objective, others perhaps somewhat subjective.

Overview

Let’s begin with some quick take-aways, especially for things that are new this year:

AMD Turin is an absolute beast. It's so fast that it's usually a good value even when priced higher.
Intel Granite Rapids is a solid step forward, avoiding the inconsistent performance of its predecessor (Emerald Rapids).
On the ARM side, Google Axion shows up as a genuinely high-performance option, while Azure Cobalt 100 and Ampere AmpereOne M add more variety and better value tiers than older ARM offerings.
Hetzner offers amazing value, with Oracle following (with a great free tier too).
From the big-3, GCP and Azure battle it out in price/performance, when AWS is usually more expensive.

General Tips

Upgrade to the modern CPU types when possible. Older VMs are slower and tend to be more expensive due to higher operational costs.
Plan your usage and consider making reservations (3-year preferred) to lower costs where applicable. Remember, you can get free 1y reservation prices through 3rd parties (e.g. DoIT) if you are with AWS.
Leverage spot VMs as much as possible. They are essentially the only way the cloud can compete with the cost of buying and running your own servers. Check prices periodically and across all regions that interest you to find the best deals.
Remember that vCPUs are not always comparable: ARM systems and very few x86 systems like AWS's C8a/C7a and GCP's t2d, provide a full CPU core per vCPU. Most others give you a full core per 2x vCPUs.

Recommendations per use-case

I'll further comment with my picks for various usage scenarios:

Budget solution: If Oracle's free tier is not enough for your project, look at Hetzner - especially if you are fine with non-US regions and perhaps limited availability to use the shared-CPU types (preferably CPX or CAX). However, if you can use spot instances, Azure can offer you ARM instances and Oracle and GCP can offer you either ARM or AMD instances with almost comparable performance/price.
Best overall value (performance/price) for non-shared CPU: If you can't use spot instances (best value) or reserve for 1+ years, Oracle pretty much tops the performance/price charts with both ARM and x86 (AMD) options. And the charts do not even include the free 4x A1 vCPUs that can make a big difference for small projects. If reservations are an option, in the common case of multi-threaded workloads, you can match OCI (or even beat it when committing for 3 years) with Azure Dpls_v6, GCP c4a/n4d or perhaps AWS C8a. If you only care about single threaded performance, look at GCP n4d/c4d.
Maximum performance: It's clear that Turin-powered VMs from all large providers are on a different level to anything else. The best one performance-wise is the non-SMT AWS C8a.

Summary per cloud provider

Finally, I'll make some comments per provider. Besides, we can't always pick a provider or switch, so we have to try to work with what's available to us.

AWS: They were the top ARM performers in my previous comparisons, but as Graviton5 is still in private beta, Google Axion now leads. However, they offer the fastest compute cloud VMs overall in the C8a with non-SMT EPYC Turin. That VM family's performance makes AWS's traditionally higher prices provide some decent value, especially if you can use spot VMs, or reserve (or use a service like Flexsave). Because AWS does not discount older gens as deeply as others, the C8a is usually the best value even for Spot instances.
GCP: It's pretty clear that you should be on GCP's 4th gen ARM or AMD instances if you care about both performance and value. The n4d Turin is the best bet for most cases - in all my tests so far it performs pretty much the same as c4d at a lower price. Only if your work load can use all cores on your system, the Axion c4a will be a bit better still, but that's about it for standard instances. For spot instances things change per-region and usually previous gen instances will give you the best performance per price. It could be t2d or c3d like in the tested region, but there are also regions where c2d or n2 (as long as you set min_cpu_platform="Intel Ice Lake") come out ahead.
Azure: Who'd have expected Azure developing in-house ARM CPUs, but here we are and Cobalt 100 might not be as fast as Google Axion, but it's not far behind and it's offered at competitive prices, so you'll get similar - or possibly better for spot instances - value compared to GCP.
Oracle: I definitely recommend Oracle for whoever has small projects where a 4-core ARM VM, which Oracle provides for free, covers most of the requirements. Their non-free instances are also very competitive, with their on-demand prices comparable to 1-3 year reservation prices from the "Big 3" (Oracle doesn't do reservation discounts), with the best value being the AmpereOne M A4 and Turin E6 for ARM and x86 respectively. The A4 are not yet widely available and if you have a free account you will have further limitations as to what VMs you can provision.
Akamai: The Linode shared-CPU instances still offer a good value (second to only Hetzner and Oracle if you only consider on-demand pricing), however you need to build and check each instance to make sure it gets an EPYC Milan (4-digit CPU code ending in 3, check it with cat /proc/cpuinfo on Linux), otherwise you might be paying the same for less. In most of my test instances I would indeed get a Milan though. At least for dedicated instances you now select a "generation", so basically you pick the CPU. They are not as good value as the Linodes, but the G8 is the best of them in both performance and value, although it's kind of a bizarre Turin setup where single-thread performance is on a lower tier than any other provider, but SMT gives a surprisingly (and inexplicable to me) high boost when multi-threading.
DigitalOcean: Although they still provide decent value, they have fallen quite behind Akamai, as there have not been any upgrades. In fact, there is quite a bit of over-subscribing, so performance drops lower than usual. I still use their Premium AMD shared-cpu instances for projects due to the history of reliability I've had with them, but if I want faster servers I have to look elsewhere, which is a shame. They also have the easiest upgrading an instance from one type to another with one click (as long as the target has at least as much disk space). As with the other lower cost providers, you do not know exactly what performance level a VM will have until you provision it. If you'd want to give them a try, feel free to use my affiliate link to sign up and get $200 in credits while supporting the free weather service 7Timer!.
Hetzner: I am very suspicious of extreme "budget" cloud providers, but Hetzner has many longtime satisfied users. Their reputation seems quite solid - most complains I've read online are from banning people, usually with good reason. Their prices seemed too good to be true, so I suspected oversubscribing. Well, they don't seem to be worse than the likes of Akamai or DigitalOcean, VMs seemed to perform at reasonably stable performance levels for hours to days at a time, perhaps with the exception of the absolute cheapest, the CX23. Interestingly, the dedicated core CCX13 had more performance fluctuation than the shared-core CPX22, as long as that one comes with the fast EPYC Genoa I was seeing when testing. Unfortunately, the CPX22 is available only in eu-central and ap-southeast, but if that's OK with you it is the best value and fastest overall.

Finally, remember that choosing a cloud provider involves considering network costs, fluctuating prices, regional requirements, RAM, storage, and other factors that vary between providers. This comparison will only assist with part of your decision.

NOAA::Aurora for Space Weather Forecasts

Dimitrios Kechagias — Thu, 25 Dec 2025 01:09:01 +0000

With the current solar maximum, I wanted to add aurora forecasting features to my iOS weather app, Xasteria. Instead of fetching text files from NOAA, I thought it would be nice for my weather proxy server to handle that. Hence I developed NOAA::Aurora and released it to CPAN.

The module provides a convenient interface to the NOAA Space Weather Prediction Center (SWPC), handling data fetching, parsing, and caching.

Usage

Simple OO interface, you can get short term aurora probability either as a map:

use NOAA::Aurora;

my $aurora = NOAA::Aurora->new();

# Short-term forecast map for the Northern hemisphere
$aurora->get_image(hemisphere => 'north', output => 'aurora_n.jpg');

Or as a probability:

# Get short-term aurora probability for a given location
my $probability = $aurora->get_probability(lat => 51, lon => -2);

You can also get a forecast timeseries for the next 3 or 27 days:

# Let's switch to RFC 3339 format for the timeseries
my $aurora = NOAA::Aurora->new(date_format => 'rfc');

# Get 3-day kp forecast as a timeseries
my $forecast = $aurora->get_forecast();
# [
#   {
#     'kp'   => '3.67',
#     'time' => '2025-12-25 00:00:00Z'
#   },
#   {
#     'kp'   => '2.67',
#     'time' => '2025-12-25 03:00:00Z'
#   },
#   ...
# ]

# Get 27-day outlook as a timeseries
my $outlook = $aurora->get_outlook();
# [
#   {
#     'ap'   => '28',
#     'flux' => '130',
#     'kp'   => '5',
#     'time' => '2025-12-22 00:00:00Z'
#   },
#   {
#     'ap'   => '22',
#     'flux' => '135',
#     'kp'   => '5',
#     'time' => '2025-12-23 00:00:00Z'
#   },
#   ...
# ]

Google Cloud SQL: x86 N2 vs ARM C4A

Dimitrios Kechagias — Sat, 08 Nov 2025 00:24:37 +0000

The recent upgrade of our production Cloud SQL databases to Enterprise Plus required an in-place process to minimize downtime for our 2TB instance. This meant that we couldn't explore the new C4A ARM option offered under Enterprise Plus offers, as it currently does not support in-place upgrades (I am told Google is working on it though).

Given how impressed I was when testing Google Axion-powered C4A GCP VMs, I thought I should explore the C4A Cloud SQL performance for initial use in our test environment, with an eye toward potential future production use.

Enterprise Plus: C4A vs N2
Import Performance
Latency
SELECTs
- Very fast (<10ms)
- Fast (<200ms)
- Slow (>200ms)
- Very Slow (>20s)
UPDATEs
INSERTs
DELETEs
Addendum: Enterprise N4
Conclusions

Enterprise Plus: C4A vs N2

In my testing, the 4th gen C4A cores, based on Google's own Axion ARM CPU, prove both faster than the Intel Cascade Lake / Ice Lake N2 CPUs and they offer a full physical core per vCPU instead of just a Hyper-Thread. This will surely have some throughput advantage.

Additionally, C4A as a 4th gen instance has access to the newer "Hyperdisk" storage with fully configurable IOPS and throughput (instead of having them scaled depending on size). This flexibility comes with a slightly higher cost, as shown in the comparison below:

Instance	N1-14	N2-8	C4A-8 low	C4A-8 mid	C4A-8 high
vCPU #	14	8	8	8	8
Machine Type	N1 (x86)	N2 (x86)	C4A (ARM)	C4A (ARM)	C4A (ARM)
RAM	64GB	64GB	64GB	64GB	64GB
Storage	600GB (SSD)	600GB (SSD)	600GB (Hyperdisk)	600GB (Hyperdisk)	600GB (Hyperdisk)
IOPS	15000	15000	4200	7000	15000
Throughput (MB/s)	288	288	318	700	1000
Price/h	$1.25	$1.23	$1.24	$1.36	$1.63

The 600GB storage size matches the sample database used. For price parity, I’ve included an N1 Enterprise instance with nearly twice the cores. The C4A was tested in three different Hyperdisk configurations:

low: for price-parity, but close to minimum possible Hyperdisk specs, so not recommended.
mid: ~10% higher cost, more realistic balance.
high: ~30% price premium, used to test scaling with high IOPS.

Benchmarks were performed using my MCE::Benchmark
Perl module to run randomized queries with varying thread counts, simulating different load levels.

A 32x C4D VM in the same region and zone as the MySQL 8.0 Cloud SQL instances handled the benchmarking, connecting via Perl DBI/DBD::mysql.

Import Performance

The first task was importing the ~600GB MySQL database from a GCP bucket:

Even the C4A-mid shows over 35% advantage vs the N2 and twice that over the N1 instance. This is a significant difference, cutting hours from the import of such a large database.

Latency

In my previous post, I briefly mentioned the latency differences between N1 and N2. Here’s a closer look at "SELECT 1" response times across 1–64 threads:

Impressively, the C4A delivers an average round-trip time of just 0.1ms, making the N2’s 0.25ms seem sluggish by comparison. The N1, with a much slower 1.5ms latency, processes far fewer requests per second and thus doesn’t experience contention even at 64 threads.

SELECTs

Randomized SELECTs based on real workloads were grouped into four categories by execution time.

Very fast (<10ms)

For small in-memory queries, let's see the max throughput at various concurrency levels:

Having twice the hardware cores vs N2, the C4A saturates slower and maxes out at over twice the throughput. We can also see the same data as a query response time:

It's clear that under load (8+ threads) C4A does not slow down nearly as fast as N2.

Fast (<200ms)

Similar behaviour, also in-memory queries, N2 saturates here at 4 threads, C4A increases throughput past 8:

Slow (>200ms)

For more complex queries involving larger joins, the C4A’s advantage narrows. At 64-thread concurrency, performance collapses as the workload becomes I/O bound:

Interestingly, the N2’s storage (with higher theoretical IOPS) outperforms the mid-tier Hyperdisk in these extreme scenarios. This does not imply that maximum Hyperdisk specs are required - if your MySQL workload depends that heavily on disk I/O, you are probably doing something wrong.

Very Slow (>20s)

For very slow reporting queries, once warmed up, C4A maintains a 25–30% lead over N2.

For very slow report queries, after the database is warmed up, we can see a 25-30% advantage of C4A over N2:

It may also be interesting to see a "warmup" run on a cold database, where data will be read from the disk:

As we saw before, in extreme disk access cases such as this cold start, N2’s higher base IOPS outperform the mid-tier Hyperdisk configuration.

UPDATEs

In a mix of updates, including compound indexes, which means both CPU and disk access are involved, the N2 seemed to keep up better with the C4A:

The low-spec Hyperdisk configuration quickly becomes a bottleneck; at least the mid level is needed for balanced performance. N1 trails significantly once again.

INSERTs

A mix of complex inserts, with the disk access playing a bigger role:

It's quite interesting to see how throughput increases with load, until there is some sort of storage-related bottleneck hit. It happens quickly with the low Hyperdisk, then with the mid Hyperdisk, while the high Hyperdisk config manages to keep up farther than the N1/N2 solutions.

DELETEs

I tested 2 types of mass DELETE statements - one with a LIMIT x loop, the other with passing ids as bind variables:

Two types of bulk DELETE operations were tested: one using a LIMIT x loop and another using bound ID variables (single-threaded for consistency).

There is a near 20% advantage for C4A, as long as the IOPS of the Hyperdisk are not set too low.

Addendum: Enterprise N4

I performed the above tests over the summer. In the meantime Google released Enterprise instances based on their 4th Gen x86 architecture (Intel Emerald Rapids, Hyperdisks), so I thought I'd add a quick comparison between the N4 Enterprise and N2 Enterprise plus for completeness. For about the same price you can get 50% more N4 cores than N2, keeping the Hyperdisk IOPS at the mid levels we used in the previous tests. Similar to the C4A, you cannot currently convert an N1 or N2 instance to an N4 due to the different storage.

Instance	N4-12	N2-8
vCPU #	12	8
Machine Type	N4 (x86)	N2 (x86)
RAM	64GB	64GB
Storage	600GB (Hyperdisk)	600GB (SSD)
IOPS	7000	15000
Throughput (MB/s)	480	288
Price/h	$1.27	$1.23

Here are the results of the most common types of in-memory SELECTs:

As expected, the N4 cores are a bit faster, and you can get more for the price so give you a significantly higher throughput, making them a better value (it would mean forgoing the extra Enterprise Plus benefits of course).

I tried some tests with DELETEs and some very deep SELECTs with CREs, which seemed to variously go one way or the other between the two types, but I couldn’t achieve consistent enough results in time to include them, so I will need to come back to this, along with an N4 to C4A comparison.

Conclusions

The takeaways as I see them:

For maximum in-memory query performance & throughput, you should go to C4A. It will cost a bit more as you should pair it with a Hyperdisk that has reasonable specs.
In the possibly less common case where disk I/O is the main concern, N2 may be your budget solution with the disk I/O performing on par or better than a mid spec Hyperdisk at a lower price. However, for I/O performance regardless of budget, the Hyperdisk of a C4A can be configured to be much faster at an additional cost.
The addition of the N4 option for Enterprise with better performance makes the N2 less attractive if you don't mind losing the Enterprise Plus features. The C4A will still have an edge in performance, due to slightly faster cores (and full core/vCPU), but I will have to follow up with some benchmarks on it.
Overall, I would be choosing between N4 & C4A for new deployments. Existing non-Hyperdisk instances are not currently directly upgradeable to them, so it is not as easy to take advantage of them in existing deployments.

Is Cloud SQL Enterprise Plus Worth It? Evaluating the MySQL upgrade at SpareRoom

Dimitrios Kechagias — Mon, 08 Sep 2025 03:15:23 +0000

At SpareRoom, MySQL is a core part of our infrastructure and it's been running on Google Cloud SQL for about five years. Earlier this year we faced a major change: upgrading to MySQL 8. This gave us the option of switching from Enterprise instances (old N1 machine family), to Enterprise Plus, which features the newer N2 machines.

Enterprise Plus is more expensive, has very specific configuration options (vCPU in powers of 2, fixed 8GB RAM/vCPU), several new features, but rather vague promises of performance improvements. Without clear benchmarks, it wasn’t obvious whether the extra cost was justified, or what the right size of instance should be.

This was obviously the time to set up my own benchmarks.

Quick table of contents:

The Benchmark Setup
- Top-5 Queries
- DELETEs and Schema Changes
Real World Performance
Verdict

The Benchmark Setup

As I always do, I focused on workloads that reflect real-world pain points for us:

Top queries: quick (tables fit in memory) but very high-volume SELECTs (searches, views, clicks, etc.) that drive peak load.
Batch deletes: archiving and clearing out large log tables, which must fit limited nightly windows.
Schema changes: migrations that tie up our pipeline for the longest.

Our baseline DB was a 24 vCPU / 90GB RAM Enterprise instance. The upgrade tool defaults to "next up", so 32 vCPU / 256GB Enterprise Plus (more than double the cost, so not viable). Instead, we compared our baseline to:

Enterprise Plus 16x vCPU (most reasonable candidate, DC on/off versions)
Enterprise Plus 8x vCPU (scaling baseline)
Enterprise 28x vCPU (rough cost match to 16x Plus when keeping baseline 90GB RAM)
Enterprise 16x vCPU (direct comparison to 16x Plus)

DC above is Data Cache, an optional locally attached SSD storage that can improve read performance. All the above in a table:

Instance	vCPUs	Machine	RAM (GB)	Storage (GB)	IOPS	Price/h
Enterprise x16	16	N1	64	1500	25,000	$1.64
Enterprise x24	24	N1	90	1500	25,000	$1.97
Enterprise x28	28	N1	90	1500	25,000	$2.40
Enterprise Plus x8	8	N2	64	1500	15,000	$1.36
Enterprise Plus x16	16	N2	128	1500	25,000	$2.37
Enterprise Plus x16 DC	16	N2	128	1500 + 750 (DC)	25,000	$2.54

Benchmarks were run from a fast 64 vCPU C4D VM in the same region and zone as the Cloud SQL servers.

Top-5 Queries

Using Cloud SQL Query Insights, I pulled our top-5 queries and ran them in a weighted benchmark (queries balanced by duration) with randomized bound variables. The benchmark ran with my MCE::Benchmark Perl module, which loads it on multiple threads to simulate any amount of load. Results with Data Cache on vs off were inconsistent and since in-memory SELECTs are not supposed to benefit from it anyway, I excluded it from analysis.

After warming up the DB, I ran the benchmark on 12 threads to see performance when the DB is at around or below 50% load (i.e. normal operation at peak usage) as well as 32 threads to get an idea of full load behaviour:

At medium load (12 threads):

Enterprise Plus 16x was ~45% faster than Enterprise 24x/28x.
Enterprise Plus 8x was small enough to be close to full load, but still kept up with the big Enterprise instances.

At high load (32 threads):

With 75% more cores, Enterprise 28 finally caught up at high load. But even in this scenario, which is the worst case for Enterprise Plus, it did not show a disadvantage.
The Enterprise Plus 8x kept up with Enterprise 16x (while costing ~17% less).

DELETEs and Schema Changes

Deleting millions of rows (25k at a time in a loop), modifying the definition of a column and adding an index were measured next. The number of cores has very little importance (especially for the schema changes), it's more CPU type and storage that matters:

DELETEs are 25-40% faster on Enterprise Plus, depending on whether you have Data Cache enabled.
Schema changes are at a more modest 5-15% difference, with Enterprise Plus always having a small edge.

Real World Performance

Armed with the above results, we switched most of our DBs to (smaller) Enterprise Plus instances. The above Enterprise 24x became an Enterprise Plus 16x. Using Query Insights I saved the performance of the heaviest queries during the last week of Enterprise and compared them with the performance of the same queries during the first week of Enterprise Plus to calculate performance gains per query. The two weeks were reasonably comparable in traffic and the 32 query types plotted below represent about 360 million calls for each respective week:

The average performance gain is about 35%. This is the actual execution time, vs the total round-trip time of the synthetic benchmark above that was closer to 45%. Some of the gap is likely due to network latency, which also improved slightly on Enterprise Plus.

I will follow up with a post exploring latency and performance differences further, along with comparing non-x86 options.

Verdict

Overall, Enterprise Plus cost us ~20% more (due to the limited configuration options). However, the performance gains, as well as the extra benefits (better SLA, near-zero downtime maintenance, 35 day Point-in-Time recovery, index advisor and more), made it a clear win.

If you’re running Cloud SQL on Enterprise and considering the move:

Don’t assume you need the same or higher core count like the upgrade forces you to. Smaller Enterprise Plus can outperform bigger Enterprise, so downsize after your upgrade.
Expect a tangible real-world performance boost.
The reliability/maintenance and other features are a nice bonus on top.

Benchmark::MCE on CPAN

Dimitrios Kechagias — Wed, 13 Aug 2025 23:00:25 +0000

I recently refactored the multi-core benchmarking framework I've been using for my Perl CPU benchmark suite (Benchmark::DKbench) and released it as a separate module: Benchmark::MCE.

Why spin it out? Because the harness can do more: it can be used to write custom benchmark suites of any type, generate massively parallel workloads for stress testing, or run throughput benchmarks against services and APIs.

The exact scenario that prompted me was a comparison of Cloud SQL database instances. We wanted to see how a 16-CPU Enterprise Plus instance would compare to a 24-CPU Enterprise instance under heavy load. One way to do that is to write one or more functions that run randomized, typical/heavy queries (e.g. random searches for SpareRoom ads in our case), then use Benchmark::MCE to time them running on dozens of parallel MCE workers to simulate high load:

use Benchmark::MCE;

suite_run({
  threads => 64,   # Parallel workers
  scale   => 100,  # Workload multiplier (x times to call code)
  bench   => {
    Name1 => sub { ...code1... },
    ...
});

This is the simplified syntax, but since it was part of a benchmark suite I've been developing for a while now, there are many more features (verifying correctness, iterating stats, single/multi-core scaling etc).

AI Image Creation: ChatGPT vs Gemini vs DALL·E vs Grok

Dimitrios Kechagias — Thu, 19 Jun 2025 01:29:59 +0000

Over a year ago I published a comparison of Google's Duet AI image generation with Microsoft's DALL·E 3 powered image creator. The focus was image generation for presentations, articles or apps and the results were promising, even though there were spectacular failures in a few subjects. I am revisiting the exact same prompts with the "current crop" of AI generators.

OpenAI ChatGPT Plus (4o): While the popular chat originally used DALL·E 3 exclusively, it recently switched to its 4o Image Generation for paid accounts and this is the version I will be testing (on a "Team" account). The free accounts seem to still be limited to DALL·E 3 (in addition to slow generation during peak). One image is generated per prompt. Only Microsoft Image Creator (DALL·E 3) remains unchanged since the last test — all others are new or upgraded.

Google Gemini (Imagen 4): It is the successor to the Duet AI I was testing in the previous comparison. It is a significant improvement and the free chat plan does include image generation (with a daily limit). For this comparison I used Gemini Enterprise from within Google Slides, which uses the same engine just generates 4 images per prompt by default. I did the comparison with the Imagen 3 engine and had to redo all the Gemini images as the improved Imagen 4 was released while I was writing the report.

Microsoft Image Creator (DALL·E 3): Free for 15 image generations daily (with 4 images per prompt), using DALL·E 3. This is the only engine also used in my previous comparison, performing pretty much the same.

xAI Grok: The free Grok chat allows you 10 images per 2 hours with its Grok 3 image generation, which is quite generous and should be enough for most non-pro use cases. A single image is created per prompt.

Table of Contents for convenience:

The Comparison Tests
- The Camels
- Astronomy
- Technical Drawings
- Logo/Splash Graphic Design
- Blog Header Image
- Sloths with Headphones
Conclusion

The Comparison Tests

In my previous comparison I tried to recreate slide deck images I had used in various talks, app icons and header images to dev.to articles. I am using the exact same methodology to see whether there has been progress since last year.

There are 12 comparison image rounds in total where all engines get the same prompts. Unlike last year, I did not tweak prompts based on results. This time I used the exact same prompts from the 2024 test for a more controlled and direct comparison.

As before, each round was rated from 0 to 10 based on the proximity of the result to expectations, suitability for the intended purpose and adherence to instructions. The ratings are subjective, but the images are included so you can draw your own conclusions.

The Camels

For the last few years I've been giving talks at the Perl and Raku conference. My presentations always feature some sort of camel, as that's the most recognised Perl symbol, so I tried to recreate a couple of camel images I have used in the past.

Running Camel

For a Software Performance Optimization talk, an image of a running camel was used (an inexpensive stock photo):

The simplest prompt I could think of:

a happy camel running

The results:

ChatGPT is simple and quite good - I got a cartoon, but I did not actually specify otherwise. Microsoft's is also not realistic, but kind of very specific and weird style. Gemini gives both a cartoonish and 3 realistic camels running, while Grok's don't seem to be running, and they are too zoomed-in to tell for sure.

Specifying:

a photo-realistic happy camel running, single colour background

ChatGPT and Gemini are pretty much what I had in mind. DALL·E is not photorealistic and Grok did not really improve at all with my second prompt, not even the background was simplified.

Let's generate some scores:


ChatGPT	10/10
Gemini	10/10
Image Creator	6/10
Grok	4/10

Google already had good results last year, but ChatGPT is added as a top performer as well.

Camel with Glasses

A camel with glasses was used for the Fast Perceptual Image Hashing talk, created from two stock images:

It's a bit rudimentary as I am neither a designer, nor did I want to spend much time on it. Perhaps an AI generator could have managed this with an appropriate prompt:

A smiling camel looking at us through big blue glasses, single colour background, photo-realistic

At this point I feel like I was too lax with scoring last year, as I had given Duet AI a 10, while ChatGPT and Google's own Gemini are now clearly better. Grok has the issue of the glasses being a bit off center compared to the eyes, which may be realistic in how glasses would not fit a camel that well, but that's not really what we are after, is it? :)

In the end I decided to retroactively adjust last year scores by -1 to show the meaningful improvement. I came close to adjusting a couple more scores, but that was a bit of a slippery slope, so in the end I adjusted only the most egregious examples (this and the sloths at the end) by -1.


ChatGPT	10/10
Gemini	10/10
Image Creator	7/10
Grok	9/10

Astronomy

I used to do talks for my local astronomy club, so I picked some of the slides I used for a "Choosing Binoculars" talk.

Photos of Astro Objects

This slide with examples of objects as you'd view them with binoculars was going to be a long-shot:

Without the "how would they look through binoculars" element, I tried to give a list of objects to see if I could get the sort of "astrophotos on canvas" style above:

A compilation of photos one each of the astronomical objects: the moon, pleiades, orion nebula, andromeda galaxy, the Double Cluster, comet Lovejoy, arranged randomly on a canvas with slight overlaps

ChatGPT is the only one that gets the right number of images, even though it just does repeats giving 2x Andromeda galaxies and 3x Pleiades. Grok is visually interesting, but not close to what I asked, while Google's and Microsoft's solutions have tons of not overly realistic objects, with Gemini adding some labels full of typos.

Repeating last year's second attempt:

A compilation of 6 photo-realistic photos, one each of the astronomical objects: the moon, pleiades, orion nebula, andromeda galaxy, the Double Cluster, comet Lovejoy, arranged randomly on a bigger canvas, some overlap is allowed

Gemini does probably worse than last year. Adding the label "Pleades" (sic) to something random does not make it the Pleiades... And it still cannot count. Interestingly, the previous Imagen 3 engine Gemini was using until a few days ago did a bit better. DALL*E can't do anything useful as we saw last year and Grok, again visually interesting, with realistic-looking objects, but not getting the "photos on canvas" instructions. ChatGPT actually does well, if there was no repeat of one of the 6 images, it would have been perfect.


ChatGPT	8/10
Gemini	2/10
Image Creator	1/10
Grok	3/10

Stack of Binoculars

This photo is from the same presentation:

4 binoculars stacked on top of each other from largest (bottom) to smallest (top), with their lens pointed towards our viewpoint, photo-realistic

I think we established last year that DALL-E 3 does not understand binoculars. Add Grok to this category, the results are photorealistic but outlandish binocular-inspired depictions. Google's Imagen 4 update is a big improvement (just a few days ago I was getting results close to last year's Duet AI), with usable results. ChatGPT's latest solution gets it right on the first try.

Alternative prompt with more hints:

4 pairs of binoculars of various types, stacked on top of each other from largest (bottom) to smallest (top) with their lens pointing to viewer, photo-realistic

I was very lenient last year with Duet AI managing to produce one usable image on the second try, ChatGPT and Gemini go far beyond by producing good results from the get go and improving with more hints. Bing and Grok are alien to the concept of binoculars.


ChatGPT	10/10
Gemini	10/10
Image Creator	1/10
Grok	2/10

Lawn Chair Binocular Mount

Again on the same slide deck, there was an image of one of the various DIY "lawn chair binocular mounts" that can be impressive and sort of amusing:

They are sometimes called "bino-chairs" and require some design creativity and ingenuity, so it was a different type of test for the AI engines.

man on lawn chair using hands-free binocular mount

Image Creator gets a lenient 5 as it gets a sort of tripod in one of the images (but not completely hands-free). ChatGPT and Gemini gave great images, I will deduct one point from Gemini for the distinction of ChatGPT getting the "hands-free" aspect with no extra direction (Gemini gets there easily if you add to the prompt). Grok would have been spot on, if the binoculars were not the wrong way around!


ChatGPT	10/10
Gemini	9/10
Image Creator	5/10
Grok	6/10

Technical Drawings

For technical talks, explanatory illustrations are often required. E.g. the following image from the Perceptual Image Hashing talk shows which cells of a 6x6 matrix are used for a specific hash:

draw a symmetric 6x6 square matrix with white lines, make the top-left cell black, also the cells that are below the bottom-left to top-right diagonal also black, and the rest blue, 2d art style

At this point, Grok started malfunctioning. First it started giving me images that incorporated the previous prompt for no reason:

Even including "forget context" etc instructions in the prompt along with the instructions did not help, but a "Forget previous images. Start from scratch." prompt did make it exclaim it "understood" and it will start afresh, then giving me a broken image link. When I complained I can't see it, I finally got a result. Not a great result though:

So, Grok, Gemini and Image Creator are trying to hard to go wildly off script. ChatGPT almost got it perfect. There's a small error about which diagonal the black squares fall under, but it's close, it even automatically switched to square format output.

Going to the very basics, replacing even the word matrix with "grid" to see if the others can be helped.

plain 6x6 square grid, solid colour background, vector drawing

ChatGPT nails it, Gemini gets it in 2 out 4 examples, while the other two are just wildly off.


ChatGPT	9/10
Gemini	4/10
Image Creator	1/10
Grok	1/10

Logo/Splash Graphic Design

Moving on to a couple of logos / splash screens I designed for the iOS apps I develop as a hobby. It's not typical slide deck graphics, but it could still be relevant if you are putting together a new project / product and creating a presentation for it.

Polar Scope Align Icon

Polar Scope Align is an iOS app for amateur astronomers & astrophotographers. It's quite popular in its niche and it is often praised for a well-designed UI with a focus on functionality. The image assets themselves, such as icons, are rather simple as I am not a designer. Here is the older (left) and newer (right) icon of the app:

Hopefully, with the right prompt, something that resembles the older & simpler icon on the left could be within the abilities of the AI engines.

red crosshairs with circle around them, centered on the middle of the 7 stars of the Little Dipper, the Little Dipper should barely fit the circle, clip-art style

ChatGPT gets the style very well, except in reverse colours - not sure why it went with black stars on white. It does not get the actual constellation - but no other generator did either, with Gemini being the closest in style. Grok did not do clip-art as instructed, while Image Creator is visually interesting but not close to what I was asking.


ChatGPT	5/10
Gemini	3/10
Image Creator	2/10
Grok	2/10

Xasteria Icon

Next up, the icon / splash screen for the app Xasteria Astronomy Weather:

solid dark blue sky, having several yellow 4-pointed stars of various sizes, each designed using hyperbolic curves, but all with their points at top/bottom/left right orientation, a third of the sky covered by a dark grey mountainous range silhouette and a big white X that is the same shape as the 4-pointed stars but rotated 45 degrees and takes up 80% of the width of the scene, clip-art style

ChatGPT is the best on the first try. It got the stars and mountains right, only the X is almost, but not exactly right, going with parabolic instead of hyperbolic curves. Gemini's X is squared, so a bit worse. Grok changed its aspect ratio a bit for some reason (wider, and kept doing that for about half the subsequent tries), and gave me some weird stars and asymmetric X. Correct colours though, so I'd call it an improvement over the weird Microsoft attempt.

To help Dall·E last year I had asked ChatGPT to optimize the prompt and I got this one to try:

create an image of a serene landscape with a solid dark blue sky. Populate the sky with several yellow 4-pointed stars of various sizes, each designed using hyperbolic curves. Ensure that all stars have their points at top/bottom/left/right orientations. Dedicate one-third of the sky to a dark grey mountainous range silhouette. Additionally, include a prominent white X shape in the scene. The X should be the same shape as the 4-pointed stars but rotated by 45 degrees, taking up 80% of the width of the scene. The X should maintain the hyperbolic curve design.

The results are mostly worse, except Gemini which gets the X with the hyperbolic curves perfectly in at least one attempt.

The last attempt involved going back to the original prompt, but tweaking the description of the "X" to make it more explicit.

solid dark blue sky, having several yellow 4-pointed stars of various sizes, each designed using hyperbolic curves, but all with their points at top/bottom/left right orientation, a third of the sky covered by a dark grey mountainous range silhouette, at the foreground a big white X that is also designed using hyperbolic curves and takes up 80% of the width of the scene, clip-art style

It actually did worse than the original here, except Gemini which is at a similar level and Dall·E which is a small improvement.


ChatGPT	8/10
Gemini	9/10
Image Creator	4/10
Grok	5/10

Blog Header Image

After all these quite specific images, I thought I'd try more creative generation and see what the AI engines can come up with when given titles of articles - dev.to articles I've posted to be exact.

Google Cloud & AMD EPYC

First, is the performance review of the latest AMD EPYC powered GCP instances. I did a simplified title last year as Duet AI was getting confused, I'll repeat the same:

An image that can serve as a title for a Google Cloud and AMD EPYC presentation

You'll notice I say "title", when I really should have said "header" or similar. I did not notice, as the image generators did not take it literally in the last comparison - possibly because they were lousy with text. Here comes ChatGPT 4o though:

Text looks good and, interestingly, the top half is kind of what I went with myself. However, I will change "title" to "header" for this comparison, as I had expected some sort of graphics would be included:

ChatGPT gave a very simple design, but it is just right, even getting Google's logo and the EPYC font right. Gemini is trying harder to impress, but modifies logos etc in the process. It does know a header image should be on a wide aspect ratio. Image Creator is an improvement from last year, no garbage text. Grok is just uninspired - generic. I'll base the points to what I gave Image Creator last year (I was a bit lenient again).


ChatGPT	10/10
Gemini	9/10
Image Creator	7/10
Grok	4/10

Compute Cloud Provider Comparison

Next was the Compute Cloud Provider performance and price comparison.

an image that can be used as a header in a compute cloud provider price & performance comparison

ChatGPT is again a simple design, gets the text right and the drawing is very on point. Gemini is usable as long as it does not try to add too much text, at which point we start getting gems like "COMPANES" (sic). Image Creator is similar to last year, no text attempted so usable results although a bit too "imaginative. Grok decided to give me a single image for the first time, and it's not great, as there are some weird typos in the title and the chart even weirder.


ChatGPT	10/10
Gemini	6/10
Image Creator	7/10
Grok	2/10

This Article

For this article a generic prompt was attempted:

Header image for blog post: "AI Image Creation: ChatGPT vs Gemini vs DALL·E 3 vs Grok"

ChatGPT was reasonably clear, the others rather disappointing, although Grok did get most of the text OK - these too were the only ones that could spell DALL·E. Gemini and DALL·E could not spell anything.

Since I didn't get good results, I gave more explicit prompts, such as asking for a painter writing a different phrase for each engine on a canvas:

painter writing [name of ai service] on a canvas.

ChatGPT does well as usual, a bit artistic output. Gemini is great, with realistic images and correct spelling / Google logo. Microsoft's service gets the spelling of "Microsoft" slightly off in 2/4 tries. Grok gives me one version before the painter has written anything. Maybe it's intended as "progression"?

I'll give average marks from the two attempts above (and the separate marks in parenthesis next to them).


ChatGPT	9/10 (8+10)
Gemini	6/10 (1+10)
Image Creator	4/10 (2+6)
Grok	5/10 (3+6)

Sloths with Headphones

Finally, I tried something fun for my videoconferencing background. First thing that came to my mind was:

sloths wearing headphones, photo-realistic

Gemini and Grok gave me what I wanted. Some playful, reasonably realistic sloths (plural) wearing headphones. ChatGPT gave me what looks like the passport photo sheet of a single sloth, not really natural or much realistic. Image Creator also had trouble with plural, half the attempts featured a single sloth. I did award a 10/10 last year, this is the second case I am revisiting to subtract 1, as I was too lenient and this year’s top two performers clearly improved upon last year's best:


ChatGPT	10/10
Gemini	6/10
Image Creator	7/10
Grok	10/10

Conclusion

Let's take a look at the cumulative scores:

Test	ChatGPT	Gemini	Image Creator	Grok
Running Camel	10	10	6	4
Camel Glasses	10	10	7	9
Astrophotos	8	2	1	3
Binoculars	10	10	1	2
Bino Chair	10	9	5	6
6x6 Matrix	9	4	1	1
PS Align Icon	5	3	2	2
Xasteria Icon	8	9	4	5
GCP / EPYC	10	9	7	4
Cloud Comparison	10	6	7	2
Drawing Phrase	9	6	4	5
Sloths	10	6	7	10
Total	109/120	84/120	52/120	53/120
%	91%	70%	43%	44%

This year's follow-up confirms that AI image generation tools have made noticeable progress - from sub-50% scores, we got to at least one solution (subjectively) scoring over 90%.

The top performer is ChatGPT, the new 4o model is the most dependable of all the solutions. It is the only one that can count, do technical drawings and the only one that has fully solved text rendering, while being the best at interpreting prompts. It went with the "less is more" approach, often giving the simplest, yet most appropriate image. It's the only one though that is not accessible in a free version. That may be just for now though, as it's quite new.
Gemini (Imagen 4) marked a clear improvement over last year’s Duet AI. It can mostly render text (not yet consistently) and can finally draw multiple binoculars without merging them into a monstrosity. It still has problems counting (with perhaps a small improvement over Duet AI) and misses fine prompt details, but it's available even on the free version. Plus, its integration into the Google Docs suite is a nice convenience.
Microsoft's Image Creator seems to use pretty much the same (DALL·E 3) engine like last year. It actually got lower marks, possibly due to luck - I only used the results of the first attempt, so some luck is involved. It's still good for creative results, but it's not accurate, can't do text, so it's rather limited for serious uses.
Grok (Grok 3) showed promise with photorealistic visuals and some artistic "flair", but was the most inconsistent, occasionally misinterpreting prompts, producing malformed compositions, or displaying contextual confusion.

Re-running the exact same 12 test rounds as last year highlighted that prompt interpretation, factual accuracy and visual clarity remain difficult to balance across models. Some were good in freeform design, others in precision, but none yet do both equally well, although ChatGPT got close. Still, the overall quality is clearly up from 2024.

So, for most professional use cases, especially when accuracy matters, ChatGPT 4o can provide great results, with Gemini being a decent alternative most of the time. Image Creator remains a decent free option for creative use, while Grok does show some interesting potential - it often goes a different way compared to the others.

How Geekbench 6 Multicore Is Broken by Design

Dimitrios Kechagias — Wed, 07 May 2025 00:39:29 +0000

As a developer, performance is very important to me (clearly, I am not a front-end dev - hah!). It's crucial for the company I work for, too, affecting both cost and user experience. I've been regularly performing and publishing cloud VM CPU comparisons to share my insights. Although my primary tool is my own DKbench suite, I tend to include Geekbench 5 in the comparison, mostly due to the abundance of available published results. This is despite Geekbench 6 being available for some time now. You might wonder why I haven't switched: Simply put, I found that Geekbench 6 multi-core is fundamentally "broken", and I thought I'd explain why.

Table of Contents:

Geekbench 6 Fails at Multi-Core Scaling
Geekbench 6's Shared Task Model
What Should a CPU Benchmark Measure?
Poor Implementation of Multi-threaded workloads
- The Text Processing benchmark
Conclusion

Geekbench 6 Fails at Multi-Core Scaling

Geekbench 6 barely scales on multi-core systems. This is unlike Geekbench 5, which, although not linear at scaling, continuously benefits from additional cores. To demonstrate, here is a comparison using Google Cloud C3D VMs with SMT disabled (vCPU = full core), showing the scaling behaviour of DKbench, Geekbench 5 and Geekbench 6 across 2 to 180 cores:

The dotted line is theoretical max / ideal scaling.

For clarity, here's the DKbench vs Geekbench 6 data in table form, along with Geekbench 6's efficiency versus ideal scaling:

Cores	DKbench Scaling	Geekbench 6 Scaling	Geekbench 6 % of Ideal
2	2.0	1.8	89.91%
4	4.0	3.2	79.92%
8	7.9	4.9	61.27%
16	15.2	7.9	49.54%
32	30.4	10.5	32.69%
48	45.5	11.4	23.66%
64	60.0	12.1	18.84%
90	82.6	12.1	13.46%
180	158.8	10.3	5.73%

Geekbench 6 scaling starts poorly and pretty much flattens at 32-64 cores or more, at which point you get a performance of around 10-12x that of a single core. Performance shockingly even declines beyond 32 cores, with 180 cores performing worse than 32!
For comparison, Geekbench 5 manages a 63x over single core performance on those 180 cores (and DKbench a cool 159x).

Geekbench 6's Shared Task Model

Let's dive into some technical details to figure out the reason behind this behaviour. Geekbench helpfully publishes some internal details, and there is a "Multi-Threading" section which explains:

Geekbench 6 uses a “shared task” model for multi-threading, rather than the “separate task” model used in earlier versions of Geekbench. The “shared task” approach better models how most applications use multiple cores.

Basically they say that in the previous versions of Geekbench, in Multi-core mode they would create more work to give to each core separately. In Geekbench 6 they have one task that they try to serve with multiple threads communicating with each other (IPC), which is indeed how, for example Photoshop would try to apply a filter on an image using more cores.

There are some fundamental issues with this approach:

Home usage: While the "shared task" approach may closely model a (specific) single app, typical user environments involve multiple apps at the same time, while the OS is running dozens of background tasks.
Server usage: The "shared task" idea is even less relevant for most cases of servers made for parallel tasks (e.g., processing multiple users/images/etc concurrently), not singular tasks processed faster.

What Should a CPU Benchmark Measure?

Let's go back to the basics for a moment. What exactly is a CPU benchmark for? I'd say a useful CPU benchmark typically does one of two things::

Application-Specific Benchmark: Tests the performance of specific software. Ideal when workloads are predictable. This type of benchmarking, in multi-core context will tell you whether you can expect performance gains for your app by simply adding cores.
Generic Benchmark: Measures general CPU capability by stressing all parts of the CPU in diverse workloads, offering insights into performance across various scenarios. In multi-core mode, these tests should similarly load all cores of the CPU, showing any the limitations of the processor design (lower "all core" boost speed, heat throttling etc).

There are some benchmarks that fall in-between. I.e. specific applications that are quite good at stressing a single or multiple cores of a CPU. The Cinebench benchmark is such an example.

Geekbench traditionally fit the generic benchmark category, providing useful rough comparisons, where the overall score usually agreed with what I would get from my custom benchmarks. Geekbench 6 breaks this in Multicore, for me it is no longer a generic benchmark:

Geekbench 6 Multicore simply measures the performance of Geekbench's particular implementation of very specific workloads.

Poor Implementation of Multi-threaded workloads

Even if we accept the premise of the "shared task" model, Geekbench 6 does a notably poor job implementing it, leading to decreasing performance when adding cores. From the internals document it seems that their approach to multi-core scaling often involves arbitrary and fixed scaling of workloads, typically setting multi-core tasks at exactly four times the single-core workload, regardless of CPU size. This approach explains the respectable 80% scaling observed up to 4 cores. Realistically, competent multi-threaded software would dynamically scale concurrent workloads to match available cores.

The Text Processing benchmark

It gets worse than this. I looked for the least scalable benchmark of the suite, which (surpisingly, as I was expecting some sort of non-parallelizable algorithm) is "Text Processing":

Cores	Text Processing Scaling
2	1.182
4	1.303
8	1.346
16	1.280
32	1.300
48	1.278
64	1.277
90	1.279
180	1.274

This is so bizarre. Their "text processing" representative benchmark scales to only about 1.35x single-core performance, peaking at 8 cores, then declining afterward.

It's even more bizarre if you read the internals doc:

The Text Processing workload loads numerous files, parses the contents using regular expressions, stores metadata in a SQLite database, and finally exports the content to a different format. It models typical text processing tasks that manipulate, analyze, and transform data to reformat it for publication and to gain insights. The input and output files are stored using an in-memory encrypted file system.
[...] and processes 190 Markdown files as its input.

So they describe it as processing 190 Markdown files (using regular expressions, storing metadata in SQLite, and exporting results) while taking virtually no advantage of parallel processing! The nearly flat scaling strongly suggests severe implementation bottlenecks e.g. perhaps due to something like poorly managed global write locks on SQLite or similar that serializes the whole process. There are no more details to figure out what they did wrong, but they did do it very wrong!

This benchmark literally implies CPUs with more than 4 cores provide no benefits for "text processing" tasks...

Conclusion

Geekbench 6’s multi-core benchmark is not merely flawed, it's fundamentally broken (and mostly by design). Its adoption of the "shared task" model and poor implementation techniques make it an ineffective representation of any real-world CPU performance. For more realistic and scalable benchmarks, I'd say stick to Geekbench 5, or try others - maybe even give DKbench a try (there's a docker version so you don't have to install anything).

C4D VMs on Google Cloud: Breaking Records Again with EPYC Turin

Dimitrios Kechagias — Wed, 16 Apr 2025 23:32:32 +0000

Over the past year, Google Cloud has been rolling out its fourth-gen compute VMs. First came the Intel-powered c4, which did much better than past Intel instances in my 2024 Cloud Comparison, but ultimately wasn't a significant leap compared to Google's own excellent c3d. The Google Axion-powered c4a ARM instances were released next, showing excellent multi-threaded performance and value.

Last week, during Google Next, the 5th Gen AMD EPYC (Zen 5) c4d instances were announced. You can follow the announcement link to apply for a preview, or wait a bit longer until general availability.
I've been testing them in preview for SpareRoom and they seem to be one of the most impressive releases in recent memory.

Table of contents:

The Comparison
Performance per 2x vCPU
- DKbench Suite
- FFmpeg Video Compression
- Linux Kernel Compilation
- OpenSSL (AVX 512)
Performance of Full Size Instances
- DKbench
- 7zip Compression
- Compilation
- OpenSSL
Conclusion
Addendum: Test methodology
- Benchmark::DKbench
- OpenBenchmarking.org (phoronix test suite)
- FFmpeg compression test

The Comparison

The new c4d comes in 3 variants, standard with 3.5GB RAM/vCPU, highcpu with 1.5GB/vCPU and highmem with 7.5GB/vCPU. vCPUs are SMT threads (2 threads per physical core), and you can specify from 2 vCPUs to a massive 384, which is 2 processors of 96 SMT cores (192 threads) each. As usual, it's a custom Zen 5 chip (EPYC 9B45) with a core clock of 4.1GHz.

I've set up a comparison picking the most relevant GCP VM types:

Type	CPU	vCPUs	RAM / vCPU
c4d	AMD EPYC Turin	2-384	1.5-7.5GB
c3d	AMD EPYC Genoa	4-360	2-8GB
t2d	AMD EPYC Milan	1-60	4GB
c4	Intel Emerald Rapids	2-192	2-7.5GB
c3	Intel Sapphire Rapids	4-176	2-8GB
n2	Intel Ice Lake	2-128	1-8GB
c4a	Google Axion	1-72	2-8GB

Performance per 2x vCPU

For an apples-to-apples comparison, I will use 2x vCPU instances. For c3/c3d Google's "visible cores" limiter is necessary, as 4x vCPU is the minimum. For most instances 2x vCPU means 1 core / 2 hyper-threads, except t2d and c4a which have no SMT/HyperThreading. That gives an advantage to those two types for multi-threaded loads, as they have twice the physical cores, but you don't pay more as the per vCPU price is not very different.
Note that I swapped the t2d with the n2d for EPYC Milan for this, as the latter can go up to 112 cores (224 threads) vs just 60 for t2d.

DKbench Suite

The DKbench CPU test suite is used to measure performance of the type of generic compute workloads we run on our servers. The test methodology is available as an addendum at the end of this post. The way to read the chart is that you are looking at green for max performance (single-thread load), and orange for value (multi-thread load on a 2x vCPU instance):

If you've seen my multi-cloud comparisons, you'll recognise that this is by far the fastest single core performance on a cloud server. It is over 40% higher than the fastest third gen (c3d). It's also almost 20% faster than the c4, a gap that widens in multi-threaded loads (as usual, Intel's HT is not as efficient).
The value of non-HT types is also evident, an Axion c4a core is 40% slower, however you pay per vCPU and you get a full core per each, which gives you more performance if your workload can fully utilize them. The aging t2d Milan is similar - much slower per core, but can still compete in value, which is why we use it for several applications to this day.

Moving on to some less generic benchmarks that use both vCPUs:

FFmpeg Video Compression

In the FFmpeg/lib264 compression test, c4d does at least as good with over 40% gains over c3d. Thanks to better SMT it even manages a 35% speed advantage over c4 in the multi-threaded comparison.

Linux Kernel Compilation

The Linux Kernel Compilation benchmark is not CPU only, the rest of the system, including the disk, is involved, but still we see close to 20% and 40% gains over c4 and c3d respectively.

OpenSSL (AVX 512)

The OpenSSL RSA4096 encryption test is AVX-512 heavy. Here is an idea of how c4d performs on such workloads:

We see 25% gains over c3d and 17% over c4. Not as impressive as the other benchmarks, but still significant.

While I consider the 2x vCPU tests the most telling, as they make it easy for me to extrapolate price & performance of our commonly used 4x-32x vCPU instances, there will be applications where a full-size instance will be used. So let’s see what a whole Turin processor (or two) can do.

Performance of Full Size Instances

As mentioned above, the c4d maxes out at 192 cores / 384 threads - a modest bump over the previous maximum of 180 cores / 360 threads on the c3d. However, paired with faster cores, I expected standout performance - and the c4d didn’t disappoint.

DKbench

The full-size c4d delivers over 30% more processing power compared to the already-impressive 360-thread c3d. Even more remarkable, a single 96-core (192-thread) EPYC Turin processor in c4d outperforms the Intel-powered c4's full 192-thread configuration by over 40%.

7zip Compression

At a compression speed of 1,732,434 MIPS it breaks the current openbenchmarking.org record of 1,658,291 which is held by a 2x EPYC 6845 system (320 cores / 640 threads!).

Compilation

At 18.3s it matches the current best times on openbenchmarking.org which come from other 2-processor AMD Zen 5 systems.

OpenSSL

Let's look again at this AVX 512-capable benchmark:

The c4d brings another 25% of performance over the c3d, similar to what we saw on the 2x vCPU test.

Conclusion

The c4d is the kind of new release we are always hoping for, essentially providing a one-click (or single-line code change) ~40% performance boost. Historically, moving to newer CPU generations on Google Cloud typically doesn’t come with increased cost, as maintaining older, less efficient hardware is generally more expensive for providers. Pricing isn’t confirmed yet, but traditionally GCP does not price AMD instances higher than Intel ones, so at significantly better performance than the c4, it should offer great value for most applications. Exceptions may include cases where you maintain a full load across all vCPUs and prioritize price-performance ratio - there, non-SMT instances like the Axion c4a (or even the older EPYC Milan t2d), could have the edge.

Addendum: Test methodology

You can find the full benchmark results listed here.

All instances were set up with a 10GB "Hyperdisk" or "SSD persistent disk" using Google's Debian 12 bookworm image in us-east-4.

Some system packages were installed to support the DKbench and phoronix test suites:

sudo su
apt-get update
apt install -y wget build-essential cpanminus libxml-simple-perl php-cli php-xml php-zip

Benchmark::DKbench

The DKbench suite recorded 5 iterations over 19 benchmarks. If you have followed my previous cloud performance comparisons, it is a suite based in perl and C/XS, which is meant to evaluate the performance of generic CPU workloads that the typical SpareRoom job or web server will run. It is very scalable, which is good for evaluating massive VMs.

To setup the benchmark with a standardized environment you would do:

cpanm -n BioPerl Benchmark::DKbench
setup_dkbench -f

To run (5 iterations):

dkbench -i 5

OpenBenchmarking.org (phoronix test suite)

To setup the phoronix test suite:

wget https://phoronix-test-suite.com/releases/phoronix-test-suite-10.8.4.tar.gz
tar xvfz phoronix-test-suite-10.8.4.tar.gz
cd phoronix-test-suite
./install-sh

To run the benchmarks I used:

phoronix-test-suite benchmark compress-7zip
phoronix-test-suite benchmark build-linux-kernel
phoronix-test-suite benchmark openssl

FFmpeg compression test

# For ARM instances - replace 'arm64' with 'amd64' for x86:

wget https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-arm64-static.tar.xz
tar -xJf ffmpeg-release-arm64-static.tar.xz --wildcards --no-anchored 'ffmpeg' -O > /usr/bin/ffmpeg
chmod +x /usr/bin/ffmpeg
wget https://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_720p_h264.mov
time ffmpeg -i big_buck_bunny_720p_h264.mov -c:v libx264 -threads 1 out264a.mp4
time ffmpeg -i big_buck_bunny_720p_h264.mov -c:v libx264 -threads 2 out264b.mp4

Benchmark CPUs Easily with the dkbench Docker image

Dimitrios Kechagias — Thu, 27 Mar 2025 02:21:48 +0000

I developed Benchmark::DKbench to use in my Cloud VM CPU comparisons. It's a great tool for general CPU benchmarking. In single-thread it correlates quite well with SPEC or Geekbench 5+, but it also scales efficiently to hundreds of cores on large cloud VMs and is open source. The only drawback was some setup required (a working Perl system and some basic libraries and CPAN modules).

To simplify the process and ensure a consistent benchmarking environment across different systems, I created a ready-to-use Docker image.

Quick Start

If you have Docker installed, you can be benchmarking in seconds:

docker run -it --rm dkechag/dkbench

It will download, run and connect to the container from where you simply run:

dkbench

That's it!

Docker Image Details

Docker Hub: hub.docker.com/r/dkechag/dkbench
GitHub Source: github.com/dkechag/dkbench-docker
Architectures: amd64, arm64

Sample Output

Output of base command on a c4a-highcpu-72 (Google Axion) VM:

--------------- Software ---------------
DKbench v3.00
Perl v5.36.0 (threads, multi)
OS: Debian GNU/Linux 12.10 (bookworm)
--------------- Hardware ---------------
CPU type:  (aarch64)
CPUs: 72 (72 Cores)
----------------------------------------
DKbench single-thread run:
Benchmark           Score               Pass/Fail
Astro:               1108               Pass
BioPerl Monomers:    1090               Pass
CSS::Inliner:        1135               Pass
Crypt::JWT:          1201               Pass
DBI/SQL:             1385               Pass
DateTime:            1416               Pass
Digest:               925               Pass
Encode:              1302               Pass
HTML::FormatText:    1121               Pass
Imager:              1344               Pass
JSON::XS:            1227               Pass
Math::DCT:           1211               Pass
Math::MatrixReal:     937               Pass
Moose:               1138               Pass
Moose prove:         1354               Pass
Primes:              1071               Pass
Regex/Subst:         1073               Pass
Regex/Subst utf8:    1164               Pass
Text::Levenshtein:   1282               Pass
Overall Score:       1183
----------------------------------------
DKbench 72-thread run:
Benchmark           Score               Pass/Fail
Astro:              79101               Pass
BioPerl Monomers:   77803               Pass
CSS::Inliner:       77056               Pass
Crypt::JWT:         86454               Pass
DBI/SQL:            100131              Pass
DateTime:           101748              Pass
Digest:             66419               Pass
Encode:             89550               Pass
HTML::FormatText:   75762               Pass
Imager:             95798               Pass
JSON::XS:           88109               Pass
Math::DCT:          86547               Pass
Math::MatrixReal:   67094               Pass
Moose:              81572               Pass
Moose prove:        76786               Pass
Primes:             63421               Pass
Regex/Subst:        76028               Pass
Regex/Subst utf8:   83210               Pass
Text::Levenshtein:  91956               Pass
Overall Score:      82344
----------------------------------------
Multi thread Scalability:
Benchmark               Multi perf xSingle      Multi scalability %
Astro:                  71.41                   99
BioPerl Monomers:       71.36                   99
CSS::Inliner:           67.88                   94
Crypt::JWT:             71.98                   100
DBI/SQL:                72.32                   100
DateTime:               71.87                   100
Digest:                 71.78                   100
Encode:                 68.76                   96
HTML::FormatText:       67.59                   94
Imager:                 71.27                   99
JSON::XS:               71.81                   100
Math::DCT:              71.48                   99
Math::MatrixReal:       71.58                   99
Moose:                  71.67                   100
Moose prove:            56.72                   79
Primes:                 59.23                   82
Regex/Subst:            70.88                   98
Regex/Subst utf8:       71.49                   99
Text::Levenshtein:      71.70                   100
----------------------------------------
DKbench summary (19 benchmarks, 72 threads):
Single:              1183
Multi:              82344
Multi/Single perf:  70.99x  (67.59 - 72.32)
Multi scalability:  98.6%   (94% - 100%)

Happy benchmarking!

Create a static mirror of your DEV blog

Dimitrios Kechagias — Mon, 24 Mar 2025 01:32:14 +0000

I started using DEV at the suggestion of Perl Weekly, and I was quite pleased with it - until I discovered that links to dev.to are effectively "shadowbanned" on several major platforms (Reddit, Hacker News, etc.). Posts containing DEV URLs would simply not be shown to users, making it impossible to share content effectively.

To work around this, I thought I would need a way to publish my DEV articles on my own domain so I could freely share them. There are some DEV tutorials out there that explain how to consume the API using frontend frameworks like React, however I don't enjoy frontend at all and I did not want to spend much time on that.

My solution was to get a simple Perl script that builds static versions of the articles, along with an index page. A Perl 5 script will run anywhere, including an old shared linux hosting account I still keep on IONOS, and I really like the speed of static sites.

I thought this is an ideal task to start with ChatGPT. Indeed, after 10-15 mins in a few prompts I had a sort-of working solution without having to open up the API documentation, nor writing any CSS. I then spent an hour or two fixing bugs, refactoring some of the ancient-looking code and tweaking / adding features (e.g. tag index pages) - tasks that I enjoy.

Here is the result. You can find the Perl script and assets in this repo.

It will run on pretty much any Perl 5 version (tested down to 5.10) with some basic CPAN modules (LWP::UserAgent, JSON, Path::Tiny).

To use it, check the project out from the repo and specify at least your DEV user name when calling the script:

./dev_to_static.pl -u <username>

# or to also specify a target directory and name your blog:
./dev_to_static.pl -u <username> -t <directory> -title="Blog name"

Try option -h to get more help.

You can run it on your web host directly, or run locally and copy the resulting directory to your host.

A nightly cron can update the site with new articles.

M2 vs M4 Mac Mini: Is it Worth the Upgrade?

Dimitrios Kechagias — Tue, 11 Mar 2025 13:58:04 +0000

Almost two years ago, I upgraded from an M1 Mac Mini to the M2 Mac Mini and compared it to my M1 Pro MacBook. The M2 was a modest 10-20% improvement over the M1, but it was far from the M1 Pro's multi-core performance - for demanding workloads, it wasn't a significant upgrade.

Now, Apple has skipped the M3 entirely and jumped straight to the M4 Mac Mini. On paper, it's already a more attractive package: the base model finally starts with 16GB RAM instead of 8GB (with no price increase), addressing a major criticism, and we are getting a CPU with faster clock, an extra 2 cores and faster RAM:

CPU	M2	M4
Transistors	20B	28B
Max CPU Clock	3.49 GHz	4.4 GHz
CPU Cores	4+4	4+6
GPU Cores	10	10
RAM Bandwidth	100 GB/s	120 GB/s

The CPU spec differences may not seem spectacular, but let's have a look at how they translate into real-world performance gains and whether they might make the M4 a worthy upgrade from an M1 or M2.

CPU Comparison

Synthetic CPU Benchmarks

I will start with a couple of generic CPU benchmarks, my own DKbench and the popular GeekBench 5 (note I don't use version 6 because IMHO it's broken for multi-core):

The M4 dominates more than the CPU spec comparison table would indicate, it looks like the M4 core has an improved IPC over the M2. The single core advantage at 30-35% is quite a bit more than the clock speed increase and the multi-core performance difference touched 50% in DKbench. However, under full load on the multi-core DKbench, the M4's fan gets noticeably (although not annoyingly) loud. But we'll get into that again later. For now, on to 3D rendering:

Here, the M4 is over 50% faster in single-core and over 60% faster in multi-core, an amazing showing.
The M4 also supports the GPU Cinebench benchmark where it scores 3971 - the M2 was not supported by this Cinebench release in order to compare.

Real-World Developer Workloads

On to real-world tasks that concern developers. I timed decompressing the Xcode 16.2 XIP file, running static analysis on Xcode for my my astronomy app and compiling it (4 times to get a comparable sum):

The most useful (to us devs, as we do it multiple times per day) task - compilation - is the most impressive one, with an over 50% performance gain. It should be clear by now, the performance difference is real and would be noticeable in daily work.

Video Compression

As the Mac Mini is ideal for a media center, video transcoding would be a common task. I am trying an FFmpeg H264 transcode (same method as here), plus a Handbrake SuperHQ HEVC transcode of the same video:

The single-core improvement is not bad at 23%, but again the multi-core performance at around 50% is very impressive.

CPU Result Table

Here are all the above results collected in table form:

Single-Core Benchmarks	M2	M4	M4 Advantage
DKbench 2.9	1430	1847	29.2%
Geekbench 5	1940	2623	35.2%
Cinebench 2024	113	174	54.0%
FFmpeg x264	306.8	248.4	23.5%
Average			35.5%

Multi-Core Benchmarks	M2	M4	M4 Advantage
DKbench 2.9	7437	11084	49.0%
Geekbench 5	9006	12989	44.2%
Cinebench 2024	587	963	64.1%
FFmpeg x264	78.6	51.2	53.5%
Decompress XIP	76.2	60.5	26.0%
Xcode Compile	51.9	34.3	51.3%
Xcode Analyze	50.3	35.4	42.1%
Handbrake HEVC	338	234	44.4%
Average			46.8%

GPU Comparison

I wouldn't say Macs are gaming machines, due to not having support for the majority of games, and no support for dedicated graphics cards for most models. However, the integrated GPU of the Apple Silicon processors is quite powerful, so I still wanted to see how much improvement there was:

The Unigine Heaven benchmark is quite old, but it does show there was a fair increase in performance (over 20%) - I had a similar result in Civilization VI which I did not include above. The gains in the Compute benchmark are a bit higher. In any case you would not really want to upgrade to the M4 just for the graphics, but it's still good to see an improvement.

SSD Speed

If you recall, going from the M1 to the M2 on the base (256GB) SSD configuration Apple reduced the number of NAND chips from 2 to 1 effectively halving the read speed, a decision that caused quite a stir. Let's see how the M4 base config does:

It's good news, we are back at good levels of performance, finally like M1 or better. Yes, basically matching the original M1 Mac Mini from 5 years ago is considered good news and I know how that sounds strange! They could at least have bumped up the minimum configuration, but I'll take the RAM increase any day!

Form factor, power & cooling

The old Mac Mini was small, the new one is tiny. A couple of USB-A are sacrificed, but we get 5 total USB-C/Thunderbolt, up from 2. It might be more future-proof, but it does mean you can't get away without hubs/adapters for those USB-A devices you still have. At least they kept ethernet (with a 10Gb option too), 3.5mm audio and HDMI (bumped up to v2.1 from 2.0) ports.

The much smaller size is matched with a hotter, more power consuming CPU:

	M2	M4
Idle Power (W)	7	4
Max Power (W)	50	65
Max Thermal Out (BTU/h)	171	222

This results in the need for powerful fan(s) to dissipate this heat. As noted above, when running DKbench on 10 cores, I could see my core temperatures quickly reach 100C, at which point the fan would engage at around 3000 rpm, quickly bringing the temp down to 70-80C. This took me by surprise as it is quite noticeable - I had never heard any Mac Mini (having both M1 & M2) make any perceivable sound before. At least it is an air/suction sound, not any annoying whirring etc. It is not a huge deal - if I ran DKbench at 6 threads it would not engage the fans and STILL be over 10% faster than the M2, but I have no idea why they chose to go this way. I am pretty sure no sane Mac Mini user would say "oh, it's too big - make it smaller, even if it's louder".

Conclusion

It's clear that the M4 Mac Mini is a substantial upgrade over the M2. The newer CPU alone offers significant performance improvements across the board, especially for heavy tasks - a developer or content creator will actually notice differences in performing daily tasks like compiling, rendering etc. Moreover, if we are looking specifically at the low cost base config models, twice the RAM and faster SSD with the M4 is a further advantage.

However, I can't help but notice a worrying trend. When the M1 launched, it was a game-changer, offering twice the performance of x86 chips at a fraction of the power consumption, with near-silent cooling. Since then, Apple has kept pushing performance higher—but at the cost of increasing power draw. The Mac Mini's power ceiling has jumped from 39W (M1) to 50W (M2) to 65W (M4). At the same time, the chassis has gotten smaller, making cooling more difficult.

This reminds me of the previous decade, when Intel was releasing power-hungry CPUs to boost performance, while Apple made laptops thinner and thinner. The result? 2019 MacBook Pros that throttled under load and had fans constantly running, especially in warm environments. We're obviously not there yet, but hearing the M4 Mini's fan spin up under heavy load makes me wonder if Apple is starting down a similar path.

For now, though, the M4 remains an excellent upgrade — but I hope future Apple Silicon devices continue to prioritize the efficiency and silent operation that made the M1 such a game-changer.

Google Axion: A New Leader in ARM Server Performance

Dimitrios Kechagias — Sun, 08 Dec 2024 16:06:16 +0000

Although our current cloud deployment at SpareRoom is x86, I’ve had the opportunity to test Google’s first in-house ARM server CPU "Axion" for a few months before its recent public release. Without giving too much away upfront - let's just say I was not left unimpressed. I’ll let the numbers, presented as charts, show how Axion compares in both performance and value with the best offerings in Google and Amazon clouds.

Table of Contents:

The Contenders
Test setup
Performance Results
- Benchmark suite results
- Compilation
- 7zip performance
- Video compression
- OpenSSL (AVX-512)
- Summary: performance delta vs Genoa & Graviton4
Performance / Price
- On Demand & Reserved
- Spot Instances
Conclusion

The Contenders

Here are the contenders for this comparison, the best/most relevant drawn from my recent Cloud VM Comparison test, with prices updated for the 3rd week of November 2024:

Instance Type	CPU type	HT / SMT	Price* $/Month	1Y Res.* $/Month	3Y Res.* $/Month	Spot* $/Month
Amazon C7a	AMD EPYC Genoa	-	77.36	51.97	35.39	26.20
Google c4a	Google Axion	-	57.69	38.89	27.29	24.50
Amazon C8g	AWS Graviton4	-	66.45	44.60	28.75	10.80
Google c4	Intel Emerald Rapids	Y	64.49	41.52	30.34	27.23
Google t2d	AMD EPYC Milan	-	64.68	41.86	30.76	11.77
Google c3d	AMD EPYC Genoa	Y	57.12	36.87	27.02	24.28

^{* Monthly price for 2x vCPU / 4GB RAM / 30GB disk instance, except t2d with 8GB RAM. For c3d, 4x vCPU is the minimum so price extrapolated to 2x vCPU.}

Amazon had the fastest ARM server VMs as we saw, featuring the Graviton4, and I am including the new compute-optimized type C8g. On the x86 front, I selected their C7a, featuring non-SMT AMD Genoa, which remains the fastest x86 VM in terms of per-vCPU performance.
From Google, I compared against three x86 types: the SMT/HT-enabled AMD Genoa (c3d) and Intel Emerald Rapids (c4), and the older AMD Milan (t2d). The t2d, while older, remains competitive in per-vCPU metrics due to its lack of SMT.

Test setup

I used the same methodology as my recent cloud comparison test, as it was detailed here with one addition: a real-world FFmpeg video compression benchmark.

Here’s how I set it up:

# For ARM instances - replace 'arm64' with 'amd64' for x86:

wget https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-arm64-static.tar.xz
tar -xJf ffmpeg-release-arm64-static.tar.xz --wildcards --no-anchored 'ffmpeg' -O > /usr/bin/ffmpeg
chmod +x /usr/bin/ffmpeg
wget https://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_720p_h264.mov
time ffmpeg -i big_buck_bunny_720p_h264.mov -c:v libx264 -threads 1 out264a.mp4
time ffmpeg -i big_buck_bunny_720p_h264.mov -c:v libx264 -threads 2 out264b.mp4

Performance Results

Benchmark suite results

DKbench is probably the most telling benchmark for the general workloads we use our servers with, and Geekbench 5 is added because of the wide availability of comparison results:

Immediately we see that Axion not only edges out Graviton4, it’s surprisingly close to the C7a, the fastest x86 instance overall.

DKbench is a good indicator of general performance, but we'll get on with some more specialised testing.

Compilation

For a developer-specific workload, I compiled Perl on two threads:

Axion came second only to Amazon's Genoa, and the difference was marginal.

7zip performance

This is the most impressive showing for Axion, leading in both compression and decompression.

Video compression

As mentioned above, I am transcoding using FFmpeg/lib264. lib264 is a very mature library and that should be well-optimized for Intel/AMD, so I was interested to see how well the new Google CPu could do:

And the answer is, not bad at all. It falls behind of Emerald Rapids & Genoa per single thread, but not by a significant margin. And given that for video compression we don't really care about single-threaded runs, only the C7a is actually faster per vCPU.

OpenSSL (AVX-512)

Moving on to an even more heavily-optimized for Intel/AMD CPUs benchmark, OpenSSL can use the latest AVX-512 instructions for increased performance. This is basically the worst case usage scenario for ARM as the architecture has much more limited SIMD extensions (NEON):

Here, Axion improves over Graviton4 as in all tests, but cannot keep up even with the older x86.

Summary: performance delta vs Genoa & Graviton4

Let's have a look at the performance delta of the Axion vs Genoa (in purple) and Graviton4 (in yellow) for all the benchmarks we ran (skipping the special case of OpenSSL):

There are consistent gains over the Graviton4 (from 3 to 15%). On the other hand, Genoa maintains the lead for most tests, with the maximum difference at 15%, but Axion keeps much closer than that in general and even bests Genoa in some cases. I would say that, for most uses, Axion will be closer to Genoa than it is to Graviton4.

Performance / Price

The main reason Amazon & Google developed their own ARM solutions is to provide the best possible value (for themselves and their customers). Hence, a look at performance / price is possibly even more useful than raw performance. I will be looking at multicore performance with DKbench, as with it's varied benchmarks gave reasonably balanced performance results.

On Demand & Reserved

Looks like it's mission accomplished for Google. Axion is by far the best value amongst the tested VMs, both for On demand and 1y/3y reserved pricing.

Spot Instances

Spot prices vary wildly, both with time and location. Based on Eastern-US pricing on the 3rd week of November when I was compiling the results, this is what we see:

I don't know if Amazon is doing this on purpose, but they are giving Graviton4 at an unbeatable spot price for US-east, where Axion has availability, when it is priced almost 2x in US-west, where Graviton4 at an unbeatable spot price for US-east is not yet available! In any case, for good value on Axion spot instances you'll have to wait for wider availability, right now on Google you'd have to go with the Milan Tau instances if you wanted the best value.

This chart is mostly to make you research spot prices, as there are always great deals to be found, especially if you are not limited to a specific region. The deals change often, so try to keep track of pricing.

Conclusion

Google’s Axion CPU proves to be an exceptional contender in the ARM server space, offering stellar performance and value. Expect an almost 10% performance improvement over Graviton4. In addition, while it trails behind x86 CPUs in some specialized workloads (e.g. AVX-512), it is not far behind the best x86 CPUs in the majority of tasks, posing as a viable alternative for those seeking to switch to ARM but keep top-tier performance levels.