The underlying hardware of cloud computing is changing

There is a fundamental shift happening in the underlying hardware that powers the cloud. Cloud providers like AWS have increasingly allowed software teams to think less about hardware, particularly with serverless offerings like AWS Lambda, that take the responsibility of managing operating systems and servers out of the hands of developers. But no matter how useful these paradigms can be, the bottom line price, and performance of these services is limited by the hardware that run them. Costs associated with space, heat and power draw are ultimately what gets factored into the prices that AWS charges their customers.

The AWS graviton2 instances are reported to deliver 40% better price performance than traditional EC2 instances. There are no changes to the way AWS customers like us interact with graviton2 instances at a surface level: no clever scaling, or better sharing of resources with other customers. What has changed is the underlying processors that AWS is using, a processor that has been designed in-house, rather than the chips manufactured in the past by Intel

A background in processor architecture

The central processing unit (CPU) is the heart of a computer, where data is crunched and decisions are made. The software we write, regardless of whether it is performing spatial analysis, or presenting an interface to a user, gets broken down into instructions that can be executed by the CPU. These instructions may involve interaction with other hardware (like reading data from a hard disk/solid state drive) or perform direct processing (like performing mathematical operations on data).

The architecture of a processor controls how these instructions are performed by the processor. While there are many different CPUs on the market, there are only a few common architectures, in order to allow for greater interoperability of CPUs with software, and other hardware. The architecture has a significant impact on the price and performance of the resulting CPU, playing a big role in how many operations are required to perform a particular task, as well as the speed of those instructions.

The graviton2 CPUs designed by Annapurna Labs (a subsidiary of Amazon), utilise an ARM architecture, rather than the x86 architecture that has dominated the desktop and cloud CPU market in the past. Of course the ARM architecture is nothing new to the world of computing, having its roots in the Berkeley RISC project of the 1980s, as well as being the architecture of choice for smartphones, routers, and embedded devices.

Late last year, Apple made a similar shift to both in-house CPU design and the ARM architecture, with the release of the M1 processor. By all reports, the move to M1 has been a huge success, with their M1 range of computers delivering better performance, as well as reducing Apple’s dependence on Intel for innovation in processor design.

What’s the catch?

The shift to ARM processors is not all smooth sailing for developers. It’s true that ARM devices, including the graviton2 processors, receive good support from the Linux operating systems that we rely on. But as we go further up the “stack”, the recent dominance of x86 processors for desktop and server computing means that ARM platforms have not enjoyed the same support from the maintainers of software libraries.

Libraries written in low-level languages like C and C++ (of which there are many) need to be compiled down to machine code that can be run by a CPU. Generally, library maintainers distribute pre-compiled binaries to save developers the lengthy process of compiling the code themselves. Unfortunately, binaries compiled for one architecture, cannot be run on other architectures. Given that the demand for ARM binaries for certain libraries has previously been low, they are not always available or up-to-date. Given this, deploying software on graviton2 instances may require waiting for binaries to be made available, or manually compiling code for ARM.

Container images, which are widely used to package and isolate software, similarly need to be rebuilt for ARM, if a pre-built ARM image does not exist

In the most extreme cases, library developers may have used optimisations specifically designed to run on x86 processors, and in these cases, more significant work will be required on the end of the library maintainers to make the code “portable” to ARM systems.

What this all means

You may not share our interest in CPU architectures, multi-threading, and instruction sets; but your AWS bill does. That’s why Spatial Partners will be evaluating the use of graviton2 instances in suitable future projects, to ensure we can deliver the maximum performance at the lowest cost.

For many applications, the move will be rather seamless, especially when working in a high-level language like Python, and where the software doesn’t rely heavily on external system libraries. For more complex solutions with many dependencies, the trade-off between ease of deployment and price performance will need to be carefully considered

Tom Clancy is a Software Engineer at Spatial Partners

At Spatial Partners we discover and deliver full solutions for any complicated data challenge. We are your problem-solving partner, and we thrive on complexity.