Cloud architectures can be rated based on 4 Dimensions of Scale. These dimensions represent the architectural flexibility that has been designed into a software system based on the needs and/or constraints of the business. Knowing the dimension of scale for your software can be useful both to understand its current limitations, and to identify areas of potential future investment.
1st Dimension: Scale Up (Minimal Scalability)
The first dimension of scale is the simplest: scale up one or more resources that constrain a software application. Resources can be physical or virtual, and may include items such as CPU, memory, disk, or I/O. Scaling up is often the easiest solution to a scalability challenge, and is frequently the go-to option for many operations teams. If the JVM is throwing an OOM at high search volumes, increase memory; if the MySql database is maxing out CPU, add more virtual cores.
A software architecture based on the 1st dimension of scale will eventually hit a wall when growth exceeds the capacity to upgrade. For example, this blog is on an Amazon t1.micro running both Wordpress and MySql. Guess what dimension this is running at?
2nd Dimension: Scale Out (Limited Scalability)
Cluster-based architectures are the most common way to scale out. These solutions can be either stateless (e.g. Apache web servers) or stateful (e.g. Swift object store), with the latter adding substantial complexity to the software implementation and operations. Over the last decade, scale out architectures have become pervasive, and many popular open source projects support the 2nd dimension of scale (e.g. Elasticsearch, OpenStack Swift, Cassandra).
While architectures that scale out are often highly flexible (e.g. just add another node in the cluster), they are unfortunately limited by the constraints of a single cluster. These constraints vary by software product, but often are based on proximity (e.g. the need to maintain low network latency between nodes in the cluster), or cluster size (the inability of a cluster to efficiently manage state synchronization above a certain node count). For example, OpenStack Swift runs efficiently within a single data center, but substantially degrades when you attempt to span a single cluster across multiple geographies.
3rd Dimension: Multi-Cluster (Highly Scalable Within Single Pod)
The third dimension of scale is the ability of a software system to seamlessly (or near seamlessly) support multiple clusters. The decision to direct traffic to a specific cluster is typically handled by client calling convention, or by a network / application-level load balancer. Well architected architectures leveraging the 3rd dimension of scale often operate as a grid, and are capable of infinite scalability without substantial additional operational overhead. Unfortunately few open source and commercial products today support a 3rd dimension of scale, so this scalability is often achieved by implementing a proprietary software layer on top of a clustered solution.
4th Dimension: Pods (Infinite Scalability)
The 4th dimension of scale is the ability to replicate the 3rd dimension for an entire software stack while maintaining centralized administration and control. The replication of the architecture is typically called a pod, and the ability to support a pod-like architecture is useful to businesses that require physical or logical separation in different deployments of their architecture (e.g. by line of business, channel partner, or service offer). While most software architectures do not need to achieve this level of scalability, occasionally this level of scale is required to support a business need (e.g. Facebook, Google).
Last Word
Most complex software architectures will consist of multiple functional components, each of which interoperate to meet the requirements of the system. Each component of a software architecture can be rated independently for its dimension of scale, with the aggregate scalability being the lowest dimension for a component servicing essential business functions.
There is no right answer to software scalability. The dimension of scale your system supports will be driven by the needs and constraints of your business.
One of my favorite scaling lines from the 2007 Amazon Dynamo paper reads as follows:
“...customers should be able to view and add items to their shopping cart even if disks are failing, network routes are flapping, or data centers are being destroyed by tornados.”
To which I always think: all that for a shopping cart? ;)
Related Posts: Defining a Cloud Reference Architecture