At Superb Internet Corp., not only are we one of the world’s oldest web hosts (in business since 1996), but we are also the world’s most experienced virtualized hosting expert, as we were the very first to offer Virtual Private Server (VPS) – the predecessor to today’s cloud - commercial hosting in 1999. Being a recognized and well known innovator and leader in the industry - always staying true to our motto, “Ahead of the Rest”® - meant we set a very high standard to meet when it came to cloud hosting: equal or better performance and functionality than dedicated, practically limitless real-time scalability, coupled with true 100% uptime.
After nearly 10 years of development, our Superb Cloud platform was released in 2014. (Yes, it took much longer than we’d have liked. But, we were determined to provide a true 100% uptime, better-than-dedicated-in-every-way cloud, vs. simply mislabeled non-HA (not high availability) VPS service as cloud or otherwise provide something that is not a true 100% uptime and unrestricted functionality, equal performance to dedicated hardware, cloud.)
This report provides a top level overview of some of the easier to portray performance tests and explains the results of the extensive, ongoing comparative performance testing that we have conducted of our and our competitors’ cloud VMs. In order to ensure that we always remain “Ahead of the Rest”® when it comes to cloud performance and value, and provide our customers with an “apples to apples” comparison, we are using a number of accounts with competing Cloud Hosting Providers (CHPs) as well as a regular CloudServer-312 account on the Superb Cloud, on which we run, either hourly or continuously, a series of standardized and well recognized performance tests. All accounts – with us and competing HSPs – are standard auto-provisioned customer accounts, the very same as anyone would purchase online.
In this report we present illustrative data compiled over a 90 day period in late 2015. We leave the decision and ultimate interpretation of the results and what it means to you – our valued and respected customer or prospective customer.
Cloud Hosting Comparison Basis
The table below illustrates the accounts that were used in the testing. We have matched up comparable features as much as possible; however, as you can see, there are some differences in the configuration – and even more so in price, due to the lack of standardization in cloud hosting services.
||Amazon Web Services
|CPU Performance (Passmark)
|Number of Cores
|SSD disk space
||$70 per month all included
||$178 per month with no Internet connectivity
$1068 per month with comparable 10TB/m
|$100 per month with no Internet connectivity
$550 per month with comparable 10TB/m
|$68 per month with no Internet connectivity
$988 per month with comparable 10TB/m
Note: Pricing as of February 24, 2016
Our goal: Real world tests – real world results. Plain and simple.
Our testing team have used universally accepted and acknowledged open source or third party performance testing applications and tools. We then briefly explain what each test measures, and what it means, using real life examples.
We installed a standard CentOS 6.5 64-bit image and then started running our tests. Our tests have been run continuously on an hourly basis, creating a log of the results of each tested cloud VM.
Our goal with these tests is to provide you with information to allow you to make your own informed decisions based on the results and your unique needs of a cloud hosting. If you would like the specific tests results or to discuss this more in depth with us, we’ll be happy to give provide it to you and to talk to you further. You are welcome to contact us anytime.
If you remember those three and a half inch floppy disks you used as coasters for your morning coffee, or how well they flew across the room when you threw them just right – if you remember those, then you’ll know all about Disk IO bottlenecks.
Disk I/O encompasses the input/output operations on a physical disk. If you’re reading data from a file on a disk, the processor needs to wait for the file to be read (the same goes for writing).
The killer when working with a disk? Access time. This is the time required for a computer to process a data request from the processor and then retrieve the required data from the storage device. Since hard disks are mechanical, you need to wait for the disk to rotate to the required disk sector.
At Superb Internet we take Disk I/O seriously. We could have taken the easy way out and built SAN servers on a 10GigE network, but why use centralized storage, which is a relic of the mainframe era of several decades ago? (That doesn’t prevent most other HSPs from using obsolete technology, however. We think that it’s a bit funny how others’ “cloud” is mainframe-like, with all client data stored on a central storage array, which is also a single point of failure in any cloud; only the processing is distributed.) We have always lead the industry as an innovator; we like the latest (but stable) technology and speed, so we built our systems with Local Hypervisors using modern Distributed Storage technology and with underlying InfiniBand® 40Gb/s packet-loss and jitter free networking (typically only used in extremely high performance supercomputing applications, until now almost never used in consumer or business level computing).
We could have built our Hypervisors with centralized storage (SAN) like the others, thus saving considerably on our cloud infrastructure costs, but the red line in the graphs below would have been lower. The result of using modern block-level Distributed Storage technology, which keeps multiple copies of all data, is local disk I/O level performance (same as you would get on a top-of-the-line dedicated server) without any bottlenecks or any single point of failure. Our investment in modern Distributed Storage technology and InfiniBand® clearly provides the best performance, hands down.
Here’s the test command we ran:
$ dd if=/dev/zero of=sb-io-test bs=1M count=1k conv=fdatasync
The Results Speak For Themselves
Fig. 1 - Test results over a week
Fig. 2 - Test results over a day
If you need high-end dedicated server level disk I/O performance (for example, you are running/using a database, a mail server, a web server, an application server or otherwise have any other use which frequently reads or writes data, as opposed to stores it in and retrieves it from RAM), your choice is a dedicated server (foregoing the High Availability/100% uptime and next to infinite, instant scalability and numerous self-service advantages of the cloud) or a cloud account with us; other CHPs fall many folds below the par when it comes to disk I/O.
Why the performance variance on the red line? Our block-level Distributed Storage technology will use local (on the currently active hypervisor) data blocks whenever possible – that is the top level that you see; the bottom level in the red line is when the redundant data blocks on another hypervisor, separate from the active one currently housing the computing of the VM are used. Notice the slight (less than 30%) variance in performance local disk vs. remote disk? That is the InfiniBand® ultra high performance networking technology at work – which, among other things, is completely packet loss and jitter free and has many 10s of fold lower latency than even the lowest theoretical minimums of 10 Gigabit Ethernet. (With others using conventional SANs, the performance loss is 60%, 80% or more, as you can clearly see in the graphs above. Even at our lowest performance, we more than two fold outperform others.)
BYTE UNIX Benchmarks (Version 5.1.3)
UnixBench was created in 1983 and to this day is still being used as a core benchmarking methodology by many. We don’t need to worry about all those fancy programming languages used today, we only want to make sure that the performance is there to enable all those fancy programming languages our customers use to be at their best and deliver the fastest results; that is where the standardized UnixBench tests come in. We like UnixBench as a benchmarking test, not because it gives us the best scores – which it does, but because it’s the same test your Grandfather would have used to test his 386 machine back in the day – reliable, tried, tested and true. Simply put, the UnixBench test lets you how much “horsepower” your cloud VM has: a well-recognized standard performance indicator.
Fig. 3 - 1 Parallel Process Test Results over a week
Fig. 4 - 1 Parallel Process Test Results over a day
Fig. 5 - 2 Parallel Process Test Results over a day
Fig. 6 - 2 Parallel Process Test Results over a week
(So there’s no confusion, all tests were run with the default options.)
Once again, the results speak for themselves. The UnixBench tests show that our cloud accounts have considerably more dedicated and unshared “horsepower” allocated and available, at all times, than others’ comparable “specs on paper” cloud accounts. Put in other words, to get performance equivalent to our CS-312 2-core cloud account, you would need to buy three to four of AWS t2.medium 2-core accounts; that is, 6-8 cores at AWS are equivalent to 2 cores with us, and to get comparable performance and inclusions of our $70/month account, you would need to pay $2964 – $3952/month with AWS (about 42 to 56 times more).
Lesson learned? Not all cloud accounts are created equal; in fact, the performance difference can be staggering. The headline specs are of little meaning. It is the actual performance (which is never disclosed by the vast majority of CHPs) that must be measured and compared side-by-side, in order to determine an accurate “like for like” performance comparison cloud account sizing.
Let’s talk IOPS.
Here at Superb Internet we know that we can’t always win at everything (though we never stop trying), but what we can do is make sure that all of our customers always get all the resources that they pay for. We don’t oversell, we don’t put all of your eggs (data) in one basket (on one central filer), we make sure your services (E-mail, web sites, databases, applications, or whatever you run) are served fast and efficiently.
If you play any online multiplayer shooter or action games, you know about ping times and you know how inconsistent ping times (jitter) create ‘lag kills’ (at least that’s my excuse for having to restart over and over and over). IOPS is to disk latency what network latency is to ping times. Like a good game of “Call of Duty”, good IOPS is about consistency and not a roller coaster.
Fig. 7 - IOPS Test Results over a day
Fig. 8 - IOPS Test Results over a week
While our IOPS performance is not the highest, it is consistently consistent (yes, “double consistent”) – so you always know what you are going to get; there are no surprises. With AWS and Softlayer/IBM, the IOPS varies greatly, thus giving you highly variable and unpredictable performance, while with Microsoft/Azure it is consistently low, with only a few occasional higher performance spikes.
In short, this means that while you can easily architect a cloud hosting solution, comprised of one or multiple VMs, with a consistent and easily forecastable performance (thus delivering a consistent and predictable user experience), using other CHPs cloud it is practically impossible to design for a consistent and predictable user experience, given the wild variations in the disk IOPS, due to the inherently oversold platform where your “neighbors” use of their cloud VMs can greatly impact your performance and thus your user/customer experience.
At Superb Internet we love what we do, we do it better, and we’ve been doing it longer than anyone else. Which is why when we say “we get it” – we really get it. After all, we have years more experience with virtualized/cloud hosting than any other CHP. Our “Ahead of the Rest”® standard has always remained unwavering: we never lower the bar and hold ourselves to a consistently high standard, as we believe in either providing the very best to our customers, or in not providing any service at all – there are no compromises.
Instead of going along with the others’ highly obfuscated and greatly oversold ways, we strongly believe in the Transparent Cloud™: in providing a consistent and always predictable performance, non-oversold cloud to our clients. That not only makes it much easier for our clients to architect and design their cloud hosting infrastructure, it also delivers a consistently predictable user/customer experience – a key requirement for most commercial cloud deployments.
The examples above have illustrated that, depending on what performance metric you use (each client application is unique and thus the most important performance value will differ from client to client), to get performance equivalent to our comparable specs cloud account, you may need to spend as much as 56 times (!) more at AWS. It is not only the various hidden fees at other CHPs that may get you (and end up unexpectedly costing you considerably more than budgeted), but so will the other CHPs unpredictable and highly variable cloud performance.
If you take away nothing else from this major CHPs comparative study, just remember this:
- Not all clouds are created equal;
- The published specs are practically meaningless; therefore, you cannot compare CHPs based on the published specs;
- Always – always – run tests for your specific applications/use when comparing multiple CHPs, in order to arrive at the quantity and/or size of accounts to compare at different CHPs which provide for equal performance (as opposed to equal or roughly similar “headline specs”).
We live, work, and play in the real world and we’ve made it simple for you. Unlike others’ obfuscated cloud (where you have no visibility inside it), our cloud is the Transparent Cloud™: there are no surprises - you always know what you will get, as we strongly believe in full disclosure (transparency) and are committed to no overselling – so with each cloud account you get the same as at least two like configuration dedicated servers set-up in an HA (High Availability) cluster with two redundant load balancers (all that you get with each and every cloud VM, where just like with a dedicated server pair, your performance is guaranteed, dedicated and consistent).
Simply put, with us you get the latest modern Distributed Storage and InfiniBand® ultra-high performance networking technology, non-oversold and guaranteed fastest computing power, and “no surprises, no hidden fees” simple, budget-friendly pricing, saving you potentially over 50 fold (yes, over fifty times) over other competing CHPs for similar performance cloud. In short, with our specs you get several fold higher performance, along with several fold lower price, than with seemingly alike cloud accounts at other CHPs.
The reason for this is simple: we are in this for the long term. Most of our customers have been with us for many years, with a high number of customers who’ve been with us for over a decade (including an impressive group of clients who’ve been with us, by now, for nearly two decades – where we’ve witnessed their exponential growth and their highs and lows, just as they have witnessed and experienced the same with us). We strongly believe in “once a Superb customer, a Superb customer for life.” We base our service specifications, pricing, service quality and delivery, support, and all else on building a life-long business relationship with each and every client. Transparency and consistency is at the core of it. Our goal is not to manipulate our service in a way as to gain the most customers, but to design, build and deliver it in a way as to retain and help each and every customer grow and leave them wanting and needing no more. When you are our customer, you are not just a customer: you are our partner, and we are your partner in your business success.