Cloud Raxak Report on Security Compliance on Amazon, Google, IBM Clouds

Executive Summary

The vast majority of U.S. organizations are not prepared to properly respond to a cyber-attack, according to a new study by the Ponemon Institute.  The Cyber Resilient Organization report surveyed 600 IT and security executives, and found that 75% of them do not believe their company is cyber-resilient and only 32% feel they can properly recover from a cyber-attack.

With this level of uncertainty around security strategy and interest in moving applications to the public cloud, Cloud Raxak developed the Public Cloud Security Compliance report.  The report looked at the security readiness of the three major cloud platforms using the NIST/DISA STIGs for security compliance.

Since 1998, Defense Information Systems Agency (DISA) has enhanced the Department of Defense’s security systems by providing Security Technical Implementation Guides (STIGs). The STIGs contain technical guidance for locking down information systems and software that might otherwise have vulnerabilities to malicious attacks. These guidelines are an excellent starting point for ensuring security compliance of your infrastructure and specify the desired security state of your computer assets.

Below we have summarized the key findings in the Cloud Raxak Security Compliance Report for Amazon EC2, Google Cloud Engine, and IBM’s SoftLayer

Cloud Raxak Security Compliance Report for Q3 2015

Testing Parameters

Cloud Raxak ran a series of  security compliance tests using standard builds of Linux distributions available on multiple cloud service providers, including Amazon, Google and IBM.  Below is a summary of the key testing parameters.

  • Created new VMs using the standard distributions for Amazon EC2, Google Cloud Engine, and IBM’s SoftLayer.
  • Raxak Protect v1.95 SaaS engine was used to check the security compliance of each VM.
  • The DISA STIG Mission Critical Classified profile was used as the security compliance standard.


Security Compliance Settings

Approximately 50% of the settings from the commercially available builds on Amazon, Google and SoftLayer, do not follow the DISA recommendations. This varies by Linux version as shown below (averaged across cloud providers where appropriate.)

Success Failed Manual













36.73% 44.44%


Table 1: Average compliance results by OS


Severity of Security Settings

On the positive side, we found that most of the high-severity settings were correct out-of-the-box. The bulk of the variation was due to the medium and low severity settings—but these are significant in that they leave open a potential vulnerability in an otherwise controlled environment.

Severity Success Failed Manual
High 77.21% 13.97% 8.82%
Medium 51.08% 35.16% 13.76%
Low 31.44% 53.66% 14.90%
Table 2: Average compliance results by severity 


New VM compared to VM in Use for 6 months

It is instructive to compare how these clean new builds compare with similar checks made on VMs that have been actively in use for a while. Variations result from well-meaning users changing settings in order to get work done effectively. Other legitimate changes happen as standard packages and operating system updates get applied. Finally users may accidentally create files with incorrect or overly permissive states (for example, world writable files). Such variation is called “drift” and needs to be actively managed over the lifecycle of the compute assets. The tables below contrast the results for a customer’s development machine running Ubuntu 14.010 that had been in use for over 6 months with a clean new Ubuntu 14.10 machine on Google Compute Engine (GCE).

Severity Passed Failed Manual Total
High 14 2 1 17
Medium 52 65 22 139
Low 25 60 14 99
Totals 91 (36%) 127 (50%) 37 (15%) 255 (100%)
Table 3: 6 month old customer development server with Ubuntu 14.04
Severity Passed Failed Manual Total
High 12 2 3 17
Medium 58 52 29 139
Low 29 49 21 99
Totals 99(39%) 103(40%) 53(21%) 255(100%)
Table 4: New virtual machine on GCE with Ubuntu 14.04 


Our analysis of virtual machines on commercial public cloud infrastructures from Amazon, Google and IBM SoftLayer, showed that none of the machines we tested conformed to the lock-down recommendations published by DISA (or the similar rules in NIST SCAP). These recommendations form a superset of the requirements for Healthcare (HIPAA), Finance (FFIEC), Retail (PCI) and other regulatory agencies.  

Further, we have found that settings on compute assets drift over time as users modify settings for a variety of (mostly genuine) reasons. This shows the value of continual checking of configurations against desired states and automated mechanisms for bringing systems back into compliance when they are found to deviate. 

Given security compliance is 40% of the cost of managing cloud applications, determining how to automate cloud security compliance will be critical to maintaining positive ROI.  What’s the solution?

Raxak Protect, is a SaaS based automated security compliance solution.  Raxak Protect provides out-of-the-box industry standard profiles based on NIST and DISA STIGS or any custom security profile you chose for your industry or unique application workload.  It not only automates the application and management of your security profiles, but it keeps up with the rapidly changing security compliance landscape so users don’t have to.

For more information, download the Cloud Raxak Protect Security Solution document. 

For more information on the Ponemon Institute Cyber Resilient Organization report. 



Back to Top