Cloud Firewalls Tested: Are Expensive Solutions Failing You?

The latest independent tests by CyberRatings.org have sent shockwaves through the cybersecurity community, revealing that some of the priciest, cloud-based firewalls may not deliver the robust protection many organizations expect. In a comprehensive evaluation spanning more than 2,000 vulnerabilities and 2,500 attack simulations across 27 evasion techniques, the results indicate that native firewalls from major cloud providers can be outclassed by third-party alternatives.

Cloud-Native vs. Third-Party: The Testing Breakdown​

CyberRatings.org—a nonprofit dedicated to unbiased security testing—put various cloud network firewalls through their paces. Notably, the tests evaluated all layers of network defense by simulating real-world exploitations and evasions. The verdict? While vendors like Fortinet and Check Point achieved perfect scores, the cloud-native firewalls from Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP) stumbled significantly.
Key findings include:

• AWS’s firewall blocked a meager 0.59% of exploits, with its signature mismatch issues crippling its defense capabilities.
• Both AWS and GCP ended up with an overall security effectiveness score of 0%, largely because their defenses were bypassed easily using various evasion methods.
• Microsoft Azure’s firewall fared slightly better in blocking evasion attacks (78% success in one area) but ultimately was penalized due to a lack of HTTPS decryption. Without proper TLS/SSL support, Microsoft’s solution also received an overall score of 0%.
• Cisco’s cloud security offering, despite being the most expensive per megabit per second, only managed a 53.5% overall effectiveness score.

This stark disparity underlines a critical point for decision makers: expensive doesn’t always equate to effective, and relying solely on cloud-native security might leave organizations exposed.
Summary of Cloud-Native Testing:

• AWS: 0% overall (0.59% exploit blocking)
• GCP: 0% overall (despite decent exploit blocking rates, evasion techniques easily bypassed)
• Microsoft Azure: 0% overall due to missing HTTPS decryption needed for encrypted traffic
• Cisco: Moderate performance (53.5% overall), not matching the high expected premium

The Evasion Factor: When Encrypted Traffic Becomes a Blindspot​

One of the more significant technical issues surfaced in the testing was the handling—or mishandling—of encrypted traffic. With the growing reliance on HTTPS, having robust decryption capabilities built into firewall defenses is no longer optional; it is essential.

• Microsoft’s firewall, for instance, was flagged for its glaring omission of HTTPS decryption. Without the ability to inspect encrypted data, Microsoft became virtually blind to a swath of malicious activities.
• Similarly, AWS and GCP failed to stop evasion attempts because the tests showed that their native firewalls could not handle complex evasion scenarios often encountered in modern cyberattacks.

These vulnerabilities are not mere technical oversights. They represent real-world risks for organizations already grappling with a plethora of cybersecurity threats. Even if exploit blocking rates seem acceptable on paper, the inability to inspect encrypted traffic leaves a gaping hole in security posture.
Key Evasion Insights:

• Native cloud firewalls struggled with evasion techniques across multiple layers.
• Lack of integrated HTTPS decryption means encrypted exploits go undetected.
• Third-party firewalls, which often include advanced decryption and threat intelligence, offer significantly enhanced protections.

The Third-Party Advantage: Why Independent Security Services Matter​

CyberRatings.org’s tests lend strong support to the argument for integrating third-party security services. Leading vendors such as Fortinet, Check Point, Versa Networks, Palo Alto Networks, and Juniper Networks not only scoured nearly every potential attack vector but also demonstrated near-perfect exploit and evasion prevention.
Benefits of Third-Party Cloud Security:

• Comprehensive exploit and evasion protection tested against thousands of vulnerabilities and attack techniques.
• Enhanced inspection of encrypted traffic via built-in HTTPS decryption.
• Continuous improvements based on independent testing and real-world threat simulations.
• Up-to-date threat intelligence that adapts quickly to emerging cyber threats.

Vikram Phatak, CEO of CyberRatings.org, bluntly stated that third-party firewalls outclass the offerings from even major cloud providers. The research suggests that organizations relying on native solutions not only risk suboptimal protection but may also experience stagnation in performance improvements—as seen with AWS’s stagnant performance since 2014.
When it comes to safeguarding sensitive systems, especially those running critical Windows workloads or facilitating enterprise-level operations, the implications for IT administrators are staggering.

Implications for IT Departments and Windows Administrators​

For many Windows administrators, the focus has traditionally been on Windows 11 updates and Microsoft security patches to safeguard endpoints and network infrastructure. However, the findings from CyberRatings.org’s tests call for a broader perspective—one where network security incorporates vigilant cloud security controls.
Consider these factors:

• Integration: Modern IT environments are hybrid, combining on-premises Windows servers with cloud-based services. An effective defense strategy must cover every vector.
• Comprehensive Security: Relying solely on native cloud firewalls may leave blind spots. Integrating third-party solutions can bolster the overall security architecture, ensuring that exploits and evasions are comprehensively addressed.
• Cost-Effectiveness: While expensive, not all premium solutions deliver reliable protection. With third-party vendors consistently scoring higher in independent tests, organizations may find a more favorable cost-to-security ratio by opting for these specialized services.

IT departments, particularly those managing Windows networks, should assess their security frameworks in light of these findings. As organizations modernize their infrastructure, the adoption of third-party security measures could be the difference between robust protection and a vulnerability waiting to be exploited.
Checklist for a Secure Cloud Strategy:

1. Evaluate current network firewall performance against both exploits and evasions.
2. Consider third-party solutions like Fortinet and Check Point, which have consistently demonstrated superior performance.
3. Ensure that any firewall solution includes robust HTTPS decryption to inspect encrypted traffic.
4. Regularly review independent testing and cybersecurity advisories to keep up with evolving threat landscapes.
5. Integrate cloud security with existing on-premises solutions for a holistic approach.

Balancing Cost, Performance, and Reliability​

One might ask: Why invest in third-party cloud security if native firewalls are bundled with the service? The answer is embedded in the principle of layered security—a widely accepted best practice in IT security frameworks. As cyberattacks grow more sophisticated, a layered approach that incorporates advanced third-party protections becomes not only sensible but necessary.
Factors to Consider:

• Price vs. Performance: Cisco’s offering, despite its high price, delivered disappointing performance relative to third-party alternatives. This discrepancy underlines the importance of independent testing data in making informed decisions.
• Adaptability: Cyber adversaries are continuously evolving their tactics. Solutions that consistently update their security signatures and hone their detection algorithms, as seen with top-performing third-party vendors, are better suited for the long haul.
• Trust but Verify: Relying solely on vendor claims can be risky. Independent testing, like that of CyberRatings.org, provides an unbiased view of what organizations can truly expect from their security investments.

Organizations are well-advised to reexamine contract terms, service level agreements (SLAs), and security provisions when dealing with cloud service providers. Relying on vendor-provided tools without supplemental security measures may result in a false sense of security.

Industry Reaction and Future Directions​

The industry reaction to these findings has been swift. Cybersecurity professionals and IT decision makers are increasingly advocating for a blended security approach—one that marries cloud-native capabilities with specialized third-party tools.
Looking ahead, we can expect:

• A surge in third-party cloud security service adoption as organizations seek more comprehensive protection.
• Increased pressure on cloud providers like AWS, Azure, and GCP to enhance their native security offerings, especially regarding encrypted traffic inspection.
• Greater scrutiny in the procurement processes for cybersecurity services, with independent testing becoming a key criterion in vendor selection.

The evolving threat landscape means that yesterday’s metrics might not suffice to protect tomorrow’s networks. Organizations must stay agile, continuously recalibrating their security strategies to incorporate both cutting-edge native services and the proven strengths of third-party solutions.

Conclusion: A Call to Reevaluate Cloud Security Strategies​

CyberRatings.org’s independent tests have exposed critical weaknesses in some of the most trusted cloud-native firewalls on the market. For organizations of all sizes, these results are a wake-up call: adopting third-party cloud security services can dramatically enhance protection, especially when dealing with sophisticated penetration tactics and evasion strategies.
For Windows administrators and enterprise IT departments alike, the message is clear:

• Don’t assume that premium pricing guarantees premium security.
• Evaluate your current security stack critically, balancing native cloud features with proven third-party solutions.
• Keep abreast of cybersecurity advisories and continuously integrate lessons learned from independent testing.

By embracing a layered security strategy that includes effective third-party firewalls, organizations can not only safeguard their IT environments—from Windows desktops to cloud-based servers—but also build a resilient infrastructure capable of adapting to an ever-changing cyber threat landscape.
In the fast-evolving field of cybersecurity, the adage “prepare for the worst, hope for the best” remains incredibly relevant. As businesses reel from these sobering test results, it might just be time to rethink and reinforce your cloud security strategy with solutions that have been independently validated to perform in the real world.

---

Source: CyberScoop Independent tests show why orgs should use third-party cloud security services