Continuous Quality and Testing to Accelerate Application Development // 1

How to assess your current testing maturity level and practice continuous testing for DevOps

Continuous Quality and Testing to Accelerate Application Development

Table of Contents03 Introduction

04 Why Is Continuous Quality and Testing Maturity Important to DevOps?

05 Continuous Testing Engineers Quality into DevOps

07 Best Practices for Well- Engineered Continuous Testing

08 Continuous Testing Maturity Levels

12 Continuous Testing Maturity Assessment

13 How to Get Started with DevOps Testing?

14 Continuous Testing in the Cloud

20 Conclusion

21 About AWS Marketplace and DevOps Institute

Level 1: Chaos

Level 2: Continuous Integration

Level 3: Continuous Flow

Level 4: Continuous Feedback

Level 5: Continuous Improvement

Contributors21

Choosing the right tools for Continuous Testing

The Right Tests at the Right Time

20 Get Started

On-demand Development and Testing Environments with Infrastructure as Code

Introduction

A successful DevOps implementation reduces the bottlenecks related to testing. These bottlenecks include finding andsetting up test environments, test configurations, and test results implementation.

These issues are not industry specific. They can be experienced in manufacturing, service businesses, and governmentsalike. They can be reduced by having a thorough understanding and a disciplined, mature implementation ofContinuous Testing and related recommended engineering practices.

The best place to start addressing these challenges is having a good understanding of what Continuous Testing is.

Marc Hornbeek, the author of Engineering DevOps, describes it as:

“A quality assessment strategy in which most tests are automated and integrated as a core and

essential part of DevOps. Continuous Testing is much more than simply ‘automating tests.’”

In this whitepaper, we’ll address the best practices you can adopt for implementing Continuous Quality and Testing onthe AWS Cloud environment in the context of the DevOps model.

We will explore:• How you can effectively implement Continuous Testing and enable your engineering and quality assurance (QA)

teams to get seamless self-service access to both the environments and tooling using AWS• How to develop a roadmap for Continuous Testing implementation based on maturity and assessment models• How to get started with DevOps testing

These suggested guidelines can help you reduce costs and time-to-market and allow your developer teams to stay focused on delivering value.

Continuous Quality and Testing to Accelerate Application Development // 4

Why Is Continuous Qualityand Testing MaturityImportant to DevOps?

The acceleration in DevOps adoption has fundamentally transformed how testing is done. Before the advent of DevOps, most quality testing happened at the end of the cycle. Now, testing increasingly takes place in parallel with development and is engineered for quality.

A comprehensive set of tests are, in aggregate, a prescription of expected behaviors for applications within the production environments. The expected behaviors are more than application features and functions. Non-functional requirements, such as performance, maintainability, and reliability, are also essential. Continuous tests are designed to cover both functional and Non-Functional Requirements (NFRs). The capabilities needed to operate an application in each of its deployment environments are part of Continuous Testing.

Although Continuous Integration adoption is becoming ubiquitous – with 88% of enterprise software companiesadopting it in 2020, test suites are still plagued by a growing backlog of failing tests. Failures in integration andautomated tests cause bottlenecks in the development pipeline and substantially increase engineering costs. When engineered well, Continuous Testing can provide impressive benefits, including reduced time-to-market, improved quality, reduced cost, improved innovation, and team satisfaction.

According to a May 2020 research report published by UNDO1:

35%% of respondents say they deploy code changes hourly.

$$61billionbillion to $1.2 trillion annually total salaries spent in enterprise value

lost for shareholders per year.

1The Business Value of Optimized Pipeline – UNDO 2020

88%% of enterprise software companies adopting Continuous

Integration in 2020 compared to 70% in 2015.

26%% of developer time is spent an average of 13 hours reproducing

and fixing falling tests, which equals 620 million developer

hours a year wasted on debugging software failures.

Continuous Quality and Testing to Accelerate Application Development // 5

Continuous test engineering requires a complete re-think of software QA test strategies, so they align with Dev-through-Ops pipeline stages. Fortunately, best practices for DevOps testing are available and can be learned and applied to any product or service.

Five tenets underlie the best practices for DevOps test best practices are indicated in Figure 1 and explained below.

Continuous Testing Engineers Quality into DevOps

RelevanceFocus on the most importanttests and results

Fail oftenRun tests frequently and with many different conditions

Test FastArrange tests to runin quick cycles

Shift LeftConduct each test as early in the

pipeline as possible

Fail EarlyArrange the tests so that the

most likely problems are foundin the earliest possible stage in

the DevOps pipeline

Figure 1: Continuous Testing Tenets

Continuous Quality and Testing to Accelerate Application Development // 6

The CD pipeline starts with the release of candidate artifacts delivered from the CI pipeline into an artifact repository. The CD pipeline is automated, and artifacts are deployed to a pre-production staging environment for system and user acceptance testing. These tests are automated as much as possible to minimize processing delays and bottlenecks. Once the tests pass, the artifacts are assessed with release policies to win approval for deployment. The release acceptance criteria are often determined by a Change Approval Board (CAB) and implemented as codified policies (if possible).

Once approved, the artifacts are deployed to production according to a deployment schedule. To minimize risk, the deployments are conducted using the safe Green/Blue and Dark Launch methods. Once deployed, additional post-deployment tests may be used to evaluate software options using A/B testing and Canary testing methods.

Testing, an essential portion of DevOps, is responsible for continuous assessment of incremental changes and is part of

establishing a culture and environment where building, testing, and releasing software can happen rapidly, frequently,

reliably, and safely.

Figure 2: DevOps Continuous Testing Blueprint

DevOps Testing – the Primary Key to DevOps and Continuous Delivery” by Marc Hornbeek

Continuous Quality and Testing to Accelerate Application Development // 7

Best Practices for

Continuous Testing concepts must be mastered. Every test that can be automated should be automated. Tests should be executed in production-equivalent test environments. The results must be assessed in each stage of the end-to-end pipeline to determine if the artifacts are acceptable or need remediation before promotion.

Culture for Continuous Testing is critical. Quality is everyone’s responsibility. Leaders need to sponsor, inspire, fund, and motivate cross-functional teams to collaborate, learn, implement, and improve Continuous Testing practices.

Test strategies for Continuous Testing include specific objectives. Automated and even manual tests are triggered at each pipeline stage. Elastic cloud test environments are automatically orchestrated to match testing requirements of each change. Tests are shifted-left an accelerated as early as possible to find customer- relevant verdicts fast and early. The use of the A/B feature flag, Blue-Green, and Canary deployments strategies help validate user alternatives and performance safety in production.

Test Frameworks and Tools are selected for integration and performance. A test framework acts as a “backbone” for integrating and automating tasks including test plans, all types of test tools, test artifact version management, test resources, test data, test schedules, test environment orchestration, test results, test reports, and test results analysis.

Test Planning includes test strategies that describe test requirements, management policies, resources, topologies, and priorities for each product module to meet test strategy requirements. Test plans define test cases and priorities for each product module to meet test strategy requirements.

Test Automation is used extensively. Automated tests are created per requirements described in test strategies and plans, using a test automation creation strategy such as TDD. Tests and test tools are automated, orchestrated, operated, and results analyzed through an API by test cases scheduled by a framework.

Test Management and Analysis. Manage resources (financial, labor, schedules, and flexible scaling) to meet specific CT quality, delivery, and response time goals determined by key stakeholders.

Well-EngineeredContinuous Testing

Continuous Quality and Testing to Accelerate Application Development // 8

Continuous TestingMaturity Levels

Many organizations have not achieved the level of continuous test maturity described above. For an organization to

mature its continuous test capabilities, it needs to know its current state of maturity and the target of its future state.

There is no standard continuous test maturity model. A useful way to approach it is to apply a five- level maturity model adapted from the “standard“2. The following paragraphs describe the five levels of continuous test automation maturity,

as defined by Marc Hornbeek.

There are three dimensions to consider at each level: People, Process, and Technology. Technology, such as better test tools, will not accomplish a higher maturity level unless the People and Processes use them effectively.

Continuous Testing Maturity Level 1: Chaos

Figure 3 shows the key characteristics of People, Process, and Technology, evident at this maturity level. There is littleindication of Continuous Testing skills at this level, test automation processes are not apparent, and test automation technology has significant gaps.

Typical outcomes are releases that are unpredictable, reactive, and the pipeline is wasteful. Response to resolve theproblems found by testing is slow.

People Process Technology

• Silo team organization

• Little knowledge of Continuous Testing

• Blame, finger-pointing

• Testing not part of planning

• No test standards

• Few automated tests

• Missing tools to test performance of applications, pipelines and infrastructure

2The original software Capability Maturity Model (SW-CMM) was developed by the Software Engineering Institute (SEI) at Carnegie Mellon University in the mid-1980s.

Figure 3: Continuous Testing Maturity Level 1: Chaos

Continuous Quality and Testing to Accelerate Application Development // 9

Continuous Testing Maturity Level 2: Continuous Integration

Figure 4 shows the key characteristics of People, Process, and Technology, evident at this maturity level. At this level, there is some knowledge of Continuous Testing. Processes and technology well support the integration and build processes for Testing. End-to-end Testing, especially delivery and deployment stages, are not well supported with automated Testing.

At this stage, there are deployment quality problems and significant gaps in the infrastructure and process of the testing system. The test processes tend to have insufficient fidelity to indicate the source of failure unambiguously. This results in a situation where the reaction to unknown sources of failure is chaotic. End-to-end pipeline failure events must be correlated manually.

• Limited knowledge of CT, Ad-hoc training

• Some Dev/QA co-ordination

• Most tests other than build tests are manual

• Minimal test version management

• Version management

• Automated build test

• Painful but repeatable releases

People Process Technology

Figure 4: Continuous Testing Maturity Level 2: Continuous Integration

Figure 5 shows the key characteristics of People, Process, and Technology, evident at this maturity level. At this level,test automation is used from end-to-end across the pipeline. Dev and QA teams cooperate to ensure a good level oftest coverage is automated. Release standards are using automated testing metrics.

Release standards are using automated metrics and some sophisticated measures correlating application test resultsperformance to business risk.

Typical outcomes include repeatable quality releases with some bottlenecks and inefficiencies. There is sufficient testing to identify the most likely source of failure. Proactive testing results are integrated into one place for analysis.

Continuous Testing Maturity Level 3: Continuous Flow

Continuous Quality and Testing to Accelerate Application Development // 10

Figure 5: Continuous Testing Maturity Level 3: Continuous Flow

Figure 6: Continuous Testing Maturity Level 4: Continuous Feedback

• CT skills and training program

• Risk management

• Dev/QA joint plan

• Collaboration using shared test metrics

• Goals: SLI/O/As

• Mentors and Guilds

• E2E CI/CD pipeline, tests visible

• Test/release standards

• Test management

• E2E performance trend drive test design

• Focus on removing test bottlenecks

• Most tests automated for app, infra, pipeline

• Release metrics use test results

• Test environment orchestration

• Predictive testing

Continuous Testing Automation Maturity Level 4: ContinuousFeedback

Figure 6 shows the key characteristics of People, Process, and Technology, evident at this maturity level. At this level, Continuous Testing processes extend from end-to-end across the pipeline. Metrics are set for test automation. The culture includes training and mentoring for test automation. Automation is applied to test environment orchestration and analytics. There is a focus on end-to-end testing rather than spot tests. Automation is applied to testing extensively.

Release standards are using automated metrics and some sophisticated measures correlating application test resultsperformance to business risk.

Typical outcomes include repeatable quality releases with some bottlenecks and inefficiencies. There is sufficienttesting to identify the most likely source of failure. Pro-active, testing results are integrated into one placefor analysis.

People Process Technology

People Process Technology

Continuous Quality and Testing to Accelerate Application Development // 11

People Process Technology

Figure 7 shows the key characteristics of People, Process, and Technology, evident at this maturity level. At this level, there is a high level of knowledge and confidence regarding continuous test automation. Dev and QA teams are tightly integrated to optimize knowledge and efficiency. End-to-end tests focus on the end customer experience and more sophisticated risk-based strategies. Test creation strategies leverage automation.

Dev, QA, and Ops teams collaborate using shared testing strategies. End-to-end tests focus on the end customer experience and more sophisticated risk-based strategies.

Typical outcomes include extremely high confidence and satisfaction by all stakeholders and innovation-driven culture. Tests can predict, with a high accuracy, where improvements are needed to avoid failures in the applications, pipelines, and infrastructure components. The test systems are self-correcting, allowing for experimentation. Achievement at this level provides a platform for autonomous continuous improvement strategies in which automation and intelligence drive innovation. “Here is the fix for the thing that will break.”

Continuous Testing Maturity Level 5: Continuous Improvement

• Experimentation

• Integrated Dev/QA

• E2E user experience focus

• Risk based test design

• Automated test creation and test results analysis

• E2E value stream test analysis, orchestration and execution

• Intelligent test creation

Figure 7: Continuous Testing Maturity Level 5: Continuous Improvement

Continuous Quality and Testing to Accelerate Application Development // 12

Continuous Testing Maturity Assessment

While the above five levels of Continuous Testing maturity provide a practical guide for defining maturity againstcharacteristics of People, Process, and Technology, they are not an absolute measure of maturity. Organizations, orspecific applications within an organization, may match some of the characteristics for different levels.

The following Continuous Maturity Model Assessment Model (Figure 8) is a useful tool to determine the “best fit” forthe maturity of an organization or application within an organization. Matching the people’s characteristics, process,and tech columns that best describe your organization provides you with a quick assessment of your dominant level oftesting maturity. You can also use this as a fast way to determine areas most in need of improvement to get to the nextlevel of maturity.

Figure 8: Continuous Testing Maturity Assessment Model

People Process Technology

Chaos

Continuous Integration

Continuous Flow

Continuous Feedback

Continuous Improvement

• Silo team organization

• Little knowledge of test automation

• Blame, finger-pointing

• Testing not part of planning

• No test standards

• Few automated tests

• Missing tools to test performance of applications, pipelines and infrastructure

• Test automation skills and training program

• Risk management

• Dev/QA joint plan

• E2E CI/CD pipeline, test visible

• Test/release standards

• Test management

• Most tests automated for app, infra, pipeline

• Release metrics use test results

• Collaboration using shared test metrics

• Goals: SLI/O/As

• Mentors and Guilds

• E2E performance trends drive test design

• Focus on removing test bottlenecks

• Test environment orchestration

• Predictive test analytics

• Experimentation

• Integrated Dev/QA

• E2E user experience focus

• Risk based test design

• Automated test creation and test results analysis

• E2E value stream test analysis, orchestration and execution

• Intelligent test creation

• Limited knowledge of testing, Ad-hoc training

• Some Dev/QA co-ordination

• Most tests other than build tests are manual

• Minimal test version management

• Version management

• Automated build tests

• Painful but repeatable releases

1

2

3

4

5

Continuous Quality and Testing to Accelerate Application Development // 13

How Do You Get Startedwith DevOps Testing?

A primary key to success with DevOps is continuous test engineering. As this whitepaper has explained, DevOpsContinuous Testing is much more than automating tests or integrating a tool into the continuous delivery toolchain.The following 7-step methodical approach can help you realize effective and efficient DevOps Continuous Testing:

1

2

3

4

5

6

7

Skill yourself and your team on DevOps testing. Explore formal training.

Select a pilot application.

Implement test infrastructure, test frameworks, and tools according to the solution roadmap.

Train cross-functional team members on test creation methodologies and tools.

Monitor progress and put in place continuous improvement procedures.

After a pilot project with a one product application and team, expand the implementation to more teamsand applications.

Create a dedicated role to guide your organization through DevOps testing or engage a DevOps test consultant and appoint a DevOps test architect to conduct an assessment of your current practices and define a roadmap. The solution roadmap should define a test strategy for the product. The strategy will define the test coverage strategy, how test will be created, which test will be conducted at each stage of the continuous delivery pipeline, how test results will be handled, and metrics to monitor testing performance.

Continuous Quality and Testing to Accelerate Application Development // 14

Continuous Testingin the Cloud

Cloud computing offers flexibility to test software services on the most appropriate environments and, at the proper time, throughout the software development cycle. The most effective software testing relies on the execution of tests that most accurately represent real-world user conditions.

Early in development, basic verification testing of code on a local laptop or SaaS IDE is acceptable. But as soon as code is compiled and merged with that of other developers, validating that the resulting services meet user expectations under various functional, performance, scalability, and security conditions become critical consideration of overall quality and user satisfaction. When utilizing a mature Continuous Integration/Continuous Delivery (CI/CD) pipeline, it is possible to trigger test jobs at any point in the delivery pipeline and automate the deployment and tear-down of appropriate cloud-based test environment to run the tests using the Infrastructure as Code approach.

Using AWS, engineering and QA teams can get seamless self-services access to both the environments and tooling they need to implement Continuous Testing effectively.

Figure 9: Continuous Testing with AWS

Continuous Quality and Testing to Accelerate Application Development // 15

Choosing the right tools for Continuous Testing

A significant pipeline strategy aligns DevOps phases with established practices and easily integrates with your chosen tools. In addition to the people and process elements of your Continuous Testing strategy, you’ll need to ensure that you have the right testing tooling that integrates with your pipeline.

Engineering and QA teams can use DevOps Solutions available in AWS Marketplace to design and deploy secure, scalable modern applications and infrastructure that can accommodate your current and future business needs as application usage grows. AWS Marketplace is a digital catalog with thousands of listings from independent software vendors that enables you to quickly and easily find, test, try, buy, and deploy software to orchestrate your entire DevOps toolchain on AWS.

The next section explains how to perform Continuous Testing across the application delivery pipeline and introduces the

most commonly used third-party testing tools to help get the job done.

Flexible consumptionand contract models

Quick and easy deployment

Helpful humans to support you

Continuous Quality and Testing to Accelerate Application Development // 16

One of the most significant benefits to developing applications in the cloud is defining your development, test, andproduction environments (infrastructure and software configuration) in machine-readable definition files. The filesare managed in version control, like your application code, so that you can automate the creation and tear-down ofconsistent production-like environments for development and testing.

This practice is key to building quality into your applications as quality starts with the code your developers commit.Suppose development teams can work in self-service production-like environments without waiting long periods foroperations to prepare them. In that case, developers can write, test, and run code to receive immediate feedback as theywork.

Enabling development and QA teams with self-service access to needed environments facilitates frequent codecommits, integration, and automated testing required for high-functioning DevOps pipelines. Adopting an Infrastructureas Code approach that is version-controlled and immutable also prevents configuration drift and errors that surfaceafter the release because the development, testing, and production environments are all identical.

On-demand Development and Testing Environments with Infrastructure as Code

Infrastructure as Code (IaC) tools available in AWS Marketplace:

HashiCorp TerraformTry it >

Quali’s CloudShell ColonyTry it >

SaltStack EnterpriseTry it >

View all IaC tools and learn more >

Puppet EnterpriseTry it >

Chef AutomateTry it >

Continuous Quality and Testing to Accelerate Application Development // 17

The Right Tests at the Right Time

Continuous Integration and Continuous Delivery (CI/CD)Most DevOps activities revolve around a source code repository, and most of them utilize Git as the repo of choice.

Continuous Integration orchestrates the builds of code into applications and services. It also automates the triggering of

tests and other tasks, including the actual deployment of testing and production environments by integrating with your

Infrastructure as Code tools.

Try it >Armory

Try it >GitHub Enterprise

Try it >Harness

Try it >GitLab

CI/CD tools available in AWS Marketplace:

View all CI/CD tools and learn more >

Try it >CloudBees Products

Try it >Circle CI - CI/CD

Continuous Quality and Testing to Accelerate Application Development // 18

Quality and Test Management Continuous Testing involves complex of interrelated artifacts and processes that often require their own management and execution frameworks. These solutions keep track of test requirements, specifications, scripts, and results sets. They can also control test execution, role base tasks, test results, and comprehensive quality reporting and analysis. DevOps solutions include integrations with key AWS Developer tools and other CI/CD tools.

Functional TestingTesting the functional inputs and outputs of software and services really picks up when integration and complexity regressions and bugs that will impact users. Cloud-based functional test automation enables teams to build and automate tests of this type. By deploying cloud-based test environments, hardware and network variables can be considered in the tests.

Load Testing, including Performance Testing These important test types measure the user experience and system breaking points as services scale to the needs of any number of users — from tens to millions. To any scale and form different regions, clouds resources instant availability has provided the best platform for driving continuous and on-demand load testing tools.

Mobile Testing Because users expect that every website and application will run on their mobile device, Continuous Testing must include the mobile user experience and mobile scalability, and even localization. With the proliferation of different mobile device types and the two primary mobile operating systems (iOS and Android), testing can be best accomplished with a mix of actual physical devices combined with device emulators that run on cloud servers.

Production TestingFormerly taboo, testing in production — with the proper tools, training, guidelines, and rollback capabilities — has become a relatively standard and accepted approach for organizations focused on end-to-end quality and reliability. Chaos Engineering, which includes randomly shutting down servers, injecting latency, dropping packets, and breaking connections, creates real-world scenarios that can best be replicated on actual production infrastructure. Feature flagging, which allows the testing of new features by toggling on/off for select users, and closely related A/B testing, which presents different capabilities to different users, enables the live testing of services to determine both functionalities as well as user adoption in live production environments.

Continuous Quality and Testing to Accelerate Application Development // 19

TestableTry it >

Gatling FrontLineTry it >

Micro FocusLoadRunner Cloud

Try it >

Genymotion CloudTry it >

View all testing tools and learn more >

Tricentis qTestTry it >

Micro Focus ALM OctaneTry it >

CloudBees Feature Management

Try it >

GremlinTry it >

LaunchDarklyTry it >

Micro Focus UFT OneTry it >

Testing tools available in AWS Marketplace:

Load and Performance Testing: Mobile Testing:Quality and Test Management:

Production Testing:

Functional Testing:

Continuous Quality and Testing to Accelerate Application Development // 20

Conclusion

Get StartedAdvance your Continuous Testing practice with the right tools and more knowledge on DevOps best practices.

To ensure your teams have the best tools for the job, explore and try new solutions in AWS Marketplace.

The acceleration to DevOps transformed the way applications are developed and delivered and how they are tested.

Successful implementation of the blueprint for well-engineered Continuous Testing depends on understanding the

stage of your maturity level and implementing the appropriate best practices outlined in this whitepaper.

The first step requires an in-depth assessment across all three dimensions of the model: people, processes, and technology, which will enable you to develop a roadmap for your own version of Continuous Testing.

As you develop your roadmap, consider cloud computing’s flexibility to test software services in the most appropriate

environments throughout the development cycle. Continuous Testing in the cloud enables your development and QA

teams with self-service access to needed environments.

To implement Continuous Testing effectively, engineering and QA teams can get seamless self-service access to both the

environments and the tooling using AWS. The resulting continuous quality management and testing will reduce costs

and time-to-market and ensure development teams stay focused on delivering customer value.

Find solutions 46% faster*

Find the market-leading tools you need to orchestrate your ideal DevOps toolchain.

Adopt new tools 53% faster*

Try leading-edge developer tools and simplify procurement with integrated AWS billing.

Deploy your way, 48% faster*

Deploy DevOps solutions with methods that best fit your use case using containers, SaaS, AMIs, APIs, or CloudFormation Templates, and more.

Trust AWS verified technologies

Count on tools that are selected for AWS interoperability and are regularly scanned for security vulnerabilities.

Key benefits of building your DevOps practice with solutions in AWS Marketplace

*2020 Amazon Web Services Marketplace online survey with 200 IT decision-makers and influencers in the U.S.

To get started, visit:https://aws.amazon.com/marketplace/solutions/devops/continuous-testing

About AWS Marketplace

About DevOps Institute

AWS Marketplace is a curated digital catalog that simplifies software discovery, procurement, provisioning, and

management. With AWS Marketplace, customers can also utilize features that speed up product evaluation, improve

governance and cost transparency, and enhance control over software spend. AWS Marketplace offers third-party

solutions across software, data, and machine learning tools that enable builders to find, and deploy solutions to

expedite innovation.

Customers can launch pre-configured solutions in just a few clicks in both Amazon Machine Image (AMI) formats and SaaS subscriptions, with entitlement options such as hourly, monthly, annual, and multi-year contracts.

AWS Marketplace is supported by a global team of solution architects, product specialists, and other experts to help IT teams connect with the tools and resources needed to streamline migration journeys to AWS.

DevOps Institute works to advance the human elements of DevOps. We create a safe and interactive environment

where members can network, gain knowledge, grow their careers, support enterprise transformation, and celebrate

professional achievements.

We are not just an information source; we connect and enable the global member community to drive human transformation in the digital age.

Contributors: Marc Hornbeek

Ambassador, DevOps Institute

CEO and Principal Consultant

at Engineering, DevOps Consulting

Kanchan Waikar

Senior Partner Solutions Architect,

AWS

Alex Jones

Sr. Digital Demand Generation Manager,

AWS Marketplace

Brad Johnson

Global DevOps Category Leader,

AWS Marketplace