design and develop a fuzzer framework to security test a
Post on 01-Jun-2022
3 Views
Preview:
TRANSCRIPT
Miskolci Egyetem
Gépészmérnöki és Informatikai Kar
Általános Informatikai Tanszék
Design and develop a fuzzer framework to security test a Cloud based application
Balázs Attila
CVXTCT
1164 Budapest, Bányász utca 11.
Design and develop a fuzzer framework to security test a Cloud based application
I
Table of Contents 1. Preface ...................................................................................................................... 1
1.1 The Company .................................................................................................................. 1
1.3 Explanation of topic ......................................................................................................... 2
1.4 Task description .............................................................................................................. 3
2. The importance of the testing in the development process ........................................ 5
2.1 Software quality assurance and its impact on security .................................................... 5
2.2 The Agile software development ..................................................................................... 5 2.2.1 Scrum methodology ...................................................................................................................... 6 2.2.2 Definition of “Done” ..................................................................................................................... 7
3. Cloud based web-applications .................................................................................... 8
3.1 Well architected framework ............................................................................................ 8
3.2 Abstraction of a distributed system ................................................................................. 9 3.2.1 Communication between components ......................................................................................... 9
3.3 Definition of Cloud ........................................................................................................ 10
4. Testing in practice, introduction of test methods ...................................................... 11
4.1 Testing the functionality ................................................................................................ 11 4.1.1 Unit test ........................................................................................................................................ 11 4.1.2 Component test ........................................................................................................................... 12 4.1.3 Smoke test .................................................................................................................................... 12 4.1.4 Functional test ............................................................................................................................. 12 4.1.5 Mutation based testing ............................................................................................................... 13 4.1.6 System stress test ......................................................................................................................... 13 4.1.7 Code coverage ............................................................................................................................. 14
4.2 Nonfunctional testing ..................................................................................................... 14 4.2.1 Security tests ................................................................................................................................ 14 4.2.2 Static code analysis ..................................................................................................................... 15
5. Fuzz testing ............................................................................................................. 16
5.1 fuzzer types, methods, features ...................................................................................... 16
6. Research a fuzzer tool .............................................................................................. 19
6.1 Considerations for selecting a fuzzer ............................................................................. 19 6.2.1 Kitty / Katnip fuzzer tool ............................................................................................................ 20 6.2.2 Sulley fuzzer tool .......................................................................................................................... 20 6.2.3 Peach fuzzer tool ......................................................................................................................... 21 6.2.4 Burp as a possible fuzzer ............................................................................................................ 22 6.2.5 American Fuzzy Loop ................................................................................................................. 22 6.2.6 Protocol learning, simulation and stateful fuzzer ..................................................................... 22
6.3 The chosen fuzzer .......................................................................................................... 23
7. Technologies involved .............................................................................................. 25
7.1 Resources in use at Amazon Web Services .................................................................... 25 7.1.1 Amazon Elastic Compute Cloud (EC2) – virtual machines .................................................... 25
Design and develop a fuzzer framework to security test a Cloud based application
II
7.1.2 Networking in nutshell at AWS ................................................................................................. 26 7.1.3 Serverless technology .................................................................................................................. 26 7.1.4 Databases, RDS, DynamoDB ........................................................................................................ 27 7.1.5 Simple Storage Service ................................................................................................................. 27 7.1.6 How to control AWS resources .................................................................................................... 28
7.2 Development environment ............................................................................................. 28 7.2.1 The programming language ......................................................................................................... 28 7.2.3 Project dependencies ................................................................................................................... 29 7.2.4 Integrated Development Environment ....................................................................................... 29
7.3. Further technologies involved ........................................................................................ 30 7.3.1 Docker container .......................................................................................................................... 30 7.3.2 Redis in-memory database .......................................................................................................... 31 7.3.3 Google Protocol Buffer ................................................................................................................ 32
8. Software specification ............................................................................................. 34 8.1 Functional requirements ................................................................................................................. 34 8.2 Nonfunctional requirements .......................................................................................................... 35
9. Software design ....................................................................................................... 36
9.1 Software architecture ..................................................................................................... 36 9.1.1 Kitty core ...................................................................................................................................... 38 9.1.2 Katnip component ........................................................................................................................ 39 9.1.3 Framework core and the config parser ....................................................................................... 40 9.1.4 Protobuf parser ............................................................................................................................ 41 9.1.5 Protobuf message validator ......................................................................................................... 45 9.1.6 Protobuf target ............................................................................................................................. 47 9.1.7 Protobuf controller ...................................................................................................................... 47 9.1.8 Protobuf runner ........................................................................................................................... 48 9.1.9 HTTP target ................................................................................................................................... 48 9.1.10 HTTP controller .......................................................................................................................... 49 9.1.11 HTTP runner ............................................................................................................................... 49 9.1.12 DNS target .................................................................................................................................. 50 9.1.13 DNS controller ............................................................................................................................ 53 9.1.14 DNS runner ................................................................................................................................. 53 9.1.15 Redis component ....................................................................................................................... 53 9.1.16 Reporter component .................................................................................................................. 54 9.1.17 Utilities component ................................................................................................................... 54 9.1.18 Runner ........................................................................................................................................ 55
10. Installation and integration into CI system ............................................................. 56
10.1 The executor environment ............................................................................................ 56
10.3 Install the application ................................................................................................... 56
10.4 Integration into Continuous Integration System ............................................................ 57
10.5 Results after the first runs ............................................................................................. 59
11. User Guide to operate ............................................................................................ 60
11.1 Configuration ................................................................................................................ 60
12. Further development possibilities .......................................................................... 63
13. The future: increase app resilience with symbolic execution. .................................. 64
13. Summary ............................................................................................................... 65
Design and develop a fuzzer framework to security test a Cloud based application
III
14. Összegzés ............................................................................................................... 66
15. Bibliography .......................................................................................................... 67
16. Printed appendix .................................................................................................... 69
16.1 Appendix 1. .................................................................................................................. 69
16.1 Appendix 2. .................................................................................................................. 70
16.2 Appendix 3. .................................................................................................................. 71
16.4 Appendix 4. .................................................................................................................. 72
Design and develop a fuzzer framework to security test a Cloud based application
1
1. Preface Public facing internet services could be much more vulnerable and could face more
suspicious activity rather than private ones. Because of the properties of public service, we
should consider and keep in mind specific factors. One of those factors is the enormous
amount of load which needs to be served in a strict time window. It occurs in a smooth, easy
to handle way sometimes peak growing as fast as it is a virtue. That is, the nonfunctional
requirements are getting more emphasis on functional requirements. This means tighter
restrictions. This approach needs a higher level of quality insurance.
My employer is the Sophos Ltd., which is a worldwide IT security company. I work for them
at the Budapest engineering site as a software engineer. My basic tasks are related to DevOps
methodology.
It was evident that I process a theme which helps the quality and trough of our product as
well as my skills too. It aids my team to manage higher quality.
1.1 The Company Sophos was founded by Jan Hriska and Peter Lammer. The main profile is the cybersecurity,
which embodied in several software products and devices such as antivirus software,
firewalls, hardware-based infrastructure protection like intrusion detection platforms, routers
and so on. Complete hardware and software-based security system against cyber criminals
designed for small and mid-size companies across the Planet. Sophos Central is a centralized
solution for companies to manage their entities across their company network. Abingdon is
the flagship site, the basis of the Company, which also has got Sophos Labs.
It has several other sites around the Globe (Vancouver, Ahmedabad, Bangalore, Sidney,
etc.). The Budapest engineering site has a Sophos Labs where the threat researchers are
seeking after threats by reversing compiled code using special tools to develop the AV
software effectiveness with new researches.
Design and develop a fuzzer framework to security test a Cloud based application
2
1.3 Explanation of topic To achieve the highest possible standard of the functionality defined in the software-
specification and to assure the operation safety, the testing is a mandatory and fundamental
process. Those tests results might reveal high- or low-level software faults arise from wrong
design patterns or in from the point of low-level view failures might arise from wrong
program code, which possibly could be a syntax or semantic fault which was not covered
during the process of development. A programmatical error, which is not covered in time,
could lead to runtime exception, which could cause harmful program and system state or
behavior and might open backdoors for criminals or other threats. This could lead to
endangered software and system security. So, testing can help systems avoid becoming a
victim.
During the thesis, I have been introducing several testing methods. Most of them are high-
level views or abstractions. One of them will be the Fuzz testing, which I will be autopsy in
deeper. In nutshell, some words about the Fuzz testing are indispensable in an introduction.
So, the essence of this type of testing method is to flood the system with dummy data (or
mutation of valid request data) it did not prepare, in a way that the hidden failures get into
the field of view.
A possible question could be “Do we need and why we need another fuzzer software because
solutions are all over the internet which is already developed by open source society.”. The
answer is the unique solution we use, for instance Protobuf over HTTP(s) which so far no
one has implemented (according to my research) in a fuzzer yet, or the DNS protocol we use
interpreted differently, or a fuzzer framework which is easy to configure no matter how
complex the protocol is. So, teams can apply it easily. fuzzer solutions on the market mostly
inflexible to use them in an agile environment or harsh to configure and manage them.
Another important aspect is the Continuous Integration and Continuous Delivery,
Continuous Testing methods and their tools used by teams over the Company. The easy to
set up and secure integration is a must-have attribution of any tool. Of course, a fuzzer
framework must have these attributes too. Teams must be able to use it with low cost and be
able to integrate into CI/CD processes.
Design and develop a fuzzer framework to security test a Cloud based application
3
Accordingly, I decided to search an opensource fuzzer project, which is suitable to build on
a frame. The tilt factor is the architecture of the fuzzer which allows me to implement the
framework and the protocols we want to fuzz. The final goal is a well-equipped fuzzer
framework integrated into our automation processes.
1.4 Task description The objective is to design and implement a fuzzer framework to security test a Cloud-based
application. To understand the task, it is necessary to give some abstraction about the system
I want to test. Imagine an endpoint with antivirus software installed on it. In the past, usually,
it grabs the current threat definition database from a server. Nowadays, users expecting fast
answers and fast resolutions, and of course, they make more and more load on the network
as time goes on. If a client running on an endpoint needs information about a suspicious
object (Potential suspicious or threat object could be a website, an email or it’s attachment,
an application, domain and so on.) it will start a lookup in the direction of a cloud service to
ask whether the suspect is a known threat or not. The server in the Cloud have the ability to
query the most maintained database. The service in the Cloud is a part of a distributed system
and has a lot of interface in conglomeration with several Cloud components. I want to
security test that distributed system with the fuzzer framework.
I have the main task, which is describe the above items, I divide it into smaller ones to handle
them quickly and will be introducing them in this document from step by step. Those steps
are the next ones as they appear in logical sort order. The first task is to set up the
requirements against the fuzzer. Some of those requirements are that the fuzzer implements
the Object-Oriented Programming; it is robust, modular or component-based. It has got a
well-designed and easy to understand easy to automate template mechanism to create the
mutations of the data. It is template-based, and its template mechanism is not stiff. It has
already implemented network protocols and or can extend them with new ones, or there is a
possibility to override and or mix them.
The second task is to research the opensource market to grade the reachable fuzzer solutions
according to the requirements that have been set up. After I make the sort, I will be describing
all of them and make a decision, which I will be making a detailed description to prove my
decision.
Design and develop a fuzzer framework to security test a Cloud based application
4
In the implementation task I will be designing and implementing a fuzzer framework based
on the chosen fuzzer, which will have a configuration file to parse the system set up
according to what the user wants to Fuzz, for instance, DNS, Protobuf, HTTP based systems.
So, there will be three main components of the framework. I will be making a high-level and
low-level design and implement the method according to the designs. The integration tasks
are about creating documentation, which is about how to handle, maintain, debug, and
integrate the framework. To incorporate the structure into the CI system will be another task
of the post work. The latest job is about to describe further developing possibilities.
Design and develop a fuzzer framework to security test a Cloud based application
5
2. The importance of the testing in the development process The second chapter is about the software testing which importance is unquestionable. Still,
it is worth examining some aspects such as tools, methods which support the impact of
testing on security and quality.
2.1 Software quality assurance and its impact on security The two terms described in the title go hand in hand, and they have an impact on each other.
Obviously, under a certain quality threshold, there is no security and no functionality. Fact
that the quality has an impact on the safety and functionality.
The software quality assurance can interpret as the internal and external features of the
software. The external quality is defined based on the functional software specification and
which software performs in real-life scenario and how useful it is for users or services which
they use. For instance, the software could be called over its API; in this aspect, the
functionality means that the software responses and behaves like that as the API
documentation describes it. The internal quality is the code quality. No matter how proper
the functionality by the specification is. If we do so, it with a messy codebase, that means,
in one hand, that the code is hard to maintain, and on the other hand, it is hiding
vulnerabilities, failures, security flaws.
The quality assurance aims to evaluate the code through testing and compel the standards to
achieve software security.
2.2 The Agile software development The agile method is one of the most powerful tools for teams to handle everyday challenges
in the developing process. The toolset provided by the agile approach is the basis of the
quality workflow.
Agile meaning: „Agility is the ability to both create and respond to change in order to profit
in a turbulent business environment.”
Design and develop a fuzzer framework to security test a Cloud based application
6
In practice, it provides a framework for developer teams to assure the synchrony with their
actual tasks to each other. In small atomic pieces, they work and do iterations from sprint to
sprint from day to day to make designs, solve problems, implement solutions. [1]
2.2.1 Scrum methodology If the agile methodology is the framework, then the Scrum holds the toolset to manage kinds
of stuff. Definition of Scrum: Scrum is a type of toolset which allows teams to assign
complex tasks to its members to solve them with creativity and in high quality and value.
The roles are the Product Owner, who is in charge of the value of the product.
Development team, members who implement the logic. Scrum Master's part is to manage
the Scrum methodology.
The most significant part of the Scrum methodology is the Project, the next level is
the Sprint. Usually, a sprint is a 2-4 week long period. During the Sprint, tasks are escalated
to developers. A developer can hold one to more tasks per Sprint, depending on the task's
complexity. Sprints started with Sprint meetings during its planning. The responsibilities are
escalated to developers. The type of the task is depending on the state of the Project, it can
be a design task, or an implementation one, or a bug fix and so on.
Each task has a technical complexity, which is usually translated into Story Points; these
points determine the degree of difficulty of the task currently under in definition. Teams
Scrum policy mostly decides the Story Pointing rule, which means if a job has voted as a
high value of Story Point, it could be better to break the task smaller apart. Voting, teams
vote each task Story Point during the estimation of the task, which could be the part of a
Sprint meeting. Estimated duties are included in the Scrum supporter system.
During the Sprint teams holds a daily ritual called Stand Up. Stand Up about to report to the
Scrum Master and to each other about the things they done previously, and they further plan
to the upcoming day. If someone has a blocking factor, the Scrum master try to resolve and
involve necessary resources to manage the problem. To achieve easily the aims with the
Scrum there are several software tools available for teams which are really effective. These
software tools can interact and communicate with each other. For instance, the ticketing
system can open up a fresh new branch in the code repository handler system to support the
developer to manage the administrative stuff that comes with the start of the task. The CI
system also has the ability to put messages on a form of commit for the repository system to
Design and develop a fuzzer framework to security test a Cloud based application
7
indicate whether the code developed on a specific branch is built successfully or not. The CI
and repository system can insert messages about the building process into the ticket handler
system. Known statuses that we developed for us during a task are the “in progress,” “in
review,” “in test,” “completed”. In the state of “in review,” members judge each other work
to verify its legitimacy for company policy or professional consideration.
It’s evident that the process ensures the ability to handle the Project and provide the ability
to interevent any time if it is necessary. The product is made on small iterations and can
adapt new solutions if the needs are changing over time. Team members getting involved in
every aspect of the Project from designing to implementing no matter what component they
will work during the creation. This last approach has been also developed by us.
The Project closing in general followed by a Retrospective, where the team members can
discuss they comment about the past project, those comments can be negative or positive
ones regarding the process, how they feel things could be done better. [2][3]
2.2.2 Definition of “Done” A simple, useful list about how to start a task and how to reach success on it. It could depend
on how the company restructures it through its policy, but mainly it holds the following
elements in our case:
• Task understood;
• Feature implemented;
• Unit tests are passing;
• Build/code is evaluated by the CI system;
• Code is evaluated by tests in the CI system;
• Well documented;
• Code approved by at least two team members, merge into master branch;
• Ticket closed;
Design and develop a fuzzer framework to security test a Cloud based application
8
3. Cloud based web-applications Modern software architectures, designs, and implementations shape the market through the
demand they generate. Server providers try to pick up the pace to adjust their supply to fit it
to the new demand. The Continuous Delivery and Integration and the Agile software
development as well as the economics and society’s multiplicators, the demand for the
novelty generates the evolution from the view of demand and supply on the market of the
server providers. This is a powerful breeding ground for technology to evolve. The field of
on promise systems slowly narrows. If a company wants to participate in the competition, it
will do better to migrate the infrastructure into a highly available, scalable cloud basis. The
best technology to achieve this aim is Cloud technology.
3.1 Well architected framework The AWS (Amazon Web Services) allows everyone to access their documentation about how
to design a well-architected framework if a reliable Cloud infrastructure is a need.
It defines the fundamentals of a Cloud-based infrastructure. One of the main pillars is the
infrastructure as code which is a principle about how sysadmins and developers should
handle the infrastructure. They recommend that every piece of the infrastructure must be
written in code, and every simple modification on it must be through code. The infrastructure
code has to be version controlled. To achieve this principle, there is a powerful tool called
CloudFormation, which is a type of language developed by AWS to provide the ability for
developers to write down the infrastructure as code. Another recommendation is to document
the code to make it more understandable for others. Worth noting here too the small extent
of steps to take when planning to make any modification on the infrastructure. Rollback is
easier. [4]
Why do these recommendations are so important when we talk about a well architected
framework? Because these recommendations are providing the basis and a design pattern to
achieve a trustworthy Cloud infrastructure that is maintainable in any cases.
Design and develop a fuzzer framework to security test a Cloud based application
9
3.2 Abstraction of a distributed system In IBM wording, the distributed system is a bunch of independent computers deployed with
independent software-components on them, which seems for the outsiders as one system.
Machines and components are interacting with each other on the network they operate. [5]
Distributed system’s greatest advantage is scalability, redundancy, reliability. Cloud systems
mostly scale in the vertical direction. Vertical scalability is an approach to involve resources
if load rise. For instance, a system stands about ten virtual machines which serve the clients,
horizontal scalability means that if peak arises up, the system utilizes more instances to
maintain the service and bear the load. Before the Cloud technology, engineers used to
design vertical scalability, which means using greater virtual machines or bare metals instead
of small ones to maintain the service under higher load. [6]
Redundancy is an option provided by Cloud services, of course not mandatory but highly
recommended to use in the design. In practice, AWS provides “MultiAZ” which is a multiple
availability zones inside a region. Mostly every region has a minimum of 3 “AZ”, but there
are zones with much more “AZ” equipped. The architect’s responsibility is whether to
implement them or not. Another feature is the recoverability and reliability. AWS has got a
term called “shared responsibility” which means that they are in charge of the underlying
infrastructure, mostly the border is the hypervisor if thinking in virtual machine terminology,
from the networking view this border is the Virtual Private Cloud internet gateway. [7]
It is a general problem for developers to solve and provide communication between services
and maintain data persistency across the Cloud. Service discovery protocol is to manage the
communication and interaction between components. It could be a Load Balancer, or/and a
bunch of DNS record zone to be able to route the appropriate traffic from producer to
consumer. Databases and transaction systems, in-memory databases are technologies whit
developers able to maintain data consistency. The messaging system is in charge of manage
to handle the message traffic between services acting like an Inter-Process Communication
on the network.
3.2.1 Communication between components An interesting fact that the first service developed by Amazon was the SQS. This highlights
that the most demanded Cloud service is the communication between services. The Simple
Design and develop a fuzzer framework to security test a Cloud based application
10
Queue Service is a message queuing service that enable developers to send, store and receive
messages across distributed systems and let them to focus on the workload. [8]
Components can send SQS messages which act as a pipe. On one side, the producer puts the
message on it, and on the other side, the consumer removes it. In steps:
1. X component puts its message into the message provider, at this time a counter starts
which measure the elapsed time;
2. Y component removes the message from the queue and reads its content. Another
counter starts to maintain the elapsed time from the reading;
3. Y component process the message then removes it permanently from the queue;
a. There is a Dead Letter Queue implemented on every SQS to store the
messages which is defined in the cloud configuration. The default value of
the amount of message retention is 14 days.
Developers can define the processing order logic according to the order of the incoming
messages, which could be for instance, First in First Out model. The whole SQS has got an
API and SDK toolset to develop feature-rich applications.
3.3 Definition of Cloud On-premise systems are mostly owned by companies to serve their services all over the
world. If a company decided to step into a new market on the Globe or the demand starts to
grow for their service, they should upgrade and scale they datacenters or should build a new
one, which has got a huge impact on their costs. While sysadmins can keep in touch
physically with their infrastructure and hold the responsibility to operates it, then the Cloud
is provided by a third party who is responsible for all factors to assure the operation. The
Cloud is much more flexible than on-premise systems.
Design and develop a fuzzer framework to security test a Cloud based application
11
4. Testing in practice, introduction of test methods Heading to the implementation of fuzzer Framework, it is necessary to create an overview
of how testing looks like in practice and what type of methods available on the paleta of
testing.
4.1 Testing the functionality Functionality is why software birth, its aim, set of objectives, features. The function is the
co-ordination of interactor components, which are having objects communicating with each
other. How to behave an object is regulated by methods and functions. No matter which
software layer or component is the target of the inspection, those pieces must serve and
assure the functionality described in the specification. By the specification, the software
must output the expected response to the given input.
The smallest unit whereat a test can proceed is a function or method. The term came here,
unit testing. The unit test is explicit. As the process moves away from the lower levels to the
higher, the testing transforms into functional testing and be more implicit. [9]
Possible stages of preparing and running a test:
1. Declare functions, expectations;
2. Prepare input data set based on specification;
3. Define expected output according to specification;
4. Define test case(s), rather more case than less;
5. Run test case(s);
6. The compartment of the result and the expected result;
4.1.1 Unit test A unit test is about testing the building elements of a component which are classes, methods,
functions, interfaces.
Every attribution of a function is known by its API documentation or the document string,
generally known its parameter signature, their type, the return value, and its type. In practice,
Design and develop a fuzzer framework to security test a Cloud based application
12
a unit testing framework supports to manage the tests. Mostly a function handles and runs
the unit test of a target function called test case.:
def test_proto_path_config(self): proto_path = self.config.get_generic_proto_path() self.assertEqual(proto_path, '/framework/data/proto/')
In the Python unit test framework, an assertEqual method will compare the functions return
value with a given expected value if equal occurs, then the test passed. Of course, there is a
whole assertXY library for each unit test framework. It can test numeric, alphanumeric types,
booleans.
4.1.2 Component test
The component or module test is about to test an independent component itself. To make a
component test, it is necessary to know its interfaces and its use through the API
documentation. [10]
4.1.3 Smoke test During the CI/CD process, the smoke test provides the ability to immediately indicate if the
software functionality deviated because of the wrong code. It covers not the whole
functionality but the main ones and runs after the code compile in the Continuous Integration
system pipeline. If a code fails, the smoke test the CI pipeline must fail too, because of the
error that occurred in the code. [9]
4.1.4 Functional test The functional tests might run under the same circumstances as the production environment.
It must equal the production infrastructure on each level and aspect, which means if the
production environment is running in the Cloud, it is recommended to run those tests with
the same deployment. The intermediate solution could be a local environment equipped with
fake services which simulate the Cloud environment. Sometimes it takes more effort to
simulate locally.
Design and develop a fuzzer framework to security test a Cloud based application
13
The phase of functional testing the product must feature complete state. Or has features that
are currently under development and fast finished. By the API documentation, can a
functional test to design.
4.1.5 Mutation based testing With a specified testing tool, Quality Engineers can proceed a mutation-based test. For
instance, it mutates the operators and operands inside a decision branch. Its advantage is the
ability to evaluate the test cases written before because it will reveal conditions that were not
covered properly through the test case.
Assume that the code which is under in test has a method that passing every test case, and
the coverage report showing around 90% and everyone is happy with that. The next level of
Quality Assurance could be the question: Do we design the test case properly? The. Mutation
based tests will support to unmask the underlying logical failures.
function highTemp(temp) { if (temp > 1000) { return true; //too hot! } else { return false; //it’s a very nice… } }test(“test highTemp”) { expect(highTemp(500)).False(); expect(highTemp(1000)).False(); // this is the appropriate case } If the mutator starts to mutating the operators, there can be a state when it changes the > to
>=, but with the first case where the function got value 500 will not reveal the problem
because the case will always pass, in the second case the value is 1000 which can lead the
decision into problem if the mutator switch the operators to >= because the case will fail if
1000 >= 1000 with expectation of False result it will cause True. The conclusion is the
perfect parameter for this test case is the value on the level of 1000. [11]
4.1.6 System stress test This test is to measure the resistance of the system against a high load. It can reveal design
problems regarding component communication for instance locks, mutual exclusions,
database connectivity problems, worker process problems during high stress. Infrastructure
Design and develop a fuzzer framework to security test a Cloud based application
14
resistance information could extract too by stress test because revealing network-level
problems, file system issues. While the system is at the edge of the tolerance, latency
measurements, response time collection is also an objective. The reaction of Cloud services
such as Autoscaling according to CPU utilization or database connection or any composition
of them could be a basis of measurement.
4.1.7 Code coverage Code coverage is a tool to measure how perfectly is the coding unit tested. During the tests,
a third-party tool called mostly coverage maintains the process and measure which part of
the code was driven via the unit tests. After the test is succeed a coverage report will be
available to evaluate the degree of coverage. It depends on the Company’s policy where the
threshold line is defined that they could say the coverage is good enough. Coverage about
90% is a good one in Enterprise.
4.2 Nonfunctional testing Beyond the functional testing after convinced about the functionality, the glance is turning
to the system security. Security could be reached among others with the code quality, and a
nonstandard code can hold noxious calls, variables, memory management. Coding
conventions without caution of use could result unmaintained memory addresses. These and
many other faults could lead to code quality, which is presumably open for threats.
4.2.1 Security tests Penetration tests are trying to crack the system in no matter what way and does not care
about the weapon. It can proceed with system infrastructure and software as well. To
penetrate a system, the knowledge of network protocols is a must-have. Specialists and their
tools are known the methods of how to test a system. If a software system contains third-
party modules, it is indispensable to test them too. The objective from the operator’s side is
to maintain the running versions, upgrade them if new releases are released and patch them
in time.
Design and develop a fuzzer framework to security test a Cloud based application
15
4.2.2 Static code analysis Static code analysis is an approach of testing when the test evaluates the code with a toolset
that is made to recognize common errors, vulnerabilities, cognitive complexity, code
maintainability, nonstandard code usage, and another aspect. Some of these terms may need
an introduction, the first is the cognitive complexity, which is a way of measuring code
understandability, which occurs when the developer uses too many decision branches,
embedded scopes, iterations, which reduce the ability to understanding the code. [12] The
code smell means that how good the code is measured by a linter which is seeking for
nonstandard code usage and so on. One of these linters is Pylint, which will be in use during
the implementation by me. Another well-known is the Flake8 which is also a Python based
static code analyzer.
Most of these tools can be run locally by an IDE to get rid of failures as soon as possible,
but for enterprise projects, there are also solutions to automate them to doing analysis on a
huge amount of code lines. The maintainability means that how subtly divided the
component is. In practice, reusable code, small and clearly formulated methods, well divided
logic and structure can lead to a well-structured narrative which makes the code easy to
consume for humans.
Design and develop a fuzzer framework to security test a Cloud based application
16
5. Fuzz testing Fuzz testing is a nonfunctional test type mostly used to testing the system security, one of
the most powerful strategies for identifying security issues.
The fuzz (fəz) verb meaning is “make or become blurred or indistinct”. During a fuzz test,
the testing tool creates mutations of a known valid or a given input value and sends them in
the direction of the system through a specific channel. When fuzzing the system, the main
aim is to observe its response and behavior to explore hidden security failures or abnormal
program operation or system performance ruin.
The history of Fuzz testing started in the ‘50s, but the really first widespread method was
developed by Apple in 1988, which was called the Monkey test. The method based on the
theory if a monkey randomly starts hitting a computer’s keyboard after “t” time, there could
be a combination hits that possibly contains Shakespeare’s all works. [13]
The first Unix adaptation of the testing tool got the name Fuzz. It was able to create mutated
output into a file, then send that input for a specific target, which was mostly a CLI software.
For instance, if we grab the well-known “ls” CLI tool which has the default usage:
usage: ls [-@ABCFGHLOPRSTUWabcdefghiklmnopqrstuwx1%] [file ...]
With a fuzzer it is possible to fuzz its flags like “ls -%/+!+%”/=../../../ foo.bar” and so on as
it does not hit a state which causes an error in the software mechanism because of the lack
of exception handling or anything else.
As time passed it was used to test Graphical User Interfaces (GUI). In 2012 Google
introduced the Clusterfuzzer, which is a tool to testing Cloud based infrastructures. Earlier,
the AFL (American Fuzzy Loop) uses a generic algorithm. So, the Fuzz testing method is
based on that every software hides its failures, which are waiting just to explore them.
5.1 fuzzer types, methods, features
fuzzer types in terms of function and features:
• General or mutation-based: It creates its mutations from nothing by a given
specification or mutates a basic dataset given by the tester or the combination of both
approaches.
• Protocol based: beyond the ability to randomize the data, it is able to use protocols
such as TCP, HTTP.
Design and develop a fuzzer framework to security test a Cloud based application
17
• The most important feature is how the fuzzer acts whit the system, which depends on
whether it knows and controls the inside structure and functionality. If we know its
inner structure, then call it whitebox fuzzing. If no information about the system, then
it called blackbox fuzzing. The tool I will implement to is a blackbox fuzzer, but we
will see it has the ability to be a whitebox one.
By the whitebox approach, the tool has information about the systems inside structure. It
knows its inside functionality it can influence the system, for instance, observing services,
gather their log data, restarting daemons if necessary. Because of more information it has it
is able to send sophisticated mutations for the system.
By the blackbox approach, the fuzzer takes no care about the structure of the system, the
only aim is to send as many mutations as it can and take observations. In order to fuzz an
application is it has to know the application programming interface.
Assume a system with a REST API over HTTP, which described by the API documentation
as it follows: “somedomain.com/foo/bar/addItem” with HTTP POST method clients can
send data in JSON format “{“item_id” : 1234}”, the item_id variable is an integer type
in the system according to the documentation but we have no information about whether it
is an unsigned 16 bit integer or a signed 32 bit integer and of course we have no information
about what happens if the client send for instance “-2343535345343” or
“2.2342424243234”. The question is, did the developers implemented a mechanism to
protect these variables? Indeed, basically, client has no idea what type of programming
language is being used on the server side. Fuzzer has to have the ability to bring to light
whether there is a mechanism implemented in the software to protect the variable or not. On
one hand, it could be but the other hand there could be a query which ruins the application,
which will initiate system exit in better case, the worst scenario could be an illegal SQL
query or a terminal access, of course not in this simple example but it could happen.
Steps of the Fuzz test:
• Identify and define the target system
• Define the communication protocol
• Define inputs according to the API documentation and system specification
• Generate data
Design and develop a fuzzer framework to security test a Cloud based application
18
• Run tests (Fuzz Data).
• Transfer cases to the target
• To observe the behavior of the system via its response or logs.
• Report problems if it occurs.
Design and develop a fuzzer framework to security test a Cloud based application
19
6. Research a fuzzer tool The open source society maintains several tools to run such tests on diverse programming
languages through standard protocols.
6.1 Considerations for selecting a fuzzer Before getting into the research, expectations against the fuzzer shall be defined.
Expectations based on the specification of the framework I will implement, which are from
the framework point of view, is the fuzzer must be platform-independent, provide a client –
server model, implements network protocols, implements an object-oriented approach
which allows building a framework around.
According to my research, basically, there are two main types of fuzzer out there according
to their use, the ones are out of the box solutions equipped with network protocols to test
network applications, or to fuzz desktop applications the second group is more plastic,
having the possibility of further development. Evident, that the first group is out of scope for
this project.
I have also identified features that are not yet important but would be in the future, for
instance, the ability to maintain a session between client and server. [14]
The requirements of the fuzzer summarized in the following table.
Modularity Each part of the fuzzer stands on its own, which means parts
are reusable. Extensibility If wanted to extend the fuzzer and would implement
something new, then it could be done without the touch of the core elements.
Rich data modeling It supports rich sets of datatypes at low and high level. Stateful Take care of the message order and able to maintain a session
while simulating the client or server role. Client–server model To have the ability to simulate server or client to fuzz client
or server application. Platform independent or cross-platform
Supports a language, which is platform-independent such as Python and the attribution of working in Linux environment.
Logging system Well architected logging module extended with a reporting tool.
Usability, maintainability
Open source fuzzer basis in Python language.
Table 1 fuzzer requirements
Design and develop a fuzzer framework to security test a Cloud based application
20
A further expectation is to be well documented and maintained actively. Because of the
Python 2.7 deprecation at the start of the year 2020, the Python 2.7 implementation is a strict
exclusion.
6.2.1 Kitty / Katnip fuzzer tool Kitty/Katnip is an open-source modular fuzzer, which was developed by Cisco. The core
module is the Kitty component, and the Katnip holds the modules implemented by society
such as file fuzzer, TCP fuzzer, UDP fuzzer. The aim of the develop of Kitty was to provide
a core component which is able to receive new type of targets and protocols.
According to its developers it is not an out of the box fuzzer. On its own, it cannot run fuzz
tests as long as the user does not set it up or implements its own solution. Kitty provides only
the mutation mechanism to creating data to send in the direction of the target, a template
system with a bunch of fields, monitoring system, and controller system to control the target
during a whitebox fuzzing. The components and classes are well divided; each functionality
border is easy to recognize and easy to build up a case. Great advantage is the Katnip, which
is another layer on top of Kitty to provide higher level solutions. [14]
Modularity Yes Extensibility Yes Rich data modeling Yes Stateful Yes Client-server model Yes Cross-platform Yes Logging system Yes Python3 Yes
Table 2 Features of Kitty fuzzer
6.2.2 Sulley fuzzer tool Sulley is a fuzzer engine and a framework in one package it is extendible, and protocol-based
also written in Python. Unfortunately, it is not maintained for 6 years ago by the readme at
the repository page. The document recommends using the BooFuzz tool, which is actively
maintained. Why I mention Sulley despite it is not maintained anymore is the reason that it
was the one of the first fuzzer, which succeeds. The BooFuzz also mention it and inform the
visitor about that the BooFuzz wanted to “fuzz everything” as the ancestor did. [15]
It seems that they actively develop it, but checking the structure, I was not convinced that it
is obvious and comfortable to extend it with such a framework and toolset I have planned.
Design and develop a fuzzer framework to security test a Cloud based application
21
In practice, for instance, the target component charges for too many objectives which they
should not be in the same class. Another functions like the logging component I found it too
complex for a fuzzing and another example is the Connection class, which constructor has
more than 20 arguments, which I found too many to handle it easily.
Modularity Yes Extensibility No Rich data modeling No Stateful No Client-server model Yes Cross-platform Yes Logging system Yes Python 3 Yes
Table 3 Features of Sulley fuzzer
6.2.3 Peach fuzzer tool Maintained by Mozilla Security, and there is no link to the repository or any download link.
Any try leads to a contact form and to marketing. That means the mainline of Peach fuzzer
is a commercial product. But it has a fork named MozPeach with the goal to continue to
deliver Peach as open-source project for the community. They only just have to work on
Python 3 support and replacing deprecated python code. According to this, it is not a
welcome solution to my framework. Even if it already supporting Python 3 because of its
structural simplicity, which not fits to the objective. Getting into the code, it is too stiff to
cooperate with another components it is a good light-weight solution in its own. It uses XML
to describe the data model and have the term “Pit” to define how the fuzzer obey with the
target. [16]
Modularity No Extensibility No Rich data modeling No Stateful Yes Client-server model Yes Cross-platform No Logging system No Python 3 -
Table 4 Features of Peach fuzzer
It does not make sense to inspect deeper because of deprecated Python code.
Design and develop a fuzzer framework to security test a Cloud based application
22
6.2.4 Burp as a possible fuzzer Basically, the BurpSuite is a special Linux distribution installed whit a bunch of security
tools to test systems protection. Researchers and security professionals use it with a
predilection to test their systems or simulate a threat or during the process of research
standing in the middle of the threat and its command server to tap the traffic and grab the
communication that they do between each other. This method called Man in The Middle.
Burp has a subsystem that can act as an HTTP server or client, can terminate TLS, and able
to send any type of HTTP method. It can mutate those requests or responses too. I just
wanted to mention because it is a renowned tool but except for HTTP it is not a good starting
point. [17]
6.2.5 American Fuzzy Loop Maintained by Google is a free software tool to assist the detection of software bugs. It has
significantly another approach as the current objective needs. The AFL mostly is to test
binary. To achieve this, it is necessary to compile the binary with a special tool. There is no
way to implement this approach on a network based fuzzer framework but worth to mention
it because of its importance on binary testing. [18]
6.2.6 Protocol learning, simulation and stateful fuzzer The PULSAR was so far the embarrassingly special solution with its sophisticated network
protocol learning capability. The tool is capable of modeling a protocol through machine
learning techniques such as clustering and hidden Markov models. Really modern and
innovative solution. The program requires the user to provide a Wireshark .pcap file to
analyze. Those types of files are from Wireshark program after running a network session,
which is taped. After the protocol has been analyzed by the tool, it wanted to try to figure
out its rules. Unfortunately, it has a lot of dependencies and really hard to deploy and
configure, it not to mention the resources that machine learning algorithms needs. The
further problem is that it uses the “R” framework for statistical computing instead of
implements the Machine Learning algorithms via a Python framework such as TensorFlow.
Another problem is how the model can carry out the analysis of encrypted traffic because
the Wireshark .pcap file has not provide information about TLS communication except the
handshake, but that is come after it will be hiding because of the specialty of TLS. It is clear
Design and develop a fuzzer framework to security test a Cloud based application
23
that the TLS handshake is a part of the protocol but there could be special headers and service
messages after the handshake which are mainly encrypted.
The presence of deprecated Python2.7 code is shortcut every question described above.
Modularity No Extensibility No, but learning mode Rich data modeling No, but learning mode Stateful Yes Client-server model Yes Cross-platform Yes, but hard to deploy and configure Logging system Yes Python3 No
Table 51 Features of Protocol learning fuzzer
6.3 The chosen fuzzer After reviewing of each fuzzers codebase and architecture the best solution to implement a
fuzzer skeleton is the Kitty/Katnip because of the structure and relationship of modules, the
robustness, and the approach of how they interact and how easy to instrument a basic fuzzing
with them. This is the only fuzzer from the above which has modules that can overwrite and
could connect with the framework I will design. During the initial tryouts it turned out that
its parts are designed to be able to maintain a sustainable and long-term project with potential
of further feature developments. Its structure allows to act on several level. It provides the
core fuzzer module, a sophisticated Graph Model which orchestrates the Template on the top
of a sophisticated templating module, which can also be influenced through a framework.
Figure 1 Relation Between Modules, Source: Kitty documentation
The templating model stands from fields that could be low-level binary values or high-level
types. It is possible to create targets or derived from existing ones depending on the protocol
we want to fuzz. The controller module gives the opportunity to maintain the target server
or services as we would start to run whitebox fuzzing. It has a web interface, which has a
graphical surface to follow the process but easily can reach its methods to automate the
testing. The monitor can observe the victim’s behavior through its network traffic or memory
Design and develop a fuzzer framework to security test a Cloud based application
24
consumption. The Kitty was refactored in 2019 to get rid of deprecated Python2.7 code and
keep the project alive. [19]
Design and develop a fuzzer framework to security test a Cloud based application
25
7. Technologies involved During the development, I use a diverse technological environment that can be divided into
two parts. The first one is related to the target services on which the tests are performed. This
includes Cloud technology and network protocols. The second group is related to concrete
development, such as programming language, integrated development environment. And
there are overlapping technologies as well because inevitable to implement network
protocols or some Cloud services to manage to perform the tests.
7.1 Resources in use at Amazon Web Services I will introduce Amazon Web Services (AWS) resources that are being used related to the
objective.
There are services used by my employer with protocols, which are the DNS, HTTP, and
Protobuf over HTTP(s). These are basically held by EC2 instances, Lambdas, and Elastic
Container Service in the Cloud. During the development, I have to simulate these services
to support the development, and for this reason I have to create my mock services. Some of
them run at localhost and others in the Cloud. It was important to simulate to have all
properly implemented protocols prior to integration. I will introduce such technologies that
accessories such as API Gateways, CloudFormation, DynamoDB and so on.
7.1.1 Amazon Elastic Compute Cloud (EC2) – virtual machines The Amazon Elastic Compute Cloud service is a virtual machine service to utilize computing
resources to solve objectives depending on what the task is, which could be a simple web
application server or a machine learning optimized instance among others. AWS offers a
wide variety of operating systems including Linux distributions. These EC2 machines are
the same ones as we were used to VMware. They have launch templates with definition
whom to utilize them. The autoscaling group with scaling policies can regulate the number
of instances in operation. Several types of Load Balancer are available to balance the load
between virtual machines depending on the need of the application and its protocol. In that
system that is being tested, there are bunch of machines with different roles regulated by the
autoscaling group. Tests include internal and external communication.
Design and develop a fuzzer framework to security test a Cloud based application
26
7.1.2 Networking in nutshell at AWS The basic building block of an AWS network is the Private Virtual Cloud (VPC) for which
the AWS is responsible, and this is true all the services provided by AWS. It is up to the
engineer and of course, the market they would serve to that into which region they deploy
the application. AWS provides several regions around the Globe; regions have Availability
Zones (AZ), mostly three AZ per region. It is recommended to take advantage to utilizing all
AZ to achieve high availability. The AWS also provides subnets per AZ, Load Balancers and
gateways are available too. As we have become accustomed to in on-premise systems, it is
possible to form network security with ingress and egress roles, access control lists via
security groups. API Gateways are for applications, which are implementing REST. It
supports containerized or serverless workload and EC2 as well. It handles all the incoming
requests and transfers it after authentication to the appropriate resource. Route53 provides
the domain naming system across the Cloud in cooperation with CloudFront, which is a
service on the edge of the Cloud and its responsibility is to transfer traffic to the appropriate
endpoint. [20]
7.1.3 Serverless technology It is a possible scenario that one of the development teams will perform a fuzz test on
Lambda based service, but the main reason is why I deployed one is to have a system with
HTTP(s) with REST API to try out the fuzzer.
The deployed Lambdas per REST function are provided by AWS via a preconfigured
CloudFormation template, which is a service-specific language to deploy and maintain
infrastructure as code written in JSON or YAML format. Worth to note that is available with
Figure 2 Lambdas per REST function, Source: Aws account
Design and develop a fuzzer framework to security test a Cloud based application
27
the CDK in Python, which is a more developer-friendly way to handle infrastructure as code.
But after all, the Lambda is a serverless technology where developers don’t need to maintain
infrastructure, the underlying system is provided by AWS. Our only responsibility is to write
the handlers. They utilize just the case when a trigger comes from a direction, which could
be a Cloud Watch event, a Simple Notification Service notification, or a Simple Queue
Service message. After the trigger triggers the Lambda function, it does its task then go back
to the idle state. There is no networking and no server with Operating System.
In this task, I use them with API Gateway which was also a part of the sample template
provided by AWS. That configuration allows me to run tryouts with the fuzzer framework.
It is allowed to perform any test on an account and AWS infrastructure what we own
according to AWS policy. Obvious was in no way flooded.
7.1.4 Databases, RDS, DynamoDB There is a way to set up and scale relational databases or non-relational databases in the
Cloud, depending on the data structure designed by architects. RDS is mostly server-based
and supports MySQL, Postgre, Oracle, MariaDB. DynamoDB is a key-value store and
supports the document database as well. Both of them support replication between zones.
The Lambdas that have been deployed put the data they process into Dynamo tables (Figure
3.). [21][22]
7.1.5 Simple Storage Service To maintain objects in the Cloud ,AWS developed the service called Simple Storage Service
(S3), and regardless to the other services, it is region independent because of every S3 bucket
name is basically a domain name. Buckets are the storage where applications can upload
objects. It maintains object retention policy, access control, and versioning of objects is an
Figure 3 Lambda Table items, Source: Aws account
Design and develop a fuzzer framework to security test a Cloud based application
28
advantage too. The maximum size of an object to upload in one session is 4 Terra Bytes.
[23]
The fuzzer framework system use this service to archive fuzz session data (Figure 4.).
7.1.6 How to control AWS resources The AWS resource control use cases need to be distinguished depending on the goal. A
resource can be creating or modify via the web console, and it is recommended to do massive
deployment with CloudFormation which can perform with CLI tool. There could be cases
when automating something than in a shell script more comfortable to use aws-cli tool. AWS
SDK and Boto library giving the ability to control resources in a programmatic way. In this
context, the term control could be meaning a deployment or a resource configuration or
interact with resources inside an application. During the work, I use aws-cli which has been
configured to the Frankfurt region (eu-central-1) with access key and secret key, to control
S3 bucket creation and object upload inside from Python script it is managed by Boto 3
library. [24]
7.2 Development environment This subchapter is about what type of developer tools has been involved during the
development. It describes the programming language and other related technologies such as
IDE, data-interchange formats, libraries, code analyzers, virtual environments.
7.2.1 The programming language To develop the framework, it was clear to use Python3.7 because it is an industrial standard
at my company, and it was a restriction in the specification to use Python. There is no extra
expectation about the speed, so it is not necessary to implement, for instance, in C++. The
Python community is big enough to find well described tutorials and necessary module
Figure 4 Objects listed in S3 bucket, Source: Aws account
Design and develop a fuzzer framework to security test a Cloud based application
29
documentation if needed. Of course, this was the language in what I already have small
experience and have the willingness to extend my knowledge.
7.2.3 Project dependencies Python dependencies can be managed by several tools. The most common is the PIP, which
I also use during the project. The dependency list is maintained by the requirements.txt. The
method to create one is a really basic step, just run the command:
“pip freeze > requirements.txt”
– then it creates the dependency list with versions.
7.2.4 Integrated Development Environment The environment I was set up to work is containing the following. The Operating System is
OSX 10.15.2 x86_64-apple-darwin19.0 kernel with shell Zsh 5.7.1. For code editing, I prefer
PyCharm. Python version is 3.7.1 wrapped into a virtual environment. For running unit tests,
I use TOX ,which is a virtual environment builder, and it is able to bootstrap as many Python
environments as configured to evaluate the code on several environments. To analyze code,
I installed Pylint and Flake8 into the virtualenv an as well into the PyCharm to get rid of
most of the code lint errors during the coding to avoid pushing wrong quality code into the
repository system. To double-check the code, I prefer SonarQube (Figure 5.) which I run in
a Docker container on loclahost and periodically performing scans on project level.
Figure 5 SonarQube code quality measurements about Fuzzer Framework
Design and develop a fuzzer framework to security test a Cloud based application
30
Python coverage tool is periodically checking how perfect is the code coverage by unit tests.
An excerpt of the current coverage can see at Appendix 1. For version controlling on
localhost there is a git client, which has remote endpoint at GitHub.
I prefer using git branching, so I have a branching model for this task which is not so
sophisticated but conductive. There is a master branch, which I maintain and do not allowed
to push code straight into the master branch, this is for avoiding accidental code push into
master. Each component has its own branch, and after the feature is completed then it is
available for merging back into master.
Project packages handled by pip and dependencies are under the project root in
requirements.txt file. Because of the Kitty concept, it is no need to touch the Kitty core code
however the Katnip module necessary to deploy in developer mode using the “-e” (editable)
flag while installing it with the PIP packaging tool. It allows me to edit and extend its
components locally.
7.3. Further technologies involved Some specific services I use to manage it with Docker as I mentioned earlier. Docker is a
containerization mechanism to run processes independently on the operating system. It was
designed to run microservices. It is a great advantage that it is able to run any service under
a time of an average git clone or any download consume. The image can be pulled from the
docker repository than deploy itself into the container.
7.3.1 Docker container In practice the Redis container deployment described by a docker-compose.yml:
version: '2' services: redis: image: 'bitnami/redis:5.0' environment: # ALLOW_EMPTY_PASSWORD is recommended only for development. - ALLOW_EMPTY_PASSWORD=yes # REDIS_DISABLE_COMMANDS=FLUSHDB,FLUSHALL ports: - '6379:6379' volumes:
Design and develop a fuzzer framework to security test a Cloud based application
31
- 'redis_data:/bitnami/redis/data' volumes: redis_data: driver: local%
Then the only thing is to do is to run the command:
docker-compose up -d
After this step, the deployment starts it downloads the Redis docker images from the
maintainer’s repository, then start the container.
7.3.2 Redis in-memory database
The Redis is an in-memory database and message broker that is designed to high-
performance data storing. In this project, I use it to combine the Python basic data structures
with it for experimental purposes. Its key/value approach is similar to Python’s dictionary
idea.
In this application the HASH type was implemented to put the Protobuf API module name
into memory. This information gathered from the pb2_api Protobuf descriptor Python
object.
HASH types are key/value pairs and allows the developer to assign collections to the HASH
name. To explain its simplicity let’s have a look at a HAHS creation when a meta_data
object, which has key/value pairs: meta_data = {
"time_of_analyze": "2020-03-08-02:51:47",
"module": "xxrrr_pb2.py",
"UUID": "5dac2afe-60df-11ea-8a13-a45e60d02ac5",
"protobuf_syntax": "proto3",
"protobuf_package": "test_proto_msg"
}
Being put into the Redis memory as a HASH type. Where the first parameter of the hmset is
the name of the HASH, the second parameter is the object to refer with the name. r.hmset('some-id', meta_data)
True
Can be called up at any time: r.hgetall('some-id')
Design and develop a fuzzer framework to security test a Cloud based application
32
{'time_of_analyze': '2020-03-08-02:51:47', 'module': 'xxrrr_pb2.py',
'UUID': '5dac2afe-60df-11ea-8a13-a45e60d02ac5', 'protobuf_syntax':
'proto3', 'protobuf_package': 'test_proto_msg'}
The main benefit of the in-memory server is that the memory access is significantly faster
than a disk performance, it is allowing the application in the future to expand performance.
7.3.3 Google Protocol Buffer
The Protocol Buffer is developed by Google to efficiently and securely transfer data between
services. Currently it has two live version the Protobuf version 2 and Protobuf version 3.
Efficiently means that it could be in some circumstances faster than the JSON or XML packed
data transfer during HTTP request because of a Protobuf compiled messages in binary form,
definitely it allocates less space. Securely means that the binary form of data sent on the
network could be harder to decode if someone tapes the communication despite it provide
some extra security if sending it on an unencrypted channel that can carry the risk that
someone catches the payload and crack it. Highly recommended to use encrypted
communication even if the message is a compiled binary, there are tools in the GitHub
universe to decipher Protobuf binary message. On the other hand, security means that
debugging such communication could be circumstantial or difficult compared to a
conventional payload like JSON or XML.
It uses messages to transform the payload into Protobuf binary. To create the messages, it
uses a compiled API to pair the appropriate field with the corresponding value, then it creates
a binary form of it. The protocol buffer raw file must be compiled by the Protobuf compiler
to produce an output that gives the ability to access the API.
Designing an interpretative which is able to understand any Protobuf message structure
without prior knowledge of API in runtime is such a big challenge. During the work the
Google factory functions supports my work to iterate through for each message and its
accessary. The most useful tool is the DESCRIPTOR it helps to construct a mechanism that
attached to a message object can provide information about. Another gorgeous feature of
Python is the getattr() method which allows to construct a method call from mix of objects
and strings.
In under average circumstances, the Protobuf API have been imported during the work while
the code birth and the developer knowing everything about including the message types,
Design and develop a fuzzer framework to security test a Cloud based application
33
nested messages, fields, field types, he can decide easily whether a message or field is
optional, required or repeated one. All of this information has been provided to manage the
development process without any interrupt. In my case, the API is appearing after the fuzzer
have been launched, in runtime, which means I have to deploy several techniques to grab the
information that is mandatory to have a fuzz session with Protobuf message. The effort that
I have to invest is greater but in long term there will be a generic logic which is able to handle
Protobuf messages.
Design and develop a fuzzer framework to security test a Cloud based application
34
8. Software specification The initial software specification was not so detailed it was mostly equal with the task
description, which can be found on page 7. , that is about to design and develop a fuzzer
framework which is able to fuzz HTTP(s), Protobuf, DNS requests. Mostly it was a sketch
about how the configuration file should look like and what kind of parameters should it
contain. Actually, it is a high-level requirement definition and gives me a starting point
because the configuration sketch defines the expected functions. There was not any other
restriction regarding the technology except the programming language.
8.1 Functional requirements The functional requirements are based on the received configuration sketch what
transforming into a requirement list it contains the followings:
1. Generic attributes:
a. The application is able to run a complete fuzz test (DNS, or HTTP, or Proto
Buf) based on the provided configuration parameters.
b. The application must be able to run in verbosity mode to assist to debug;
c. Must be able to handle the target hostname;
d. Must be able to handle protocols depending on the target;
e. A noncritical requirement is to handle the TLS connection. The main reason
for this is that during the tests it is not necessary to use encryption because,
in this case, the services communicating with each other on the intranet.
Nevertheless, if someone wants to test right in the cloud, it is a must-have to
use TLS;
f. The application must be able to archive test results as artifacts. In this point
there were no more restriction or specification;
2. User must be able to select thru the configuration to on which protocol want to run
tests;
3. Protobuf type specification:
a. User be able to place compiled pb2 files into the project directory, and the
application must be able to decompile them, and there must be a mechanism
which decides the structures and types of the Protobuf API;
Design and develop a fuzzer framework to security test a Cloud based application
35
b. Preparation for the further Protobuf compile, currently not necessary;
c. Support the logic with further information in the configuration file is allowed.
For instance, restrict which classes have to be tested from the API, or provide
extra information which are cannot gather by decompiler;
d. Values to fuzz provided by the fuzzer except Protobuf version 2 default
values;
4. DNS type specification:
a. The ability to provide the TLD;
b. A field to define domain labels which the application will fuzz;
c. A timeout about how many seconds the fuzzer should wait for the response;
d. DNS record types: A, NS, TXT;
5. HTTP type specification:
a. The application must able to send HTTP requests with the following methods:
GET, POST. For further prepare the ability for DELETE, HEAD;
b. A field for the query string or REST path;
c. Provide a field and mechanism through the templating system to be able to
fuzz not only the query string or rest path or the payload but the URL too;
d. Provide a field for the content type description and the payload;
Restrictions regarding the subject of the fuzzed target are that never fuzz the target host and
port. The application must be able to integrate into a CI system that must be able to control
it via Groovy language or shell script.
8.2 Nonfunctional requirements There were no extra nonfunctional requirements, but in anticipation, it is a must-have that
the application could run in a virtual machine environment with a normal resource
consumption to provide the stable operation and not to overload the node which serves its
functions.
Design and develop a fuzzer framework to security test a Cloud based application
36
9. Software design In this chapter based on the specification, and on the Kitty fuzzer I will introduce the
designing steps. I will describe each component, their relation, and their communication and
functions. The first step is to create a “big picture” about the application to see how the
components interact to proceed with the goal. After that, I will introduce each component on
its level.
9.1 Software architecture
To design this specific application, it is an important to find connection points in the Kitty
API where or with the framework can interact with. Those points are identifying the basic
architecture of the application. So, I have to adjust the components and their behavior to
work with the Kitty API. To achieve this, it is a must-have to implement object-oriented
programming and with its use compose the application.
Regarding the specification the main “big picture” to start the design procedure as is follows:
• Available (third_party) components:
o Kitty core component takes care of the fuzz mechanism;
o Katnip component as it is extended with:
§ Protobuf target;
§ DNS target;
§ HTTP target;
• Components needs to be implemented:
o A runner that instruments the components depending on the protocol type.
o The core component holds the base object, which is the ancestor of objects,
every other inherited from this;
o Models to describe or hold data structures:
§ Maintain configuration file;
§ Maintain and hold the Protobuf API files;
o Services to do several tasks:
§ Configuration parser class;
§ Protobuf API parser class;
Design and develop a fuzzer framework to security test a Cloud based application
37
§ Protobuf message validator class;
§ File search class;
§ Protobuf compiler class;
§ Redis client class to maintain in-memory database related methods in
one place;
§ The artifact archiver class;
o Runners who orchestrate the fuzz test on each protocol:
§ HTTP runner class;
§ Protobuf runner class;
§ DNS runner class;
o Controllers to control a specific target. Not implemented in this version:
§ HTTP controller class;
§ Protobuf controller class;
§ DNS runner controller class;
o Reporter subsystem:
§ Creates a report for the Jenkins CI;
§ Creates a lazy report to the standard output (stdout);
o Utility component:
§ Helper class and methods;
The above order almost defines as well the project structure on the file system level.
The cloud_configs directory holds mock servers which I cloned from several maintainers at
GitHub for server simulation. Some of them I extended with features, for instance, with
Protobuf message processing received via HTTP POST message.
In the following subchapters I will describe each component and its classes and the features
they serve. Before I am hitting into the negotiation for each component, here comes a
common step to describe the relationship thru a relation model.
The terms used by the application can be found at Table 1. To design the Protobuf parser
and the message_skeleton it is necessary to create rules and concepts to handle the logic and
assure that the components speak the same language. The implemented logic can work along
these lines.
Design and develop a fuzzer framework to security test a Cloud based application
38
Table 1 Terms
Name Description Protobuf API Refers to a compiled *_pb2.py file that implements the messages, and fields defined in
the raw .proto file. Message skeleton
It is a digestible and simplified form of a Protobuf API for the fuzzer produced by the message validator component.
pb2_api Refers to the object that holds information about the Protobuf API after the pb2_parser component decompiled it;
Info string The term represents a Protobuf message and field hierarchy: __marker__.package.FooClass.FooMessage.FooField
__marker__ Help to decide from the actual level whether it is an optional, repeated, or nested. Optional obviously not mandatory, repeated indicates messages that can occur many times with no limitation.
9.1.1 Kitty core The Kitty core component provides the basic mutation mechanism to create mutated data
from zero or based on the user provided default values through its low-level field mutator
then the high-level Graph Model. With the Graph Model, it can manage by the programmatic
way how the fuzzer iterate over the templates and sub-templates, which one should be the
first and which one follows which. Worth to note, the templating system and the Graph
Model have a lot of potentials. Depending by the structure of the protocol or the target
system it allows to form a template no matter how complex it is.
Because all the components already have been described in an earlier chapter (6.3) it is
enough to introduce how the application will use the Kitty as a module and what type of
components will be in use generally. Each use will be explained in each component’s
description in the following subchapters.
To install Kitty, it is a pretty easy process it needs only a command via PIP packaging
manager: pip install kitty
After Kitty is installed, it can call in any component with the Pythonic method, for instance: from kitty.model.low_level.field import String
It will import the String field type. Fields are the atomic building blocks of a template, and
they can represent strings, numbers, integers, unsigned integers, signed integers in several
lengths. The application will use several components from the Kitty core, but it will do it
mostly through the Katnip, which is basically a toolset written over the Kitty core to
implement several targets and fields by default. This will be one of the connection points to
combine the framework application and the fuzzer tool.
Design and develop a fuzzer framework to security test a Cloud based application
39
In high-level abstraction, according to Katnip API documentation to build a fuzzer it should
be the next components to have:
• Template which describes a message that the accepted by the target;
• Target class, which will the fuzzed message actually transfer to the target (in our case
targets are mostly hosts with an interface, and a protocol);
• A controller if the fuzzing is a whitebox type and it can access the fuzzed target
service;
• A GraphModell which will traverse each sub template if the template is that complex;
• A fuzzer instance, which transfer the control to the core component and does the rest
of the job such as producing the mutations described by the template. So, the core
component provides the basic mechanism to build up a complete fuzzer.
• It has got an interface which is a common browser GUI to visualize the actual steps;
• The latest method that needs to be invoked is in charge to start the fuzz session;
All right, this was a basic fuzzer construction, this is what the framework application what I
develop wants to use to achieve the DNS, HTTP and PROTOBUF fuzzing. In my approach,
the pattern is fast similar as described by the API documentation, but in a bit more
sophisticated way because it wants to implement several protocols, each protocol should
have a well separated class. Other utilities regarding to artifact production and the separated
logging of the framework also needs individual classes. It will have a configuration, possibly
a data generator in the future and so on. It is an existing aim to develop this application from
year to year and serve feature requests that I receive. So, it really needs a framework.
9.1.2 Katnip component The Katnip installation is a bit different from the way of how Kitty was installed because its
targeting component expands with new target types during the development, so I have to
have access to its components at project level. In such a case, one of the most practical ways
to develop an active component is to install it editable. Katnip already has a TCP target
module that performs TCP communication over sockets, and it has a UDP target that does
the same mechanism but on UDP channel. Technically while coding the UDP or TCP there
is no need of extra effort from the developer except that simple definition on the socket to
use user datagram protocol while peers are communicating. Both targets are derived from
Design and develop a fuzzer framework to security test a Cloud based application
40
the Kitty ServerTarget class, and they act like a blueprint to rule the communication protocol
definitely.
9.1.3 Framework core and the config parser The framework core stands from two file, one is the fuzz_object.py which implements the
basic fuzzerObject class. This is the base class, most of the others are derived from this. The
framework core follows the Kitty object design pattern and it is an absolute reference of it
except the Redis which is a parameter in the fuzzerObject’s constructor, thereby the
StrictRedis client object is available to serve in aa early step. The fuzzerObjet class is also
responsible for managing the separate logging from Kitty’s logger to avoid interflowing with
it. This achieves that the fuzz session runs separated logging regarding to the tests and the
framework related operations.
The config parser uses one independent class, which is the GenerateUUID that implements
a very simple method to generates a unique ID for each fuzz test session. This unique ID
appears in system-wide and its objective is to assure the identification of each session.
The ConfigParser class is derived from the FuzzObject, and it is in charge of making the
configuration parameters accessible to the appropriate components via its class methods. To
avoid the spreading of multiple configuration objects in the memory of course with the same
data, that is does not make sense, it uses a toolset from a common class which is basically a
decorator function that prevents the unnecessary object creation called singleton. The
singleton is an object which exists in only one instance no matter how many time a function
call it. The tool can be found under the folder “utils” called decorators.py. [25] As the code
snippet showing the singleton decorator on Figure 6.
Design and develop a fuzzer framework to security test a Cloud based application
41
is a function which is wrapped into another function with the support of functools library
that returns a decorator and invoke the wrapped function which is decorated and returning
with the wrapped function. Technically, it evaluates whether the wrapped function has a live
reference to a live object or not.
9.1.4 Protobuf parser Definitely, the ParsePb class is one of the most complex ones because of the logic it
represents to decompile and preprocess the given Protobuf API which will after the
procedure more suitable for further processing. The idea is realized through several steps. It
is an important thing to control the dataflow simply because the fuzzer, during the start of
the application, initially, it has no information regarding to the complexity of the given
Protobuf API that provided by the user, during the process it must be ensured about its
structure. No matter how difficult the complexity is by the Protobuf API, the logic must be
able to parse it. That’s why I found that processing in more steps the Protobuf API is a good
approach. The pb2_parser class aims to proceed with the first step.
Figure 6 Singleton decorator accroding to RealPython's tutorial
Design and develop a fuzzer framework to security test a Cloud based application
42
The raw format of the Protobuf is a file with the .proto extension. A raw Protobuf version 3
file on the Figure 7. showing its syntax. At first glance, the data language rules are clearly
visible.
By its structure, it implements messages. A message stands from fields, a field can be any
well-known type such as INTEGER, STRING, or special one the ENUM, which is a list of
possible enumerated values. It is possible to use nested messages from the actual Protobuf
file or import other packages through messages and calling them whenever it needed. A field
type can be a message type too, which was defined earlier in the raw Protobuf file.
The ParsePb class reads and processes the compiled Protobuf and its power is that it can
read and decompile those APIs without any intervention or outside help. An excerpt from a
compiled Protobuf version 2 file on Figure 7. Depending on the Protobuf version, there are
differences that need to be taken care of during the design. Protobuf version 2 supports
default values while this feature was deprecated in version 3. Extensions are also
unsupported by the newer version; they were let the developers to declare a range of fields
Figure 7 Raw Protobuf 3 file - source Google
Design and develop a fuzzer framework to security test a Cloud based application
43
that are maintained for third-party extensions. Basically, it is a placeholder for external
messages. The Oneof type also a difference that makes sense. It provides a mechanism when
a message having many optional fields, but only one allowed to reserve the field, then the
Oneof enforce the behavior and apply only one message for the field. Setting a member of a
Oneof will drop all other previously added members.
Switching to Protobuf version 3 things are getting simpler, nevertheless, the logic must be
able to handle both.
The ParsePb has also been derived from the FuzzObject, and it uses the CheckProtoEnv
class which is the part of the Utilities component and its objective is to check whether the
environment is Protobuf capable or not. Through the Google Protobuf library, it consumes
compiled Protobuf API files and produces JSON output for the MessageValidator class. It
loads the Protobuf API module in runtime because the logic provides a generic way to read
and understand the API content. It needs only the path where the compiled Protobuf files can
be found. It puts the produced API structure in the memory as an object, and as well as JSON
file under <project_root>/bin/pb2_api_json path. This structure will be the blueprint for
the MessageValidator class to inspect and reveal the dependencies and hierarchy inside the
Protobuf API file.
Figure 8 Excerpt of a Compiled Protobuf 2 file, source Google
Design and develop a fuzzer framework to security test a Cloud based application
44
To crawl the information from the API the logic have to do the followings:
- Import *_pb2.py in runtime;
- Instantiate the component with an instance variable;
- Starting a loop to check after every possible message or field type occurrence:
o foo_object.DESCRIPTOR.message_types_by_name.keys() – will
returns between one and ‘N’ an object list that will show me the top-level
messages in the API
o Then I have to inspect for each message whether they have any other sub
message, or nested message, or fields, if fields occurs, what types they are?
o The logic should instantiate them in runtime as:
§ foo_message = getattr(foo_object, ‘Message_Name’)
o Then construct the descriptor again by calling foo_message.DESCRIPTOR.nested_types_by_name.keys()
o Repeat until more descendants can be found.
- A further objective is to maintain their relationship and find them a structure that fits
and provides a practical way to deal with them. To construct and pack the blueprint
of the API for the fuzzer to generate the proper data into the appropriate place is also
a major objective.
The greatest challenge regarding to this component is that the Protobuf API file parsing is
not as clear as ,for instance, parsing a JSON or XML structure. It has currently a deficiency
Figure 9 JSON file from the Protobuf API
Design and develop a fuzzer framework to security test a Cloud based application
45
that requires to provide ENUM relation via configuration to connect ENUM fields to the
appropriate message or field. An excerpt of the JSON type Protobuf API description file is
on the Figure 9.
It is true that the Google provide a MessageToJson function to get JSON format data from a
Protobuf message. But it is work only if the message itself is available in any form. In our
case, initially the application has no idea regarding to the message content or its structure.
In under normal circumstances an application applies the Protobuf as a whitebox, well
known component which is clearly defined across the components, and mostly it does not
change without any preliminary indication.
To reach out to the objectives I used to manage the problems with various toolset, to autopsy
a components structure, its classes, or methods in runtime, I voted to the Inspect library.
After the application able to get the first level classes, which named to ancestors or
top_levels I continue to go backward with collecting the method names from the API to
extract more information. Studying the Google’s relevant codes it was clear that the
application needs a mechanism which can iterate over a Protobuf API and has the right
intelligence to decide whether the actually under investigation level is a class, or a
MessageDescriptor, or EnumDescriptor perhaps a field and if it is, what kind of data type
represent it.
The MessageDescriptor describes a message defined earlier in a raw .proto file. It can give
miscellaneous attributes about the message as well the ContainingType that tell us whether
the actual message has a nested one or not, or about the Oneof, field type, or further nested
messages inside in it. So, the underlaying mechanism of the Protobuf is the key to understand
it. [26]
9.1.5 Protobuf message validator The ValidateMessage class is derived from the RPutCollections class, which is a helper class
for the Redis operations, and it is derived from the FuzzObject class. It has two main
objectives; one is to create a skeleton for the fuzzer about the Protobuf messages and the
other one is to validate whether the Protobuf APi parsing was successful or not. It discovers
the relationships between each message depending on the level they can be found during this
procedure; it also instantiates the messages and their fields and loads them with dummy data.
At the end of the procedure there will be a skeleton string and a test Protobuf message. The
Design and develop a fuzzer framework to security test a Cloud based application
46
skeleton string gives the ability for the fuzzer to easily decide the order of the fields to fuzz
and its types. It is also an encouragement for the ProtobufTarget class to instantiate each
message and put the fuzzed data into the appropriate message field. The skeleton is basically
an ordered list with strings. Strings delimited with dots and it describes the Protobuf message
relation see on this excerpt:
testRequest=main_pb2.TestRequest
TestRequiredRequestData = testRequest.TestRequiredRequestData.add()
TestRequiredRequestData.a_req.TYPE_STRING
It’s clear that this is a more straightforward form of a Protobuf API compared to the one
created by the ParsePb class. At this point, if we observe each state of an initially compiled
Protobuf API related to this skeleton, I mean compare the Protobuf API to the JSON API
description and to the skeleton, it is evident that in this form the messages are reproducible
any time anywhere without conflict.
The objective was to design an intermediate object group that supports the pairing, sorting
data. Currently, Python data structures such as list, dictionary support to compute, and
maintain the hierarchy between each message, or class. The helper structures are: Group of
nested messages, group of optional messages, repeated, and enum’s are also grouped. Groups
are starting points for further subdivisions. Then the component maintains the ancestor,
which is mostly the top-level class it should carry on the preceding messages. During other
further steps it dived the structures onward per message type. For instance, there will be
nested_optional, nester_repeated, repeated_imported message, and there will be
repeated_nested, repeated_optional, and so on every combination mapped to assure the
hierarchy. During the skeleton creation, the component simultaneously validates its job by
creating Protobuf message, decode then encode it back. The message creation without
critical error confirms that the process was successful.
The component puts its logs into the framework log file and stdout as well it also gives us
the ability to run the session with verbosity mode to inspect errors if it occurs.
The component basically works form three data source, the first is the configuration file the
second is the Redis in-memory database where some of the Protobuf related attributes are
stored. It uses the JSON descriptor files and as well the Python object which describes the
Protobuf API. The ability to force the fuzzer to fuzz only one class at the top level era it is
Design and develop a fuzzer framework to security test a Cloud based application
47
possible because if someone wanted to blackbox fuzz an application with this protocol, it is
not necessary to take care of the response messages, so it is allowed to exclude classes from
the mechanism in certain conditions such as the response message.
9.1.6 Protobuf target The ProtobufTarget class provides functionalities that are essentials to proceed a Protobuf
fuzzing on a given target. First, it uses the message skeleton to create each Protobuf message
in runtime. It is possible because of Python’s getattr, setattr methods which are in use
application wide. It implements two generic function to handle method calling and attribute
setting. After the messages are instantiated, it pairs them with the fuzzed data, which are
provided by the ProtobufRunner class; at this point the application use the setattr Python
method to set the attribute of a function. Worth to note, it could be a real scenario that the
fuzzer core could send such a fuzzed data, which could rise a type error, so proper exception
handling is a must-have. After the message is available it encodes it to bytes.
To put the Protobuf message into the finish line the application constructs a HTTP POST
method message with the appropriate HTTP headers and put it on wire via socket.
To get the proper result, the session needs a response from the target, so the fuzzer initialized
with the state of expect response, which is basically an abstract method in the Katnip’s
ServerTarget class which also overwritten by the application because the sophisticated
failure reporting. So, the response receiving maintained by a function which reads the
response from the socket and transfer it to the transmit function where, according to the
expectations, functions are unpacking every HTTP response header seeking for HTTP 50x
and 40x response codes. If any of them occurs, a handler will create the appropriate report.
Those reports will be processed by the Reporter component.
It also implements supporter classes and methods to assist the unpack mechanism to decide
hierarchy described by the message_skeleton.
9.1.7 Protobuf controller The controller class for Protobuf basically done by the Kitty API and its representation is
almost equal as it describes to how to set up a controller. Currently it checks whether the
Design and develop a fuzzer framework to security test a Cloud based application
48
actual target is alive or not, but it has got no influence on it. It is only a preparation of
whitebox fuzzing in the future. It uses all the Kitty and Katnip components.
9.1.8 Protobuf runner The ProtobufRunner class is the point where the Kitty and the framework marriage has been
made and this statement is true for every upcoming runner. In one hand, information came
from the framework side and goes to the Kitty fuzzer templating system, goes to the Katnip
Protobuf target than the direction again to the framework application. The template
mechanism receives the skeleton and grabs the types that it starts to fuzz. It recognizes what
kind of data type should to fuzz. According to the structure of skeleton it creates a dot
delimited string for instance: 1234.foo.bar.bazz.123 and sends to the
ProtobufTarget which creates a Protobuf message from them. The mechanism runs as many
times as the template and the input data mutation number is. If there is a structure which
describes, for instance, 500 mutations, the mechanism will turn 500 times. During the
integration it is possible to fine tune the template mechanism and the GraphModel to achieve
the proper mutation combination.
The rest of the class is about the fuzz session handling according to the Kitty/Katnip API
documentation as it was described earlier. The ProtobufRunner class invoked by the main
Runner class according to the configuration definition.
9.1.9 HTTP target The HttpTarget Class inherited from the Kitty ServerTarget class, constructed according to
its API documentation, and is in charge to send HTTP requests in the direction of the given
target using sockets. It also maintains the responses and seeking for HTTP error codes in the
HTTP response header and reports them if an exception occurs. Exception in this
terminology means that any of the 50x or 40x response codes can be found in the HTTP
response header. Obviously that the reporting mechanism is able to report in several other
conditions, it is only up to the needs. Currently every target initiates a report if the payload
raises an exception during the send or receive procedure. The case mostly is type error or
Design and develop a fuzzer framework to security test a Cloud based application
49
wrong byte start wrapped into try/except/finnaly scope to log the type errors and avoiding
the termination.
9.1.10 HTTP controller The HttpControler class representation, and the existence of it is equal with the
ProtobufController class. It is a basis of a further whitebox fuzzing. Currently it checks the
given target HTTP server whether it is available or not.
9.1.11 HTTP runner
The HttpRunner class implements the HTTP GET and POST methods according to the HTTP
version 1.1. It is able to fuzz the HTTP server target via REST API or the conventional way
during a POST method with form data or using a query string during a GET method.
It has two main templates constructed in it, each of them defines on HTTP method. It decides
the method according to the configuration provided by the user. The main challenge in this
runner class is to define a proper HTTP header which is good and sophisticated enough for
a well-equipped webserver. The best service to measure its perfection is the AWS
CloudFront.
A well-formed HTTP GET header according to the HTTP standard can see on Figure10.
screenshot taken in Postman:
On each fuzzer template, the application has to implement these forms, taking care of every
detail, for instance, the two double new lines at the end of protocol header, or the first line
must exactly be in the same form and order and so on. To simulate the structure of any HTTP
method, the application uses the Kitty template and its fields. An excerpt of the template,
which builds the HTTP’s ‘GET / HTTP/1.1’ header field can see on the next code snippet
on Figure 11.:
Figure 10 HTTP GET header, Postman
Design and develop a fuzzer framework to security test a Cloud based application
50
Technically a field is a list of methods, and each method represents a segment of a line.
Depending of the type of the actual segment the application calls the function, which
provides the functionality to fuzz the data value in it. It is clear that it simulates the new line
at the end of the method definition.
It has the ability to fuzz the whole protocol instead of fuzzing only the given path or payload.
So, if the feature requested through the configuration, the fuzzer can fuzz the protocol related
fields too. This feature gives the ability to cover errors if the webserver binary has any hidden
software bug or there is a misconfiguration elsewhere or wrong server logic implemented if
someone use a self-made webserver. The templating mechanism has the ability to fine-tune
it during the integration because of its flexibility and easy adaptation.
Basically, it generates fuzzed data by the default values provided via configuration, these
values are the path and the payload.
9.1.12 DNS target The DnsTarget class is also based on a factory extension called UdpTarget which is
implemented by the fuzzer framework, and it can be found under the targets in the
framework. It is also derived from ServerTarget, which is a core Kitty class. The reason for
the existence of a DNS fuzzer is that the Cloud service has specific versions which are
providing information about threat objects via DNS query responses. These are mainly “A”,
“NS” and “TXT” records:
• “A” record responses IP address;
• “NS” responses the name server;
Figure 11 HTTP GET template header
Design and develop a fuzzer framework to security test a Cloud based application
51
• “TXT” responses text record;
The following Figure 12. shows the DNS records in AWS Route53:
The first difference regarding to a TCP based connection is that in this case the application
uses sockets with user datagram as it seems in the following code snippet on Figure 13.:
The class receives the fuzzed data from the DnsRunner via its templating mechanism, in this
case the fuzzer mutates only the domain labels which are came from the framework’s
configuration. To dominate the data stream, it is necessary to take care of exceptions because
the fuzzer core in some cases changing the default character coding from ‘UTF-8’ to ‘iso-
8859-1’ that immediately raises a UnicodeDecodeError with wrong byte position. It is
obvious that useful to handle these exceptions. To put under pressure the victim server and
observe how to behave for a query that might never exists in real-world scenario but worth
to know what happens across the borders.
Further challenge is to construct a DNS query according to the given record type. The record
type is also provided by the configuration file. In the response side the objective is to grab
the DNS response code to decide whether there was any problem or not. But let’s see a DNS
request header first on the following code snippet in Figure 14.:
Figure 13 Socket creation for UDP, Katnip original method
Figure 12 Route53 DNS records
Design and develop a fuzzer framework to security test a Cloud based application
52
And the query’s response message at Figure 15.:[27][28]
Figure 15 DNS Response message layers.
The DNS target contain the implementation of a DNS query that is following a low-level
approach. Knowing the fuzzer core behavior I considered that is important to not use factory
library (DnsLib) for composing DNS query messages instead I implement the query from
layer to layer and concatenate the whole question message.
It is important that the fuzzer must fuzz only the domain labels and never fuzz the TLD and
the domain name:
• Domain name: somedomain; TLD: com
o Never fuzz: somedomain.com
Any preceding domain labels are allowed to fuzz in any order. For instance, from the domain
foo.bar.baz.somedomain.com the first three part is allowed to fuzz.
Back on the road, I voted for the factory module DnsLib process the DNS response and use
it to create the necessary reports, logs; there is nothing serious at first sight. But the responses
mostly arrive in disrupted structure (wrong start byte) too because of non-valid characters.
In order to handle these scenarios, the component inherits the DNSRecord and DNSBuffer
Figure 14 DNS Header format, Source RFC 2929
Design and develop a fuzzer framework to security test a Cloud based application
53
classes from DnsLib module and take care to override the appropriate methods to adjust its
bearing to the new circumstances.
The response codes which the logic triggers are the: FORMERR, SERVFAIL, NXDOMAIN,
NOTIMP, REFUSED, YXDOMAIN, XRRSET, NOAUTH, NOZONE. Initially the logic holds
all the response codes in a list and compare the index number with the response code number.
If the response code is not 0, the framework creates a report from the complete query and
response. With this mechanism, the application being able to collect problems during a DNS
fuzz session. Of course, it creates a report if any other common error occurs and raises an
exception. It is also providing the ability to extend or fine-tune the template structure or to
extend the reporting points, which means it is an easy task to introduce new error or
exception types to report.
9.1.13 DNS controller The DnsController class as all the followings is only an open door to implement a whitebox
fuzzer with a server agent. Currently it is only evaluating whether the target is alive or not.
9.1.14 DNS runner The DnsRunner class implementation makes the least problem because the domain label
structure representation in the template is obvious. It derived from the FuzzObject and
implements its logging in combination with the Kitty object logger. DNS also has a counter
to decide to after how many times of target failures should the controller forgive the
connection initiation. As a result of the DNS protocol the application uses timeout settings
which determine a time window to wait for the response. The rest of the class must apply
the anatomy of a fuzz session according to Kitty API documentation and as it is implemented
in other runners.
9.1.15 Redis component The RPutCollections class is derived from the FuzzObject and used by the ValidateMessage
class to put Protobuf API component name into the in-memory database. It is basically an
experimental class to try out to how easy the work could be with an in-memory server and
Design and develop a fuzzer framework to security test a Cloud based application
54
of course provide a class for the further developments. The Redis Python client connection
made by the FuzzObject. The class constructor waits the Protobuf API object which is
provided by the ParsePb class. They meet each other in the Runner class.
9.1.16 Reporter component After the application has been integrated into the CI system, it is a valid expectation that it
could be generating reports to publish them under the CI job’s reporting surface. These are
artifacts which are stands from in our case the Kitty logs the framework logs, the Protobuf
API JSON files, reports created by the Kitty web interface if a predefined error has occurred.
This report files are JSON files.
The reporter component transforms the application reports into artifacts in cooperation with
the FrameworkUtils class, which has several methods to collect each output into one specific
and easily identifiable folder. The class called PutArchiveToS3 that move the artifacts into
an AWS S3 bucket, and it is carrying on whether the bucket is exists or not.
The Reporter class privilege is to read and parse the reports generated by the Kitty web-
interface and produces a stdout summary and an XML file. The XML has to be compatible
with the Jenkins unit test publisher plugin. The Reporter class implements the mechanism
via the xml.etrre.cElementTree Python library.
There are several reusable methods that the mechanism listed above using, for instance,
searching for files, using UUID.
During the integration, the JUnit compatible XML has needed structural changes as I
planned earlier.
9.1.17 Utilities component This component builds to maintain helper classes and methods in one place to support the
framework with general features they provide.
• CheckProtoEnv class is to test the local environment whether it is capable of running
Protobuf related stuffs such as read or compiling;
• Decorators is a powerful collection of functions to help debugging code;
• GenerateUuid is a singleton class which creates a UUID for each fuzz session to help
keep track of them;
Design and develop a fuzzer framework to security test a Cloud based application
55
• Utils as mentioned several times above give the ability to use functions system-wide
by components such as the message decoder, encoder and file system related
operations, it also implements helper functions to search specific value by key in a
dictionary, or specific key by value, or list all values or all keys; It might better to
name it in the future to FrameworkLib;
9.1.18 Runner The Runner class integrates the Kitty/Katnip components and the fuzzer Framework
application to instrument a fuzz session according to the given configuration state, which
can be a DNS fuzz, a HTTP fuzz or a Protobuf fuzz session. It derived from the FuzzObject
and uses most of the classes implicit or explicit. This is the point where the Protobuf API
parser components has been utilize each other. It defines the function called tear down,
which triggers the arhiver components and the Redis memory release after a fuzz session
function.
Design and develop a fuzzer framework to security test a Cloud based application
56
10. Installation and integration into CI system To properly install and set up the framework, it is necessary to define its environment. In
this special case, when we speak about a complete integration into a CI system, it needs some
design decisions. The first thing that we should care about is the environment that runs the
framework, the next one is how to install on it, and the last one is the CI system job definition.
And of course, a target system, which is the victim of the fuzz test. According to this order
I will introduce the high-level design of the integration of the fuzz framework application.
10.1 The executor environment The virtual machine environment which serve the application is a deployed Centos7 Linux
node inside the Company’s VmWare cluster, which is an independent node from the CI
system. The mandatory Operating System packages are the Python 3.7 , PIP package
manager and Python virtualenv package, which allow us to bootstrap the necessary Python
environment. The CI will expect that the machine is always up and running and
preconfigured with the application.
To install the prerequisites the administrator should do the following steps:
1. Deploy a CentOS 7 Linux, with networking in the same subnet as the CI system;
2. Deploy Python 3.7 and the PIP package manager and the virtulenv package with a
common packaging manager like yum:
a. yum install python37, pip;
b. python3 –version; - to check it was properly deployed;
c. python3 -m pip –version; - to check pip version;
3. Install virtualenv:
a. python3 -m pip install virtualenv;
After these steps, there are no more configuration left from the virtual machines side.
10.3 Install the application After the virtual machine has been installed, the first step is to check out the fuzz_framework
repository from GitHub at https://github.com/netlounge/fuzz_framework and stay on the
master branch. Now it’s time to activate the virtual environment inside the project directory
with the command: virtualenv -p python3 .venv; It is not required but recommended
Design and develop a fuzzer framework to security test a Cloud based application
57
because of easy to handle Python packages and dependencies inside virtual environment.
There is a step by step guide inside a README.md under the project directory to fulfill the
integration or standalone installation: fuzz_framework/README.md
Going beyond the installation after running a test session from cli on the target system to try
out whether the application is working correctly or not, it is ready to integrate into the CI
system.
10.4 Integration into Continuous Integration System The CI system, in our case is the Jenkins CI, which was written in Java language, and it is
basically an engine that can receive wide variety of plugins to manage automated jobs to
support the software development’s process. A job is a configuration set or a script, which
automates several process depending on the definition it has. It can be a build job to building
source code to binary then it can run unit tests, functional, non-functional tests on it, or it
can mix up more work step into one huge pipeline as it looks on the Figure 16.
As it seems clearly it builds the source files into binary, runs unit tests separated the C++
and the Python related ones. Some of the steps are running in parallel mode. It creates
documentation from the code docstring placed inside the code, creates deployable RPM
packages from the binary and publish them as artifacts then grab a virtual machine from the
VmWare cluster and deploy the code, runs further tests then it publishes the results again.
These chapter’s objective is to design such a pipeline for the fuzz tests. Because of the
complexity the pipeline implementation, I will negotiate it from designers’ perspective
which are approximately as follows:
Figure 16 A Jenkins Pipeline view, source Sophos
Design and develop a fuzzer framework to security test a Cloud based application
58
1. First of all, we assume that there is a working AWS Cloud elsewhere pre-deployed
and available to run these tests; If so, the one possible first step is to scale-out the
deployed stack and grab information via aws cli about the state to know when the
stack is updated successful, and the servers are serving.
2. The following step can be to configure the fuzzer to run a DNS fuzz test on the target
system; The configuration can be done with a tool, which is able to modify a JSON
file; The test can run several times if it is needed and if more initial default DNS
query is available from a file; Another script can trigger the fuzzer application to run
the test;
3. The second step is to collect the actual tests artifacts and evaluate the results;
4. During the following steps, the Job starts the second one protocol but with the
Protobuf related configuration;
5. The last step is to tear down the system, collect application logs from the Cloud
services, scale it down, publish the collected application logs and framework logs,
results produced by the fuzzer framework;
6. Cleaning up on the Virtual Machine to preserve its initial state for the next run;
Jenkins gives the ability for the developers to scripting the above steps with the Groovy
language which is developed by the Apache Foundation to control the Jenkins scripted
pipeline. The high-level design of integration and data flow can be found at Appendix 2.
(Fuzzer Framework high-level integration and data flow chart). It demonstrates the on-
premise infrastructure where the Jenkins CI operates in the separated corporate network. The
application and its network connections, source code stream and interactors depict its
relations and place in the infrastructure. The data flow indicates the protocol types in the
direction of the AWS VPC. I tried to describe the VPC as obfuscated as I can to hide the
systems details and its structure, but even so the high-level sketch, some basic components
are well recognizable to understand the data flow targets.
It might worth speaking about a second decision that could be another integration setup. Its
existence is based on that to maintain a full Cloud deployment could raise the team budget
from period to period into a high rate. Definitely, an almost perfect condition could be to use
shared deployments in the Cloud to share its resources across several tests. In practice, one
deployment, is cheaper than two or more it’s obvious, but in case of one deployment other
Design and develop a fuzzer framework to security test a Cloud based application
59
facts are deserving to consider. One of those considerations is that one deployment used by
huge test cases receives higher load. Instances are being upscaled in a wider time window
with surely high load, database tables, or RDS instances also scaling during peak and stress.
These things are generating really high costs. Beyond the costs, engineers who are
responsible for the DevOps roles must manage each case to schedule the Jobs properly to
avoid deadlocks on parallel tasks and provide proper mutual exclusion against each other.
Based on these considerations, it could be an absolutely accepted solution to running a virtual
machine on a conventional Linux box to simulate the Cloud environment with fake services.
These services are fakes as their name refers to them and provided by AWS to give the ability
for the engineers to develop Cloud applications at their localhost to saving them time and
costs. The idea is at Appendix 3 (Fuzz Framework On-Premise integration).
After the integration into CI Pipeline the ongoing process of the fuzzer while runs a DNS
fuzz test is on Figure 17.:
Figure 17 Ongoing DNS fuzz test, source SOPHOS Jenkins CI Pipeline.
10.5 Results after the first runs After the fuzzer has been integrated and the first tests are initiated it has found a minor but
interesting issue while ran HTTP GET test. The requested data have been passed inside the
server core logic without validation and that somehow forced the logger subsystem to create
an empty log record, with such kind of flaw a threat can obfuscate its presence in the system.
It uses 7 test suites with two network protocols, the DNS on A, NS, TXT, and HTTP GET
launch every Saturday.
Design and develop a fuzzer framework to security test a Cloud based application
60
11. User Guide to operate To configure and operate properly, there are some further steps that the user must do before
running the first fuzz session. These steps are true in the case if the framework is a part of
the CI system. The CI must know, of course programmatically way, that what configuration
fields should modify to get the expected operation. After the runs and even if an error occurs
during the session, the log analysis is a must-have.
11.1 Configuration The starting point to set up the fuzzer, is the configuration file, which is under the project
folder framework/data/config/framework_config.json it has four main object type. The
generic object is about the system-wide settings:
• verbosity: set 1 to run on verbose mode;
• target_host: the host, which is the victim of the fuzz session;
• target_port: protocol specific port but it can be different according to the
implementation of the target listening port;
• tls: set true if the TLS is a must-have while fuzzing HTTP(s) methods;
• web_port: is a Kitty specific port where the Kitty web interface is available;
• archive_to_s3: if true, the framework will archive the results into an S3 bucket;
The next block is about the protocol type, set the appropriate one to true and be careful to
set only one protocol to true;
Protobuf related fields are about to configure the Protobuf fuzzing over HTTP:
• proto_path: is the path where the raw Protobuf files can be found, currently not
implemented to compile them;
• pb2_path: indicates for the system the compiled Protobuf API place;
• modules: The main Protobuf API component name, which holds all the imported
Protobuf packages;
• classes: The top-level message class name according to the Protobuf API structure,
it could be more than one on the top level;
• enum2class: The only insufficiency from the Protobuf parser component is that it
cannot decide to which ENUM filed belongs to which class, this is the only strict
point where the system needs help from the user; The preceding two field which is
Design and develop a fuzzer framework to security test a Cloud based application
61
the module and the classes are only to let the users decide the circle of the module
and class to fuzz; These two approach also a basis for a further development to
sophisticate the fuzz session during a Protobuf fuzzing;
The DNS fields are the followings:
• tld: The top-level domain name and the domain to query;
• default_labels: These domain labels are available represented by a DNS record and
this is the basis of the fuzzed query;
• timeout: The time window size what the fuzzer should wait for a DNS response;
• A: to ask A record;
• NS : to ask name servers;
• TXT: to ask TXT record;
DNS able to ask only one record type in a fuzz session.
HTTP related fields are:
• GET to fuzz a HTTP GET method;
• POST_PUT is to fuzz a HTTP POST method;
• fuzz_protocol: set true to fuzz not only the HTTP payload and path but the HTTP
header fields too;
• content_type: to define the HTTP header content_type;
• payload: Define the json format of a REST payload or form key value pairs;
With the appropriate configuration file, there is an open way to start a fuzz session. During
the session, the framework creates output files tagged with the actual session’s ID. At the
end of the session it puts all together into a folder under the framework/bin folder named
also to the session’s ID. Under this folder, every report and output log can be found for
further inspection if necessary. To debug the system, run it in verbosity mode and read the
log files produced by Kitty and the framework. The framework logs every essential step on
each stage it starts and finishes, and logs exceptions where it could be helpful. Each log line
contains a timestamp the component name, the file name and the line number where the
appropriate function step can be found. Whit these logging combined with the Kitty logs and
with the session ID it is an easy process to inspect errors or failures.
Design and develop a fuzzer framework to security test a Cloud based application
62
A complete fuzz session output folder content after a Protobuf session on Figure 18.:
To deep analyze the results:
In one hand, the framework will generate a result.xml , which is available in Junit compatible
format for the CI system. Each report generated into standalone file during the fuzz session
if any predefined error occurs. If the report in the CI system or on the stdout shows errors,
which can be interesting, then the result outputs will help to investigate deeply.
A report file name format was designed to help easily combine the information aggregation
to follow up the error. Worth to note that information provided in base64 code inside the
report files, but the Python’s Base64 library can decode it with the method b64decode().
The project contains a docs folder where the framework’s generated API documentation can
be found to help understand and deep analyze the mechanisms used by the framework if
someone needs it. An excerpt of the API Documentation can be found under Appendix 4.
Figure 18 Session result data, source Fuzz framework
Design and develop a fuzzer framework to security test a Cloud based application
63
12. Further development possibilities There are a lot of potential further development possibilities in this project, including a more
complex templating mechanism to create accurate mutation for the targets. It could also be
an objective to extend HTTP methods or the DNS query types to follow up the targets
evolution or extend the field type recognition mechanism when fuzzing Protobuf messages
or support more sophisticated Protobuf message structure. Some improvement around the
Protobuf message validator to get rid of its forced support from the configuration file, which
means it should handle the Protobuf message hierarchy from the view of enumerated fields.
The proper exception handling cases during the fuzzed data transfer could also be the field
of future improvement. Most of these are solvable during the integration because many of
them can be done with low effort.
To put the application into a higher level, it could be a possible feature request in the future
to run the framework as a background process or behind a http server and accept REST API
calls to reconfigure it in runtime to set up easily for the next fuzz session in a more elegant
way. From the background process approach it could be also a feature to run it in multi-
threading mode. A mechanism to check whether a Protobuf API has any change and
recompile it if changes are occurs. Store the file hashes of the API files to compare each
other to each version to decide is there any change on them.
A standalone seeder component which allows to produce input data in more sophisticated
way.
Design and develop a fuzzer framework to security test a Cloud based application
64
13. The future: increase app resilience with symbolic execution. In the near future of the field of fuzz testing, groundbreaking innovations are born according
to the newest studies. These studies do not argue that the blackbox fuzzing methods are not
good enough, but they agree that the technology is ripe for it to run symbolic execution,
which is basically the whitebox fuzzing that I also mention several times as a future
possibility. Google and Microsoft are pioneers in that kind of technology, and they make
software products more resilient with that type of testing. One method is that the test cases
are predefined by the testers bypassing the time that a simple fuzz test can take during brute-
forcing the application instead they focus on automatically generated test cases with the help
they can take a walk through on specific paths to support more accurate reveal of software
failures. Others went further and provide existing models to symbolic execution. One of
these methods is the Compositional fuzzing, which influences the system in very low-level
by isolating functions and apply seeders on them with special fuzzer drivers. They also take
care of the complex crash report, which can replay the test cases to generate reports about
the vulnerabilities. They are replacing functions with exploit test-cases to drive the code into
pathways leading to vulnerabilities.
While no test methods can ever explore every software errors, but there is the ability to
evolve the tests in parallel with the software to have more reliable and inclusive test result
set to which convince the responsible coworkers that the application is quite resistant.
[29][30]
Design and develop a fuzzer framework to security test a Cloud based application
65
13. Summary The topic of my thesis is to security test a Cloud-based application with a fuzzer application.
During the task, I researched an adequate fuzzer tool on the open-source software market
than implemented a framework on the top of the fuzzer to proceed fuzz tests and of course
the network protocols also implemented into the tool. In the first chapter, I have introduced
the task and provided some information about my employer. With this approach, the
justification of the task’s equity was done. The next chapter is about the importance of testing
in software development. The chapter also introduced the Agile software development and
its methods. Starting from the fact that the framework’s objective is to fuzz a Cloud
application, I introduced the Cloud, mainly the Amazon Web Services, and I gave an
abstraction about the distributed systems. The subsequent chapter was about the software
testing started with the unit testing method and discussed all the major tools up to the fuzz
testing. The fuzz testing was inspected in more detail.
After the introduction of fuzz testing, I started to describe the design procedure with the
research process of the possible fuzzer tools whereat I have implemented a framework. I
made an aspect system, that provided the benchmark between each fuzzer tool to help
ranking them and making a decision. I compared them pro and contra according to the
attribution and I made the decision to choose the one which fits the most for the expectations.
After these steps, I presented the technologies which were involved explicit or implicit into
the development. I discussed the Cloud resources and the integrated development
environment and the programming language that I choose.
The last two chapters were about the integration I made a clue about how to integrate it into
a CI system and gave advice about what kind of steps shall be done to implement in a specific
use case. I touched the installation procedures and the prerequisites too. Finally, I wrote a
User Guide to guide the prospect user to cope with the framework during operate with it. I
did notes about the configuration and the trouble shooting possibilities.
Design and develop a fuzzer framework to security test a Cloud based application
66
14. Összegzés A szakdolgozatom témája egy felhő alapú alkalmazás biztonsági tesztelés egy fuzz
rendszerrel, amelynek keretében a tesztelést vezérlő keretrendszert magam fejlesztettem,
valamint a megvalósítandó protokollokat magam implementáltam egy választott fuzzer-be.
A dolgozat első fejezeteiben a feladat ismertetésén túlmenően szót ejtettem a vállalatról ahol
dolgozom, így alátámasztva a feladat létjogosultságát hiszen az alkalmazásra szükség van a
minőség és biztonság fokozása céljából. A következőkben ismertettem a szoftver tesztelés
jelentőségét illetve az Agilis szoftverfejlesztést és módszertanát. Abból kiindulva, hogy a
tesztelt alkalmazás felhő alapú, bemutattam általánosságban a felhő technológiát, azon belül
is az Amazon Web Service-et, illetve az elosztott rendszert. A következő fejezet a szoftver
tesztelésről szólt ahol is a unit teszteken keresztül eljutottam a funkcionális és nem
funkcionális tesztelésen át a Fuzz tesztelésig ez utóbbi teszt bemutatására egy teljes fejezet
áll rendelkezésre.
A Fuzz tesztelés ismertetése után elkezdtem a feladat tervezését ami a lehetséges fuzzer
eszközök felkutatásával indult, a fejezetben felállítottam egy szempont rendszert ami alapján
értékeltem és választottam lehetséges eszközt a szabad szoftverek piacán rendelkezésre álló
fuzzer-ek közül, az egyes fuzzereket pro és kontra követelmények szerint elemeztem és
bemutattam, majd a szempontokat kiértékelve kiválasztottam az általam fejlesztett
keretrendszer számára legalkalmasabbat. Ezt követően ismertettem a fejlesztésbe közvetve
vagy közvetlenül bevont technológiákat kapcsolódóan a felhőrendszerhez valamint a
fejlesztői környezethez.
A következő fejezetben specifikáltam a szoftver funkcióit majd ennek alapján elkezdtem a
tervezést amin keresztük az implementálandó feladat problémáiba nyújtok betekintést.
Komponensről komponensre haladva ismertettem azok funkcióját, kapcsolatát más
komponensekkel, helyüket a rendszerben. A leírt terv szoros részét képezi az
implementációnak.
Az utolsó fejezetekben kifejtettem az alkalmazás integrációját, támpontot és egy lehetséges
use case-t valamint egy CI szkript tervet nyújtottam az integráció megvalósításához.
Végül a felhasználót segítő dokumentációt készítettem és ismertettem a lehetséges jövőben
fejlesztéseket, melyek közül némelyek alapja már elő van készítve mások komolyabb
beavatkozást igényelnek.
Design and develop a fuzzer framework to security test a Cloud based application
67
15. Bibliography [1] J. Highsmiths, Agile Software Development Ecosystems. Addison-Welsey, 2002. [2] Michael James, “Scrum Methodology.” https://scrummethodology.com. [3] D. CLAIRE, “What is Scrum?” Atlassian. https://www.atlassian.com/agile/scrum. [4] AWS Well-Architected. Amazon Web Services, Inc., 2019. [5] “Distributed computing,” Distributed Computing, 2020.
https://www.ibm.com/support/knowledgecenter/SSGMCP_5.2.0/com.ibm.cics.ts.doc/dfhtm/topics/dfhtm09.html.
[6] S. Jignesh, “Vertical Scaling and Horizontal Scaling in AWS,” 2018. https://dzone.com/articles/vertical-scaling-and-horizontal-scaling-in-aws.
[7] Architecting on AWS Student book, Module 8., 6.4. AWS, 2019. [8] “Amazon SQS.” https://aws.amazon.com/sqs/. [9] “Functional Testing Types – Examples, Tips & More,” QA Symphony.
https://www.qasymphony.com/blog/functional-testing-types/. [10] Ficsor Lajos, Kovács László, Kusper Gábor, and Krizsán Zoltán,
Szoftvertesztelés. www.iit.uni-miskolc.hu, 2001. [11] Yotam Kadishay, “Tests Coverage is Dead — Long Live Mutation Testing,” Apps
Flyer, 2018. https://medium.com/appsflyer/tests-coverage-is-dead-long-live-mutation-testing-7fd61020330e.
[12] G. Ann Campbell, Cognitive Complexity. SonarQube, 2018. [13] “Fuzzing.” Wikipedia, [Online]. Available: https://en.wikipedia.org/wiki/Fuzzing. [14] Cisco, “Kitty documentation.” https://github.com/cisco-sas/kitty. [15] “Sulley fuzzer documentation,” OpenRCE. https://github.com/OpenRCE/sulley. [16] Mozzilla Security, “Peach fuzzer documentation.”
https://github.com/MozillaSecurity/peach. [17] “Burp Suite Documentation.”
https://portswigger.net/burp/documentation/desktop/tools/intruder/using. [18] “America Fuzzy Loop.” https://github.com/google/AFL. [19] Cisco SAS team, “Kitty - Framework Structure.” [Online]. Available:
https://kitty.readthedocs.io/en/latest/base_structure.html. [20] “Amazon Virtual Private Cloud (VPC).” https://aws.amazon.com/vpc/. [21] “Amazon RDS.” https://aws.amazon.com/rds/. [22] “Amazon DynamoDB.” https://aws.amazon.com/dynamodb/. [23] “Amazon S3.” https://aws.amazon.com/s3/. [24] “Amazon CloudFormation.” https://aws.amazon.com/cloudformation/. [25] “Primer on Python decorators,” RealPython.com, 2019.
https://realpython.com/primer-on-python-decorators/. [26] Google, “Protocol Buffer developer documentation,” Google Protocol Buffer.
https://developers.google.com/protocol-buffers/docs/reference/csharp/class/google/protobuf/reflection/message-descriptor.
[27] D. Eastlake, E. Brunner-Williams, and B. Manning, RFC 2929 - Domain Name System (DNS) IANA Considerations. The Internet Society, IANA, 2000.
[28] “Let’s hand write DNS messages.” 2017, [Online]. Available: https://routley.io/posts/hand-writing-dns-messages/.
[29] Mel Liaguno, “Developers: Fuzzing, Symbolic Execution with Regression Testing Offers App Resilience,” The News Stack, 2019. https://thenewstack.io/developers-fuzzing-symbolic-execution-with-regressive-testing-offers-app-resilience/.
Design and develop a fuzzer framework to security test a Cloud based application
68
[30] Saahil Ognawala, Fabian Kilger, and Alexander Pretschner, Compositional Fuzzing Aided by Targeted Symbolic Execution. Faculty of Informatics, Technical University of Munich, 2019.
Design and develop a fuzzer framework to security test a Cloud based application
69
16. Printed appendix 16.1 Appendix 1.
Code Coverage of the application generated during unit tests with Python coverage library
Coverage report of parse_pb.py component
Design and develop a fuzzer framework to security test a Cloud based application
70
16.1 Appendix 2.
Fuzzer Framework high-level integration and data flow chart
Design and develop a fuzzer framework to security test a Cloud based application
71
16.2 Appendix 3.
Fuzz Framework On-Premise integration
.
Design and develop a fuzzer framework to security test a Cloud based application
72
16.4 Appendix 4.
API Documentation preview generated by Sphinx.
top related