challenge to add openstack api validation framework · openstack community development is hard...
Post on 13-Oct-2020
3 Views
Preview:
TRANSCRIPT
Challenge to add OpenStack
API Validation Framework
2013.05.31
Ken’ichi Ohmichi
NEC-Soft, Ltd.
(oomichi@mxs.nes.nec.co.jp)
Page 2 NEC Corporation 2013
Agenda
▌OpenStack
▌OpenStack community development
▌OpenStack functions through RESTful API
▌Issues of RESTful APIs
▌Fail-Fast architecture
▌API validation framework
▌Development process
▌Schedule for Havana
OpenStack
▌ The Open Source Cloud Operating System▌ Controls virtual machines, storage/networking resources. ▌ Consists of multiple functional components.� Compute: Nova� VM image: Glance� Identity: Keystone� Networking: Quantum (will be renamed)� Block storage: Cinder� Object Storage: Swift� Dashboard: Horizon� Metering: Ceilometer� Orchestration: Heat
OpenStack community development
▌ The community is very active.� OpenStack community consists of many developers, users and operators.� They are cooperating each other on mailing-list, IRC and development tools such as Gerrit.� The community releases new version of OpenStack every six months, and the functions increases by each version.
OpenStack community development
▌ The development routine
1. Submit a bug-report / blueprint to OpenStack Launchpad.2. “git clone” necessary source code from OpenStack github.3. Modify the source code as you like.4. “git commit” with the id of bug-report / blueprint.5. “git review” for posting a patch into Gerrit system.6. (Reviewers) review the patch and submit their feedbacks with score on
Gerrit.Score +2: ApproveScore +1: Looks good do me, but someone else must approveScore -1: I would prefer that you didn’t merge this.Score -2: Do not merge
7. Apply reviewer’s feedback to your patch, and retry “git review” for reposting it.
8. (Reviewers) re-review the patch. When core reviewer marks it as +2, the patch is merged to OpenStack github. Congratulations☺
OpenStack functions through RESTful API
▌ Control functions through RESTful API directly.� EX) We can create a virtual machine by sending API request with following parameters to http://<openstack>:8774/<tenant-id>/servers with POST method. The parameters can be specified with JSON/XML format.
{"server": {
"flavorRef": "1","imageRef": "70a599e0-31e7-49b7-b260-868f441e862b","name": "new-server-01"
}
▌ Control functions with dashboard or command line.
▌ These tools uses RESTful API in the background. Create vm by
pushing here.
Dashboard
Page 7 NEC Corporation 2013
Issue/Solution of RESTful APIs
▌Issue� Not all API parameters are completely validated, so some API
operations execute without parameter validations.
▌Solution� By validating every API parameters just after receiving API request, it
makes better OpenStack quality.� Because the basic design of OpenStack is “Fail-Fast” architecture.
Fail-Fast Architecture
▌ Fail-Fast system does not retry/cleanup if something wrong happens.� Ex) If something wrong happens after adding a DB record during one API operation, it does not remove the added record. As the result, incomplete record remains in DB.
▌ The antonym is Fail-Safe.▌ The implementation cost of Fail-Safe tends to higher:� “What should be cleaned up when error handling?”� “What should it do when double errors (something wrong in error handling)?”
▌ On Fail-Fast system is easy-debugging because developers can know what happens easily.
Op-01
Op-02
Op-03
Record-01
add
Fail-Fast
remains
Op-01
Op-02
Op-03
Record-01
add
Fail-Safe
removeddel
Page 9 NEC Corporation 2013
API validation framework: Goals
▌I have proposed API validation framework to OpenStack community for comprehensive validation.
▌This framework goals are:� To validate every API parameter.� To return an error response before API operation, if invalid API
parameter. � To unify the error message format of the response, if the same
cause.ex) “.. is not string.", ".. is not integer."
▌By comprehensive validation just after receiving API request, itmakes better OpenStack quality because it can reduce incomplete situations like incomplete record remaining.
Merit
Page 10 NEC Corporation 2013
API validation framework: Validation Point (discussing now)
▌The API validations are executed before each API method.
▌The proposed framework is implemented with jsonschema library.▌This framework can validate requests of both XML and JSON, because jsonschema handles “dict” data after JSON/XML deserialization.
� I heard often “Can this framework handle XML request?” due to its library name.
� The answer is “Yes, it can.”
Nova
Client RouterJSON/XMLDeserialization
request
(2) Execute the selected validation
(1) Select a validation and a method
corresponding to API
(3) Execute the selected method
API ValidationAPI ValidationAPI Validation
API methodAPI methodAPI method
Page 11 NEC Corporation 2013
API validation framework: API Schema (discussing now)
▌“API schema” = API parameter data format.� Ex) The API schema of the “Change Password” API.
{'type' : 'object','properties' : {
'changePassword': {'properties' : {
'adminPass': {'type': 'string', 'required': True},}
}}
}
▌Need many API schema definitions because OpenStack has many APIs (hundreds).
Page 12 NEC Corporation 2013
API validation framework: Merits
▌In addition to the goals, there are two merits of the framework.
▌Clarification of API Parameter Definitions� The type and acceptable range and length of each parameter are clarified in the API
schema definitions.� By defining the schemas of every API, developers will be able to create applications
more easily using Nova APIs.
▌Cleaned-up Code� The amount of Nova code can be reduced by merging validations and error response
methods.� Ex) the “min_count” parameter of “create a virtual machine instance” API
try:min_count = int(str(min_count))
except ValueError:msg = _('min_count must be an integer value')raise exc.HTTPBadRequest(explanation=msg)
if min_count < 1:msg = _('min_count must be > 0')raise exc.HTTPBadRequest(explanation=msg)
'min_count': {'type': 'integer', 'minimum': 1},
Development process
▌ March, 2013� Created some prototype for Nova.� Proposed the prototype to OpenStack community.� Could listen positive responses.
▌ April, 2013� Had a session of OpenStack Design Summit for this framework in Portland, Ore.� Could make consensus for its purpose.� Could know it is good that the first target of this framework is Nova v3 API.� Nova v3 API will be implemented for Havana, and compatibility issues do not happen even if applying strict validation.
Page 14 NEC Corporation 2013
Schedule for Havana
▌Now we are discussing what is best validation mechanism.� We need more time for consensus.
▌We have 4 months until Havana-3 meaning deadline for new code.� Make Framework Core Patches.� Make API schema Definition Patches.
• This is very hard work, because Nova has many APIs.• Merge by middle of August.
we
MayApr JulyJune SepAug Oct
AD 2013
h3h1 h2
rc1rc2
rc3 Havana
Framework core
Deadline for new code
API schema definition
▌ OpenStack community development is hard work, but contribution to the community is very interesting because the community consists of skillful developers and they give good questions, advices, and answers.
▌ I am glad if many people are into OpenStackcommunity.
Thanks
Page 16 © NEC Corporation 2013
Page 17 NEC Corporation 2013
Related URLs
▌Blueprint “Nova API Validation Framework”� https://blueprints.launchpad.net/nova/+spec/nova-api-validation-fw
▌Framework core patch for Oslo� https://review.openstack.org/#/c/25884/
▌Framework core for Nova� https://review.openstack.org/#/c/25358/
▌Nova v3 API� https://blueprints.launchpad.net/nova/+spec/nova-v3-api
▌How to contribute to OpenStack� https://wiki.openstack.org/wiki/How_To_Contribute
top related