“kuality” assurance what does that look like? scott heise indiana university kfs - quality...
Post on 29-Dec-2015
219 Views
Preview:
TRANSCRIPT
“Kuality” Assurance
What does that look like?
Scott Heise Indiana University KFS - Quality Assurance Manager
Paul Sandoval University of Arizona KRA – Lead Testing Coordinator
Kenton Hensley Cornell University KRA – Lead Business Analyst
Scott Heise
Indiana University
Kuali Financial System
(KFS)
Quality Assurance Manager
“Kuality” Assurance
Functional Specifications
Application Development
Open Source License
Requirements
Accessibility Guidelines
Kuali Architecture/Standards
Unit TestCreation
Automated Unit Testing
Code ReviewOpen Source
License ReviewFunctional/
Regression Testing
Integration Testing
ModuleSign-off
Accessibility Review
Performance Testing
Functional Test Plan
Automated Unit Testing
Functional Specifications
Application DevelopmentUnit TestCreation
Automated Unit Testing
ModuleSign-off
Manual Testing
Functional Specifications
Application Development
Functional/Regression Testing
Integration Testing
ModuleSign-off
Functional Test Plan
Testing
Functional Specifications
Application DevelopmentUnit TestCreation
Automated Unit Testing
Functional/Regression Testing
Integration Testing
ModuleSign-off
Functional Test Plan
Code Review
Application Development
Kuali Architecture/Standards
Code Review
ModuleSign-off
Accessibility Review
Application Development
Accessibility Guidelines
ModuleSign-off
Accessibility Review
License Review
Application Development
Open Source License
Requirements
Open Source License Review
ModuleSign-off
Quality Assurance Framework
Functional Specifications
Application Development
Open Source License
Requirements
Accessibility Guidelines
Kuali Architecture/Standards
Unit TestCreation
Automated Unit Testing
Code ReviewOpen Source
License ReviewFunctional/
Regression Testing
Integration Testing
ModuleSign-off
Accessibility Review
Performance Testing
Functional Test Plan
License Review
Application Development
Open Source License
Requirements
Open Source License Review
ModuleSign-off
Licensing
• Software distribution licensing– Open Source Initiative (OSI)
Educational Community License 1.0 (ECL)• Original code
– Individual Contributor License Agreements (CLAs)– Corporate Contributor License Agreements (CCLAs)– © Labeling the code
• Third-Party code– ECL-Compatible License– Follow license requirements– Acknowledgments– License folder
Usability – Before Coding
• Indiana UXG – Usability Group– Developed Standards/Guidelines for the UI– Initial Mocks reviewed by Kuali Financial
Functional Council• Decisions made:
– Editable data entry lines– Page Level Help – no field level help
– Usability sessions with users from each school via Breeze
Usability – During/After
• Marked as Usability
• Reviewed and Prioritized by Usability Prioritization Committee
• Usability Dev team formed during QA
• Usability issues that affect infrastructure, reviewed and time allocated during next release, e.g. – error handling framework
Accessibility Review
Application Development
Accessibility Guidelines
ModuleSign-off
Accessibility Review
Accessibility
• Guidelines
• Test by sampling
• “Issues” approach to resolution
Paul Sandoval
University of Arizona
Kuali Research Administration System
(KRA)
Lead Testing Coordinator
KFS Testing
Functional Specifications
Application Development
Functional/Regression Testing
Integration Testing
ModuleSign-off
Functional Test Plan
KFS TeamOrganization Board
Project ManagerFunctional Council Technical Council
LeadSubject Matter
Expert
Business Analyst
DevelopmentManager
Subject Matter
ExpertsDevelopers
Partner Institution
- - - -
Lead Architect
Testing Coordinator
Quality Assurance Manager
TestingTeam
Module Team
ProjectAnalyst
LeadDeveloper
Testing Coordinator
Subject Matter
ExpertsDevelopers
Partner Institution
TestingTeam
ConfigurationManager
Testing Structure
• Testing Teams– One per module– Module Testing Coordinator
• Write/assign scenarios to testers• Train testers on Kuali, Confluence, JIRA• Review Bugs: Duplicate? Training? Usability?
– Testers from all schools – 15 plus/module
• KFS Testing Coordinator– Monitors issues application wide– Assists Module TCs
Process
• eDoc/process ready for testing
• Scenario created/assigned – JIRA
• Bugs reported in JIRA
• Issues fixed marked as resolved
• Testing environment released weekly
• Release notes created - Confluence
• Testers test close/reopen
• Rinse and Repeat!
Tools - JIRA
Tools - Confluence
Lessons Learned
• Face to Face sessions
• Never too many testers
• Training for testers
• Pre-Testing to ensure a fairly stable environment
• Weekly meetings with testers
• Testing is the BEST way to learn!
KRA
Testing and Usability
Process
• Functional Specs (SMEs)
• Code (Developers)
• Testing (Developers, Lead Testing Coordinator, Module Testing Coordinator, SMEs, Testers – includes Coeus consortium members)
Testing
• Module Testing Coordinators– One per module– Write/assign scenarios to testers from each
school in JIRA– Train testers
• Testers test – report bugs in JIRA
• Module Testing Coordinators review– Bug, Duplicate, Training opp, Usability
Bugs
• Reported in JIRA
• Fixed by Developers
• Resolved with next Build
• Confluence used to help testers keep track of resolved bugs ready to be tested
• Testers Test – Close/Reopen
Lessons Learned
• One person was designated to write Spec Docs in the SME group in KFS. In KRA, everyone is taking an active role in writing Spec Docs. This promotes ownership in the process!!
• SME Members are going to take a more active role in testing.
• Face to Face testing – Training
KRA
Quality, Timelines and Synergy
Can these work together?
QA is Continuous!
• QA exists throughout the entire software development life cycle!
– Planning and Communication Tools• Microsoft Project, Confluence and Jira, Breeze,
Skype, Face-to-face meetings – Analysis
• Functional and Technical– Licensing– Coding
• Code Reviews, Development Standards
QA is Continuous!
– Testing• Unit Testing, Regression Testing, Integration Testing
– Usability and Accessibility• HTML Mock Development, Focus Groups, Design
Critiques, W3C Priorities, Accessibility Testing
– Documentation• Functional and Technical
– Builds and Releases
Timelines
QA impacts Timelines
• Project planning is continuous along with QA• Coeus represents the base functionality that will be
delivered in KRA• Enhancements are reviewed, estimated and submitted
for functional approval. If accepted, the project plan is adjusted.
• Functional specifications are standard• SME groups follow similar processes when interacting
with SME group members and the development team• A user interface group monitors HTML mock
development and usability/accessibility issues
QA impacts Timelines
• Focus groups and design critiques are conducted with Faculty and staff roles across multiple schools
• HTML mocks and jsps are validated for W3C Priority 1 and 2 issues and corrected
• Code reviews are standard and code is reviewed weekly• Unit testing is standard and a continuous test process
runs daily• Multiple test environments exist to ensure that
automated and manual testing is performed• Documentation is standard – functional specifications,
technical gap documents, integration documents, user and help documents
Lessons Learned
• Developers should test the web application on a regular basis
• Code should be reviewed as functionality is developed and a final code review should be required for completion
• Unit Testing guidelines should be set early and followed• Functional testers should test functionality as early as
possible• Project team members must embrace QA processes• Coeus participation is essential! Coeus documentation
is essential
“Kuality” Conclusions
• QA exists throughout the software development life cycle
• While processes may vary between projects, the goal is to ensure that the software is built in a timely manner and meets specified functional requirements
• Reviewing and making changes based upon lessons learned can improve the project
Questions?
Contacts
Please feel free to contact the following individuals about the KFS, KRA or Coeus projects:
• Jim Thomas, Project ManagerIndiana University – jthomas@indiana.edu
• Andy Slusar, Senior Project ManagerCornell University – as833@cornell.edu
top related