Testing Mozilla Web Properties

  

Stephen Donner6/24/2009

Meet the Team

Stephen Donner - stephend@mozilla.comKrupa Raj - kraj@mozilla.comRaymond Etornam Agbeame - retornam@mozilla.com

Technologies we test

• AJAX• Drupal

o Spread Firefoxo QMO

• Apacheo .htaccess - bug 478952

• Open-source search engineso Sphinx (SUMO)

• TikiWiki (SUMO is its biggest consumer)• Media types

o Oggo Flash

.swf vs. .flv

Tools We Use

• Firebug• Web Developer toolbar• Live HTTP Headers• SQL Injection• Selenium

o IDEo RC

• VMWare Fusion• Acunetix• W3C

Gotchas

• Staging server vs. productiono Different environments

Configs (permissions, etc.) Application versions

o Caching Memcache NetScaler / Zeus

o DB setups Slave vs. master

Replication lago Legacy browsers / non-standard configs

IE 6, anyone? JavaScript disabled

Gotchas, part 2

• Partner integrationo GetSatisfactiono reCaptchao Mozilla Service Week

• Add-on promotionso Fashion Your Firefox

Pain Points

• Tight scheduleso Quarterly goals from other parts of the org

• Lack of clear specso Wireframes vs. mockups

(vs. full user-interaction testing)• Project direction / ownership• Out-of-band testing requests

o Ops/IT - hardware failure, new-technology evals, regression-testing of new hardware

Automated vs. Manual Testing

• Balance need to rapidly execute/ship vs. long-term stability• Ramp-up cost• Can't automate everything•  Quite a few projects are one-off (time investment)

o Impact Mozillao Fashion Your Firefox o Operation Firefox

• Good automation candidateso Searcho APIs (services.addons.mozilla.org)o Static content

Headers/footers/legal/privacy policies/EULAs Login / logout / registration/ forgotten password

Exception of Captcha systems

Grades / Levels of Testing

• Acceptance check • Full Functionality Test• Regression testing • Basic Functionality Test

 (Exhaustive list: http://www.softwaretestinghelp.com/types-of-software-testing/)  

What we use Selenium for

• Verify page titles• Verify content pieces (text, links, images)• Pagination• Search terms persist in textfields• ~ 15 commands

Example of manual testing

Mozilla Service Week (http://mozillaservice.org)

Selenium demo

• AMO Basic Functionality Test• AMO Search

How can you help?

• Vet content• Test browser compatibility• Sanity check our testing• Write unit tests

Resources

• Public mailing list: webqa@mozilla.org• Homepage:

https://wiki.mozilla.org/QA/Execution/Web_Testing• IRC channel: irc.mozilla.org/webqa

Questions?  Comments?  Suggestions?

 

Thanks for your interest!