acceptance testing plone sites and add ons with robot framework and selenium
DESCRIPTION
Acceptance tests are high-level tests to verify the completeness of user stories (end user features) for a developed product. It is said that while lower level unit and integration tests confirm that the product has been made right, only the acceptance tests confirm that the product actually does the right thing. Robot Framework is a generic test automation framework for acceptance testing and acceptance test-driven development. The key feature of Robot Framework is its extensible test syntax, which allows to write tests in a business-readable domain-specific language, including BDD (behavior driven development) style for executable acceptance criteria. Selenium is a browser automation framework. It allows to use real browsers in acceptance testing so that all effects of CSS and JavaScript taken into account, including possible differences between different browsers. This talk walks through alternative ways for using Robot Framework and Selenium to perform acceptance testing, including * testing live sites (readonly) * testing sandboxed Plone sites with live data (beforestorage + demostorage) * testing sandboxed Plone add-ons (plone.app.testing + plone.app.robotframework) * testing Sphinx documentation (while generating screenshots)TRANSCRIPT
Acceptance testing!Plone sites and add-ons with Robot Framework!
and Selenium
Asko Soukka, [email protected] University of Jyväskylä, Finland
Hello!+ Asko SoukkaTwitter: @datakurre IRC: datakurreEmail: [email protected] Web: datakurre.pandala.org
+ Software developer at University of Jyväskylä Finland
+ Plone Foundation Member
MenuTerminology!+ Acceptance testing + Robot framework + Selenium !Acceptance testing Plone!+ Just do it with a live instance + Do it safely with a dedicated instance + Do it correctly in a p.a.testing sandbox + Do it with while writing your
documentation with screenshots !
Acceptance testing
Functional testing of user stories!!!
Scenario: As an editor I can translate a document Given a site owner and a document in English When I translate the document into Catalan Then I can view the document in Catalan
!!
Unit tests !!
!
Integration tests !
Acceptance tests
Robot Framework!Keyword driven testing in your own domain specific language!!Extend with Python, Java or XML-RPC
$ pip install robotframework $ pybot —-help $ pybot my_test_suite.robot
Robot Framework*** Settings *** !Library string WITH NAME string !*** Test Cases *** !Convert string to uppercase ${value} = string.upper twist and shout! Should be equal ${value} TWIST AND SHOUT!
!*** Settings *** !Library ... Resource ... !*** Variables *** ${NAME} value!*** Test Cases *** Scenario title Some keyword argument ${NAME} !*** Keywords *** !Some keyword [Arguments] ${arg1} ${arg2} Another keyword ${arg1} ${arg2} !!
test_suite.robot!
Selenium
Browser automation framework!!
All browser and service logos and trademarks on this slide are property of their respective owners.
Selenium2Library
$ pip install robotframework-selenium2library $ pybot —-help $ pybot my_selenium_test_suite.robot
Selenium2 (WebDriver) keywords!for Robot Framework!
Selenium2Library*** Settings *** !Library Selenium2Library !*** Test Cases *** !Front-page should contain search box Open browser https://www.jyu.fi/ Element should be visible ... css=form[name='searchform'] input [Teardown] Close all browsers
!
Testing Plone with live data[buildout] parts += testinstance![testinstance] recipe = plone.recipe.zope2instance ...eggs = ... zc.beforestorage Products.PrintingMailHost ... before-storage = now demo-storage = true shared-blob = false
Testing Plone add-ons*** Settings *** !Resource plone/app/robotframework/selenium.robot Library Remote ${PLONE_URL}/RobotRemote !*** Test Cases *** !Scenario: Powered by Plone remains visible Open test browser Go to ${PLONE_URL} Page should contain Powered by Plone [Teardown] Close all browsers
Testing Plone add-onsimport unittest !import robotsuite from myproduct.testing import MYPRODUCT_ROBOT_TESTING from plone.testing import layered !!def test_suite(): suite = unittest.TestSuite() suite.addTests([layered( robotsuite.RobotTestSuite(’test_hello.robot'), layer=MYPRODUCT_ROBOT_TESTING)]) return suite
Testing Plone add-onsfrom plone.testing import z2 from plone.app.testing import FunctionalTesting from plone.app.robotframework.testing import \ REMOTE_LIBRARY_BUNDLE_FIXTURE !MYPRODUCT_ROBOT_TESTING = FunctionalTesting( bases=(MYPRODUCT_FIXTURE, REMOTE_LIBRARY_BUNDLE_FIXTURE, z2.ZSERVER_FIXTURE), name="MyProduct:Robot")
Testing Plone add-onsfrom plone.testing import z2 from plone.app.testing import FunctionalTesting from plone.app.robotframework.testing import \ REMOTE_LIBRARY_BUNDLE_FIXTURE !MYPRODUCT_ROBOT_TESTING = FunctionalTesting( bases=(MYPRODUCT_FIXTURE, REMOTE_LIBRARY_BUNDLE_FIXTURE, z2.ZSERVER_FIXTURE), name="MyProduct:Robot")
Testing Plone add-ons
extras_require={ 'test': [ ... 'plone.app.robotframework', ], },
http://docs.plone.org/search.html?q=robotframework
To BDD or not to…...Scenario: As an editor I can translate a document Given a site owner and a document in English When I translate the document into Catalan Then I can view the document in Catalan !*** Keywords ***
! A site owner Enable autologin as Site Administrator ...
Testing Plone with live data!eggs = ... plone.app.robotframework collective.monkeypatcher ... zcml-additional = <configure xmlns="http://namespaces.zope.org/zope" xmlns:monkey="http://namespaces.plone.org/monkey"> <include package="collective.monkeypatcher" /> <monkey:patch description="Enable Robot Framework remote library" class="Products.CMFPlone.Portal.PloneSite" original="RobotRemote" replacement="plone.app.robotframework.testing.RobotRemote" ignoreOriginal="true" /> </configure>
REVISITED!
Sphinx integrationextensions = ['sphinxcontrib_robotframework']
.. code:: robotframework ! *** Settings *** ! Library string WITH NAME string ! *** Test Cases *** ! Convert string to uppercase ...
$ pip install Sphinx $ pip install sphinxcontrib-robotframework $ pip install robotframework-selenium2library
.. figure:: ploneconf.png .. code:: robotframework ! *** Settings *** ! Library Selenium2Library ! *** Test Cases *** ! Show Plone Conference 2014 website Open browser http://2014.ploneconf.org/ Capture page screenshot ploneconf.png [Teardown] Close all browsers
!.. figure:: portal-footer.png.. code:: robotframework ! *** Settings *** ! Resource plone/app/robotframework/server.robot Resource Selenium2Screenshots/keywords.robot ! *** Test Cases *** ! Scenario: Powered by Plone remains visible Setup Plone site ... plone.app.testing.PLONE_ZSERVER Go to ${PLONE_URL} Page should contain Powered by Plone Capture and crop page screenshot ... portal-footer.png id=portal-footer [Teardown] Run keywords ... Tear down Plone site ... Close all browsers !
More+ Robot Framework
http://robotframework.org/
+ Selenium2Library (for Robot Framework) https://github.com/rtomac/robotframework-selenium2library
+ Overview of Plone-related Robot ecosystem http://datakurre.pandala.org/2013/09/meet-robot-family-for-plone-developers.html