s6 the guardian of line today automation test
TRANSCRIPT
The Guardian of LINE TODAY Automation Test
Kent Han
• LINE TODAY
• Challenges
• CI and Automation Test
CI Pipeline
Unit Test
Functional Test
Service Monitoring
• Wrap Up
AGENDA
LINE TODAY
FastTrack RegularTrack
2016.03 - Fast Track as Proof Of Concept 2016.06 - Regular Track as long-term product
GLN - Global LINE News
GLN - Global LINE News
TW 18m
TH 36m
IR 3m
ID 32m
JP 58m
TW TH ID IR
(RTL)
GLN-GlobalLINENewsGLN - Global LINE News
Fast Track - 02/26 (TW, TH, ID, US, MM) Regular Track - 06/16 (TW, TH, ID) 10/04 (IR)
3/2 3/5 3/8 3/113/143/173/203/233/263/29 4/1 4/4 4/7 4/104/134/164/194/224/254/28 5/1 5/4 5/7 5/105/135/165/195/225/255/285/31 6/3 6/6 6/9 6/126/156/186/216/246/276/30 7/3 7/6 7/9 7/127/157/187/217/247/277/30 8/2 8/5 8/8 8/11 8/148/178/208/238/268/29 9/1 9/4 9/7 9/10
DUBDPV
Fast Track
Regular Track
ChallengesSwiftly adjust to survive, or die
Feature requests are diverse
Agile is the Only Solution
How Fast is Your Agile
CI + Automation Test
Jenkins 2.0
CI Pipeline by Jenkinsews
Testing Pyramid
https://watirmelon.blog/tag/testing-pyramid/
Unit test vs. Integration test • Speed • Stable • Easy to troubleshoot
Testing Pyramid
PullRequestmustbecheckedbyunittest
Unit Test
Static Code Analysis• Checkstyle • FindBugs • PMD
• Duplicate Code
Testing Tools
• Python
• Robot Framework
• Selenium
• Appium
• Jenkins
• Gatling
Functional Test
Why Did We Adopt Robot Framework?
• Human-readable test cases for Maintainability
• Python and Java support for Versatility
• --rerunfailed for flaky tests
*** Test Cases ***Add A Comment To An Article [Tags] comment RAT New An Article In Category Add Comment To The Article 1 Times Comment Count Of The Article Is 1
Reply A Comment To A Comment [Tags] reply RAT New An Article In Category Add Comment To The Article 1 Times Comment Count Of The Article Is 1 Reply To The Comment With Text Comment Comment Should Be Included In Article
Vote Up To A Comment [Tags] vote RAT New An Article In Category Add Comment To The Article 1 Times Comment Count Of The Article Is 1 Vote The Comment With UP
Robot Framework Report
Content Working Flow
Feeding Test
• Average 5,500 articles imported a day
• Should ensure the input stream
• Fake articles for testing
Thumbnail
Image
Video
Youtube
<?xml version='1.0' encoding='UTF-8'?><articles> <UUID>c11ae751-b180-11e6-96c0-985aeb8c0520</UUID> <time>1461117682767</time> <article> <ID>c11b1973-b180-11e6-8d03-985aeb8c0520</ID> <nativeCountry>TW</nativeCountry> <language>zh</language> <startYmdtUnix>1479907723508</startYmdtUnix> <endYmdtUnix>1571117682767</endYmdtUnix> <title>20161123212843_all_medias</title> <category> </category> <publishTimeUnix>1479907723508</publishTimeUnix> <contents> <image> <description>Mountain is beautiful!</description> <url>yama_1.png</url> </image> <video> <description>Kobe Top 1</description> <thumbnail>thumbnail_1.jpg</thumbnail> <url>kobe_1.mp4</url> </video> <video> <description>Youtube</description> <url>https://www.youtube.com/embed/c7rCyll5AeY</url> </video> <text> <content> Google GoogleAlphabet
Larry Page Alphabet Google Sundar Pichi
Google Larry PageGoogle ......
Alphabet </content> </text> </contents> </article></articles>
CMS Test
• OA Digest
• Timeline Digest
• Category publication
• When new features are added, basic functions should not fail
Front-end Test
• Selenium covers the most
• Appium for the mobile-only feature
• API test as a tool to generate testing data
• Internal tools
iMON - application error
NSight - system error
• Health check script
Web response time
RabbitMQ management api
ElasticSearch cluster/node/stats/status api
Service Monitoring
0
100
200
300
400
v2.1 v2.2 v2.3 v3.0 v3.1 v3.2 v3.3
major bugs total bugs
Bug Stats
Wrap UpQuality is everyone’s KPI
Find a pain point in retrospective then improve in next sprint
THANK YOU