performance and stability testing \w gatling
TRANSCRIPT
With Gatling
Performance & Stability Testing
Development Testing
@MeDmitry Vrublevsky, Software developer @
/in/dmitryvrublevsky
Roadmap
- Mission overview
- Gatling introduction
- Some usage example
Mission
Given: Graph database (Neo4j)
Challenge: execute query as FAST as we can
Neo4j
- Written in Java
- Native graph storage
- Extendable
So?
- Default API is too slow
- Implement custom extension
- Control all the things
1. Developed
2. Deployed
3. ???
4. Success!
Performance
Specification
Do “X” operations
in “Y” seconds
Specification
Do “10000” query operations
in “10” seconds
POST /extension/query Body: “MATCH (root)-[:HAS*]->(child)”
Response: { // JSON data }
Client
LB
DB1 DB2 DB3
- our extension
Problems?
Client
LB
DB1 DB2 DB3
- our extension- possible problem
Load testing framework
Free & Open source
High performance
Friendly DSL
Nice html reports
Coding ?
Bundle
1. Download bundle 2. Extract 3. Ready
http://gatling.io/#/download
Optional
- Maven integration - SBT plugin - Gradle plugin
package com.neueda.exampleimport io.gatling.core.Predef._import io.gatling.http.Predef._import scala.concurrent.duration._class Example extends Simulation { val httpConf = http.baseURL("http://www.example.com") val example = scenario("Example") .exec( http("get_example_index") .get("/") .check(status.is(200)) ) setUp( example.inject(rampUsers(10) over (10 seconds)) ).protocols(httpConf) }
package com.neueda.exampleimport io.gatling.core.Predef._import io.gatling.http.Predef._import scala.concurrent.duration._
class Example extends Simulation {}
val httpConf = http.baseURL("http://www.example.com")
val example = scenario("Example")
http("get_example_index") .get("/") .check(status.is(200))
setUp().protocols(httpConf)
example.inject( rampUsers(10) over (10 seconds) )
package com.neueda.exampleimport io.gatling.core.Predef._import io.gatling.http.Predef._import scala.concurrent.duration._class Example extends Simulation { val httpConf = http.baseURL("http://www.example.com") val example = scenario("Example") .exec( http("get_example_index") .get("/") .check(status.is(200)) ) setUp( example.inject(rampUsers(10) over (10 seconds)) ).protocols(httpConf) }
================================================================================2015-04-07 23:02:54 5s elapsed---- Example -------------------------------------------------------------------[##################################### ] 50% waiting: 5 / running: 0 / done:5---- Requests ------------------------------------------------------------------> Global (OK=5 KO=0 )> get_example_index (OK=5 KO=0 )================================================================================
================================================================================---- Global Information --------------------------------------------------------> request count 10 (OK=10 KO=0 )> min response time 255 (OK=255 KO=- )> max response time 302 (OK=302 KO=- )> mean response time 274 (OK=274 KO=- )> std deviation 13 (OK=13 KO=- )> response time 95th percentile 293 (OK=293 KO=- )> response time 99th percentile 300 (OK=300 KO=- )> mean requests/sec 1.08 (OK=1.08 KO=- )---- Response Time Distribution ------------------------------------------------> t < 800 ms 10 (100%)> 800 ms < t < 1200 ms 0 ( 0%)> t > 1200 ms 0 ( 0%)> failed 0 ( 0%)================================================================================
================================================================================2015-04-07 23:02:54 5s elapsed---- Example -------------------------------------------------------------------[##################################### ] 50% waiting: 5 / running: 0 / done:5---- Requests ------------------------------------------------------------------> Global (OK=5 KO=0 )> get_example_index (OK=5 KO=0 )================================================================================
================================================================================---- Global Information --------------------------------------------------------> request count 10 (OK=10 KO=0 )> min response time 255 (OK=255 KO=- )> max response time 302 (OK=302 KO=- )> mean response time 274 (OK=274 KO=- )> std deviation 13 (OK=13 KO=- )> response time 95th percentile 293 (OK=293 KO=- )> response time 99th percentile 300 (OK=300 KO=- )> mean requests/sec 1.08 (OK=1.08 KO=- )---- Response Time Distribution ------------------------------------------------> t < 800 ms 10 (100%)> 800 ms < t < 1200 ms 0 ( 0%)> t > 1200 ms 0 ( 0%)> failed 0 ( 0%)================================================================================
Reports \o/
class Test extends Simulation { val httpConf = http.baseURL("http://neo-database:7474") val test = scenario("GetGraph") .exec( http("execute_query") .post("/extension/query/execute") .body(StringBody("MATCH (root)-[:HAS*]->(child)")) .check(status.is(200)) .check(jsonPath("$.results").count.is(100)) ) setUp( test.inject( rampUsersPerSec(0) to (1000) during (60 seconds) ) ).protocols(httpConf) }
http("execute_query") .post( "/extension/query/execute" ) .body(StringBody( "MATCH (root)-[:HAS*]->(child)" )) .check( status.is(200) ) .check( jsonPath("$.results").count.is(100) )
1
2
3
4
rampUsersPerSec(0) to (1000) during (60 seconds)
Req
uest
s / S
econ
d
0
150
300
450
600
Seconds
0 10 20 30 40 50 60
When request/second goes up
And some threshold reached
Then we receive Timeout error
Everything is ok?
Maybe we can do higher load?
How to spot the problem?
Drop all
non-essential
parts
Client
LB
DB1 DB2 DB3
- our extension
0
250
500
750
1000
0 10 20 30 40 50 60
Client - Server communication
problems
• Incorrect server setup
• Unconfigured networking
• Low max open connection limit
Learned
- Performance testing should be done - Test results should be interpreted
properly - Test results should be verified in
different environments
We need to test different queries
Feeders!
val feeder = Array( Map("query" -> "..."), Map("query" -> "..."), Map("query" -> "...") )
.queue
.random
.circular
val test = scenario(“GetGraph") .feed(feeder) .exec(http("execute_query") .post("/extension/query/execute") .body(StringBody("${query}")) .check(status.is(200)) .check(jsonPath("$.results").count.is(100)) )
CSV JSON JDBC
Sitemap Redis
Custom
http://gatling.io/docs/2.1.4/session/feeder.html
csv("data.csv").random
Our own feeder
- Custom feeder - Lazy loads data - Performant
Dynamic user load?
http://gatling.io/docs/2.1.4/general/simulation_setup.html
test.inject( nothingFor(5 seconds),
atOnceUsers(10),
rampUsersPerSec(10) to (100) during (20 seconds),
constantUsersPerSec(100) during (40 seconds),
rampUsersPerSec(100) to (500) during (20 second),
constantUsersPerSec(100) during (60 seconds))
Checks
http://gatling.io/docs/2.1.4/http/http_check.html
- status - currentLocation - header - regex - xpath - jsonPath
regex("Invalid Page title").notExists
status.is(200)
jsonPath("$.posts").exists
regex("""<div class="article">""").count.is(10)
Realtime monitoring
http://gatling.io/docs/2.1.4/realtime_monitoring/index.html
Execution progress visual feedback
Graphite (InfluxDB) +
Grafana
Recorder
http://gatling.io/docs/2.1.4/http/recorder.html
HTTP
http://gatling.io/docs/2.1.4/http/recorder.html
- HTTP Protocol - SSL - WebSocket - SSE
Jenkins integration
https://github.com/jenkinsci/gatling-plugin
- Any IDE with Scala support is OK
- Intellij IDEA
- Eclipse (Scala IDE)
- Netbeans
Our use cases
GET /api/node/1 POST /api/node UPDATE /api/node
Basic API tests
- Generate background load - Make stability tests
- Cluster communications - Load balancer setup - Spikes
Load generator
- Simulate significant write load - Check how database reacts
Data import
Easy Fun
Performant
