Online Feedback-based Estimation of Dynamic Page

Service Time

Ashwini KumarKaushik Veeraraghavan

Benjamin WesterKang Shin

2

Motivation

• Dynamic pages– Increasingly prevalent– Pages difficult to cache

• Service time applications– Differentiated QoS (prioritizing requests) – Load balancing– Virtual host services– Estimates: credit card payment times

3

Idea

• History– Maintain records of past service times– Predict based on existing records

• Assume a correlation between past and future– What is the invariant property?– What system state can we store to track it?

• We explore three estimators and their design

4

Outline• Motivation

• Design & Implementation

• Evaluation

• Conclusion and Future Work

5

History

1. Maintain per-URL fixed-size history table2. Data depends on the estimator used3. Tag incoming requests4. Update history when request processing finishes

6

SEDA: Haboob

7

Sirocco

8

Estimators

Scheme Store in history Estimation

Average

STQ

STQT

Length Queue

Time Service

Time Service

Length Queue

Threads Time Service

)(historyavg

Length Queuehistoryavg )(

Threads

Length Queuehistoryavg )(

9

Outline• Motivation

• Design & Implementation

• Evaluation

• Conclusion and Future Work

10

Evaluation

• How close are our estimators?

• Load patterns– Steady load: httperf

• 50ms / request

– Changing load: SURGE

0

20

40

60

80

100

0 5 10 15 20 25

Re

que

sts

Time (seconds)

11

Average Estimator (steady)

• Request takes ~35ms to complete

• Estimator should not be too responsive

12

Average Estimator (changing)

• Each point averages 20 requests

• Vertical bars indicate the addition of a handling thread

13

Average Estimator

• How to read the graph:• Enters system: t = 5.4s• Estimated service time: 1.6s• Real service time: 4.9s

14

Average Estimator

• History becomes out-of-date

• Base estimation on the current state of the system

15

STQ Estimator

• It’s better than the Average Estimator…

• Inaccurate around a thread change

16

STQ Estimator

• Estimations will be inaccurate unless the estimator knows about the coming change in the number of threads

• After a change, recently-completed requests show a mixed view of system state and will pollute the history

17

STQT Estimator

• Overshoots target because of a polluted history

• Fix: Integrate request times across the thread change

18

Outline• Motivation

• Design & Implementation

• Evaluation

• Conclusion and Future Work

19

Concluding Remarks

• Takeaways– Don’t adapt to transient delays– Use current state along with the history– Be aware of underlying changes

• Future Directions– Extrapolate requests’ service times– URL service time as a distribution

Questions?