games of prediction or things get simpler as yoav freund banter inc
TRANSCRIPT
![Page 1: Games of Prediction or Things get simpler as Yoav Freund Banter Inc](https://reader033.vdocuments.us/reader033/viewer/2022051316/56649ce15503460f949aaf61/html5/thumbnails/1.jpg)
Games of Predictionor
Things get simpler as
Yoav Freund
Banter Inc.
![Page 2: Games of Prediction or Things get simpler as Yoav Freund Banter Inc](https://reader033.vdocuments.us/reader033/viewer/2022051316/56649ce15503460f949aaf61/html5/thumbnails/2.jpg)
The Chip game
Game ends when this is empty
![Page 3: Games of Prediction or Things get simpler as Yoav Freund Banter Inc](https://reader033.vdocuments.us/reader033/viewer/2022051316/56649ce15503460f949aaf61/html5/thumbnails/3.jpg)
Plan of the talk
• Why is the chip game is interesting?
• Letting the number of chips go to infinity.
• The boosting game.
• Drifting games (let chip=sheep).
• Letting the number of rounds go to infinity
• Brownian motion and generalized boosting (nice pictures!)
![Page 4: Games of Prediction or Things get simpler as Yoav Freund Banter Inc](https://reader033.vdocuments.us/reader033/viewer/2022051316/56649ce15503460f949aaf61/html5/thumbnails/4.jpg)
Combining expert advice
• Binary sequence: 1,0,0,1,1,0,?
• N experts make predictions:
expert 1: 1,0,1,0,1,1,1
expert 2: 0,0,0,1,1,0,1
…
expert N: 1,0,0,1,1,1,0
• Algorithm makes prediction: 1,0,1,1,0,1,1
• Assumption: there exists an expert which makes at most k mistakes
• Goal: make least mistakes under the assumption (no statistics!)
• Chip = expert, bin = number of mistakes made
• Game step = algorithm’s prediction is incorrect.
![Page 5: Games of Prediction or Things get simpler as Yoav Freund Banter Inc](https://reader033.vdocuments.us/reader033/viewer/2022051316/56649ce15503460f949aaf61/html5/thumbnails/5.jpg)
Chip game for expert advice
Experts predicting 1
Experts predicting 0
No. of mistakesMade by expert
0 1 K K+1
Game ends whenthere is one chip here(rest have fallen off)
![Page 6: Games of Prediction or Things get simpler as Yoav Freund Banter Inc](https://reader033.vdocuments.us/reader033/viewer/2022051316/56649ce15503460f949aaf61/html5/thumbnails/6.jpg)
21 questions with k lies
• Player 1 chooses secret x from S1,S2,…,SN• Player 2 asks “is x in the set A?”• Player 1 can lie k times.• Game ends when player 2 can determine x.• A smart player 1 aims to have more than one
consistent x for as long as possible.• Chip = element (Si)• Bin = number of lies made regarding element
![Page 7: Games of Prediction or Things get simpler as Yoav Freund Banter Inc](https://reader033.vdocuments.us/reader033/viewer/2022051316/56649ce15503460f949aaf61/html5/thumbnails/7.jpg)
Chip game for 21 questions
element not in A
element in A
No. of lies made assuming element is the secret
0 1 K K+1
Game ends when thereis only one chip on this side
![Page 8: Games of Prediction or Things get simpler as Yoav Freund Banter Inc](https://reader033.vdocuments.us/reader033/viewer/2022051316/56649ce15503460f949aaf61/html5/thumbnails/8.jpg)
Simple case – all chips in bin 1
• 21 questions without lies
• Combining experts where one is perfect
• Splitting a cookie
• Optimal strategies:– Player 1: split chips into two equal parts– Player 2: choose larger part
• Note problem when number of chips is odd
![Page 9: Games of Prediction or Things get simpler as Yoav Freund Banter Inc](https://reader033.vdocuments.us/reader033/viewer/2022051316/56649ce15503460f949aaf61/html5/thumbnails/9.jpg)
Number of chips to infinity
Replace individual chips by chip mass
Optimal splitter strategy:Split each bin into two equal parts
![Page 10: Games of Prediction or Things get simpler as Yoav Freund Banter Inc](https://reader033.vdocuments.us/reader033/viewer/2022051316/56649ce15503460f949aaf61/html5/thumbnails/10.jpg)
Binomial weights strategies
• What should chooser do if parts are not equal?• Assume that on following iterations splitter will play
optimally = split each bin into 2 equal parts.• Future configuration independent of chooser’s choice• Potential: Fraction of bin i that will remain in bins 1..k
when m iterations remain
• Choose part so that next configuration will have maximal (or minimal) potential.
• Solve for m:
![Page 11: Games of Prediction or Things get simpler as Yoav Freund Banter Inc](https://reader033.vdocuments.us/reader033/viewer/2022051316/56649ce15503460f949aaf61/html5/thumbnails/11.jpg)
Optimality of strategy
• If the chips are infinitely divisible then the
solution is min/max optimal
• [Spencer95] Enough for optimality if number of chips is at least
![Page 12: Games of Prediction or Things get simpler as Yoav Freund Banter Inc](https://reader033.vdocuments.us/reader033/viewer/2022051316/56649ce15503460f949aaf61/html5/thumbnails/12.jpg)
Equivalence to a random walk
• Both sides playing optimally is equivalent to each chip performing an independent random walk.
• Potential = probability of a chip in bin i ending in bins 1..K after m iteration
• Weight = difference between the potentials of a chip in its two possible locations on the following iteration.
• Chooser’s optimal strategy: choose set with smaller (larger) weight
![Page 13: Games of Prediction or Things get simpler as Yoav Freund Banter Inc](https://reader033.vdocuments.us/reader033/viewer/2022051316/56649ce15503460f949aaf61/html5/thumbnails/13.jpg)
Example potential and weight
m=5; k=10
![Page 14: Games of Prediction or Things get simpler as Yoav Freund Banter Inc](https://reader033.vdocuments.us/reader033/viewer/2022051316/56649ce15503460f949aaf61/html5/thumbnails/14.jpg)
Boosting
• A method for improving classifier accuracy• Weak Learner: a learning algorithm
generating rules slightly better than random guessing.
• Basic idea: re-weight training examples to force weak learner into different parts of the space.
• Combine weak rules by a majority vote.
![Page 15: Games of Prediction or Things get simpler as Yoav Freund Banter Inc](https://reader033.vdocuments.us/reader033/viewer/2022051316/56649ce15503460f949aaf61/html5/thumbnails/15.jpg)
Boosting as a chip game
• Chip = training example• Booster assigns a weight to each chip,
weights sum to 1.• Learner selects a subset with weight
– Selected set moves a step right (correct)– Unselected set moves a step left (incorrect)
• Booster wins examples on right of origin.– Implies that majority vote is correct.
![Page 16: Games of Prediction or Things get simpler as Yoav Freund Banter Inc](https://reader033.vdocuments.us/reader033/viewer/2022051316/56649ce15503460f949aaf61/html5/thumbnails/16.jpg)
The boosting chip game
h(x) = y
h(x) <> y
.2
.2
.2
.1
.1
.1
.1
Majority vote is correct
![Page 17: Games of Prediction or Things get simpler as Yoav Freund Banter Inc](https://reader033.vdocuments.us/reader033/viewer/2022051316/56649ce15503460f949aaf61/html5/thumbnails/17.jpg)
Binomial strategies for boosting
• Learner moves chip right independently with probability ½+
• The potential of example that is correctly classified r times after i out of k iterations
• Booster assigns to each example a weight proportional to the difference between its possible next-step potentials.
![Page 18: Games of Prediction or Things get simpler as Yoav Freund Banter Inc](https://reader033.vdocuments.us/reader033/viewer/2022051316/56649ce15503460f949aaf61/html5/thumbnails/18.jpg)
Drifting games (in 2d)
Targetregion
![Page 19: Games of Prediction or Things get simpler as Yoav Freund Banter Inc](https://reader033.vdocuments.us/reader033/viewer/2022051316/56649ce15503460f949aaf61/html5/thumbnails/19.jpg)
Targetregion
Drifting games (in 2d)
![Page 20: Games of Prediction or Things get simpler as Yoav Freund Banter Inc](https://reader033.vdocuments.us/reader033/viewer/2022051316/56649ce15503460f949aaf61/html5/thumbnails/20.jpg)
The allowable steps
B = the set of all allowable stepNormal B = minimal set that spans the space. (~basis)Regular B = a symmetric regular set. (~orthonormal basis)
Regular step set in R2
![Page 21: Games of Prediction or Things get simpler as Yoav Freund Banter Inc](https://reader033.vdocuments.us/reader033/viewer/2022051316/56649ce15503460f949aaf61/html5/thumbnails/21.jpg)
The min/max solution
• A potential defined by a min/max recursion
• Shepherd’s strategy
[Schapire99]
![Page 22: Games of Prediction or Things get simpler as Yoav Freund Banter Inc](https://reader033.vdocuments.us/reader033/viewer/2022051316/56649ce15503460f949aaf61/html5/thumbnails/22.jpg)
![Page 23: Games of Prediction or Things get simpler as Yoav Freund Banter Inc](https://reader033.vdocuments.us/reader033/viewer/2022051316/56649ce15503460f949aaf61/html5/thumbnails/23.jpg)
Increasing the number of steps
• Consider T steps in a unit time
• Drift should scale like 1/T
• Step size O(1/T) gives game to shepherd
• Step size keeps game balanced
![Page 24: Games of Prediction or Things get simpler as Yoav Freund Banter Inc](https://reader033.vdocuments.us/reader033/viewer/2022051316/56649ce15503460f949aaf61/html5/thumbnails/24.jpg)
The solution when
The local slope becomes the gradient
The recursion becomes a PDE
Same PDE describes time development ofBrownian motion with drift
![Page 25: Games of Prediction or Things get simpler as Yoav Freund Banter Inc](https://reader033.vdocuments.us/reader033/viewer/2022051316/56649ce15503460f949aaf61/html5/thumbnails/25.jpg)
Special Cases
• Target func. = PDE boundary condition at time=1.
• Target func. = exponential – potential and weight are exponential at all times
– Adaboost.
– Exponential weights algs for online learning.
• Target func. = step function– Potential is the error function, weight is the normal distribution.
– Potential and weight change with time.
– Brownboost.
![Page 26: Games of Prediction or Things get simpler as Yoav Freund Banter Inc](https://reader033.vdocuments.us/reader033/viewer/2022051316/56649ce15503460f949aaf61/html5/thumbnails/26.jpg)
Potential applications
• Generalized boosting– Classification for >2 labels– Regression– Density estimation– Variational function fitting
• Generalized online learning– Continuous predictions (instead of binary)
![Page 27: Games of Prediction or Things get simpler as Yoav Freund Banter Inc](https://reader033.vdocuments.us/reader033/viewer/2022051316/56649ce15503460f949aaf61/html5/thumbnails/27.jpg)
![Page 28: Games of Prediction or Things get simpler as Yoav Freund Banter Inc](https://reader033.vdocuments.us/reader033/viewer/2022051316/56649ce15503460f949aaf61/html5/thumbnails/28.jpg)
Boosting for variational optimization
Minimize
f(x)
x
y
![Page 29: Games of Prediction or Things get simpler as Yoav Freund Banter Inc](https://reader033.vdocuments.us/reader033/viewer/2022051316/56649ce15503460f949aaf61/html5/thumbnails/29.jpg)
![Page 30: Games of Prediction or Things get simpler as Yoav Freund Banter Inc](https://reader033.vdocuments.us/reader033/viewer/2022051316/56649ce15503460f949aaf61/html5/thumbnails/30.jpg)
![Page 31: Games of Prediction or Things get simpler as Yoav Freund Banter Inc](https://reader033.vdocuments.us/reader033/viewer/2022051316/56649ce15503460f949aaf61/html5/thumbnails/31.jpg)
![Page 32: Games of Prediction or Things get simpler as Yoav Freund Banter Inc](https://reader033.vdocuments.us/reader033/viewer/2022051316/56649ce15503460f949aaf61/html5/thumbnails/32.jpg)
![Page 33: Games of Prediction or Things get simpler as Yoav Freund Banter Inc](https://reader033.vdocuments.us/reader033/viewer/2022051316/56649ce15503460f949aaf61/html5/thumbnails/33.jpg)