ocamlot: ocaml online testing · ocamlot: ocaml online testing ... david sheets, anil madhavapeddy,...
TRANSCRIPT
![Page 1: OCamlot: OCaml Online Testing · OCamlot: OCaml Online Testing ... David Sheets, Anil Madhavapeddy, Amir Chaudhry and ... I Repository with revision dependencies](https://reader031.vdocuments.us/reader031/viewer/2022021423/5accf4d07f8b9a6a678cecea/html5/thumbnails/1.jpg)
OCamlot: OCaml Online TestingState for the Open Source OCaml Community
David Sheets, Anil Madhavapeddy, Amir Chaudhry andThomas Gazagnaire
University of Cambridge and OCamlPro
![Page 2: OCamlot: OCaml Online Testing · OCamlot: OCaml Online Testing ... David Sheets, Anil Madhavapeddy, Amir Chaudhry and ... I Repository with revision dependencies](https://reader031.vdocuments.us/reader031/viewer/2022021423/5accf4d07f8b9a6a678cecea/html5/thumbnails/2.jpg)
OCamlot : OCaml Online Testing
The Quality Quality
A Typical Typist
The Cybernetic Curator
Just Representation
To Avalon
![Page 3: OCamlot: OCaml Online Testing · OCamlot: OCaml Online Testing ... David Sheets, Anil Madhavapeddy, Amir Chaudhry and ... I Repository with revision dependencies](https://reader031.vdocuments.us/reader031/viewer/2022021423/5accf4d07f8b9a6a678cecea/html5/thumbnails/3.jpg)
OCamlot : OCaml Online Testing
The Quality Quality
A Typical Typist
The Cybernetic Curator
Just Representation
To Avalon
![Page 4: OCamlot: OCaml Online Testing · OCamlot: OCaml Online Testing ... David Sheets, Anil Madhavapeddy, Amir Chaudhry and ... I Repository with revision dependencies](https://reader031.vdocuments.us/reader031/viewer/2022021423/5accf4d07f8b9a6a678cecea/html5/thumbnails/4.jpg)
A Caml on Every Core
I Consistent results (“It Just Works”)
I Obvious solutions (“What have I forgotten?”)
I Addictive actions (“I want to install more software.”)
I Quality functions (“Just what I wanted.”)
![Page 5: OCamlot: OCaml Online Testing · OCamlot: OCaml Online Testing ... David Sheets, Anil Madhavapeddy, Amir Chaudhry and ... I Repository with revision dependencies](https://reader031.vdocuments.us/reader031/viewer/2022021423/5accf4d07f8b9a6a678cecea/html5/thumbnails/5.jpg)
A Caml on Every Core
I Consistent results (“It Just Works”)
I Obvious solutions (“What have I forgotten?”)
I Addictive actions (“I want to install more software.”)
I Quality functions (“Just what I wanted.”)
![Page 6: OCamlot: OCaml Online Testing · OCamlot: OCaml Online Testing ... David Sheets, Anil Madhavapeddy, Amir Chaudhry and ... I Repository with revision dependencies](https://reader031.vdocuments.us/reader031/viewer/2022021423/5accf4d07f8b9a6a678cecea/html5/thumbnails/6.jpg)
A Caml on Every Core
I Consistent results (“It Just Works”)
I Obvious solutions (“What have I forgotten?”)
I Addictive actions (“I want to install more software.”)
I Quality functions (“Just what I wanted.”)
![Page 7: OCamlot: OCaml Online Testing · OCamlot: OCaml Online Testing ... David Sheets, Anil Madhavapeddy, Amir Chaudhry and ... I Repository with revision dependencies](https://reader031.vdocuments.us/reader031/viewer/2022021423/5accf4d07f8b9a6a678cecea/html5/thumbnails/7.jpg)
A Caml on Every Core
I Consistent results (“It Just Works”)
I Obvious solutions (“What have I forgotten?”)
I Addictive actions (“I want to install more software.”)
I Quality functions (“Just what I wanted.”)
![Page 8: OCamlot: OCaml Online Testing · OCamlot: OCaml Online Testing ... David Sheets, Anil Madhavapeddy, Amir Chaudhry and ... I Repository with revision dependencies](https://reader031.vdocuments.us/reader031/viewer/2022021423/5accf4d07f8b9a6a678cecea/html5/thumbnails/8.jpg)
Universal Concerns
I Compiler hypothesesI New injectivity restriction is minimally disruptive
I Build system hypothesesI Compiler commands can be diverted
I Package manager hypothesesI OPAM 1.1 is stable
I Environment hypothesesI Packages silent on OS/arch dependency can build and run
everywhere
![Page 9: OCamlot: OCaml Online Testing · OCamlot: OCaml Online Testing ... David Sheets, Anil Madhavapeddy, Amir Chaudhry and ... I Repository with revision dependencies](https://reader031.vdocuments.us/reader031/viewer/2022021423/5accf4d07f8b9a6a678cecea/html5/thumbnails/9.jpg)
Universal Concerns
I Compiler hypothesesI New injectivity restriction is minimally disruptive
I Build system hypothesesI Compiler commands can be diverted
I Package manager hypothesesI OPAM 1.1 is stable
I Environment hypothesesI Packages silent on OS/arch dependency can build and run
everywhere
![Page 10: OCamlot: OCaml Online Testing · OCamlot: OCaml Online Testing ... David Sheets, Anil Madhavapeddy, Amir Chaudhry and ... I Repository with revision dependencies](https://reader031.vdocuments.us/reader031/viewer/2022021423/5accf4d07f8b9a6a678cecea/html5/thumbnails/10.jpg)
Universal Concerns
I Compiler hypothesesI New injectivity restriction is minimally disruptive
I Build system hypothesesI Compiler commands can be diverted
I Package manager hypothesesI OPAM 1.1 is stable
I Environment hypothesesI Packages silent on OS/arch dependency can build and run
everywhere
![Page 11: OCamlot: OCaml Online Testing · OCamlot: OCaml Online Testing ... David Sheets, Anil Madhavapeddy, Amir Chaudhry and ... I Repository with revision dependencies](https://reader031.vdocuments.us/reader031/viewer/2022021423/5accf4d07f8b9a6a678cecea/html5/thumbnails/11.jpg)
Universal Concerns
I Compiler hypothesesI New injectivity restriction is minimally disruptive
I Build system hypothesesI Compiler commands can be diverted
I Package manager hypothesesI OPAM 1.1 is stable
I Environment hypothesesI Packages silent on OS/arch dependency can build and run
everywhere
![Page 12: OCamlot: OCaml Online Testing · OCamlot: OCaml Online Testing ... David Sheets, Anil Madhavapeddy, Amir Chaudhry and ... I Repository with revision dependencies](https://reader031.vdocuments.us/reader031/viewer/2022021423/5accf4d07f8b9a6a678cecea/html5/thumbnails/12.jpg)
OCamlot : OCaml Online Testing
The Quality Quality
A Typical Typist
The Cybernetic Curator
Just Representation
To Avalon
![Page 13: OCamlot: OCaml Online Testing · OCamlot: OCaml Online Testing ... David Sheets, Anil Madhavapeddy, Amir Chaudhry and ... I Repository with revision dependencies](https://reader031.vdocuments.us/reader031/viewer/2022021423/5accf4d07f8b9a6a678cecea/html5/thumbnails/13.jpg)
Evaluating the future
I Typical typist makes pull request to repository
I Curator observes proposal
I Hypothetical consequences evaluated
I Potential amendments proposed
I Repository administrator signs with 1 click
I E.g.: Weekly Core release
![Page 14: OCamlot: OCaml Online Testing · OCamlot: OCaml Online Testing ... David Sheets, Anil Madhavapeddy, Amir Chaudhry and ... I Repository with revision dependencies](https://reader031.vdocuments.us/reader031/viewer/2022021423/5accf4d07f8b9a6a678cecea/html5/thumbnails/14.jpg)
Evaluating the future
I Typical typist makes pull request to repository
I Curator observes proposal
I Hypothetical consequences evaluated
I Potential amendments proposed
I Repository administrator signs with 1 click
I E.g.: Weekly Core release
![Page 15: OCamlot: OCaml Online Testing · OCamlot: OCaml Online Testing ... David Sheets, Anil Madhavapeddy, Amir Chaudhry and ... I Repository with revision dependencies](https://reader031.vdocuments.us/reader031/viewer/2022021423/5accf4d07f8b9a6a678cecea/html5/thumbnails/15.jpg)
OCamlot : OCaml Online Testing
The Quality Quality
A Typical Typist
The Cybernetic Curator
Just Representation
To Avalon
![Page 16: OCamlot: OCaml Online Testing · OCamlot: OCaml Online Testing ... David Sheets, Anil Madhavapeddy, Amir Chaudhry and ... I Repository with revision dependencies](https://reader031.vdocuments.us/reader031/viewer/2022021423/5accf4d07f8b9a6a678cecea/html5/thumbnails/16.jpg)
The Shared Repository Management Game
I Objective: promote public health and safety
I Strategy: get inside the humans’ loops
I Loop: Observe, Orient, Decide, Act
I Advantage: scale and heterogeneity
t
![Page 17: OCamlot: OCaml Online Testing · OCamlot: OCaml Online Testing ... David Sheets, Anil Madhavapeddy, Amir Chaudhry and ... I Repository with revision dependencies](https://reader031.vdocuments.us/reader031/viewer/2022021423/5accf4d07f8b9a6a678cecea/html5/thumbnails/17.jpg)
Repository Observation
I GitHub bindings with web hooksI Pull RequestsI CommitsI Comments
I Pre-build merge
I Other event streams?
![Page 18: OCamlot: OCaml Online Testing · OCamlot: OCaml Online Testing ... David Sheets, Anil Madhavapeddy, Amir Chaudhry and ... I Repository with revision dependencies](https://reader031.vdocuments.us/reader031/viewer/2022021423/5accf4d07f8b9a6a678cecea/html5/thumbnails/18.jpg)
Dependency Orientation
I Capabilities with environment dependencies
I Packages with constraint-based dependencies
I Repository with revision dependencies
I Artifact and task interdependencies
I Independence predicate
I E.g.: Jerome Vouillon’s Android OPAM repository
![Page 19: OCamlot: OCaml Online Testing · OCamlot: OCaml Online Testing ... David Sheets, Anil Madhavapeddy, Amir Chaudhry and ... I Repository with revision dependencies](https://reader031.vdocuments.us/reader031/viewer/2022021423/5accf4d07f8b9a6a678cecea/html5/thumbnails/19.jpg)
Dependency Orientation
I Capabilities with environment dependencies
I Packages with constraint-based dependencies
I Repository with revision dependencies
I Artifact and task interdependencies
I Independence predicate
I E.g.: Jerome Vouillon’s Android OPAM repository
![Page 20: OCamlot: OCaml Online Testing · OCamlot: OCaml Online Testing ... David Sheets, Anil Madhavapeddy, Amir Chaudhry and ... I Repository with revision dependencies](https://reader031.vdocuments.us/reader031/viewer/2022021423/5accf4d07f8b9a6a678cecea/html5/thumbnails/20.jpg)
Dependency Orientation
I Capabilities with environment dependencies
I Packages with constraint-based dependencies
I Repository with revision dependencies
I Artifact and task interdependencies
I Independence predicate
I E.g.: Jerome Vouillon’s Android OPAM repository
![Page 21: OCamlot: OCaml Online Testing · OCamlot: OCaml Online Testing ... David Sheets, Anil Madhavapeddy, Amir Chaudhry and ... I Repository with revision dependencies](https://reader031.vdocuments.us/reader031/viewer/2022021423/5accf4d07f8b9a6a678cecea/html5/thumbnails/21.jpg)
Triage Decisions
I Classify common error modesI UnsatisfiableI DependencyI TransientI SystemI MetadataI External dependencyI Build errorI Multiple
I Judge severity and impact
I Generate amendments to resolve
![Page 22: OCamlot: OCaml Online Testing · OCamlot: OCaml Online Testing ... David Sheets, Anil Madhavapeddy, Amir Chaudhry and ... I Repository with revision dependencies](https://reader031.vdocuments.us/reader031/viewer/2022021423/5accf4d07f8b9a6a678cecea/html5/thumbnails/22.jpg)
State Action
I Commit task intention
I Commit build evidence
I Commit error analysis
I Commit proposed amendment
![Page 23: OCamlot: OCaml Online Testing · OCamlot: OCaml Online Testing ... David Sheets, Anil Madhavapeddy, Amir Chaudhry and ... I Repository with revision dependencies](https://reader031.vdocuments.us/reader031/viewer/2022021423/5accf4d07f8b9a6a678cecea/html5/thumbnails/23.jpg)
OCamlot : OCaml Online Testing
The Quality Quality
A Typical Typist
The Cybernetic Curator
Just Representation
To Avalon
![Page 24: OCamlot: OCaml Online Testing · OCamlot: OCaml Online Testing ... David Sheets, Anil Madhavapeddy, Amir Chaudhry and ... I Repository with revision dependencies](https://reader031.vdocuments.us/reader031/viewer/2022021423/5accf4d07f8b9a6a678cecea/html5/thumbnails/24.jpg)
Just Representation
I Agents negotiate their tasks over HTTPS
I Independent metadata layers for orthogonal aspects
I Persistence to a layer in a branch consistent store
I All byproducts serialized as s-expressions
I Branch-consistent stores like Git or Irminsule
![Page 25: OCamlot: OCaml Online Testing · OCamlot: OCaml Online Testing ... David Sheets, Anil Madhavapeddy, Amir Chaudhry and ... I Repository with revision dependencies](https://reader031.vdocuments.us/reader031/viewer/2022021423/5accf4d07f8b9a6a678cecea/html5/thumbnails/25.jpg)
Just Representation
I Agents negotiate their tasks over HTTPS
I Independent metadata layers for orthogonal aspects
I Persistence to a layer in a branch consistent store
I All byproducts serialized as s-expressions
I Branch-consistent stores like Git or Irminsule
![Page 26: OCamlot: OCaml Online Testing · OCamlot: OCaml Online Testing ... David Sheets, Anil Madhavapeddy, Amir Chaudhry and ... I Repository with revision dependencies](https://reader031.vdocuments.us/reader031/viewer/2022021423/5accf4d07f8b9a6a678cecea/html5/thumbnails/26.jpg)
Our Present Executive
I Observation available and active
I Orientation minimally prototyped
I Basic triage heuristics complete
I No amendments
I Ephemeral task system (commit evidence+analysis)
I Simple reactions
![Page 27: OCamlot: OCaml Online Testing · OCamlot: OCaml Online Testing ... David Sheets, Anil Madhavapeddy, Amir Chaudhry and ... I Repository with revision dependencies](https://reader031.vdocuments.us/reader031/viewer/2022021423/5accf4d07f8b9a6a678cecea/html5/thumbnails/27.jpg)
History So Far
I Lots of metadata bugs
I Many package bugs
I Some tool bugs
I 30+ x86-64, 30+ x86-32
I 8+ ARMv6, ARMv5tel, PPC, PPC64, Sparc64...
I Multiple Linux distros, FreeBSD, OpenBSD, OS X
I Dead Rpi, panicking *nix filesystems
I Don’t make warnings fatal!
![Page 28: OCamlot: OCaml Online Testing · OCamlot: OCaml Online Testing ... David Sheets, Anil Madhavapeddy, Amir Chaudhry and ... I Repository with revision dependencies](https://reader031.vdocuments.us/reader031/viewer/2022021423/5accf4d07f8b9a6a678cecea/html5/thumbnails/28.jpg)
OCamlot : OCaml Online Testing
The Quality Quality
A Typical Typist
The Cybernetic Curator
Just Representation
To Avalon
![Page 29: OCamlot: OCaml Online Testing · OCamlot: OCaml Online Testing ... David Sheets, Anil Madhavapeddy, Amir Chaudhry and ... I Repository with revision dependencies](https://reader031.vdocuments.us/reader031/viewer/2022021423/5accf4d07f8b9a6a678cecea/html5/thumbnails/29.jpg)
To Avalon
I opamfu for DAG analysis + memoization
I Forward migration
I Lower overhead to administer exotic workers
I Advanced automation (amendments, chat bots)
I Better site integration
I Benchmarking
I Universal experiments with differential analysisI VM-based test systems
I 500+ on-demand machine instances from Rackspace
![Page 30: OCamlot: OCaml Online Testing · OCamlot: OCaml Online Testing ... David Sheets, Anil Madhavapeddy, Amir Chaudhry and ... I Repository with revision dependencies](https://reader031.vdocuments.us/reader031/viewer/2022021423/5accf4d07f8b9a6a678cecea/html5/thumbnails/30.jpg)
OCamlot : OCaml Online Testing
The Quality Quality
A Typical Typist
The Cybernetic Curator
Just Representation
To Avalon