how to fight bit rot with types
DESCRIPTION
How to Fight Bit Rot With Types. Martin Odersky AOSD 2010. Types Can Cause Bit Rot. Types make contracts explicit, so they should help keeping systems consistent. However, if type systems are too week, they might require unnecessary duplication, which can cause bit rot. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/1.jpg)
How to Fight Bit Rot With Types
Martin Odersky
AOSD 2010
![Page 2: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/2.jpg)
Types Can Cause Bit Rot
Types make contracts explicit, so they should help keeping systems consistent.
However, if type systems are too week, they might require unnecessary duplication, which can cause bit rot.
Case Study: Scala collection framework.
![Page 3: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/3.jpg)
(quick dive into Scala
![Page 4: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/4.jpg)
Where it comes from
Scala has established itself as one of the main alternative languages on the JVM.
Prehistory:
1996 – 1997: Pizza1998 – 2000: GJ, Java generics, javac
Timeline: 2003 – 2006: The Scala Experiment2006 – 2009: Driving industrial adoption
![Page 5: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/5.jpg)
What’s Scala?
![Page 6: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/6.jpg)
Scala is a Unifier
Agile, with lightweight syntax
Object-Oriented Scala Functional
Safe and performant, with strong static typing
![Page 7: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/7.jpg)
Let’s see some code:
![Page 8: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/8.jpg)
A class ...
public class Person {
public final String name;
public final int age;
Person(String name, int age) {
this.name = name;
this.age = age;
}
}
class Person(val name: String, val age: Int) {}
... in Java:
... in Scala:
![Page 9: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/9.jpg)
... and its usageimport java.util.ArrayList;
...
Person[] people;
Person[] minors;
Person[] adults;
{ ArrayList<Person> minorsList = new ArrayList<Person>();
ArrayList<Person> adultsList = new ArrayList<Person>();
for (int i = 0; i < people.length; i++)
(people[i].age < 18 ? minorsList : adultsList)
.add(people[i]);
minors = minorsList.toArray(people);
adults = adultsList.toArray(people);
}
... in Java:
... in Scala: val people: Array[Person]val (minors, adults) = people partition (_.age < 18)
A simple pattern match
An infix method call
A function value
![Page 10: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/10.jpg)
But there’s more to it
![Page 11: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/11.jpg)
Embedding Domain-Specific Languages
Scala’s flexible syntax makes iteasy to define
high-level APIs & embedded DSLs
Examples:- Scala actors (the core of
Twitter’s message queues)- specs, ScalaCheck- ScalaFX- ScalaQuery
scalac’s plugin architecture makes it easy to typecheck DSLs and to enrich their semantics.
// asynchronous message send
actor ! message
// message receive
receive {
case msgpat1 => action1
…
case msgpatn => actionn
}
![Page 12: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/12.jpg)
The Bottom Line
When going from Java to Scala, expect at least a factor of 2 reduction in LOC.
But does it matter? Doesn’t Eclipse write these extra lines for me?
This does matter. Eye-tracking experiments* show that for program comprehension, average time spent per word of source code is constant.
So, roughly, half the code means half the time necessary to understand it.
*G. Dubochet. Computer Code as a Medium for Human Communication: Are Programming Languages Improving?In 21st Annual Psychology of Programming Interest Group Conference, pages 174-187, Limerick, Ireland, 2009.
![Page 13: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/13.jpg)
How to get started
100s of resources on the web.
Here are three greatentry points:
• Simply Scala• Scalazine @ artima.com• Scala for Java
refugees
![Page 14: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/14.jpg)
How to find out more
Scala site: www.scala-lang.org Eight books last year
![Page 15: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/15.jpg)
)
![Page 16: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/16.jpg)
Some Scala Collections
![Page 17: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/17.jpg)
Collection Traits
• object-oriented• generic: List[T], Map[K,
V]• optionally persistent, e.g.
collection.immutable.Seq
• higher-order, with methods such as foreach, map, filter.
• Uniform return type principle: Operations should return collections of the same type (constructor) as their left operand.
scala> val ys = List(1, 2, 3)ys: List[Int] = List(1, 2, 3)
scala> val xs: Seq[Int] = ysxs: Seq[Int] = List(1, 2, 3)
scala> xs map (_ + 1)res0: Seq[Int] = List(2, 3, 4)
scala> ys map (_ + 1)res1: List[Int] = List(2, 3, 4)
![Page 18: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/18.jpg)
Old Collection Structuretrait Iterable[A] { def filter(p: A => Boolean): Iterable[A] = ... def partition(p: A => Boolean) = (filter(p(_)), filter(!p(_))) def map[B](f: A => B): Iterable[B] = ...}
trait Seq[A] extends Iterable[A] { def filter(p: A => Boolean): Seq[A] = ... override def partition(p: A => Boolean) = (filter(p(_)), filter(!p(_))) def map[B](f: A => B): Seq[B] = ...}
![Page 19: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/19.jpg)
Types force duplication
filter needs to be re-defined on each level
partition also needs to be re-implemented on each level, even though its definition is everywhere the same.
The same pattern repeats for many other operations and types.
![Page 20: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/20.jpg)
Signs of Bit Rot
Lots of duplications of methods.– Methods returning collections have to be repeated for every
collection type.
Inconsistencies. – Sometimes methods such as filter, map were not specialized in
subclasses– More often, they only existed in subclasses, even though they
could be generalized
“Broken window” effect.– Classes that already had some ad-hoc methods became
dumping grounds for lots more.– Classes that didn’t stayed clean.
![Page 21: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/21.jpg)
Excerpts from List.scala
![Page 22: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/22.jpg)
How to do better?
Can we abstract out the return type?
Look at map: Need to abstract out the type constructor, not just the type.
But we can do that using Scala’s higher-kinded types!
trait Iterable[A] def map[B](f: A => B): Iterable[B]
trait Seq[A] def map[B](f: A => B): Seq[B]
![Page 23: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/23.jpg)
HK Types Collection Structure
trait TraversableLike[A, CC[X]] {
def filter(p: A => Boolean): CC[A]
def map[B](f: A => B): CC[B]
}
trait Traversable[A] extends TraversableLike[A, Traversable]
trait Iterable[A] extends TraversableLike[A, Iterable]
trait Seq[A] extends TraversableLike[A, Seq]
Here, CC is a parameter representing a type constructor.
![Page 24: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/24.jpg)
Implementation with Builders
All ops in Traversable are implemented in terms of foreach and newBuilder.
trait Builder[A, Coll] { def += (elem: A) // add elems def result: Coll // return result}trait TraversableLike[A, CC[X]] { def foreach(f: A => Unit) def newBuilder[B]: Builder[B, CC[B]] def map[B](f: A => B): CC[B] = { val b = newBuilder[B] foreach (x => b += f(x)) b.result }}
![Page 25: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/25.jpg)
Unfortunately ...
... things are not as parametric as it seems at first. Take:
scala> val bs = BitSet(1, 2, 3)bs: scala.collection.immutable.BitSet = BitSet(1, 2, 3)
scala> bs map (_ + 1)res0: scala.collection.immutable.BitSet = BitSet(2, 3, 4)
scala> bs map (_.toString + "!")res1: scala.collection.immutable.Set[java.lang.String] = Set(1!, 2!, 3!)
Note that the result type is the “best possible” type that fits the element type of the new collection.
Other examples: SortedSet, String.
class BitSet extends Set[Int]
![Page 26: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/26.jpg)
How to advance?
HK-types: cannot be enforced all the way down the type hierarchy, so cannot be defined at the roots.
Previous system: Leads to duplications and bit rot.
We need more flexibility. Can we define our own type system for collections?
Question: Given old collection type From, new element type Elem, and new collection type To: Can an operation on From build a collection of type To with Elem elements?
Captured in: CanBuildFrom[From, Elem, To]
![Page 27: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/27.jpg)
Facts about CanBuildFrom
Can be stated as axioms and inference rules:
CanBuildFrom[Traversable[A], B, Traversable[B]]
CanBuildFrom[Set[A], B, Set[B]]
CanBuildFrom[BitSet, B, Set[B]]
CanBuildFrom[BitSet, Int, BitSet]
CanBuildFrom[String, Char, String]
CanBuildFrom[String, B, Seq[B]]
CanBuildFrom[SortedSet[A], B, SortedSet[B]] :- Ordering[B]
where A and B are arbitrary types.
![Page 28: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/28.jpg)
Implicitly Injected Theories
Type theories such as the one for CanBuildFrom can be injected using implicits.
A predicate:trait CanBuildFrom[From, Elem, To] { def apply(coll: From): Builder[Elem, To]}
Axioms:implicit def bf1[A, B]: CanBuildFrom[Traversable[A], B,
Traversable[B]]implicit def bf2[A, B]: CanBuildFrom[Set[A], B, Set[B]]implicit def bf3: CanBuildFrom[BitSet, Int, BitSet]
Inference rule:implicit def bf4[A, B] (implicit ord: Ordering[B]) : CanBuildFrom[SortedSet[A], B, SortedSet[B]]
![Page 29: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/29.jpg)
Connecting with Map
• Here’s how map can be defined in terms CanBuildFrom:
trait TraversableLike[A, Coll] { this: Coll =>
def foreach(f: A => Unit)
def newBuilder: Builder[A, Coll]
def map[B, To](f: A => B)
(implicit cbf: CanBuildFrom[Coll, B, To]): To = {
val b = cbf(this)
foreach (x => b += f(x))
b.result
}
}
![Page 30: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/30.jpg)
So, no need for HK-Types?
Yes, there is, in collection factories
Factories provide common set of operations to build collections: empty, apply, fill, tabulate, iterate, ...
scala> List(1, 2, 3)res84 List[Int] = List(1, 2, 3)
scala> Vector(1, 2, 3)res5: Vector[Int] = Vector(1, 2, 3)
scala> List.tabulate(2, 2, 2)(_ + _ + _)res6: List[List[List[Int]]] = List(List(List(0, 1), List(1, 2)), List(List(1, 2), List(2, 3)))
scala> Vector.tabulate(2, 2, 2)(_ + _ + _)res7: Vector[Vector[Vector[Int]]] = Vector(Vector(Vector(0, 1), Vector(1, 2)), Vector(Vector(1, 2), Vector(2, 3)))
![Page 31: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/31.jpg)
Collection Factories
Implementation in terms of higher-kinded types:
trait TraversableFactory[CC[X] >: Null <: Traversable[X]] {
def newBuilder[A]: CC[A]
def apply[A](elems: A*): CC[A] = ...
def tabulate[A](n1: Int)(f: Int => A): CC[A] = ...
def tabulate[A](n1: Int, n2: Int)(f: (Int, Int) => A): CC[CC[A]] = ...
def tabulate[A](n1: Int, n2: Int, n3: Int)(f: (Int, Int, Int) => A) : CC[CC[CC[A]]] = ...
}
object Traversable extends TraversableFactory[Traversable] {
def newBuilder[A]: Traversable[A] = new ListBuffer[A]
}
Alternative implementation with CanBuildFrom is possible but painful.
![Page 32: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/32.jpg)
Resumé
• Higher-kinded types work 95% of the time• Unfortunately for collection methods this is not enough,
because they’d have to be defined in the root, to be inherited everywhere.
• Another case of the fragile base class problem• On the other hand, HK types work great in factories,
because a collection object can choose to inherit from a factory or not.
![Page 33: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/33.jpg)
Objections
![Page 34: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/34.jpg)
![Page 35: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/35.jpg)
Type Views
• How to explaindef map[B, To](f: A => B)
(implicit cbf: CanBuildFrom[Coll, B, To]): To
to a beginner?• Key observation: We can approximate the type of map.• For everyone but the most expert user
def map[B](f: A => B): Traversable[B] // in class Traversabledef map[B](f: A => B): Seq[B] // in class Seq, etc
is detailed enough.• These types are correct, they are just not as general as
the type that’s actually implemented.
![Page 36: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/36.jpg)
Possible Solution: Flexible Doc Comments
![Page 37: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/37.jpg)
Going Further?
• Modern software demands increasingly sophisticated type systems.– correctness, performance, parallelism.
• Tendency to cover more and more aspects of program execution:– interfaces, mutability, effects, ownership,
thread-locality, regions, ...
• Risk that programmers are left behind.• How to overcome this “type wall”?• Are better tools the right answer?
![Page 38: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/38.jpg)
Could Dynamic Typing Have Helped?
trait Iterable[A] { def filter(p) = ... def partition(p) =
(filter(p(_)), filter(!p(_))) def map[B](f) = ...}
trait Seq[A] extends Iterable[A] { }
Here Yes
![Page 39: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/39.jpg)
Could Dynamic Typing Have Helped?
Here? Much Harder
scala> val bs = BitSet(1, 2, 3)bs: BitSet = BitSet(1, 2, 3)
scala> bs map (_ + 1)res0: BitSet = BitSet(2, 3, 4)
scala> bs map (_.toString + "!")res1: Set[java.lang.String] = Set(1!, 2!, 3!)
![Page 40: How to Fight Bit Rot With Types](https://reader036.vdocuments.us/reader036/viewer/2022081506/56814e91550346895dbc3af0/html5/thumbnails/40.jpg)
Thank You
Questions?