1. Scala - where are are - where we plan to go Martin Odersky

2. Welcome to ScalaDays! 155 attendees, coming from all corners of the planet. We are packed. Sadly, many more people could not be admitted for lack of space. This shows the drive and energy of this community! The Scala team is looking forward to talk with you and learn from you during the next two days. We are particularly glad to have you here at EPFL, the place where Scala was born.

3. History (incomplete) 2001Scala design begins, as a more practical successor to Funnel. 2003First experimental release, first course taught with it. 2004Scalable Component Abstractions paper. 2005Scala 2.0, written in Scala. 2006Industrial adoption starts. 2007First Lift release. 2008First Scala LiftOff, Twitter adopts Scala. 2009Big increase in adoption, IDEs mature. 2010First ScalaDays.

4. The main idea behind Scala

5. Scala is a Unifier Agile, with lightweight syntax Object-Oriented Scala Functional Safe and performant, with strong static typing

6. The Philosophy behind Scala Put productivity and creativity back in the hands of developers. “Joy is underrated as a metric for a language’s potential success in a development organization.” a3lx@twitter - address professional developers - trust them & value their expertise - (don’t tell them how they should do their job)

7. Some users in industry

8. Open Source Ecosystem Many great projects, including: lift (David P. & friends) akka (Jonas B.) kestrel, cachet, queroulous(Twitter) migrations (Sony) sbt(Mark H.) ScalaTest(Bill V. & friends) specs, ScalaCheck, ScalaModules, ScalaQL,.... ( the list goes on, many more tools & projects presented at this conference )

9. Now: 2.8 First release candidate Scala 2.8 RC1 is out! This is primarily a consolidating release Some new language features More complete libraries Redesigns of a few features that did not work out well. Big improvements in tooling. Big improvements in stability. Laying the groundwork for the future. Scala 2.8 named and default parameters copy operation nested annotations package objects safer package nesting @specialized new collection API vectors hash tries new treatment of arrays and strings manifests better implicits better type inference continuations packrat parsing faster actors improved Swing support better IDEs interactive compiler API new scaladoc better REPL

10. Named and Default Parameters Scala 2.8 named and default parameters copy operation nested annotations package objects safer package nesting @specialized new collection API vectors hash tries new treatment of arrays and strings manifests better implicits better type inference continuations packrat parsing faster actors improved Swing support better IDEs interactive compiler API new scaladoc better REPL def newTable(size: Int = 100, load: Float = 0.5f) = new HashMap[String, String]{ override val initialSize = size override val loadFactor = (load * 1000).toInt } newTable(50, 0.75f) newTable() newTable(load = 0.5f) newTable(size = 20, load = 0.5f) Defaults can be arbitrary expressions, not just constants. Default evaluation is dynamic.

11. Copy Operation Scala 2.8 named and default parameters copy operation nested annotations package objects safer package nesting @specialized new collection API vectors hash tries new treatment of arrays and strings manifests better implicits better type inference continuations packrat parsing faster actors improved Swing support better IDEs interactive compiler API new scaladoc better REPL Problem: Case classes are great for pattern matching, but how do you transform data? Often need to update (functionally) some specific field(s) of a date structure defined by case classes. Solution: Selective copy method. Q: How to define “copy”? case class Tree[+T](elem: T, left: Tree[T], right: Tree[T]) def incElem(t: Tree[Int]) = t copy (elem = t.elem + 1)

12. Defining “copy” Scala 2.8 named and default parameters copy operation nested annotations package objects safer package nesting @specialized new collection API vectors hash tries new treatment of arrays and strings manifests better implicits better type inference continuations packrat parsing faster actors improved Swing support better IDEs interactive compiler API new scaladoc better REPL Problem: A class with n parameters allows 2 n possible combinations of parameters to copy. Default parameters to the rescue. case class Tree[+T](elem: T, left: Tree[T], right: Tree[T]) { // generated automatically: def copy[U](elem: U = this.elem, left: Tree[U] = this.left, right: Tree[U] = this.right) = Branch(elem, left, right) }

13. Nested Annotations Scala 2.7 largely followed Java annotation syntax, but left out some of the more convoluted constructions.  Nested annotations were not possible. Scala 2.8 throws out (deprecates) all special annotation syntax. Instead uses class constructors, which can have named parameters. This greatly simplifies the syntax, and allows all annotations to be expressed, including nested ones. Scala 2.8 named and default parameters copy operation nested annotations package objects safer package nesting @specialized new collection API vectors hash tries new treatment of arrays and strings manifests better implicits better type inference continuations packrat parsing faster actors improved Swing support better IDEs interactive compiler API new scaladoc better REPL @JoinTable( joinColumns = Array(new JoinColumn(name = "a name“) ) var test: String = _

14. @specialized Problem: Generics cause a performance hit because they require a uniform representation: everything needs to be boxed. Example: Scala 2.8 named and default parameters copy operation nested annotations @specialized new collection API vectors hash tries new treatment of arrays and strings package objects safer package nesting manifests better implicits better type inference continuations packrat parsing faster actors improved Swing support better IDEs interactive compiler API new scaladoc better REPL trait Function1[-T, +U] { def apply(x: T): U } def app(x: Int, fn: Int => Int) = fn(x) app(7, x => x * x) Translated by scalac 2.7 to: trait Function1 { def apply(x: Object): Object } class anonfun extends Function1 { def apply(x: Object): Object = Int.box(apply(Int.unbox(x))) def apply(x: Int): Int = x * x } def app(x: Int, fn: Function1) = Int.unbox(fn(Int.box(x))) app(7, new anonfun)

15. If Function1 is @specialized... Scala 2.8 named and default parameters copy operation nested annotations @specialized new collection API vectors hash tries new treatment of arrays and strings package objects safer package nesting manifests better implicits better type inference continuations packrat parsing faster actors improved Swing support better IDEs interactive compiler API new scaladoc better REPL trait Function1[@specialized -T, @specialized +U] { def apply(x: T): U } def app(x: Int, fn: Int => Int) = fn(x) app(7, x => x * x)... scalac 2.8 generates instead: trait Function1 { def apply(x: Object): Object def apply$$II(x: Int): Int = Int.unbox(apply(Int.box(x))) // specializations for other primitive types } class anonfun extends (Int => Int) { def apply(x: Object): Object = Int.box(applyIntInt(Int.unbox(x))) override def apply$$II(x: Int): Int = x * x } def app(x: Int, fn: Function1) = fn.apply$II(x) app(7, anonfun)

16. Scala Collections Scala 2.8 named and default parameters copy operation nested annotations package objects safer package nesting @specialized new collection API vectors hash tries new treatment of arrays and strings manifests better implicits better type inference continuations packrat parsing faster actors improved Swing support better IDEs interactive compiler API new scaladoc better REPL Collection Properties: object-oriented generic: List[T], Map[K, V] optionally persistent, e.g. collection.immutable.Seq higher-order, with methods such as foreach, map, filter. Uniform return type principle: Operations return collections of the same type (constructor) as their left operand. scala> val ys = List(1, 2, 3) ys: List[Int] = List(1, 2, 3) scala> val xs: Seq[Int] = ys xs: Seq[Int] = List(1, 2, 3) scala> xs map (_ + 1) res0: Seq[Int] = List(2, 3, 4) scala> ys map (_ + 1) res1: List[Int] = List(2, 3, 4)

17. Old Collection Structure Scala 2.8 named and default parameters copy operation nested annotations package objects safer package nesting @specialized new collection API vectors hash tries new treatment of arrays and strings manifests better implicits better type inference continuations packrat parsing faster actors improved Swing support better IDEs interactive compiler API new scaladoc better REPL trait Iterable[A] { def filter(p: A => Boolean): Iterable[A] =... def partition(p: A => Boolean) = (filter(p(_)), filter(!p(_))) def map[B](f: A => B): Iterable[B] =... } trait Seq[A] extends Iterable[A] { def filter(p: A => Boolean): Seq[A] =... override def partition(p: A => Boolean) = (filter(p(_)), filter(!p(_))) def map[B](f: A => B): Seq[B] =... } Types force duplication!

18. New Collection API Avoids type & code duplication through –uniform framework of traversers and builders. –supported by: implicits, higher-kinded types Uniform package organization reflects mutability: –scala.collection –scala.collection.mutable –scala.collection.immutable Richer and more regular set of collection operations. New collection implementations: vectors and hash tries. More info: “Fighting bit rot with types”, FSTTCS09 Scala 2.8 named and default parameters copy operation nested annotations package objects safer package nesting @specialized new collection API vectors hash tries new treatment of arrays and strings manifests better implicits better type inference continuations packrat parsing faster actors improved Swing support better IDEs interactive compiler API new scaladoc better REPL

19. Vectors and Hash Tries Trees with branch factor of 32. Persistent data structures with very efficient sequential and random access. Invented by Phil Bagwell, then adopted in Clojure. New: Persistent prepend/append/update in constant amortized time. Next: Fast splits and joins for parallel transformations. Scala 2.8 named and default parameters copy operation nested annotations package objects safer package nesting @specialized new collection API vectors hash tries new treatment of arrays and strings manifests better implicits better type inference continuations packrat parsing faster actors improved Swing support better IDEs interactive compiler API new scaladoc better REPL

20. Package Objects Motivation: The new collection design reshuffled a lot of classes. For instance: scala.List  scala.collection.immutable.List How can we keep the old aliases around? How can we migrate old to new? Easy: put type List = scala.collection.immutable.List val List = scala.collection.immutable.List into the scala package. But how to we put a type alias and a val definition into a package? Scala 2.8 named and default parameters copy operation nested annotations package objects safer package nesting @specialized new collection API vectors hash tries new treatment of arrays and strings manifests better implicits better type inference continuations packrat parsing faster actors improved Swing support better IDEs interactive compiler API new scaladoc better REPL

21. Package Objects Scala 2.8 named and default parameters copy operation nested annotations package objects safer package nesting @specialized new collection API vectors hash tries new treatment of arrays and strings manifests better implicits better type inference continuations packrat parsing faster actors improved Swing support better IDEs interactive compiler API new scaladoc better REPL Problem: Everything has to live inside an object or class. => No top-level value definitions, type aliases, implicits. Package objects make packages more like objects. They also let you define your own ``Predefs’’. package object scala { type List[+T] = scala.collection.immutable.List val List = scala.collection.immutable.List... }

22. Safer Package Nesting Java knows only absolute packages But Scala’s packages nest This can lead to an impedance mismatch. For instance: package net.liftweb java.lang.System.print(“hello”) This used to fail if net contains a subpackage java. New rules: Intermediate packages only visible in multiple package clauses. To access net.java as java, you’d need to write package net package liftweb... Scala 2.8 named and default parameters copy operation nested annotations package objects safer package nesting @specialized new collection API vectors hash tries new treatment of arrays and strings manifests better implicits better type inference continuations packrat parsing faster actors improved Swing support better IDEs interactive compiler API new scaladoc better REPL

23. New Treatment of Arrays Scala 2.8 named and default parameters copy operation nested annotations package objects safer package nesting @specialized new collection API vectors hash tries new treatment of arrays and strings manifests better implicits better type inference continuations packrat parsing faster actors improved Swing support better IDEs interactive compiler API new scaladoc better REPL Arrays were “magical” up to 2.8 –sometimes primitive –sometimes wrapped –automatic boxing/unboxing conversions based on static type. This could be confusing and slow. 2.8 removes the magic. Arrays are always Java arrays Two implicit conversions make sure that –arrays are compatible with sequences. –arrays support all sequence operations Strings are treated similarly to arrays.

24. Manifests Question: How to create an Array[T] where T is a type parameter? No magic allowed! Solution: supply the run-time type as a type descriptor called a Manifest or ClassManifest : Scala 2.8 named and default parameters copy operation nested annotations package objects safer package nesting @specialized new collection API vectors hash tries new treatment of arrays and strings manifests better implicits better type inference continuations packrat parsing faster actors improved Swing support better IDEs interactive compiler API new scaladoc better REPL def evens[T: ClassManifest](xs: Vector[T]): Array[T] = { val arr = new Array[T]((xs.length + 1) / 2) for (i <- 0 until xs.length by 2) arr(i / 2) = xs(i) arr } def evens[T](xs: Vector[T]) (implicit ev: ClassManifest[T]): Array[T] The first line is equivalent to:

25. Better Implicits Better Type Inference Type inference for implicits has been refined. Previously: Infer type first, search implicit for inferred type afterwards. Now: Implicit search can drive type inference. This makes implicits more powerful. Other advance: Type constructors of higher-kinded types can now be inferred. Scala 2.8 named and default parameters copy operation nested annotations package objects safer package nesting @specialized new collection API vectors hash tries new treatment of arrays and strings manifests better implicits better type inference continuations packrat parsing faster actors improved Swing support better IDEs interactive compiler API new scaladoc better REPL

26. Delimited Continuations Continuations capture the return stack as a data structure. Delimited continuations capture only up to some enclosing scope. Lost of advanced control structures can be built on them, e.g: –coroutines –event-based I/O –reactive programming –lightweight actors Scala 2.8 supports delimited continuations in a compiler plugin. Scala 2.8 named and default parameters copy operation nested annotations package objects safer package nesting @specialized new collection API vectors hash tries new treatment of arrays and strings manifests better implicits better type inference delimited continuations packrat parsing faster actors improved Swing support better IDEs interactive compiler API new scaladoc better REPL

27. Better IDEs The Scala plugins for Eclipse, IntelliJ, Netbeans all have matured a lot. The Eclipse and Netbeans plugins are now based on completely redesigned compiler interfaces for command completion, semantic highlighting and incremental building. If you have discounted IDEs before for Scala, maybe now is the time to try again! Scala 2.8 named and default parameters copy operation nested annotations package objects safer package nesting @specialized new collection API vectors hash tries new treatment of arrays and strings manifests better implicits better type inference continuations packrat parsing faster actors improved Swing support better IDEs interactive compiler API new scaladoc better REPL

28. Docs for Advanced Types Problem: Advanced statically typed APIs pose challenges for presentation and documentation. Scala 2.8 named and default parameters copy operation nested annotations package objects safer package nesting @specialized new collection API vectors hash tries new treatment of arrays and strings manifests better implicits better type inference continuations packrat parsing faster actors improved Swing support better IDEs interactive compiler API new scaladoc better REPL

29. Simpler Type Views How to explain def map[B, To](f: A => B) (implicit cbf: CanBuildFrom[Coll, B, To]): To to a beginner? Key observation: We can approximate the type of map. For everyone but the most expert user def map[B](f: A => B): Traversable[B] // in class Traversable def map[B](f: A => B): Seq[B] // in class Seq, etc is detailed enough. These types are correct, they are just not as general as the type that’s actually implemented. These are documented in the new ScalaDoc as use cases. Scala 2.8 named and default parameters copy operation nested annotations package objects safer package nesting @specialized new collection API vectors hash tries new treatment of arrays and strings manifests better implicits better type inference continuations packrat parsing faster actors improved Swing support better IDEs interactive compiler API new scaladoc better REPL

30. New ScalaDoc Scala 2.8 named and default parameters copy operation nested annotations package objects safer package nesting @specialized new collection API vectors hash tries new treatment of arrays and strings manifests better implicits better type inference continuations packrat parsing faster actors improved Swing support better IDEs interactive compiler API new scaladoc better REPL

31. Why 2.8, not 3.0? Scala 2.8 is numerically a point release, but feels like a major release. So why did we not name it Scala 3.0? 2.8 has been 18 months in the making, and acquired more changes on the way. Originally, we just wanted to rewrite collections. That took longer than expected. Meanwhile, a lot more additions and changes got included. But 2.8 was already announced, and referred to in books and other documentation. In light of this, we judged it too confusing to change the version number in mid-flight.

32. And now......what’s next?

33. Next: Scala on.NET Work started again to have a native Scala compiler on.NET with Visual Studio integration. Funded my Microsoft. Stay tuned...

34. Next: Reliability and Contracts Lost of things happening around contracts and verification. Peter Müller: Scala verifier Viktor Kuncak: Scala code synthesis from constraints Our group: effects Common project: ProgLab.NET

35. Next: Concurrency Actor encapsulation –How to ensure that actors do not share memory? –Solved by capability-based uniqueness type system. –Implemented in compiler plugin. –Paper in ECOOP 2010. Software Transactional Memory –CCSTM work in this conference Open question: What’s a good combination of actors and transactions?

36. Next: Parallelism PPP Challenge: How to make use of multi-cores, GPUs, clusters, clouds, to achieve better performance for normal applications? First step in this direction: Parallel collections. –will be integrated in the new collection API. –based on persistent data structures with fast splits and joins. Longer-term vision: Embedded parallel DSLs. See keynote by Kunle Olokutun tomorrow.

37. Next: The Green House Great work on Scala is springing up everywhere, no longer focussed just at EPFL. Q: How can we integrate contributions? Tension: Want to get lots of exciting new stuff contributed, yet maintain a stable and reliable distribution. Planned Solution: three-tiered: IncubatorHosts collaborative Scala projects Green houseA “cutting edge” version of the distribution with experimental features. TrunkThe stable distribution.

38. Thanks To the Scala team at EPFL: Lukas Rytz, Hubert Plociniczak, Iulian Dragos, Miguel Garcia, Gilles Dubochet, Philipp Haller, Aleksandar Prokopec, Antonio Cunei, Tiark Rompf, Donna Malayeri, Phil Bagwell, Adriaan Moors, Ingo Maier. To our external committers and frequent contributors: Paul Phillips, Miles Sabin, Ilya Sergey, Caoyuan Deng, James Matlik, Frank Teubler, Kevin Wright, Manohar Jonnalagedda, Pedro Furlanetto, Johannes Rudolph, Jason Zaugg, Seth Tisue, Ismael Juma, Mark Harrah, Colin Howe, Mirko Stocker, Spiros Tzavellas, Matt Russell, David Taylor To the community at large for your support, suggestions, encouragements.