Class Visibility ErrorsSlide 1

Class Visibility Errors Bryan Atsatt February 2009

Class Visibility ErrorsSlide 3 Class Visibility Errors Not Enough Visibility (not-found) ClassNotFoundException NoClassDefFoundError Split Visibility (split-packages) IllegalAccessError (package-private, different loaders) Too Much Visibility (duplicates) LinkageError (loader constraint violation) ClassCastException (same name, different loaders) Most developers have not seen duplicate errors… yet.

Class Visibility ErrorsSlide 4 Duplicate Class Definitions Ticking time bombs A duplicate exists when there are two or more Class instances with the same name Duplicates always have different defining loaders Frequently result in runtime errors LinkageError The JVM enforces "Loading Constraints" (5.3.4) to avoid collisions: "It is essential that any type name N mentioned in the field or method descriptor denote the same class or interface when loaded by L1 and when loaded by L2.“ ClassCastException Well known in the wild (e.g."child-first“ loading from Servlet specification, OSGi). Very confusing since type names are same. Cause and effect widely separated in time.

Class Visibility ErrorsSlide 5 Duplication Errors Oracle’s experience (>= 2003) Modules (“shared-libraries”) in application server Named, versioned loader instances Versioned imports  complex (acyclic) graph Rich reflective api and diagnostic machinery Highly successful at solving the versioning problem, but… Duplication errors suddenly common No prevention model in place Fancy diagnostics and education averted disaster

Class Visibility ErrorsSlide 6 Duplication Errors A support nightmare Subtle conditions, which few developers understand Class loading is like plumbing: ignored until it breaks, messy when it does and often requiring special expertise to fix Extremely hard to diagnose Error messages rarely contain enough information Loading often deferred until first use, stack trace confusing Loading can be highly recursive, stack often very deep Stack trace does not provide instance data, cannot determine which loader(s) are involved No useful reflective api on ClassLoader Most loader code pathways are non-debug or native so mostly useless in a debugger

Class Visibility ErrorsSlide 7 How Are Duplicates Created? Today Copies of jars in different loaders EE implementations that support child-first Poorly behaved custom loaders Tomorrow Multiple versions of a module Copies of classes in a module that are available in another Wrong import choice when multiple candidates are available

Class Visibility ErrorsSlide 8 Modules Change The Balance Modules should reduce frequency of not-found errors at runtime, but... Will increase frequency of them at compile time, initially, since dependency declarations must be precise. Tools can certainly help. Without intervention, modules will significantly increase frequency of duplication errors at runtime.

Class Visibility ErrorsSlide 9 Modules == More Loaders More loaders == more opportunities for duplicates Finer grained class partitioning A small, simple tree of loaders becomes a potentially large (even cyclic) graph. Tendency to bundle classes defensively or for convenience Multiple simultaneous module versions Unsophisticated use of version requirements Compiler can catch many error cases, but only for a specific environment.

Class Visibility ErrorsSlide 10 Error Cases Conflicting imports, one module These cases use an example “animal” module system, with a simple property syntax. Multiple versions of “akita” are present in the repository. pika: Module-Imports=akita, version=1.0; newt; akita, version=0.9 Module “pika” imports module “akita” twice, “newt” once; silly, but possible. If not prevented, duplicates from akita will be visible to pika. See AnimalModuleTest.conflictingImportsFail()

Class Visibility ErrorsSlide 11 Error Cases Conflicting module imports == LinkageError vole: Module-Imports=akita, version=0.9 wombat: Module-Imports=vole; akita Vole ctor takes Akita : public Vole(Akita akita) Wombat instantiates Vole : new Vole(new Akita()); See AnimalModuleTest.moduleImportsCauseLinkageError()

Class Visibility ErrorsSlide 12 Error Cases Conflicting package imports == LinkageError raven: Imports=akita, version=0.9 shrew: Imports=raven; akita Raven ctor takes Akita : public Raven(Akita akita) Shrew instantiates Raven : new Raven(new Akita()); See AnimalModuleTest.packageImportsCauseLinkageError()

Class Visibility ErrorsSlide 13 Error Cases Conflicting imports == ClassCastException newt: Module-Imports=akita, version=0.5 oryx: Module-Imports=akita; newt Newt ctor publishes Akita instance to static List. Oryx.toString() tries to return loader name for instance in static list. Unspecified version in oryx selects newer version. Compiler generated downcast to Akita in Oryx fails. The same modules do not fail when only the 0.5 version of akita is present. See AnimalModuleTest.moduleImportsCauseClassCast() and AnimalModuleTest.moduleImportsDoNotCauseClassCast()

Class Visibility ErrorsSlide 14 Error Cases Embedded copy == ClassCastException newt: Module-Imports=akita, version=0.5 quetzal: Imports=newt Quetzal contains a copy of classes from akita, same generics downcast results in error when it tries to reference Akita instance from the list in newt. See AnimalModuleTest.embeddedCopyCausesClassCast()

Class Visibility ErrorsSlide 15 Duplicates Acceptable if Not Shared If C does not expose classes from B2, either by re- exporting them or by sharing instances, then A ‘s import of C will not result in failures. Duplicates that are hidden implementation details are fine. If not allowed, A is forced to move to B2, or to find a version of C that does not conflict, neither of which is required here; in a complex system, this behavior would be a serious usability issue. x  y == x imports y

Class Visibility ErrorsSlide 16 So… rule out all visible duplicates? Maybe, but first, to detect them, we need to be able to determine what classes or packages are "exposed" by each module. So assume that we can at least ask each module what packages it exports. Now, we can collect transitive dependencies, discover all exported packages, and ensure that modules share a common provider where packages intersect.

Class Visibility ErrorsSlide 17 But… We Can Do Better If package level imports are possible (ever), then we want a slight refinement. Given that a single module will likely export many packages, an importer may select a subset of them; if classes from that subset don’t expose the duplicate, that importer is shielded from errors. If C exports packages x, y and z, and only y refers to classes from B, then importers of x or z don't care about the duplicate.

Class Visibility ErrorsSlide 18 Maximize Resolution Flexibility Implied dependencies To support this refinement, we need to know what package imports are “implied” by a given package export. That is, if an exported class uses a class from another package (e.g. in a method signature), as long as we can determine that fact at runtime, we can enforce the no duplicates rule. And we enforce it at a granularity that maximizes resolution flexibility.

Class Visibility ErrorsSlide 19 Error Avoided Identical scenario as raven & shrew, but declare 'implied' dependency: toad: Exports=toad, depends=akita Imports=akita, version=0.9 uakari: Imports=toad; akita Toad ctor takes Akita: public Toad(Akita akita) Uakari instantiates Toad: new Toad(new Akita()); System ensures that toad and uakari both select akita 0.9. See AnimalModuleTest.packageImportsAndDependsDoesNotCauseLinkageError()

Class Visibility ErrorsSlide 20 Summary Many failures occur only when multiple versions are present and will therefore manifest very late in the cycle. Duplication is a double-edged sword: Useful to avoid forced split of dependency graph when duplicates are hidden implementation details. Dangerous when duplicate types are surfaced. Implied dependency declarations enable disambiguation.

Class Visibility ErrorsSlide 21 Summary, continued. Early detection is critical to avoid support nightmare: compile time (one "model" environment) resolution time (actual environment) Enumeration of package level exports is required for duplicate detection and warning/failure at resolution time. Class space "consistency" model is required, but enforcement must be controlled declaratively; tools should generate this information.

Class Visibility ErrorsSlide 22 Resolution Constraints 1.A module must fail to resolve if a required import cannot be satisfied. 2.A failed module must not be used to satisfy imports. 3.Importing a module M is equivalent to importing all packages exported by M. 4.A module may declare that it both exports and imports package p ; once resolved it may either export or import p, not both. 5.If multiple providers exist for an import, resolved providers are preferred over unresolved, and higher versions over lower. 6.Modules that share a dependency must select a common provider or fail; a shared dependency on package s exists between modules M 1 and M 2 when all of the following are true: M 1 imports package p from M 2 Package p depends on package s M 1 either imports or exports package s

Class Visibility ErrorsSlide 23 Appendix OSGi ‘uses’ example from R4 specification Given A: Import-Package: q; version=”[1.0,1.0]” Export-Package: p; uses:=”q,r” B: Export-Package: q; version=1.0 C: Export-Package: q; version=2.0 Shared dependency on q means D fails to resolve D: Import-Package: p, q; version=2.0

Class Visibility ErrorsSlide 24