[2019 oracle code one] dependency hell

Surviving Dependency Hell

Speakers: Robert Scholte (@rfscholte) & Ray Tsang (@saturnism)

For more blog posts, see The Oracle Code One table of contents


Identifying problem

  • Setup two dependencies with Maven and ran
  • Errors: NoSuchMethodError, NoSuchFieldError, ClassNotFoundException
  • mvn dependency:tree – lots of output
  • mvn dependency:tree -Dincludes=com.google.guava – filters by the jar suspected
  • mvn dependency:tree -Dincludes=com.google.guava -Dverbose=true – shows conflicts and reason
  • Showed IntelliJ dependency tree graph

Picking a dependency

  • Classpath – first class wins
  • Maven – nearest wins
  • Maven doesn’t understand Semver (semantic versioning)
  • Use Maven Enforcer with upper bound. This assumes library is backwards compatible
  • Guava 21+ is backward compatible

Other problems

  • Shade – Copies files to creates problems. Problem if jars relocated. Only do as last resort
  • Classloaders

Solutions

  • Explicit dependency with version you need
  • Exclude dependency when pull in transitive dependency
  • Dependency management to specify version you need. Dependency management is a lookup table. It doesn’t actually need to exist. Can have a reference to a snapshot in there.
  • When making breaking changes – use new group id or artifact id. Use new package name. This is what commons lang did. – http://jlbp.dev/JLBP-6.html
  • BOM file for multi module projects with all transitive dependencies needed. Only use for modules within the project. – http://jlbp.dev/JLBP-15.html

<and then I needed to leave to get to my session in another building and get ready>

My take

Regardless of how good this is, I needed to leave early to get to my own session in another building right after this. I’m sorry I had to leave. It was excellent. I like that I learned about http://jlbp.dev so I can do more reading later.

[2019 oracle code one] maven stories

Broken Built Tools & Bad Habits: The Maven Stories

Speaker: Robert Schoite @rfschoite

For more blog posts, see The Oracle Code One table of contents



General

  • Instructions change over time and following practices from prior generations can get you in trouble.
  • Advice from Maven 1 and 2, may not apply to Maven 3
  • When learn new things – analyze, google/stack overflow, ask colleagues
  • When things fail – do three things when learning. Or can fix, create workarounds.
  • Workarounds tend to turn into a pattern
  • CI server is the neutral judge. Counter to “it works on my machine”
  • Apache tests latest Maven 3.0, 3.2, 2.3, 3.5 and 3.6 with java 7. And 3.6.1 with Java 7, 8, 11, 12 and 13 early access. Test on Ubunto and Windows

Why it works on your machine but not server

  • Changed code
  • OS – ex: Windows not case sensitive
  • JDK
  • Version of Maven
  • Files – regular files/directories
  • Properties – system/environment

Options to troubleshoot

  • -v version of Maven
  • -V version info and run build. Good for running on CI server
  • -e execution error messages
  • -X execution debug info

Local repo

  • No TTL (time-to-live)
  • Maven 2 – dumb cache
  • Maven 3: _remote.repositories – verify cached artifacts still exist
  • Designed for single user. CI server can be multi user and corrupt files due to concurrent writes/reads
  • Takarai Concurrent Local Repository – adds file locking to repo
  • Checksums not verified by default. -C is for strict checksums to fail if don’t match or -c for lax checksums to warn if don’t match. Shouldn’t be a problem now since binary repos have checksums. In Maven 4, will probably turn on by default.
  • Maven 2 and 3 share directory. In future, might separate SNAPSHOTS or by different remote repos

Multi modules

  • In Maven 2, not aware of reactor. Dependencies had to exist in local repo
  • Maven 3 is reactor aware so shouldn’t need install anymore. [for this scenario]
  • Can use installAtEnd/deployAtEnd experimental feature to wait until the last module runs

Flow

  • Locally run mvn verify
  • On CI Server, run mvn deploy. (hard to write without install so that is fine too)
  • This technique means all SNAPSHOTS are served by repo manager

Performance

  • Aim for clean Maaven output
  • Don’t write to System out/error
  • Don’t log during testing loglevel = off
  • Can set Maven flag to quiet if needed

Files

  • Replacing files is a waste
  • Most plugins can handle incremental execution
  • Avoid maven clean because forced rework
  • He wrote a plugin to nag you about using clean and install

Properties

  • Don’t change version of Java or a dependency on the command line

Versions

  • Maven 3.5.0 comes with CI friendly placeholders. Can specify revision, sha1 or changelist in version #. Ex 3.1.0-JIRA101-SNAPSHOT
  • Need relative path or GAV when building. Dependencies must use GAV
  • Maven 3.7 will probably require Java 8+

My take

This was good. Assuming you are familiar with Maven, there was a lot of info covered quickly but not hard to understand. If you don’t know Maven, I suspect this would have gone over your head. I learned a few new things so I’m happy.

JavaOne – Maven BOF

“Maven 5 BOF”

Speaker: Brian Fox, Manfred Moser & Robert Scholte

For more blog posts from JavaOne, see the table of contents


[I was late because we talked more about JUnit 5 after the BOF]

Only 26.4% of Maven Central traffic is from Maven. Nothing else is more than 10% though; not even Ivy or Gradle

Some projects don’t have snapshots; instead every commit is a release

Talked about version ranges. Depends on proximity to your project rather than the latest version. Important to clean up pom dependencies before Java 9 so not in module path. Use Maven dependency plugin (analyze) to find unused ones. Make sure to use latest version of depedency plugin.

Maven won’t generate module descriptor. Different purpose. Not all modules are dependencies. More info in module descriptor. What to export is a decision that needs to be decided by developer. jdeps can generate a rough descriptor to get started based on binaries.

Can have .mvn file inside projects with preferences startig in Maven 3.3.5. For example, you can specify to provide more memory.

Shouldn’t be issues going from 3.3.5 to 3.5.9

Maven (dependency) resolver is now a standalone project