Switching your Gradle builds from JCenter to Maven Central

JCenter is being decommissioned on May 1st, 2021. Since many Gradle builds use JCenter by default, this means your Gradle build file is likely to have jcenter() in it. This means you have a few months to switch to Maven Central. Don’t worry, it’s easy.

Note: If you work for a company, you are hopefully using an internal binary repository proxy. (ex: Nexus, Artifactory, etc)

What’s the difference between JCenter and Maven Central?

The main benefits of JCenter are:

  • Some artifacts are in JCenter and not Maven Central – their authors are working on moving them to Maven Central. This is unlikely to affect FRC teams, but might affect people using an artifact that is more specialized.
  • It’s easier to publish to JCenter – Sonatype has been working on making it easier for Maven Central. Some of that is intrinsic though because Sonatype does a lot of verification.
  • JCenter is faster – Remember that artifacts are cached on your machine. So once you’ve downloaded the artifacts, your build performance is the same.

How do I find the affected files in GitHub?

Searching on github for either of these does not do what you might expect

  • jcenter user:boyarsky filename:build.gradle- the code tab returns 0 results
  • jcenter org:stuypulse filename:build.gradle – the code tab returns 2 results

That’s because github search only looks in files that have been updated or returned in search results in the past year. Unfortunately, the last year has not been particularly representative of a normal year. And people edit/search the contents of build.gradle files way less frequently than other file types.

I recommend just searching for the filename. That returns all your build.gradle files so you can edit them. (And since you are probably consistent in your choice of binary repository, you can sample a few matches to see if you have to change.

  • user:boyarsky filename:build.gradle – the code tab returns 5 results
  • org:stuypulse filename:build.gradle – the code tab returns 24 results

Tip: You may also have a reference to jcenter in your settings.gradle so I recommend searching that as well.

How do I edit the file?

GitHub has good Rest APIs so you can script this if you have a lot. If you don’t have a ton, either of the following is viable. (I had 5 build.gradle files in my personal repo and 3 of them were already using Maven)

Option 1 – Use the browser

  1. Open each link from the code tab of the search
  2. Choose the default branch (ex: main/master) from the pull down – search often returns a specific commit
  3. Drill down to the build.gradle file if not already there.
  4. Click the edit/pencil icon in the top right (just above the code)
  5. Change jcenter() to mavenCentral()
  6. Enter a commit comment and save
  7. If you want to make sure your build still works, run it (ex: Travis)

Option 2 – Clone the repos

(Tested on Mac; I don’t have Git Bash on my home computer so don’t know if it works exactly the same. I have used find on Git Bash though so I think it does)

  1. Clone the affected repos (if you don’t already have them)
  2. Go to a parent directory of the github repos (ex: <userHome>/git
  3. Run a UNIX command to update the list files find . -name build.gradle
  4. Run a UNIX command to update the affected files: find . -name build.gradle -exec sed -i ” -e ‘s/jcenter/mavenCentral/g’ {} \;
  5. Commit/push the affected repos (listed in step 3)
  6. If you want to make sure your build still works, run it (ex: ./gradlew build)

[2019 oracle code one] dependency hell

Surviving Dependency Hell

Speakers: Robert Scholte (@rfscholte) & Ray Tsang (@saturnism)

For more blog posts, see The Oracle Code One table of contents


Identifying problem

  • Setup two dependencies with Maven and ran
  • Errors: NoSuchMethodError, NoSuchFieldError, ClassNotFoundException
  • mvn dependency:tree – lots of output
  • mvn dependency:tree -Dincludes=com.google.guava – filters by the jar suspected
  • mvn dependency:tree -Dincludes=com.google.guava -Dverbose=true – shows conflicts and reason
  • Showed IntelliJ dependency tree graph

Picking a dependency

  • Classpath – first class wins
  • Maven – nearest wins
  • Maven doesn’t understand Semver (semantic versioning)
  • Use Maven Enforcer with upper bound. This assumes library is backwards compatible
  • Guava 21+ is backward compatible

Other problems

  • Shade – Copies files to creates problems. Problem if jars relocated. Only do as last resort
  • Classloaders

Solutions

  • Explicit dependency with version you need
  • Exclude dependency when pull in transitive dependency
  • Dependency management to specify version you need. Dependency management is a lookup table. It doesn’t actually need to exist. Can have a reference to a snapshot in there.
  • When making breaking changes – use new group id or artifact id. Use new package name. This is what commons lang did. – http://jlbp.dev/JLBP-6.html
  • BOM file for multi module projects with all transitive dependencies needed. Only use for modules within the project. – http://jlbp.dev/JLBP-15.html

<and then I needed to leave to get to my session in another building and get ready>

My take

Regardless of how good this is, I needed to leave early to get to my own session in another building right after this. I’m sorry I had to leave. It was excellent. I like that I learned about http://jlbp.dev so I can do more reading later.

[2019 oracle code one] maven stories

Broken Built Tools & Bad Habits: The Maven Stories

Speaker: Robert Schoite @rfschoite

For more blog posts, see The Oracle Code One table of contents



General

  • Instructions change over time and following practices from prior generations can get you in trouble.
  • Advice from Maven 1 and 2, may not apply to Maven 3
  • When learn new things – analyze, google/stack overflow, ask colleagues
  • When things fail – do three things when learning. Or can fix, create workarounds.
  • Workarounds tend to turn into a pattern
  • CI server is the neutral judge. Counter to “it works on my machine”
  • Apache tests latest Maven 3.0, 3.2, 2.3, 3.5 and 3.6 with java 7. And 3.6.1 with Java 7, 8, 11, 12 and 13 early access. Test on Ubunto and Windows

Why it works on your machine but not server

  • Changed code
  • OS – ex: Windows not case sensitive
  • JDK
  • Version of Maven
  • Files – regular files/directories
  • Properties – system/environment

Options to troubleshoot

  • -v version of Maven
  • -V version info and run build. Good for running on CI server
  • -e execution error messages
  • -X execution debug info

Local repo

  • No TTL (time-to-live)
  • Maven 2 – dumb cache
  • Maven 3: _remote.repositories – verify cached artifacts still exist
  • Designed for single user. CI server can be multi user and corrupt files due to concurrent writes/reads
  • Takarai Concurrent Local Repository – adds file locking to repo
  • Checksums not verified by default. -C is for strict checksums to fail if don’t match or -c for lax checksums to warn if don’t match. Shouldn’t be a problem now since binary repos have checksums. In Maven 4, will probably turn on by default.
  • Maven 2 and 3 share directory. In future, might separate SNAPSHOTS or by different remote repos

Multi modules

  • In Maven 2, not aware of reactor. Dependencies had to exist in local repo
  • Maven 3 is reactor aware so shouldn’t need install anymore. [for this scenario]
  • Can use installAtEnd/deployAtEnd experimental feature to wait until the last module runs

Flow

  • Locally run mvn verify
  • On CI Server, run mvn deploy. (hard to write without install so that is fine too)
  • This technique means all SNAPSHOTS are served by repo manager

Performance

  • Aim for clean Maaven output
  • Don’t write to System out/error
  • Don’t log during testing loglevel = off
  • Can set Maven flag to quiet if needed

Files

  • Replacing files is a waste
  • Most plugins can handle incremental execution
  • Avoid maven clean because forced rework
  • He wrote a plugin to nag you about using clean and install

Properties

  • Don’t change version of Java or a dependency on the command line

Versions

  • Maven 3.5.0 comes with CI friendly placeholders. Can specify revision, sha1 or changelist in version #. Ex 3.1.0-JIRA101-SNAPSHOT
  • Need relative path or GAV when building. Dependencies must use GAV
  • Maven 3.7 will probably require Java 8+

My take

This was good. Assuming you are familiar with Maven, there was a lot of info covered quickly but not hard to understand. If you don’t know Maven, I suspect this would have gone over your head. I learned a few new things so I’m happy.