composite builds in gradle

In the lab today, I got a nice Gradle question. I’m answering it here on the blog so all the students can see it. (and anyone else who is curious).

The use case

We have a common library that this year (and all future year’s robots should use). However, this is the first year of competition with the library so it is likely we will need to make changes to it and rebuild at competition.

Constraint #1 – lack of wifi

For any professionals reading this wondering why we don’t use Jitpack, github or any other hosting – internet is “tricky” at competition.; It is common for there to not be any wifi in the pit and you aren’t allowed to make your own hotspot. So we need a solution that works without internet and easily lets us re-build on the same machines.

Constraint #2 – reuse

I really want the library (StuyLib) to stay in it’s own repository and not have the build file “polluted” by any changes. Why you ask? So it stays a library and not something we copy/paste from year to year.

The solution

Composite builds. Gradle has a nice system for composite builds that lets you refer to one build from another.

Directory structure

  • git clone https://github.com/StuyPulse/StuyLib
  • git clone https://github.com/StuyPulse/fantastic-spork (or any other robot)
  • That means StuyLib and fantastic-spork are in parallel directories

Updates to StuyLib

Yes, I know, I said I didn’t want updates. The only things I added were a group and version variable. That gives the artifact a proper name. (group id = com.stuypulse, artifact id = StuyLib, version = 1.0.0). StuyLib still doesn’t know it is part of a composite build.

plugins {
   id "java"
   id "edu.wpi.first.GradleRIO" version "2020.1.2"
}

// add the following two lines
group = 'com.stuypulse'
version = '1.0.0'

sourceCompatibility = JavaVersion.VERSION_11
targetCompatibility = JavaVersion.VERSION_11

Updates to Robot

At end of settings.gradle, add this line:

includeBuild '../StuyLib'

Then in build.gradle, add a dependency on StuyLib

def ROBOT_MAIN_CLASS = "com.stuypulse.robot.Main"

// add a dependency
dependencies {
  implementation('com.stuypulse:StuyLib:1.0.0')
}

Testing

First, I tested that I could build each independently:

  1. cd StuyLib
  2. ./gradlew build
  3. cd ..
  4. cd fantastic-spork
  5. ./gradlew build

Then I added a class that used existing functionality in StuyLib to fantastic-spork

package jb;

import com.stuypulse.stuylib.util.*;

public class Jeanne {
  StopWatch s;
//  String str = StopWatch.TEST;
}

I re-ran the ./gradlew build in fantastic-spork and good. Then I added one line to StopWatch:

public static final String TEST = "test";

I uncommented the line in Jeanne.java. Then for the key part. I re-ran ./gradlew build in fantastic-spork and it worked. I was able to recompile fantastic-spork and have it see a change in StuyLib without having to rebuild StuyLib.

Plus this change to StuyLib only exists on my machine which proves it is being used.

[2019 oracle code one] JPMS Modules with Gradle

Building JPMS Modules with Gradle

Speakers: Paul Bakker @pbakker

For more blog posts, see The Oracle Code One table of contents


General

  • Nobody in room yet using module system
  • Often takes several years after a feature release to be adopted.
  • Took 2-3 years after lambda use was common
  • Most open source provides a real module or at least a module name in the meta-inf

Plugin

  • Maven supported module system early on and still does.
  • Gradle did not so developed a plugin
  • https://github.com/java9-modularity/gradle-modules-plugin

IntelliJ

  • Shows modules as folders

Gradle

  • Each module has own build.gradle
  • Root build.gradle file is where the modular plugin is applied
  • external plugin so set classpath and apply to all java projects
  • Dependencies are defined in both build.gradle and module-info.java. Have to do this in Maven too.
  • Plugin adds gradle configuration when compile/test.

Flags

  • Showed exports – plugin has addExports option when compile
  • Showed opens – plugin has addOpens option when compile
  • Also supports AddModules and addReads like the Java options

Application plugin

  • Creates tar/zip with lib and bin folder
  • Supports module path

Packages

  • Split packages
  • “The classpath always works – as long as you don’t have expectations”
  • Always a problem; now in your face
  • patch-module as workaround. Puts library in patchlibs instead of being on module path

Multiple start scripts

  • Out of box, can create one zip file/one starting point
  • Module plugin allows creating custom startup scripts

Testing

  • For whitebox testing, ok to break encapsulation.
    • Classpath is ok
    • Plugin provides moduleOptions on test to choose
  • Black box testing – want module path to test how module works from outside module boundaries.
    • Use module path
    • Need additional module for this approach for tests

Java versions

  • plugin has modularity.mixedJavaReleases
  • compiler runs in two steps
  • compiles everything module-info.java using recent java version (presumably Java 9) and pulls out module-info.java
  • compiles other sources using java 8
  • the module-info.java is ignored by Java 8

My take

I found the gradle files hard/impossible to read (red/purple/green) on dark background. That made it hard for me to follow the parts I didn’t already know. I did learn some though so it was worth going. And it’s great that gradle has a JPMS plugin! I like how he mapped the command line to Gradle code.

jenkins convert to pipeline plugin

I needed to convert a freestyle job to a pipeline job for “Automating your Ci/CD Stack with Java and Groovy” Hands On Lab at Oracle Code One. It’s a really simple project, but it is still a pain to convert by hand.

I had read about the Convert to Pipeline plugin so decide to give it a shot. It’s easy enough to install a plugin.

Generated Pipeline

Below is what got generated. While I’d like to assume that credentials binding isn’t in wide use, that one didn’t surprise me. In hindsight, it makes sense because the credentials binding would need to wrap the Gradle block. I was surprised that Gradle wasn’t supported.

I am glad that the tool leaves comments for what you need to do by hand.

// Powered by Infostretch 

timestamps {

node () {

	stage ('osprey - Checkout') {
 	 checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: '', url: 'https://github.com/boyarsky/OracleCodeOne2018-HOL-Automating-Stack-Groovy.git']]]) 
	}
	stage ('osprey - Build') {
 	
// Unable to convert a build step referring to "org.jenkinsci.plugins.credentialsbinding.impl.SecretBuildWrapper". Please verify and convert manually if required.
// Unable to convert a build step referring to "hudson.plugins.gradle.Gradle". Please verify and convert manually if required.
		// JUnit Results
		junit 'osprey-project/build/test-results/test/*.xml' 
	}
}
}

Using the pipeline generator

On the pipeline job, you can click the “pipeline syntax” link. It opens a new browser tab with a pull down for common pipeline operations.

Credentials

I choose “withCredentials”, entered the variable names and selected the credential from the pull down. Then I clicked “Generate Pipeline Script” and got:


withCredentials([usernamePassword(credentialsId: 'nexus', passwordVariable: 'nexusPassword', usernameVariable: 'nexusUserName')]) {
// some block
}


Gradle

That brings us to Gradle. As far as I can tell, there are only objects for Gradle if you are using Artifactory. I am using the gradle wrapper though, so I followed the advice of this Stack Overflow post and just called gradlew.


withCredentials([usernamePassword(credentialsId: 'nexus', passwordVariable: 'nexusPassword', usernameVariable: 'nexusUserName')]) {
  sh "./osprey-project/gradlew clean build -b osprey-project/build.gradle -PnexusBaseUrl=http://nexus:8081 -PnexusUserName=${nexusUserName} -PnexusPassword=${nexusPassword}"
}

I decided to separate the publish step into a different stage and rename the stages as well.  Resulting in:


// Powered by Infostretch 

timestamps {

node () {

	stage ('Checkout') {
 	 checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: '', url: 'https://github.com/boyarsky/OracleCodeOne2018-HOL-Automating-Stack-Groovy.git']]]) 
	}
	stage ('Build') {
	    
	   withCredentials([usernamePassword(credentialsId: 'nexus', passwordVariable: 'nexusPassword', usernameVariable: 'nexusUserName')]) {
          sh "./osprey-project/gradlew clean build -b osprey-project/build.gradle -PnexusBaseUrl=http://nexus:8081 -PnexusUserName=${nexusUserName} -PnexusPassword=${nexusPassword}"
       }

		// JUnit Results
		junit 'osprey-project/build/test-results/test/*.xml' 
	}
	
	stage ('Publish') {
	    
	   withCredentials([usernamePassword(credentialsId: 'nexus', passwordVariable: 'nexusPassword', usernameVariable: 'nexusUserName')]) {
          sh "./osprey-project/gradlew publish -b osprey-project/build.gradle -PnexusBaseUrl=http://nexus:8081 -PnexusUserName=${nexusUserName} -PnexusPassword=${nexusPassword}"
       }
	}
}
}

Note: My actual build file was bigger. The parts outside the pipeline (ex: project security settings) got preserved.