from java to groovy – thinking in functional programming (via jenkins)

I found my Java knowledge to be a crutch when writing Groovy scripts. Luckily, I was able to use this crutch less and less as I gained more exposure. I was asked about this today so I decided to write up a blog on my journey. If you know how to run a Maven project and want to follow along, I’ve included instructions at the end.

Why I started using Groovy

Jenkins and Hudson are both continuous integration servers that provide a Groovy console for writing queries and updates against the object model. This is really cool as you can do something with/to hundreds of jobs with marginal effort. I had read “Making Java Groovy“, but  needed this real scenario to jump into it.

Phase 1: The “problem” with Groovy (and Java 8)

A valid Java program *is* a valid Groovy program. This is nice and convenient because you can call Java methods. It’s also a crutch because you can write Java methods rather than actually using Groovy. My first Groovy program in Jenkins was something like this:

def jobs = Jenkins.instance.items;
for (job in jobs) {
  if (job.isDisabled() ) {
    println(job.getName())
  }
}

At least it uses two pieces of Groovy. “def” to define a variable of unknown type. And “in” within the for loop. But this is an imperative program. I loop through the jobs checking a condition and printing as I go.

Incidentally, I didn’t have this problem in Clojure or Lisp when taking classes. There was nothing to fall back TO.

Phase 2: Translating after the fact

My next phase along the way to functional programming was translating my “imperative” program into functional programming. Now this step may seem like a waste of time given that I had a working program. I did it anyway so I would expose myself to the idioms of how I *wanted* to be coding this.  At the time, I was hoping it would take. And it did after a while.

But for phase 2, I was doing it very incrementally. First I made my imperative code one line. Still just as imperative. It still has a loop and if statement.

Jenkins.instance.items.each({ j -> if (j.isDisabled()) println(j.getName()) })

Then I started thinking about operations. It’s still not great because it still has “each”, but at least I’ve gotten the “if” statement out of there. I’m using “grep” and thinking about filtering first rather than looping.

Jenkins.instance.items.grep({ j -> j.isDisabled() }).each( { j-> println(j.getName()) })

My next step felt like a nice mental jump. I was able to think about the entire task in functional programming. I filter the list and then I translate to the field I want. And I’ve finally succeeded in getting the print statement out of my logic and just happening outside of it!

print(Jenkins.instance.items.grep({ j -> j.isDisabled() }).collect( { j-> j.getName() }))

Phase 3: Getting rid of some more Java syntax

Next I focused on getting rid of stray Java syntax. A lot of things required in Java are redundant in Groovy. I had omitted semi-colons from day 1. Getting rid of the rest took more time.

First I trained myself to get rid of the unneeded parens. I already have {}. No need to have parens there too. The parens make it a method call. But in Groovy, they are implied.

print(Jenkins.instance.items.grep{ j -> j.isDisabled() }.collect { j-> j.getName() })

Then I trained myself to use the shorter form of method names. Groovy automatically adds is/get to your method names so you can write them as if they are properties.

print(Jenkins.instance.items.grep{ j -> j.disabled }.collect { j-> j.name })

Finally I trained myself to indent consistently. I don’t know if this is the best way, but it helps me remember my chain of actions:

print(Jenkins.instance.items
.grep{ j -> j.disabled }
.collect { j-> j.name })

Phase 4: Thinking functionally

For a while, I’d be able to write easy code like this functionally, but have to fall back to imperative programming for debugging. Then I got used to adding print statements within steps to be able to continue on from wherever I was.  This let me think of the whole exercise as a series of steps and start coding them functionally. When I got stuck, I’d just proceed within an each statement to figure out what to do. Then I’d translate the each statement into the correct method.

They key is that I didn’t start thinking in loops anymore. i started with what I needed to do. “I want to find all the disabled jobs and get their names.” It’s a long way from “I want to loop through all the jobs and print the names of the disabled ones.”

For those wanting to run this with Maven

Jenkins is open source and has a robust plugin model. There’s lots of documentation for writing a plugin, but you don’t need to know any of it to run the console.

1) Create a Maven project/file with this pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>jb</groupId>
	<artifactId>play-jenkins-plugin</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<packaging>hpi</packaging>

	<parent>
		<groupId>org.jenkins-ci.plugins</groupId>
		<artifactId>plugin</artifactId>
		<version>1.573</version>
	</parent>

	<repositories>
		<repository>
			<id>jenkins-releases</id>
			<url>http://repo.jenkins-ci.org/releases/</url>
		</repository>
	</repositories>
</project>

2) mvn install (wait a long time for Maven to “download the internet”)

3) mvn hpi:run

4) http://localhost:8080/jenkins

5) Click Manage Jenkins

6) Click Script Console

7) Play!

ken kousen on groovy

I saw Ken Kousen deliver his intro to Groovy talk today at the NY Java SIG.  While I’ve done some small things with Groovy, I’ve forgotten a lot of what I once knew.   Some of it, i remembered/used but am including here so this isn’t random facts but a summary of what was presented. Blogging about it this time in hopes I can retain more.

First of all, Ken is a great speaker.  If you have the chance to see him in person, go.  He:

  • is entertaining – programmer humor
  • presents info clearly
  • has well thought out examples
  • switches well between the groovy console, command line, browser and slides.  While there were frequent switches, it was easy to follow
  • live examples. Surprises and mistakes and all    Icndb.com for humor in json examples provided energy towards the end

Useful links

Now on to Groovy

What is groovy

  • Groovy is like Java 3. Other jvm languages are simulating other languages like jythong or special purpose like functional languages. Groovy is a general purpose like a next generation OO language. Plus you can fall back to Java and make it “groovier” later [which is great for learning]
  • Compiles to bytecode.  Also a C compiler is available.
  • If the name of your groovy file matches the name of the class, you can only have a class in the file. Otherwise you can put multiple classes or even just scripts in the file. Ken’s convention is to use underscores in filenames for scripts to identify them.
  • Hello world is println “hi” [yes, that’s it]
  • Parens are optional “until they are not”. They have a low precedence.  Can leave out if obvious where they go
  • Semicolons are always optional
  • Perls goal is to write the shortest code possible. Groovy’s goal is to write the simplest code possible.
  • Testing and build are the place to start with groovy when introducing into a reluctant organization

Options to run

  • groovy foo.groovvy – don’t have to compile first. Should compile first when use it for real though. If want to integrate with java, must run groovyc to compile since java requires class files.
  • The class files require groovy in the claspath to run with “java” rather than “groovy”. In groovy home, embedable directory contains groovy-all and groovy-allindy. The later is for invoke dynamic in Java

Tools

  • Groovy Grails Tool Suite – for Eclipse
  • Groovy menu has convert file extensions to groovy where right click java class. This lets you refactor slowly. [Or if create java class by accident like often do]

Operator overloading

  • Every operator corresponds to a method.  This means if your class implements that method, you can use that operator.  This lets you write new Date() + 7 to get a date a week in the future.
  • ** is power operator
  • [] goes with getAt or putAt methods.  This is what lets you say s[-1] or s[0..2] instead of pure array indexes.  The getAt/putAt methods understand how to navigate.  The methods also know to call next for 0..2 and prev for -3..-1.  I also learned you can specify multiple ranges such as as s[0..2,4..5]
  • In groovy == calls the equals method. Which means just converting a java class can result in an infinite recursion. Run your tests when refactoring
  • To append: list << value

Groovy Types

  • 123.getClass().getName() is an Integer. If you make it bigger becomes long or BigInteger dynamically.
  • 2 – 1.1 uses BigDecimal so accurate unlike 2d – 1.1d which has Java roundoff error from using doubles.
  • GString “test ${name}” uses interpolation to substitute the variable name. You don’t need braces if just one variable
  • Single quotes are Java strings
  • Double quotes are groovy strings
  • Triple quotes are for multi line strings.  Useful for SQL.
  • Def means don’t know or care about type
  • Duck typing. Can call any methods on def oject. If method exists at runtime ,call it. Else throw MissingMethodException at runtime.
  • Map keys assumed to be Strings

Groovy translation

  • S.class.name works. When it looks like you are referencing a private field, groovy adds get or set to call appropriate methods. The runtime environment provides getter and setter methods so  you don’t need to write them
  • Don’t need to type return belcause last expression is automatically returned

Plain Old Groovy Object (POGO)
Generates at runtime:

  • getter/setter – If don’t want setter, add final to instance variable.  If don’t want getter or setter on instance variable, type private.
  • map based constructor
  • class and methods public by default
  • instance variables private by default

All you have to write is a class and the types.  If you implement a method, Groovy will use yours and not generate it.  If using the map based constructor, Groovy calls the default constructor and any appropriate setters.  This means you don’t need overloaded constructors.

class Person{
String first
String last
}

Person p = new Person()
p.setFirst("f")
println "$p.first"

new Person(first: 'a', last: 'b')

Walkthru of converting a POJO to a POGO

  1. Remove getters and setters
  2. Remove private from  instance variables
  3. Remove semicolons
  4. Remove constructor – need to use map constructor – affects callers [not backward compatible to simplify all the way]
  5. Remove public on class
  6. Remove import for java.util.Date
  7. Add @Canonical which includes @ToString, @TupleConstructor and @EqualsAndHashCode.  All are AST transformations
Changing types
  • [1,2] creates list
  • LinkedList a = [1] makes the desired type
  • [1] as Set also creates the desired type

Closure

listOfStrings.each { /* code using it as default loop variable */ }

map.each{ k,v -> /* code */ }

list.collect{ /* transform to new list where results make up the new list */ }.join(',')  // map reduce

Closure coercion can be used for an anonymouse inner class
{ /* code */} as FilenameFilter
Useful when an interface has only one method because uses implemetation for all methods in interfaces.  Could pass a map of closures but might as well implement interface then.

Helpful classes/methods

  • Groovy adds toURL to String class and getText to Url class masking query.toUrl().text a one liner
  • XMLSlurper lets you get a dom tree from a URL which you can then navigate as tree.root.result[o].geometry.location.lat 
  • db.rows(“selet * from table”)
  • db.eachrow(” query “){ /*code */}
  • groovy.json.JsonSlurper().parseText(json)
  • groovy.xml.MarkupBuilder to create. Similarly for xml, json and ant
Testing
  • Must test code since constraints we are used to are gone
  • Spock over GroovyTestCase – very nice framework.  See example in github/book

Random interesting tips

  • Can use map based constructor on a pojo too because groovy will call default constructor and set methods
  • foo.with{} calls foo.xxx for any methods not found in scope
  • Meta programming: add methods to class that need. Even to a java class. Complex.metaClass.plus= {}.  Every class has a metaclass.

creating a groovy project with gradle in eclipse

Last month, I went to a talk on gradle.  Today I decided to give it a shot.  My goal was to create a simple groovy project with gradle.  I did it in less than 30 minutes so getting started was fast.

Setup

I already had the Groovy Eclipse plugin.  I then installed the Gradle plugin from the Eclipse marketplace.  Yes, this could be done at the command line.  I’m used to M2Eclipse IDE integration so wanted the same for Gradle.  This step went as smoothly as any other plugin.

Creating a new gradle project

Just like Maven, the first step is to create a new Gradle project.  Since Groovy Quickstart wasn’t in the list, I choose Java quickstart.   The create request appeared to hang, being at 0% for over five minutes.  This was the first (and really only) problem.  I killed Eclipse and started over.  There was no point in doing that.  It just takes long. Apparently, this is a known issue. I tried again and after 5 minutes Gradle did download dependencies from the maven repository.

Making the Java project a Groovy project

Java quickstart does exactly what it sounds like.  It creates a project using the “Maven way” directory structure for Java.  To adapt this to a Groovy project, I:

  1. hand edited build.gradle to add
    apply plugin: 'groovy'
  2. hand edited build.gradle dependencies section to add
    groovy group: 'org.codehaus.groovy', name: 'groovy', version: '1.7.10'

    (I actually missed this step on the first try and got the error “error “you must assign a Groovy library to the ‘groovy’ configuration”.  The code was documented here.)

  3. created src/main/groovy and src/test/groovy directories
  4. gradle > refresh source folders.  This is like Maven where you need to refresh dependencies and the like to sync the Eclipse workspace.
  5. gradle > build > click build (compile and test)

Impressions of the gradle plugin

  1. I’ve mentioned a few times that it is very similar to the Maven plugin.  This is great as the motions feel very familiar and only the part that is new is gradle itself.  (Well that and refreshing my groovy knowledge – it’s been a while.)
  2. You can run your GroovyTestCase classes through Eclipse without Gradle (via run as junit test)
  3. My first build (with one class and one test class) including some downloading the internet took 1 minute and 2 seconds.  My second build took literally two seconds.
  4. I like the “up to date check” so only some targets get run.
  5. I like that you get an Eclipse pop-up if any unit tests fail.
 
This blog post also motivated me to start using my github account to make it easy to show the code.  In particular, the build.gradle file or the whole project. (This class doesn't require any programming so I think it is ok to put this online.  If Coursera complains, I will take it down.)