from java to groovy – thinking in functional programming (via jenkins)

I found my Java knowledge to be a crutch when writing Groovy scripts. Luckily, I was able to use this crutch less and less as I gained more exposure. I was asked about this today so I decided to write up a blog on my journey. If you know how to run a Maven project and want to follow along, I’ve included instructions at the end.

Why I started using Groovy

Jenkins and Hudson are both continuous integration servers that provide a Groovy console for writing queries and updates against the object model. This is really cool as you can do something with/to hundreds of jobs with marginal effort. I had read “Making Java Groovy“, but  needed this real scenario to jump into it.

Phase 1: The “problem” with Groovy (and Java 8)

A valid Java program *is* a valid Groovy program. This is nice and convenient because you can call Java methods. It’s also a crutch because you can write Java methods rather than actually using Groovy. My first Groovy program in Jenkins was something like this:

def jobs = Jenkins.instance.items;
for (job in jobs) {
  if (job.isDisabled() ) {
    println(job.getName())
  }
}

At least it uses two pieces of Groovy. “def” to define a variable of unknown type. And “in” within the for loop. But this is an imperative program. I loop through the jobs checking a condition and printing as I go.

Incidentally, I didn’t have this problem in Clojure or Lisp when taking classes. There was nothing to fall back TO.

Phase 2: Translating after the fact

My next phase along the way to functional programming was translating my “imperative” program into functional programming. Now this step may seem like a waste of time given that I had a working program. I did it anyway so I would expose myself to the idioms of how I *wanted* to be coding this.  At the time, I was hoping it would take. And it did after a while.

But for phase 2, I was doing it very incrementally. First I made my imperative code one line. Still just as imperative. It still has a loop and if statement.

Jenkins.instance.items.each({ j -> if (j.isDisabled()) println(j.getName()) })

Then I started thinking about operations. It’s still not great because it still has “each”, but at least I’ve gotten the “if” statement out of there. I’m using “grep” and thinking about filtering first rather than looping.

Jenkins.instance.items.grep({ j -> j.isDisabled() }).each( { j-> println(j.getName()) })

My next step felt like a nice mental jump. I was able to think about the entire task in functional programming. I filter the list and then I translate to the field I want. And I’ve finally succeeded in getting the print statement out of my logic and just happening outside of it!

print(Jenkins.instance.items.grep({ j -> j.isDisabled() }).collect( { j-> j.getName() }))

Phase 3: Getting rid of some more Java syntax

Next I focused on getting rid of stray Java syntax. A lot of things required in Java are redundant in Groovy. I had omitted semi-colons from day 1. Getting rid of the rest took more time.

First I trained myself to get rid of the unneeded parens. I already have {}. No need to have parens there too. The parens make it a method call. But in Groovy, they are implied.

print(Jenkins.instance.items.grep{ j -> j.isDisabled() }.collect { j-> j.getName() })

Then I trained myself to use the shorter form of method names. Groovy automatically adds is/get to your method names so you can write them as if they are properties.

print(Jenkins.instance.items.grep{ j -> j.disabled }.collect { j-> j.name })

Finally I trained myself to indent consistently. I don’t know if this is the best way, but it helps me remember my chain of actions:

print(Jenkins.instance.items
.grep{ j -> j.disabled }
.collect { j-> j.name })

Phase 4: Thinking functionally

For a while, I’d be able to write easy code like this functionally, but have to fall back to imperative programming for debugging. Then I got used to adding print statements within steps to be able to continue on from wherever I was.  This let me think of the whole exercise as a series of steps and start coding them functionally. When I got stuck, I’d just proceed within an each statement to figure out what to do. Then I’d translate the each statement into the correct method.

They key is that I didn’t start thinking in loops anymore. i started with what I needed to do. “I want to find all the disabled jobs and get their names.” It’s a long way from “I want to loop through all the jobs and print the names of the disabled ones.”

For those wanting to run this with Maven

Jenkins is open source and has a robust plugin model. There’s lots of documentation for writing a plugin, but you don’t need to know any of it to run the console.

1) Create a Maven project/file with this pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>jb</groupId>
	<artifactId>play-jenkins-plugin</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<packaging>hpi</packaging>

	<parent>
		<groupId>org.jenkins-ci.plugins</groupId>
		<artifactId>plugin</artifactId>
		<version>1.573</version>
	</parent>

	<repositories>
		<repository>
			<id>jenkins-releases</id>
			<url>http://repo.jenkins-ci.org/releases/</url>
		</repository>
	</repositories>
</project>

2) mvn install (wait a long time for Maven to “download the internet”)

3) mvn hpi:run

4) http://localhost:8080/jenkins

5) Click Manage Jenkins

6) Click Script Console

7) Play!

ken kousen on groovy

I saw Ken Kousen deliver his intro to Groovy talk today at the NY Java SIG.  While I’ve done some small things with Groovy, I’ve forgotten a lot of what I once knew.   Some of it, i remembered/used but am including here so this isn’t random facts but a summary of what was presented. Blogging about it this time in hopes I can retain more.

First of all, Ken is a great speaker.  If you have the chance to see him in person, go.  He:

  • is entertaining – programmer humor
  • presents info clearly
  • has well thought out examples
  • switches well between the groovy console, command line, browser and slides.  While there were frequent switches, it was easy to follow
  • live examples. Surprises and mistakes and all    Icndb.com for humor in json examples provided energy towards the end

Useful links

Now on to Groovy

What is groovy

  • Groovy is like Java 3. Other jvm languages are simulating other languages like jythong or special purpose like functional languages. Groovy is a general purpose like a next generation OO language. Plus you can fall back to Java and make it “groovier” later [which is great for learning]
  • Compiles to bytecode.  Also a C compiler is available.
  • If the name of your groovy file matches the name of the class, you can only have a class in the file. Otherwise you can put multiple classes or even just scripts in the file. Ken’s convention is to use underscores in filenames for scripts to identify them.
  • Hello world is println “hi” [yes, that’s it]
  • Parens are optional “until they are not”. They have a low precedence.  Can leave out if obvious where they go
  • Semicolons are always optional
  • Perls goal is to write the shortest code possible. Groovy’s goal is to write the simplest code possible.
  • Testing and build are the place to start with groovy when introducing into a reluctant organization

Options to run

  • groovy foo.groovvy – don’t have to compile first. Should compile first when use it for real though. If want to integrate with java, must run groovyc to compile since java requires class files.
  • The class files require groovy in the claspath to run with “java” rather than “groovy”. In groovy home, embedable directory contains groovy-all and groovy-allindy. The later is for invoke dynamic in Java

Tools

  • Groovy Grails Tool Suite – for Eclipse
  • Groovy menu has convert file extensions to groovy where right click java class. This lets you refactor slowly. [Or if create java class by accident like often do]

Operator overloading

  • Every operator corresponds to a method.  This means if your class implements that method, you can use that operator.  This lets you write new Date() + 7 to get a date a week in the future.
  • ** is power operator
  • [] goes with getAt or putAt methods.  This is what lets you say s[-1] or s[0..2] instead of pure array indexes.  The getAt/putAt methods understand how to navigate.  The methods also know to call next for 0..2 and prev for -3..-1.  I also learned you can specify multiple ranges such as as s[0..2,4..5]
  • In groovy == calls the equals method. Which means just converting a java class can result in an infinite recursion. Run your tests when refactoring
  • To append: list << value

Groovy Types

  • 123.getClass().getName() is an Integer. If you make it bigger becomes long or BigInteger dynamically.
  • 2 – 1.1 uses BigDecimal so accurate unlike 2d – 1.1d which has Java roundoff error from using doubles.
  • GString “test ${name}” uses interpolation to substitute the variable name. You don’t need braces if just one variable
  • Single quotes are Java strings
  • Double quotes are groovy strings
  • Triple quotes are for multi line strings.  Useful for SQL.
  • Def means don’t know or care about type
  • Duck typing. Can call any methods on def oject. If method exists at runtime ,call it. Else throw MissingMethodException at runtime.
  • Map keys assumed to be Strings

Groovy translation

  • S.class.name works. When it looks like you are referencing a private field, groovy adds get or set to call appropriate methods. The runtime environment provides getter and setter methods so  you don’t need to write them
  • Don’t need to type return belcause last expression is automatically returned

Plain Old Groovy Object (POGO)
Generates at runtime:

  • getter/setter – If don’t want setter, add final to instance variable.  If don’t want getter or setter on instance variable, type private.
  • map based constructor
  • class and methods public by default
  • instance variables private by default

All you have to write is a class and the types.  If you implement a method, Groovy will use yours and not generate it.  If using the map based constructor, Groovy calls the default constructor and any appropriate setters.  This means you don’t need overloaded constructors.

class Person{
String first
String last
}

Person p = new Person()
p.setFirst("f")
println "$p.first"

new Person(first: 'a', last: 'b')

Walkthru of converting a POJO to a POGO

  1. Remove getters and setters
  2. Remove private from  instance variables
  3. Remove semicolons
  4. Remove constructor – need to use map constructor – affects callers [not backward compatible to simplify all the way]
  5. Remove public on class
  6. Remove import for java.util.Date
  7. Add @Canonical which includes @ToString, @TupleConstructor and @EqualsAndHashCode.  All are AST transformations
Changing types
  • [1,2] creates list
  • LinkedList a = [1] makes the desired type
  • [1] as Set also creates the desired type

Closure

listOfStrings.each { /* code using it as default loop variable */ }

map.each{ k,v -> /* code */ }

list.collect{ /* transform to new list where results make up the new list */ }.join(',')  // map reduce

Closure coercion can be used for an anonymouse inner class
{ /* code */} as FilenameFilter
Useful when an interface has only one method because uses implemetation for all methods in interfaces.  Could pass a map of closures but might as well implement interface then.

Helpful classes/methods

  • Groovy adds toURL to String class and getText to Url class masking query.toUrl().text a one liner
  • XMLSlurper lets you get a dom tree from a URL which you can then navigate as tree.root.result[o].geometry.location.lat 
  • db.rows(“selet * from table”)
  • db.eachrow(” query “){ /*code */}
  • groovy.json.JsonSlurper().parseText(json)
  • groovy.xml.MarkupBuilder to create. Similarly for xml, json and ant
Testing
  • Must test code since constraints we are used to are gone
  • Spock over GroovyTestCase – very nice framework.  See example in github/book

Random interesting tips

  • Can use map based constructor on a pojo too because groovy will call default constructor and set methods
  • foo.with{} calls foo.xxx for any methods not found in scope
  • Meta programming: add methods to class that need. Even to a java class. Complex.metaClass.plus= {}.  Every class has a metaclass.

Live from TSSJS – Web Frameworks with Matt

Live blogging from TheServerSide Java Symposium again, this time with Matt Raible, who is giving a presentation entitled “Comparing JVM Web Frameworks”.  He began his talk taking feedback from the audience on what frameworks people are using, with one humerous dialogue that proceeded as follows:

“How many people are using Struts 1?”.
One gentleman raises his hand.
“Poor bastard.  Are you using Struts with WebSphere?”
The gentleman nodded.
“Why haven’t you quit your job yet?”

1.  Web Frameworks – Too many?
In the non-Java market, jQuery has risen to the top but this hasn’t happened in the Java world.  Some people have criticized Java as having too many frameworks to choose from.

Matt is personally very much against Struts 1, as well as JSF.  He talks about how the founder of both, Craig McClanahan has essentially ditched Struts/JSF for Rails in around 2007.  He also quotes James Gosling as saying he hates JSF with a link to the YouTube video, although indicates he may have been referring to JSP.

2. Choosing a framework
Matt often talks about being asked which framework to use.  He has constructed a list of 20 criteria to determine which framework to use such as:

  1. Developer Productivity
  2. Developer Perception
  3. Learning Curve – Should be able to learn in under a week
  4. Project Health
  5. Developer Availability – Will you be able to find developers to hire who know the material?
  6. Job Trends
  7. Templating
  8. Components
  9. Ajax
  10. Plugins or Add-Ons
  11. Scalability
  12. Testing Support
  13. i18n and testing
  14. Validation
  15. Multi-language Support
  16. Quality of Documentation/Tutorials
  17. Books Published
  18. REST Support (client and server)
  19. Mobile / iPhone Support
  20. Degree of Risk

Matt reviewed a number of platforms and posted the resulting matrix online.  The top 5 ones based on these scores are Grails, Tapestry, Lift, Spring, and Struts2.  Often the 5, 6, and 7 providers change every few months.

3.  Business Decisions
Often companies hire consultants to reinforce web framework decisions that they have already made.  A lot of companies have a proprietary web framework that can be successful for their product.

4.  Controversary
Matt has been criticized for his scoring by members of the community.  He has posted a detailed explanation of how he calculated these ratings online.  As a well-read GWT developer, I disagree with some of his scores for GWT, especially templating which only scored a 0.5.

5. Matt’s favorite web frameworks

  1. Grails and Groovy:  Eeasy transition for Java developers.  It is also a gateway to Groovy.  He mentions some companies and developers resist Groovy because it is targeted only for Java developers and there isn’t a high availability of jobs.
  2. GWT:Great for Java developers who hate JavaScript.  The community is open and Google has done a good job growing a vibrant developer base.  The down sides, though, are that it is slow to test and requires Java knowledge to use.
  3. Ruby on Rails: Good for web developers, rather than hard-core Java developers.  The community is excellent.  The down sides is performance, testing, and type safety are limited.
  4. Spring MVC:  Easy configuration and usage
  5. VAADIN:  Excellent theme and layout support.  Uses the GWT API for developing view.
  6. WICKET:  Great for Java Developers.  Active community with creators providing a lot of support.  One downside is very little jobs available for WICKET developers.

Active Mailing Lists
One good measure of the health of a web framework is the number of developers asking questions and involved in the community, such as mailing lists and forum websites.  Some frameworks have hundreds of questions posted a month while others have 1-2 and are clearly not being used much.

Conclusion
Matt finished asking the question “What if there is no ‘best’ web framework?”, to which he spent time presenting competing argument about how people ‘shop’ for frameworks.  He compared shopping for frameworks with shopping for food and discussed an article with a researcher who did trending for how people choose food products.  “When we pursue universal principles in food, aka web services, we are doing ourselves a massive disservice”.  In other words, pick the frameworks that work for you, not neccesssarily the “best” framework.

Matt gave one of the best presentations I’ve attended at TheServerSide Java Symposium, and covered a large variety of web frameworks, while demonstrating expertise in each.  As a fellow consultant, I appreciate that he emphasized how quickly it is to pick up these frameworks as well as how easy it is to find a job with these skills.  At one point, he was one of only 3 active GWT developers in the Denver area and would get a call anytime there was a GWT job available.

Side note:  Matt recommends all Java developers learn about and know JRebel given its power, saying if you take away only one thing from this presentation it is that you should know JRebel.