from java to groovy – thinking in functional programming (via jenkins)

Main menu:

Topics

Recent Posts

Blog

July 2014
M T W T F S S
« Jun    
 123456
78910111213
14151617181920
21222324252627
28293031  

Past Posts

Java/Java EE

JDBC

Other

from java to groovy – thinking in functional programming (via jenkins)

July 27th, 2014 by Jeanne Boyarsky

I found my Java knowledge to be a crutch when writing Groovy scripts. Luckily, I was able to use this crutch less and less as I gained more exposure. I was asked about this today so I decided to write up a blog on my journey. If you know how to run a Maven project and want to follow along, I’ve included instructions at the end.

Why I started using Groovy

Jenkins and Hudson are both continuous integration servers that provide a Groovy console for writing queries and updates against the object model. This is really cool as you can do something with/to hundreds of jobs with marginal effort. I had read “Making Java Groovy“, but  needed this real scenario to jump into it.

Phase 1: The “problem” with Groovy (and Java 8)

A valid Java program *is* a valid Groovy program. This is nice and convenient because you can call Java methods. It’s also a crutch because you can write Java methods rather than actually using Groovy. My first Groovy program in Jenkins was something like this:

def jobs = Jenkins.instance.items;
for (job in jobs) {
  if (job.isDisabled() ) {
    println(job.getName())
  }
}

At least it uses two pieces of Groovy. “def” to define a variable of unknown type. And “in” within the for loop. But this is an imperative program. I loop through the jobs checking a condition and printing as I go.

Incidentally, I didn’t have this problem in Clojure or Lisp when taking classes. There was nothing to fall back TO.

Phase 2: Translating after the fact

My next phase along the way to functional programming was translating my “imperative” program into functional programming. Now this step may seem like a waste of time given that I had a working program. I did it anyway so I would expose myself to the idioms of how I *wanted* to be coding this.  At the time, I was hoping it would take. And it did after a while.

But for phase 2, I was doing it very incrementally. First I made my imperative code one line. Still just as imperative. It still has a loop and if statement.

Jenkins.instance.items.each({ j -> if (j.isDisabled()) println(j.getName()) })

Then I started thinking about operations. It’s still not great because it still has “each”, but at least I’ve gotten the “if” statement out of there. I’m using “grep” and thinking about filtering first rather than looping.

Jenkins.instance.items.grep({ j -> j.isDisabled() }).each( { j-> println(j.getName()) })

My next step felt like a nice mental jump. I was able to think about the entire task in functional programming. I filter the list and then I translate to the field I want. And I’ve finally succeeded in getting the print statement out of my logic and just happening outside of it!

print(Jenkins.instance.items.grep({ j -> j.isDisabled() }).collect( { j-> j.getName() }))

Phase 3: Getting rid of some more Java syntax

Next I focused on getting rid of stray Java syntax. A lot of things required in Java are redundant in Groovy. I had omitted semi-colons from day 1. Getting rid of the rest took more time.

First I trained myself to get rid of the unneeded parens. I already have {}. No need to have parens there too. The parens make it a method call. But in Groovy, they are implied.

print(Jenkins.instance.items.grep{ j -> j.isDisabled() }.collect { j-> j.getName() })

Then I trained myself to use the shorter form of method names. Groovy automatically adds is/get to your method names so you can write them as if they are properties.

print(Jenkins.instance.items.grep{ j -> j.disabled }.collect { j-> j.name })

Finally I trained myself to indent consistently. I don’t know if this is the best way, but it helps me remember my chain of actions:

print(Jenkins.instance.items
.grep{ j -> j.disabled }
.collect { j-> j.name })

Phase 4: Thinking functionally

For a while, I’d be able to write easy code like this functionally, but have to fall back to imperative programming for debugging. Then I got used to adding print statements within steps to be able to continue on from wherever I was.  This let me think of the whole exercise as a series of steps and start coding them functionally. When I got stuck, I’d just proceed within an each statement to figure out what to do. Then I’d translate the each statement into the correct method.

They key is that I didn’t start thinking in loops anymore. i started with what I needed to do. “I want to find all the disabled jobs and get their names.” It’s a long way from “I want to loop through all the jobs and print the names of the disabled ones.”

For those wanting to run this with Maven

Jenkins is open source and has a robust plugin model. There’s lots of documentation for writing a plugin, but you don’t need to know any of it to run the console.

1) Create a Maven project/file with this pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>jb</groupId>
	<artifactId>play-jenkins-plugin</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<packaging>hpi</packaging>

	<parent>
		<groupId>org.jenkins-ci.plugins</groupId>
		<artifactId>plugin</artifactId>
		<version>1.573</version>
	</parent>

	<repositories>
		<repository>
			<id>jenkins-releases</id>
			<url>http://repo.jenkins-ci.org/releases/</url>
		</repository>
	</repositories>
</project>

2) mvn install (wait a long time for Maven to “download the internet”)

3) mvn hpi:run

4) http://localhost:8080/jenkins

5) Click Manage Jenkins

6) Click Script Console

7) Play!

eclipse luna

July 1st, 2014 by Jeanne Boyarsky

When I went to eclipse.org, I wasn’t greeted the cool Kepler book from last year. I did see “are you ready for Java 8″ front and center. The matrix comparing the packages is still clear. I chose the Java EE version. The download page had a warning that “Eclipse requires Mac OS X 10.5 (Leopard) or greater.” No problem. I’m on the latest version. I’m also on the latest version of Java 8.

Initial launch

When launching my workspace, I got the warning:

Warning: Workspace ‘/myWorkspace’ was written with an older version of the product and will be updated. Updating the workspace can make it incompatible with older versions of the product. Are you sure you want to continue with this workspace.

Which is fine. I’m not going backward. And all the important code is in Subversion or Git anyway. The “Failed to load the JNI shared library” error I got with Kepler is fixed. To be fair, it was fixed in Kepler SR 1, but I never upgraded. It’s nice to be able to launch Eclipse through the icon again though.

Installing the plugins

Like last year, I decided to install the plugins I need for Eclipse Marketplace so I can shed the plugins I tried out and don’t actually want. Cleaning plugin house once a year is nice. The biggest plugin I wanted to shed was the old experimental FIRST robotics plugin. It was never intended for Kepler. I installed it last month to write a presentation.

The significant plugins I use are listed in this table. A number of plugins were beta for Luna or I had to use the Kepler version. I don’t remember that problem in previous years.

Plugin Purpose
Mongrel Tomcat integration supporting Tomcat 7.  (The version of Sysdeo I was using seems to have had that too but at least Mongrel looks more active.) It looks like they used the Sysdeo source code and forked it since Sysdeo isn’t getting updates anymore.Last year, I tried Mongrel and fell back to Sysdeo. This year, Mongrel stuck. I’m happy with it.
Ecl Emma Code coverage
PMD and FindBugs Static analysis. For PMD, I had to use the update site. An install “happened” through Eclipse Marketplace, but I didn’t any of the PMD settings I was expecting. Using the update site gave me what I expected
Subversive To access Subversion repositories
Groovy/Grails Tool Suite (didn’t install) Groovy project/editor and console. At this time only the Kepler version is available which conflicts with other plugins I’ve installed. I’ll use the command line (or fall back to Kepler) for the time being
Eclipse Memory Analyzer For finding memory leaks. Last year this was only available via an update site. Now it is in Eclipse Marketplace.
Freemarker IDE Freemarker syntax highlighting and macro assistance.  Note that it is listed under the JBoss Tool Project. You pick that plugin and then unselect everything except “Freemarker IDE”
Python Python plugin/perspective (had to download one for Eclipse 3.6)
Code Recommenders I think this one is new for Luna. It’s supposed to be better than autocomplete. So far it is a nice toy. I do like that it prompts you when you use autocomplete to make it the default, enable subword matches, etc. Making it easy to see what is going on.

I have faster internet (FIOS) since last year so the downloading was faster. However, finding the right plugins took longer this time. And I still find it odd that Git is included and Subversion is not. Licenses I guess.

What excites me

  1. Java 8 support! This blog post shows the quick fixes and the like that are available. They did a really good job. The integration is very intuitive and “just works”
  2. eGit has come a long way. It even supports cherry picking now. I think I’ll still mostly use command line, but it is nice to have a visual option.
  3. Drag and drop to change order of the list of perspectives in the toolbar.

What frustrates me

  1. Nothing. I’m really happy with Luna

how to use cells as tab name variables in excel

June 19th, 2014 by Jeanne Boyarsky

I have a co-worker who is an Excel wizard. He set up a system for automatically gathering data using references to other tables. I’ve learned a lot from him like how to use lookup tables, filters, reference other files, etc. Microsoft lists out the syntax for referring to other sheets.

There was one one instance where we needed the tab name to vary based on another cell. We’ve been doing this by hand each time we add a new tab. It’s not hard, but it is boring and tedious. Being a good computer programmer, I want the computer to do tedious for me. The problem is that it doesn’t take long enough to update it by hand so I can never justify looking into it.

I recently installed Office on my Windows 8 VM because I needed to see how something looked in real Word. (vs Open Office.) Since I now have Excel, I decided to try it out. I got it working much faster than I thought.

Just like programming, I did iterations to build up to it.

Iteration 1 – sum another tab

I created file1.xlsx with two tabs: tab1 and tab2. In each, I added some numbers to the first column of each. I then wrote the formula to add them up:

=SUM([file1.xlsx]tab1!$A:$A)

Explaining what this means:

  • [file1.xlsx] – refer to another file
  • tab1 – refer to this tab in that file
  • !$A;$A – all columns in column A. The $ means to always use column A even if I copy paste the formula into multiple rows.

Iteration 2 – try to use a formula based on a String

=SUM(INDIRECT(“[file1.xlsx]tab1!$A:$A”))

  • INDIRECT – the indirect function lets you pass text to refer to an Excel reference

Iteration 3 - append strings to build the expression

=SUM(INDIRECT(“[file1.xlsx]tab”& A3 &”!$A:$A”))

  • & – concatenate strings