Category Archives: Java/Java EE

using spring RestTemplate to add issues to github

Posted on by
1

I’m using github as a project management tool for a (non-work) project.  That’s been working well for being able to track things and have it close to the project artifacts.  Milestones have dates (to remember what was committed to) and issues correspond to tasks or follow up items within the milestone.  I reached a point where I needed to add the same three tasks to over 15 milestones.  (And possibly more in the future.)

It probably would have taken an hour to do this manually.  It’s a boring task so I imagine I’d have gotten distracted.  The boredom of even contemplating the task plus the possibility that I might have to do again make me realize I should spend a little longer and write code.  Which meant the prep time was a little over an hour but the time to run the program was less than a minute.

I worked in iterations because I wanted to learn early on if this was feasible (or a time sink.)

Iteration 1 – setup a project and call a github web service

I used Maven because I knew I’d need a couple libraries and dependencies.  In Eclipse, I created a new Maven project with

  • group id = jb
  • artifact = github-issue-creator

I then added one dependency.  I knew I’d need Spring Web because I wanted to use RestTemplate.  At this point, my pom.xml looks like

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>jb</groupId>
  <artifactId>github-issue-creator</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <dependencies>
    <dependency>
      <groupId>org.springframework</groupId>
      <artifactId>spring-web</artifactId>
      <version>4.0.2.RELEASE</version>
    </dependency>
  </dependencies>
</project>

It was then time to actually call RestTemplate.  I quickly reviewed the docs to confirm my memory of the flow.  Then I wrote the simplest program I could that would call a Github web service.  I had learned from the GitHub API docs of the Octocat Hello World test repository.   I confirmed the issues query web service command in the GitHub docs and went to https://api.github.com/repos/octocat/Hello-World/issues in a browser.  Perfect – I get JSON back.

I then wrote the RestTemplate code using this URL.

package jb;

import org.springframework.web.client.*;

/**
 * View a public repo's issue via a web service on github to make sure can
 * connect. This particular repo is referenced int he documentation.
 *
 * @author jeanne
 *
 */
public class Trial1 {

	public static void main(String[] args) {
		String url = "https://api.github.com/repos/octocat/Hello-World/issues";

		RestTemplate template = new RestTemplate();

		// ignore result because just testing connectivity
		template.getForObject(url, Object[].class);
		System.out.println("success");
	}

}

You’ll notice I’m telling Spring to return an array of Objects.  I don’t need a more specific class since I’m ignoring the result.  It’s just a proof of concept – my real code will do a POST and own’t care about getting the data back.  I knew an array was needed because the JSON in the browser returned a list.  If you do need to deal with the returned values, look at this Jackson example with Spring and RestTemplate.

Iteration 2 – use my credentials to access a github web service

So far so good.  I have two more steps left to accomplish my goal.  I can either test passing credentials or try a POST.  I decided to test passing credentials first so I would be doing harmless GET operations during the testing.

I’ve only used basic authentication with RestTemplate once so I had to look up what to do.  This blog post was helpful showing how to use the Apache Commons Codec library to deal with credentials.

First, I had to add this dependency to my POM.

<dependency>
  <groupId>commons-codec</groupId>
  <artifactId>commons-codec</artifactId>
  <version>1.9</version>
</dependency>

This code obtains all the issues for myRepoName.  And no, that isn’t my GitHub password.  It isn’t my repository name for that matter.

package jb;

import org.apache.commons.codec.binary.*;
import org.springframework.http.*;
import org.springframework.web.client.*;

/**
 * Call a web service that uses authenticated user to test passing credentials
 *
 * @author jeanne
 *
 */
public class Trial2 {

	public static void main(String[] args) {
		String url = "https://api.github.com/repos/boyarsky/myRepoName/issues";

		String plainCreds = "boyarsky:notMyGitHubPassword";
		byte[] plainCredsBytes = plainCreds.getBytes();
		byte[] base64CredsBytes = Base64.encodeBase64(plainCredsBytes);
		String base64Creds = new String(base64CredsBytes);

		HttpHeaders headers = new HttpHeaders();
		headers.add("Authorization", "Basic " + base64Creds);

		// ignore result because just testing connectivity
		HttpEntity<String> request = new HttpEntity<String>(headers);
		RestTemplate template = new RestTemplate();
		template.exchange(url, HttpMethod.GET, request, Object[].class);
		System.out.println("success");
	}

}

As you can see, I encode the bytes of my username/password combination. I then set that as the basic authentication header. I had to switch to “exchange” instead of “getForObject” as noted in that blog post for this to work.

Iteration 3 – actually creating an issue

There are only four differences between this and iteration 2 from a web services perspective.

  1. From the github create issue docs, I learned the POST URL is/repos/:owner/:repo/issues.
    private static final String URL = “https://api.github.com/repos/boyarsky/myRepoName/issues”;
  2. Use HttpMethod.POST instead of HttpMethod.GET.  I actually wasted some time with this step.  I forgot to change it so the program was running without error and returning  issues rather than setting them.
  3. Pass in the JSON for the values you want to set in the issue.
  4. Now I tell Spring to excpect an Object instead of an array because the JSON returned is no longer an array.
String json = "{\"title\": \"" + title + "\", \"assignee\": \""
				+ assignee + "\", \"milestone\": " + milestone + "}";

		// ignore result because just testing connectivity
		HttpEntity<String> request = new HttpEntity<String>(json, headers);
		RestTemplate template = new RestTemplate();
		template.exchange(URL, HttpMethod.POST, request, Object.class);

I tested this once and it worked. I then added the logic to get the proper values (it is formulaic) and let it fly.

Getting the milestone id was easy – it is in the URL: https://github.com/boyarsky/myRepoName/issues?milestone=4&state=open

Conclusion
While it would have been faster to add all the issues by hand, this was certainly more interesting and more fun. And the next time I need to add an issue to each milestone, it will be trivial. Thanks github for the wonderful documentation!

 

edit: if you need to do this for two factor authentication, see my updated post

ant vs java – using ** to match directories

Posted on by
Reply

Suppose I have the following files:
/MyFiles/dir/a.txt
/MyFiles/dir/child/b.txt

In Ant, this is how you write code to output all text files in any subdirectory of a “dir” directory.

<fileset id="jb" dir="/MyFiles">
  <include name="**/dir/**/*.txt" />
</fileset>

<pathconvert pathsep="${line.separator}" property="out" refid="jb"/>
<echo>${out}</echo>

In Java, the equivalent is

public class MyPathMatcher extends SimpleFileVisitor<Path> {
  private PathMatcher matcher =
     FileSystems.getDefault().getPathMatcher( "glob:**/dir/**.txt");

 @Override
 public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException {
   if (matcher.matches(file)) {
     System.out.println("File " + file);
   }
   return FileVisitResult.CONTINUE;
 }

  public static void main(String[] args) throws Exception {
    MyPathMatcher dirs = new MyPathMatcher();
    Files.walkFileTree(Paths.get("/MyFiles"), dirs);
   }
}

The Java documentation says “Two asterisks, **, works like * but crosses directory boundaries. This syntax is generally used for matching complete paths”. Whereas in the Ant documentation, “When ** is used as the name of a directory in the pattern, it matches zero or more directories.”

The Ant approach probably feels more natural to me because I’ve been using it longer. But the Java approach seems more logical because it doesn’t have the extra slash that doesn’t actually get matched.

java tools for dependencies and build (ant vs maven)

Posted on by
2

A student at the robotics program I mentor emailed me the following question.  Since it isn’t specific to him, I decided to answer in a blog post.

As a developer, what tools does Java have to help me manage things like dependencies and building the project? For example, Python has pip and Ruby has gem+bundler, for managing dependencies.

Can you extol the virtues of Maven and ant? Why should I use one over the other? My research leads me to believe Maven supersedes ant. Are there any other alternatives? And if you prefer one, what makes it valuable to you?

Ant and Maven are the most common build tools in the Java world.  I’ve used both.  Gradle is a third that I’ve heard a lot about.  I haven’t used Gradle, but it looks cool.  The benefits of Maven, but the Groovy language.  Maven and Gradle are actually  both a build and dependency tool.  Ant is just a build tool.  You can use Ant without any dependency management tool/repository.  Or you can use Artifactory or Ivy with Ant.

Maven came after Ant, but didn’t supersede Ant.  Both have their purposes.   We use Ant at CodeRanch to:

  1. Build the forum software 
  2. Generate the book promotion documents every week

Ant serves these purposes very well to this day.  The latest release of Ant was in the past 6 months; it is far from a dead project.  The strengths of Ant include:

  1. Ant works better when internet availability is unpredictable or very slow.   With Ant, you have complete control over when and which jars are downloaded.  (To be fair, you can work with Maven in offline mode once you have the jars locally.  But there are a lot more dependencies and therefore more to download.)
  2. Ant works better when you don’t want to do things “the Maven way.”  Maven uses convention over configuration.  Which is great when you can use those conventions.  They save a lot of time.  (And most of those conventions are fine).  It so happens at CodeRanch, we don’t deploy a full war.  We deploy a zip file containing a subset of the project files.  Again because of that internet thing.  We don’t all have the ability to upload a full war without the internet connection timing out.  This could be done in Maven, but it would be odd.  And I think it would be confusing to use Maven and ignore conventions.
  3. Ant gives you complete control.

Maven has a different set of advantages

  1. It is much faster to get started with a new project.  You can use an “archetype” to get a project template.
  2. You don’t have to think about dependencies.  If you are using X and it depends on Y, both get pulled.  The dependency management is excellent.
  3. Convention means you don’t have to set up much in your build configuration.

I think Maven is going to be better for the project I was asked about here.  You want dependency management which is something Maven excels at.  And you have a greenfield project which means there is no reason you can’t use Maven’s conventions.   Also, many other tools easily integrate with Maven (or Ant) like Findbugs which will be helpful.