sonar plugins and why i had to fork the sonar web plugin to add a rule

Last weekend, a few of the CodeRanch moderators were discussing a Trac item that we’d like to get rid of the I18N messages from our HTML. It’s there at all because open source JForum had it. We haven’t been updating it and have been using it inconsistently. Also, we do want to externalize the ones in Java which is why we can’t just get rid of the file wholesale – someone could add more references from HTML. I suggested that this would be a good thing to set up a SonarQube rule and volunteered to create the rule. This was a far harder and more interesting task than I expected it to be when I said I’d do it.
webProps

The web plugin scans HTML files and has a bunch of pre-defined rules and templates for them. You can set any file extensions you’d like so I had set “.html,.html,.ftl” in Administration > Configuration > Web to hit both our HTML and Freemarker code. Alternatively, you can set this in your Sonar properties file.

Looking through the rules and rule templates in the web plugin, there wasn’t one for what I wanted – plain String or regular expression search. Nuts! There are a few existing rules that do what I want, however I couldn’t use them:

Rule Why can’t use
Some Java packages or classes should not be used in JSP files In web plugin, but only for jsp/jsp files
Disallowed methods should not be allowed In Java plugin so only works for Java code
XPath rule Only works for XML files
The text plugin regular expression rule So close! This does exactly what I need. Except (see next section)

Great, the text plugin meets my needs. But alas. You can only associate a file extension with one plugin for each Sonar run. The HTML/Freemarker files are already considered web files.

So what’s wrong with the text plugin?

The text plugin has a rule called “Simple Regex Match”. When configured with the proper regular expression, it does exactly what I want. Except for one huge problem. In order to use it, I have to associate the text plugin with HTML/Freemarker files. This is a two step procedure:

  • In the browser, Administration > Configuration > Web – set the web extensions to “foo”. Leaving it blank is insufficient as it defaults to include HTML and Sonar does not let the same file extension be associated with multiple plugins.
  • In sonar-project.properties, add the line: sonar-text-plugin.file.suffixes=.htm,.html,.ftl

You also have to run sonar-runner twice. Once with the HTML/Freemarker files associated with the web plugin to get those results and again with them associate with the text plugin to get the rule I’m trying to write. This means you need to have the code show up as two projects within SonarQube. (otherwise Sonar will tell you the rule violations were fixed for whichever analysis you run last.)

Extending the web plugin

Ok. I clearly need to extend the web plugin so that I can have a regular expression rule associated with the web plugin. The docs say the web plugin doesn’t support extending it with Java or XPath rules as does this post. I tried anyway hoping this was out of date. I was able to get my new rule to show up in Sonar, but not to run. Everything I’d need to do in order to get it to run was considered a duplicate and prevented Sonar from even starting up. For example, “org.sonar.api.utils.SonarException: Can not add the same measure twice” or “duplicate keys not allowed”. So I concede this path is a non-starter. It would have been nice to have my custom rule in a separate Maven project rather than having to fork the existing one.

Forking the web plugin

Sigh. Last resort. Time to fork. This newsgroup post pointed to a commit that shows how to add a rule to the web plugin. I wanted to add a rule template rather than a rule, but the idea is the same. It wasn’t hard to make the change within the project. It’s just that forking isn’t good in the long run as I’ll have to keep re-merging. What I needed to change in my fork:

  1. Add custom rule template class to org.sonar.plugins.web.checks.coding package. The code was pretty similar to the rule for illegal spaces:
    package org.sonar.plugins.web.checks.coding;
    
    import java.io.IOException;
    import java.nio.charset.Charset;
    import java.util.List;
    
    import org.sonar.api.utils.SonarException;
    import org.sonar.check.Priority;
    import org.sonar.check.Rule;
    import org.sonar.check.RuleProperty;
    import org.sonar.plugins.web.checks.AbstractPageCheck;
    import org.sonar.plugins.web.checks.RuleTags;
    import org.sonar.plugins.web.node.Node;
    import org.sonar.plugins.web.visitor.CharsetAwareVisitor;
    import org.sonar.squidbridge.annotations.NoSqale;
    import org.sonar.squidbridge.annotations.RuleTemplate;
    
    import com.google.common.io.Files;
    
    @Rule(
      key = "RegularExpressionNotAllowedOnLineCheck", 
      name = "Regular expression \"regex\" not allowed on web page", 
      priority = Priority.MAJOR, tags = {
      RuleTags.CONVENTION })
    @RuleTemplate
    @NoSqale
    public final class RegularExpressionNotAllowedOnLineCheck extends AbstractPageCheck implements CharsetAwareVisitor {
    
    	@RuleProperty(
          key = "regex", 
          description = "Single line regular expression to prohibit on web pages")
    	public String regex = "";
    	
    	private Charset charset;
    
    	@Override
    	public void setCharset(Charset charset) {
    	   this.charset = charset;
    	}
    
    	@Override
    	public void startDocument(List<Node> nodes) {
    		List<String> lines;
    		try {
    			lines = Files.readLines(getWebSourceCode().inputFile().file(), charset);
    		} catch (IOException e) {
    			throw new SonarException(e);
    		}
    		for (int i = 0; i < lines.size(); i++) {
    			// only support matching text within a single line to facilitate identifying line number
    			if (lines.get(i).matches("^.*" + regex + ".*$")) {
    				createViolation(i + 1, "Replace all instances of regular expression: " + regex);
    			}
    		}
    	}
    
    }
    
  2. Add the rule template to org.sonar.plugins.web.rules.CheckClasses so the web plugin knows about.
  3. Create a new HTML file in the resources localization directory.
  4. Write/update tests. I like that the sonar-web-plugin project uses JUnit tests to enforce some of there conventions like rules ending in “Check”.

Where is the code?

I’ve placed my fork on Github. It’s 2.5-SNAPSHOT because I didn’t see any need to version it. I also created a pull request for the sonar-web-plugin in hopes they are interested in merging it. The plugin was last released a year ago so I imagine I’ll be using my fork for along the foreseeable future in any case. Which is fine. CodeRanch is a volunteer run website so no rules against using forked software.

how the OCA Java Programmer certification compares to AP Computer Science A

I gave a presentation to high schoolers about the OCA Java Programmer I certification. I was asked over email how well the AP (Advanced Placement) Computer Science class provides the foundational info for the OCA cert:
My main question is wondering how close within reach the java certification would be and how useful it would be to students.  From what you said, the value of having the cert is very good.  But, I’m still wondering about how much additional work will be required to achieve a good result.  I understand your point about needing study materials etc,-  that’s all a given and something that today’s students are very used to doing (PSAT, SAT, ACT, AP, CLEP, etc.).  But, do you expect that generally an AP Computer Science course would provide the foundational info needed to pass the test, and then the student just needs practice to be proficient and quick?
I took AP Computer Science over 15 years ago. It wasn’t in Java and a lot has changed. So I looked at the College Board’s Computer Science course description to find out. I also learned that AP Computer Science AB no longer exists.
AP Computer Science vs the OCA
The OCA exam is meant to cover the basics. In some ways, the AP exam goes beyond that. For example, the AP exam covers recursion. There are a few things covered on the OCA that don’t appear to be covered on the AP exam:
  • Exceptions – checked exceptions, throw/throws, try/catch block
  • StringBuilder
  • byte, char, float and long
  • Wrapper classes
  • Importing other Java packages
  • Garbage collection
  • Overloaded methods
  • Java 8 date/time
  • Java 8 lambdas/predicates
  • Plus a few things that I’m not clear as to whether they are on the AP
    • Ternary operators
    • Switch statements
    • Break/continue

All of these is easily learned through a study guide so I think a student preparing for the AP exam is in good shape to pick of an OCA certification study guide.

Plus it is likely the student’s class is using a book which was not intended for the narrower scope of the AP exam. Which means the student would have come across some of the above anyway.

java 8 jshell mock questions

I gave a presentation showing how to make mock exam questions. We used Java 9 JShell as the example. The idea was to pick a feature nobody knew much about. We started by picking key points from this blog. Then we made questions. This post is the results. (Edited for formatting, to alphabetize answers, after confirming, etc). Good job all!

Question 1

What does REPL stand for?

  1. Random Enterprise Process Logic
  2. Rapid Enterprise Process Line
  3. Rapid Execution of Programming Logic
  4. Read Evaluate Print Loop
  5. Read Evaluate Print Line
  6. Read Execute Print Loop

Question 2

When typing into JShell interactively, it uses ______.

  1. Nashorn
  2. JRE
  3. JS
  4. JVM
  5. REPL
  6. RPEL
  7. Streams

Question 3

Which of the following compiles when typed into JShell? (Choose all that apply)

  1. Class.forName(“String.class”)
  2. Class.forName(“String.class”);
  3. Thread.sleep(1000)
  4. Thread.sleep(1000);
  5. System.out.println(“hello”)
  6. System.out.println(“hello”);

Question 4

Which of the following compiles when typed into JShell? (Choose all that apply)

  1. void method() { throw new Exception(); }
  2. void method() { throw new RuntimeException(); }
  3. void method() throw Exception { throw new Exception(); }
  4. void method() throw Exception { throws new Exception(); }
  5. void method() throws Exception { throw new Exception(); }
  6. void method() throw Exception { throws new Exception(); }

Question 5

Which of the following compiles when typed into JShell? (Choose all that apply)

  1. System.out.println(“Hello Java”)
  2. System.out.println(“Hello Java”);
  3. public void method() { throw new Exception(); }
  4. public void method() { throw new Exception(); };
  5. public void method() {System.out.println(“Hello Java”)}
  6. public void method() {System.out.println(“Hello Java”);}
  7. public void method() {System.out.println(“Hello Java”);};

Question 6

What are valid statement terminators in JShell? (Choose all that apply)

  1. ;
  2. :
  3. \n
  4. None (just press enter)
  5. None of the above

Question 7

Which of the following lines will compile and display a value if it’s the first line you type after launching JShell? (Choose all that apply)

  1. 5 + 6
  2. a + b
  3. System.out.println(“Hello”);
  4. System.out.println(“Hello”)
  5. out.println(“Hello”)
  6. out.println(“Hello”);
  7. { System.out.println(7) }
  8. { System.out.println(7); }
  9. 6 + 6; 10
  10. None of the above

Question 8

Which of the following is valid code in JShell? (Choose all that apply)

  1. system.out.println(“hello world”);
  2. system.out.println(hello world);
  3. Println(“hello world”);
  4. “hello word”
  5. hello world
  6. Answer: 5+3
  7. 5f+3
  8. new String()

Question 9

What is the output of the code below if executed in JShell?(Check all that apply)

Integer a = 5;

Integer c = a + 10;

Integer a = 7

a

  1. Compiler Error
  2. Runtime error
  3. 7
  4. Error: Duplicate variable declaration
  5. None of the above

Question 10

Which of the following commands invokes the help command in Jshell?  Choose all that apply.

  1. /h
  2. /help
  3. /?
  4. ?
  5. f1
  6. ctrl-f1
  7. help

Question 11

How would you invoke help on a particular Jshell command?  Choose all that apply.

  1. /help=command
  2. /help:command
  3. /help command
  4. /he command
  5. /h command

Question 12

double area (double radius) {

return 3.14 * square(radius);

}

area(2)

What is the correct output?

  1. 6.28
  2. 6
  3. 7
  4. Attempted to call area(double) which cannot be invoked
  5. Attempted to call square which cannot be invoked

Answers

  1. Choice 4 – Read Evaluate Print Loop
  2. Choice 5 – REPL
  3. All 6 choices are correct. The first two throw an exception, but do compile.  (This question had public in the user group lab. I wrote this question and removed it after)
  4. Choices 2 and 5. The semicolon is only optional at the end of the statement; not within a block
  5. Choices 1, 2, 6 and 7. The last two give a warning about public not being allowed, but work. The others fail because of an exception or missing semicolon within a method.
  6. Choices 1 and 4. Semicolon terminates a statement or you can omit the terminator
  7. Choices 1, 3, 4, 8 and 9. The last one is tricky. It gets executed as two statements.
  8. Choices 4, 7 and 8. JShell is case sensitive; just like Java
  9. Choice 3. (In the user group lab, there was another choice about a warning for a duplicate variable. I couldn’t reproduce the warning so removed that choice to avoid ambiguity.)
  10. Choices 2 and 3. Isn’t memorization fun?
  11. Choices 3 and 4. More memorizing
  12. Choice 4. Creating the method works, but gives a warning that square(double) must be declared. Then calling area gives “attempted to call method area(double) which cannot be invoked until method square(double) is declared”