csrf for JForum without javascript

In February, I wrote a three part series on how we fixed JForum on coderanch to protect from CSRF.  In included;

  1. Analysis
  2. Extending OWASP
  3. Problems

Remaining problems

Unfortunately, there were three remaining problems.

  1. Some mobile devices weren’t able to handle the JavaScript which added the tokens.  Meaning our site didn’t work on all mobile devices.
  2. The CSRF token was in the URL of thread links which meant people were sharing those links with tokens in them.  This isn’t a significant security risk, but it does confuse Google which is bad for SEO.  It is a tiny security risk in that if someone posts their current CSRF token, that user can be targeted for a CSRF attack until that token expires.
  3. Some people take a really long time to write a post or have lunch in the middle and the token expires.  Giving them a CSRF error page when they finally finish.

This blog post shows how we solved these problems.

Ending requirement for JavaScript to set CSRF token and getting token out of URLS

In general, there are three options for setting a CSRF token in forums/URLs: JavaScript, a filter to change the HTML at runtime before it gets served to the client or hard coding the token.

Based on our experience with mobile devices, we decided not to go with option1 (JavaScript.)  I had used a technique similar to option 2 (changing the HTML as it goes by) to transform our JForum URLs to a different format.  This turned out to be more complex on a forum where users routinely post code.  The same problem would occur here so I decided against that option.  We don’t want to be adding CSRF tokens to code user’s post in questions.  And we certainly don’t want someone to be able to inject a CSRF attack in a post!

This left me with option 3 – hard coding the token.  There were a few steps to implementing this option:

  1. The JavaScript solution added the token to URLs in addition to forms.  This wasn’t “good” in the first place in that URLs shouldn’t be changing database state.  I had recognized this as a shortcoming back in February but lacked the time to fix it then.  There’s a representative list of these pages/URLs on github.  Many were fixed by a one to one conversion of links to forms. (with POST as the method.)  A few were fixed with a generic form on the page and JavaScript that calls it.  We used the generic JavaScript form for some admin links to save bandwidth. Most of these aren’t available in the mobile view anyway.  And moderators weren’t the ones having JavaScript issues in the first place – probably because we have newer mobile devices.
    </span></span>function submitActionForForum(actionVerb, forumId) {
    var action = document.actionForm.action;
    
    action = action.replace("ACTION", actionVerb);
    
    action = action.replace("FORUM_ID", forumId);
    
    document.actionForm.action = action;
    
    document.actionForm.submit();
    
    }
    
    
  2. At this point we don’t need to add tokens to anchors.
  3. Remove AddJavaScriptServletFilter and JavaScriptServlet so tokens are no longer generated by JavaScript.
  4. Add token to all forms:
    <input type="hidden" name="OWASP_CSRFTOKEN" value="${OWASP_CSRFTOKEN!""}" />
    
    
  5. For forms containing multipart/form-data (there were less than 10 of these), add to the URL:(this is needed because the multipart request is only parsed once – later on in the process and the parameter isn’t available to us in the filter)
    ?OWASP_CSRFTOKEN=${OWASP_CSRFTOKEN!""} 
  6. In parallel to the previous two steps, I wrote a unit test to ensure I didn’t miss any required tokens AND to alert anyone adding a form to the codebase in the future about the need to add a token.  Unit test available on github.
The other problem – session timeouts
My first thought was to use AJAX to keep the session active as long as the browser is open.  This session isn’t ideal as some users keep their browser open all day.  I talked to the site owner and we agreed on a three hour timeout for CSRF tokens based on inactivity.  (This also has the advantage of the token surviving a server restart which we do for deployments.)
I implemented by creating a database table with the user id and token.  I have a TimerTask that deletes any tokens over three hours old and runs every 15 minutes.  (This isn’t strictly accurate as it keeps the token alive as long as the user session or the token is less than 3 hours old.  So if I have an active session for 2 hours 55 minutes and a server restart happens, I lose my token.  This implementation may change if it turns out to be a problem in reality.  The current filter and helper class are now online for reference.
What I learned
It’s a lot harder to protect against CSRF on a public website than one with controlled users.  This was an interesting project though.

ken kousen on groovy

I saw Ken Kousen deliver his intro to Groovy talk today at the NY Java SIG.  While I’ve done some small things with Groovy, I’ve forgotten a lot of what I once knew.   Some of it, i remembered/used but am including here so this isn’t random facts but a summary of what was presented. Blogging about it this time in hopes I can retain more.

First of all, Ken is a great speaker.  If you have the chance to see him in person, go.  He:

  • is entertaining – programmer humor
  • presents info clearly
  • has well thought out examples
  • switches well between the groovy console, command line, browser and slides.  While there were frequent switches, it was easy to follow
  • live examples. Surprises and mistakes and all    Icndb.com for humor in json examples provided energy towards the end

Useful links

Now on to Groovy

What is groovy

  • Groovy is like Java 3. Other jvm languages are simulating other languages like jythong or special purpose like functional languages. Groovy is a general purpose like a next generation OO language. Plus you can fall back to Java and make it “groovier” later [which is great for learning]
  • Compiles to bytecode.  Also a C compiler is available.
  • If the name of your groovy file matches the name of the class, you can only have a class in the file. Otherwise you can put multiple classes or even just scripts in the file. Ken’s convention is to use underscores in filenames for scripts to identify them.
  • Hello world is println “hi” [yes, that’s it]
  • Parens are optional “until they are not”. They have a low precedence.  Can leave out if obvious where they go
  • Semicolons are always optional
  • Perls goal is to write the shortest code possible. Groovy’s goal is to write the simplest code possible.
  • Testing and build are the place to start with groovy when introducing into a reluctant organization

Options to run

  • groovy foo.groovvy – don’t have to compile first. Should compile first when use it for real though. If want to integrate with java, must run groovyc to compile since java requires class files.
  • The class files require groovy in the claspath to run with “java” rather than “groovy”. In groovy home, embedable directory contains groovy-all and groovy-allindy. The later is for invoke dynamic in Java

Tools

  • Groovy Grails Tool Suite – for Eclipse
  • Groovy menu has convert file extensions to groovy where right click java class. This lets you refactor slowly. [Or if create java class by accident like often do]

Operator overloading

  • Every operator corresponds to a method.  This means if your class implements that method, you can use that operator.  This lets you write new Date() + 7 to get a date a week in the future.
  • ** is power operator
  • [] goes with getAt or putAt methods.  This is what lets you say s[-1] or s[0..2] instead of pure array indexes.  The getAt/putAt methods understand how to navigate.  The methods also know to call next for 0..2 and prev for -3..-1.  I also learned you can specify multiple ranges such as as s[0..2,4..5]
  • In groovy == calls the equals method. Which means just converting a java class can result in an infinite recursion. Run your tests when refactoring
  • To append: list << value

Groovy Types

  • 123.getClass().getName() is an Integer. If you make it bigger becomes long or BigInteger dynamically.
  • 2 – 1.1 uses BigDecimal so accurate unlike 2d – 1.1d which has Java roundoff error from using doubles.
  • GString “test ${name}” uses interpolation to substitute the variable name. You don’t need braces if just one variable
  • Single quotes are Java strings
  • Double quotes are groovy strings
  • Triple quotes are for multi line strings.  Useful for SQL.
  • Def means don’t know or care about type
  • Duck typing. Can call any methods on def oject. If method exists at runtime ,call it. Else throw MissingMethodException at runtime.
  • Map keys assumed to be Strings

Groovy translation

  • S.class.name works. When it looks like you are referencing a private field, groovy adds get or set to call appropriate methods. The runtime environment provides getter and setter methods so  you don’t need to write them
  • Don’t need to type return belcause last expression is automatically returned

Plain Old Groovy Object (POGO)
Generates at runtime:

  • getter/setter – If don’t want setter, add final to instance variable.  If don’t want getter or setter on instance variable, type private.
  • map based constructor
  • class and methods public by default
  • instance variables private by default

All you have to write is a class and the types.  If you implement a method, Groovy will use yours and not generate it.  If using the map based constructor, Groovy calls the default constructor and any appropriate setters.  This means you don’t need overloaded constructors.

class Person{
String first
String last
}

Person p = new Person()
p.setFirst("f")
println "$p.first"

new Person(first: 'a', last: 'b')

Walkthru of converting a POJO to a POGO

  1. Remove getters and setters
  2. Remove private from  instance variables
  3. Remove semicolons
  4. Remove constructor – need to use map constructor – affects callers [not backward compatible to simplify all the way]
  5. Remove public on class
  6. Remove import for java.util.Date
  7. Add @Canonical which includes @ToString, @TupleConstructor and @EqualsAndHashCode.  All are AST transformations
Changing types
  • [1,2] creates list
  • LinkedList a = [1] makes the desired type
  • [1] as Set also creates the desired type

Closure

listOfStrings.each { /* code using it as default loop variable */ }

map.each{ k,v -> /* code */ }

list.collect{ /* transform to new list where results make up the new list */ }.join(',')  // map reduce

Closure coercion can be used for an anonymouse inner class
{ /* code */} as FilenameFilter
Useful when an interface has only one method because uses implemetation for all methods in interfaces.  Could pass a map of closures but might as well implement interface then.

Helpful classes/methods

  • Groovy adds toURL to String class and getText to Url class masking query.toUrl().text a one liner
  • XMLSlurper lets you get a dom tree from a URL which you can then navigate as tree.root.result[o].geometry.location.lat 
  • db.rows(“selet * from table”)
  • db.eachrow(” query “){ /*code */}
  • groovy.json.JsonSlurper().parseText(json)
  • groovy.xml.MarkupBuilder to create. Similarly for xml, json and ant
Testing
  • Must test code since constraints we are used to are gone
  • Spock over GroovyTestCase – very nice framework.  See example in github/book

Random interesting tips

  • Can use map based constructor on a pojo too because groovy will call default constructor and set methods
  • foo.with{} calls foo.xxx for any methods not found in scope
  • Meta programming: add methods to class that need. Even to a java class. Complex.metaClass.plus= {}.  Every class has a metaclass.

getting computer vision to work on a raspberry pi

This year, I helped a local high school FIRST robotics team get computer vision working on a Raspberry Pi.   Two students worked on the problem.  At the beginning, they both worked on computer vision.  Later, one specialized on the Pi and the other on CV.  I learned a lot from a book and from playing with it.  We encountered some interesting and memorable problems along the way.

Recognizing a target

We started with sample code from another team from last year.  This helped learn how to write the code and understand the fundamentals.  It also helped with the “plumbing” code.  As hard as it was to recognize a target, this didn’t prove to be the most frustrating part.  The Pi itself mounted a number of challenges.

Parts for the Pi

We bought/scavenged parts for the Pi.  A USB keyboard, USB mouse and cell phone micro charger where donated.  A HDMI/DVI cable we needed to buy.  We borrowed a computer monitor and ethernet cable.

Finding the right jars

The Pi is built on ARM.  We needed javacv-0.2-linux-arm.jar.  It turned out there is no linux arm version in the latest javacv (0.3).  There is one in 0.2 which we used.  Which was incompatible with the versions of other tools.  (see next problem.)

Setting up the Pi

Compiling opencv on the pi takes 4 hours.  Since that’s how long a meeting is, this meant running the compile overnight.  Having to wait overnight to find out if something worked was like a taste of what punchcard programmers had to go through!

Then it turned out we couldn’t even use our compile.  We were missing the libjniopencv_core.so file. We spent a few days trying to solve this.  We wound up  using a pre-compiled for Pi version.  This is how we got version compatibility.

Updating a NetBeans Ant script

Since speed matters in a competition,we wanted to change the build run target to not compile first.  Netbeans comes with an empty looking build.xml and a useful build impl xml file.  (This is actually my favorite feature of NetBeans – that the build can easily be run outside of NetBeans.)  We easily found the run target in the build impl file.  We copied it to build.xml, renamed it and removed the compile dependency.  This wasn’t actually a problem, but it was interesting how NetBeans sets up the build file.

Starting a script on startup

We wanted the Pi to start the computer vision script automatically on boot up.  We created a file in /etc/init.d since this is a Linux (Debian) install.  Then we made a fatal error.  We forgot to add the & to run the script in the background.  So when we tested rebooting, the Pi hung.  And we couldn’t SSH to it because it hadn’t booted.  The solution was to take the pi’s sd card to another computer and edit the bootup script to use single user mode.  We could then login and edit the startup script to add the missing &.

Networking

We used Java sockets to transfer the “answer” from the Pi to the robot.  The answer being a single number representing the number of degrees off from the center of the target.  We made the mistake of testing this with both ends on a regular computer.  When moving to the robot it didn’t compile because the robot uses J2ME.  We then refactored to use the mobile version (code here).

Performance – CPU

Success.  Computer vision works.  The problem is it took 3 seconds per image.  We reduced it to 1.3 seconds per image by reducing the resolution to the smallest one the camera supports.  We shaved off another 0.1-0.2 seconds by turning off file caching in ImageIO.  We learned the problem was full CPU usage when calling ImageIO.read.

I found an interesting thread showing the “old way” of creating a JPEG using ImageIcon was much faster.  We tried it and the thread was right.  It even created an image we could open in a photo editor.  The problem is that it didn’t work with our code/libraries for image processing.  We don’t know why.  Obviously ImageIO has some default we are unaware of.  A 1 second time is acceptable so this isn’t  a problem.  But it’s an interesting conundrum.

Another interesting CPU note.  We tried compiling image magik.  It took over 3 hours on the Pi.  By contrast, it took 2.5 minutes on a Mac.