I tried running Parasoft’s JTest against the modified JForum use by JavaRanch.

The first thing I did was run all the static analysis rules and got over 47 thousand hits.  Now this doesn’t mean much as some of the rules are cosmetic things and some directly confict with each other. (Use “*” for imports and don’t use “*” for imports imply every single file with get hit for one of them.  I started running all the rules so I could drill down and see what was out there.

A few of the rules were things that I had already noticed and made me cringe when reading the code.  For example, PostAction as a cyclomatic complexity (branch count) over over 40.  Ten is high.  40 is almost unreadable.

Some of the more interesting ones:

• Found some duplicate code and string literal duplication.
• The rules were updated for Java 5 – For example, it found I overwrote toString() but not valueOf() in an enum.
• security rules for SQL injection
• I really like “remove commmented out Java code” as this is a pet peeve of mine!
• I learned that there is a problem with “double checked locking” pattern if you don’t use the volatile keyword.
• As expected, JTest found unclosed resources, missing static keywords and the like.
• There’s a bug in Java’s substring implementation that can cause an out of memory error.
• It didn’t find a specific defect I was hoping to uncover when I downloaded it.  But this was more informative than not – now I know that defect isn’t in the code without having to look through hundreds of classes!

Some of the rules offer a “quick fix” where you can automatically fix the violation through your IDE plugin.  I used this to add {} to single line conditionals in 50 places without effort.

The only complaints I ran into:

• It’s mostly updated for Java 5, but not completely.  One rule says to using StringBuffer instead of String when concatenting strings (when you should use StringBuilder.)
• It’s slow – 5 minutes – 11 rules – 519 files.  PMD goes faster.
• And of course – it’s expensive – but it’s intended for corporate users.  Thank you Parasoft for the trial to use on our non-commercial software!

The breadth of errors JTest can find is amazing.  It doesn’t just save you time manually reviewing code – it tells you about subtle errors you might not know about.

Overall, I still think that Ted Nelson had the best idea when he wrote this in Computer Lib/Dream Machines (Aperture, 1974):

The best way to start programming is to have a terminal running an interactive language, and a friend sitting nearby who already knows the language, and has something else to do but can be interrupted with questions. And you just try stuff. Till more and more you get the feel of it. And you find yourself writing programs that work.

- “Software Craftsmanship” (page 95) and available online in an article about how it applies to Ruby.

This quote was from 1974.  Since then, some things have stayed changed and some things have stayed the same.  In particular, we now have personal computers rather than interactive terminals.  Of course, a command line prompt or IDE is an interactive session – just a more powerful one.  I think the concept is just as critical as it was then.

I can think of a few ways this learning applies to me here in 2009:

1. Summer interns – When I was a college intern, I shared a cubical with an employee.  This made it easy to ask a short question when I got stuck.  (among other benefits)  The scenario was very similar to the quote – there was someone sitting nearby that I could ask without getting up that already knew the answers.  I plowed along until I got stuck and then there was someone to put me back on the right path.  I really appreciated this approach and have voluntarily shared my cubical with our summer interns once I was the employee.
2. Pair training – Training new developers by pairing with them allows them to see the big picture before learning all the details.  If the goal is to be able to develop without pairing, the new developer needs to become self sufficient.  I like the idea of a more gradual path to there.  Whether it be going back to my own desk for snippets or bringing my laptop to work on other stuff, it provides the atmosphere where the new developer can start applying knowledge, but having someone “on call” to ask.  Personally, I think it was more effective when I brought my laptop because it lowered the “cost” of asking.
3. JavaRanch – “a friendly place for Java greenhorns” – an online forum doesn’t have quite the same turnaround as someone sitting nearby, it is certainly better than nothing.  Very easy questions do tend to get answers within a few hours.  It’s the same idea of having someone available to ask when learning a new language.  The internet is great for that when there isn’t something in the office.

Not all my examples just cover learning a new language, but they do show how having someone more experienced around helps with learning other things.

Don’t you just hate threading errors?  They are hard or impossible to reproduce.  It’s hard to figure out exactly what’s going on.  It’s hard to tell if you fixed it.

The problem

We’ve had a problem at JavaRanch for about a month where some posts mysteriously didn’t appear.  The problem was at its worst early in the business day in India pretty regularly.  When unfortunately pretty much everyone who worked on the system is sound asleep.  It did occur at other times as well.  We did come up with a workaround pretty quickly – reboot the server.  I know – not much of a solution, but it let operations continue in some fashion.

We had a few symptoms:

• It often took 1-4 minutes to post.
• The page refreshed without actually storing the post although the index page indicated the post was there.
• Our e-mail server was acting up causing it to take hours to send out certain types of e-mails even after many retries.  (We didn’t realize this was a symptom until the problem was solved.  At the time, it was just another incredibly time consuming problem that needed addressing.)

What we did

We added metrics.  We monitored.  We did code reviews.

Attempt #1 – A week after the problem started occurring one moderator thought he found it.  There was an Executor class in JForum that added e-mails to send out to a queue.  If the queue was full it blocked causing the user to have to wait minutes for a post.  Which in turn tied up all the resources.  Eventually the database transaction would timeout waiting and give up.  Since the cache was already updated, it looked like the post partially went through and the database didn’t know about it.  So he added a call to Doug Lei’s concurrency library and deployed executor.discardOldestWhenBlocked();

The problem didn’t go away!  But we were so sure it was that.  Oh, no!  Now we have to find another root cause.  More metrics, more monitoring.  More trying to figure out what was going on with the mail server.  Another moderator started work on moving to a different mail server.

Attempt #2a -We moved to a different mail server.  It’s slower than the old one so still possible to back up the outgoing queue.  Just less likely.

Attempt #2b – A third moderator (me) wrote a standalone test against the executor to see if we could reproduce the problem.  We replaced the e-mail logic with Thread.sleep() for 10 seconds and then threw a bunch of threads at it.  Lo and behold the later ones all blocked.  At this point there were two possible solutions – upgrade to Java’s built in concurrency package  which doesn’t have this race condition – or call executor.discardWhenBlocked().  After consulting with Henry Wong of “Java Threads“, we decided to make the smallest possible change and call a different executor method.

Both attempts 2a and 2b were deployed over the same weekend so we don’t know for sure what fixed the problem.  We do know that 2a helped and 2b worked in simulated tests.  And most importantly, we know JavaRanch hasn’t lost posts or had long posting delays in a week.  That’s the important thing!

Lessons learned

• Listing all the symptoms in one place helps draw connections.
• Test a potential fix works in an isolated test case.  We were *so* close three weeks ago.
• Having a threading expert on hand saves a lot of time in making sure a fix doesn’t have negative side effects.  Just in explaining to Henry we thought about the problem on a deeper level.

JForum 3 (in development) does not have this problem as it does not use Doug Lei’s concurrency library.  I will report the problem for JFOrum 2.1.8.  While it’s only a problem for large installation with e-mail or resource problems, it should be in their forums too.

As I sit here listening to my final JavaOne presentation there’s one question that has lingered on everyone’s mind this week: Will there be a JavaOne next year? In case you’ve been living in a cave (or a basement coding), Oracle has purchased Sun, the creators and maintainers of the Java programming language, and Oracle has a history of rebranding and reorganizing companies it has acquired.

Despite Larry Ellison, the CEO of Oracle, appearing on stage at the key note address, I and other attendees such as Tim Bray, can’t help get the feeling this will be the last JavaOne. Let me state right now I do not have any solid facts either way; Sun and Oracle have been purposely tight-lipped about it. This article is more about the current climate of the conference from a developer perspective and some wrap up thoughts.

For starters, the most consistent comments I’ve heard all week from alumni that have attended the conference for years, is that the conference is significantly diminished in size this year. Perhaps its the state of the economy or perhaps its the state of trade shows in the industry as a whole, but one way or another, attendance and vendor presence feels trickled down at best, deserted at worst. In fact, as one attendee pointed out this morning, Sun filled many of the Pavilion booths with its own company products. Whether it was the Sun Cloud, Sun Open Solaris, the Sun Java Store, Sun’s Duke awards, Sun’s Drizzle, etc, it felt like nearly half the booths were populated with Sun employees. That’s not to say there weren’t some good vendors and demos (JBoss, Intel, Microsoft, BlackBerry … Apple still missing in action), but it definitely felt like many vendors chose to skip the show. Given Java’s open community roots, its quite possible Oracle will keep the JavaOne showing going, if only to keep the community pleased, but its hard to say for sure. I guess we’ll have to all wait and see.

Some wrap up notes about the show:

1. Java FX: One of the two big buzz words from the show was definitely JavaFX, which is sort of like a 3D/advanced graphics version of Swing that is theoretically easier to use. I say “theoretically” because I have been hard pressed to find one attendee who’s used it in a production system. Also, from what I’ve heard from other attendees the mantra “JavaFX is the future of Java UI” is a bell they’ve been ringing for 2+ years now with little coming to fruition. Sun demo-ed an easy-to-use JavaFX builder tool but the tool will not be released until end of the year at the earliest. My take on it, shared by other members of the conference I’ve spoken with, is that Sun has been so slow to get JavaFX out the door, other technologies are already light years ahead and currently used in production systems.

2. Cloud Computing: The other big buzz word of the show was cloud computing. Sun, like Google, Amazon and a dozen other companies, has started its own cloud computing system for running and storing your applications, and you couldn’t trip over a booth this year without seeing the word cloud somewhere. For those not familiar with cloud computing, its really just a fancy word for a set of virtualized servers, in which the developer knows nothing of the underlying hardware. Turns out this website has existed ‘in a cloud’ for years; who knew? The cloud topics were actually interesting since the hardware has finally caught up such that virtualization isn’t as costly as it once was so I look forward to more advances in cloud computing.

3. The Java Store: One of the more disappointing presentations was unveiling of The Java store. For starters, it felt derivative right off the bat, following Apple’s iPhone store, BlackBerry’s Store, Verizon’s Store,… the list goes on and on. Second, at its launch there’s no way to charge users for applications, everything’s free and the pricing model is undecided. How does Sun expect to encourage developers to contribute their own time if they have no idea how they will be reimbursed? They should have thought that out and implemented a pricing model in the very first version of the store, beta or otherwise.

Third, the beta was far from impressive. I had to sign up for a beta account, wait to be approved to access it (why do you need approval to use it if everything is free?), and using it was painful. I received 6-8 pop-up “Are you sure?” and “Do you trust X?” questions while installing, using the store, and installing the applications. Lastly, this is the first program on a Mac where dragging and dropping the application icon to the trash can does not actually uninstall it. The application icon you see is just a complicated link to the files that have been downloaded to your classpath. In other words, you have to use the Java Store to uninstall applications. This is another area where they need to make it simple and easy to use, else no one’s going to use it.

4. Join us and write apps for free! As I traveled from booth to booth, one of the most consistent sentiments from each company, whether it be mobile device makers such as Sprint and Blackberry, cloud operators such as Sun, copier device makers such as Ricoh, or even ’smart pens’ makers such as Livescribe, was: We have Java APIs now, come write applications for us!. While its clear Apple’s iPhone developer community success has changed the way companies find developers, I couldn’t help but wondering where they think tons of developers are just going to come aboard and write applications for them for free. Some do have solid pricing models for developers getting paid, but for the most part I think they are hoping developers magically materialize, at least on paper, so they can put on the appearance of staying competitive with Apple. I’m sure the next 2 years will be rampant with these stores failing or succeeding. For example, BlackBerry has had public APIs to develop on their mobile for years, and the quality of the software in the community, up until recently, has been awful.

Final Thoughts
If this entry seems critical of JavaOne, it’s only that I worry for the future. Make no mistake, I loved the JavaOne conference. I had a wonderful time, met a lot of really cool people, and expanded my knowledge of a lot of technologies and languages. The community is one of a kind, and everyone here is really top notch. Where else can you have conversations about serialization and cross-site scripting over breakfast? Thanks to everyone I met, the presenters in particular, and I hope to see you all next year!

Live from the JavaOne conference, I’m currently attending a session on JCA specification 1.6 enhancements. According to the specification team, the goal of the new version of the spec is to simplify writing resource adapters (RAs) for those who have not written them in the past or may be new to the specification. In this article, I highlight some of the important new features.

1. New Annotation: @Connector
An alternative for creating XML information, this can now be done simply in the annotation.

2. New Annotation: @ConfigProperty
Complicated in previous versions, the new 1.6 spec will allow for automatic discovery of config properties to ease development of connectors.

3. New Annotation: @Activation
They’ve replaced activation interface required with simple annotation property as follows:

Generic Work Context
They updated the concept of work context to accomplish tasks. A resource adapter submits a job request to the work manager, which then creates a work plan to complete the request, such as setting the proper transaction request. Backwards compatible with existing JCA 1.5 connector work solution.

More Information
This article is briefer than previous ones, in part because the slides were fast and flowing during the talk. Hopefully, I’ll update the information when the official slides are published.