Author Archives: Jeanne Boyarsky

clone a postgresql database for testing cleanly

Posted on by
4

I’m looking at writing integration tests for the back end of JavaRanch‘s JForum install.

A few “pesky” requirements/constraints

  • Multiple developers all over the word have their own local test databases filled with data in different states.  The tests must work for everyone.  Ideally they won’t leave data floating around either.
  • The tests must use PostgreSQL.  While the original JForum supported multiple databases, the JavaRanch version has been scaled down to just run with the one we need.  We do have some PostgreSQL specific SQL which rules out using an embedded database like HSQLDB or Derby.
  • Developers are using both Eclipse and IntelliJ.  Tests should care about the IDE anyway, so this isn’t a big constraint.
  • Developers are using a variety of operating systems and languages on their operating systems.  While code is in English, there can’t be assumptions as to the OS state.

Strategy

I think the best strategy is to create a second database just for testing.  The JForum database would remain untouched and a jforum_integration_test database can be created for the tests.  dbUnit can control the state of that special database.

The problem

Before I even start thinking about dbUnit, I did a proof of concept to ensure I could create a new database from scratch using the command line.  Creating a database is the easy part.  The “hard” part is that JForum doesn’t come with a schema.  It comes with an installation servlet that creates the schema.  While few people will be creating a schema for JForum, the technique I used applies elsewhere.

The procedure “before”

  1. Start up the JForum war
  2. Go to the JForum install URL and enter some information which creates the tables
  3. Run the JavaRanch customizations.

How to clone a database for which you only have a partial script

  1. Create an empty database
    createdb jforum_integration_test
  2. Arrive at the base schema
    1. Go the JForum installation URL
    2. Enter the information to create the tables
  3. Export the schema thus far
    pg_dump -U postgres jforum_integration_test > c:\temp\postgres.sql
  4. Provide instructions for the rest of the sql which were created by our developers.

How to import

Now for the easy part!

Importing this dump is a matter of a single command:

psql -U postgres jforum < "pathToWorkspace\JForum\javaranch-docs\deployment\file.ddl"

Lessons learned after

The next day I learned that this wasn’t enough.  We also needed some test data from the server.  I ran this a few times to get the relevant test data.

pg_dump --data-only --inserts -U user -W database --file roles  --table tableName

Conclusion

My next step will be to actually configure dbUnit against this new database and start writing tests.

algorithms – survival of the fittest

Posted on by
1

I’ve been volunteering with a FIRST robotics high school team as a programming mentor. Due to a “series of unfortunate events”, we didn’t get to test our logic to drive straight in autonomous mode until the night the robot went into the crate.

We wound up using an interesting approach. First we brainstormed options and prioritized them. If the programmers got the robot early, we would test in that order. Alas, we did not get the robot early. We got it for 45 minutes about an hour before packing time.

Since it takes time to load code changes to the robot, we parallelized as much as possible. The students had time to test the competing approaches before we got our turn with the robot. So we tried them all by flipping a switch and then tuned the most promising one. It was a great use of time and available resources. And we wound up with an approach that wasn’t in our initial brainstorm.

The lesson applies to us all. Be aware of your primary constraint; in this case time; and optimize to that. Punch card programmers certainly knew this and optimized per run at the machine.

I was very impressed. Good luck in the regional competition team 694!

another reason % of time writing tests is meaningless

Posted on by
5

Earlier in the month, I blogged a response to – “What would you say is the average percentage of development time devoted to creating the unit test scripts?”.  As I was telling a friend about it, I realized that I missed an important point!

The question also implies that development time is constant.  Or maybe not.   But if I tell you that I spend X% of my time on creating unit tests and your project currently spends Y time on development, I imagine you are considering the development time with unit testing takes Y + Y*X.

However, this isn’t the case.  The real time is Y + Y*X – Z where Z is the time saved in finding the defects in the code faster.  The benefits span past the current project, but this factor matters even if one just measures within the current project.

Or another way of expressing this is that “your Y is different than my Y.”  My time on development has been adjusted by creating software in a different way.

This seems like a convoluted way of expressing it.  Has anyone come across a better explanation?