Category Archives: Java/Java EE

dbunit vs recreate schema

Posted on by
Reply

As I continue looking at writing integration tests for the back end of JavaRanch’s JForum install, I faced the decision of how to guarantee a clean database.  A previous post on the topic covers how to clone a postgresql database via the command line.

I was originally thinking about using dbUnit to import data.  I may still do that if I find myself needing a lot of data.  However, I realized it was more important to be able to recreate the table structure each time.  Our database schema changes regularly as we enhance tables and I don’t want people to have to update the exported dataset. The concept is to have the test create the schema/tables/data from scratch each time it runs.

Some interesting things in this infrastructure:

  1. JUnit suite wide setup – I had so much to say on this topic that I wrote a separate blog post on it.
  2. Main method – The main method is interesting because it provides the starting point. It sets up the database (more on that later) and kicks off all the JUnit tests. It uses the JUnit runner to benefit from JUnit’s reporting mechanism. I also added logging for the database setup in case it took a long time on someone’s machine. This turned out not to be a problem. At the moment, I’m leaving it “just in case.” 
    public static void main(String[] args) throws Exception {
long start = System.currentTimeMillis();
setUp(args);
long end = System.currentTimeMillis();
System.out.println();
System.out.println("Done setting up database in " + (end - start) + "ms.  Now running tests.");
System.out.println();
JUnitCore.main("com.javaranch.test.functional.All_JForum_Functional_Tests");
}
  3. Pointing to a test database– Many developers, including myself, have test data in their “jforum” database and wouldn’t appreciate the integration tests mucking around with it. As a result, the integration tests use a special “jforum_integration_test” database. This database has its schema recreated each run of the test. Conveniently, JForum has a property for the database name that all the code uses. Having the code update this in setup is sufficient. 
    SystemGlobals.setValue(ConfigKeys.DATABASE_CONNECTION_DBNAME, "jforum_integration_test");
  4. Tear down first
    A common pattern when integration testing is to call tear down before setup. Tear down drops the “public” schema. (which is where postgresql stores everything) Setup creates the schema and loads the DDL/SQL. This is done in a DatabaseController class to keep the All_XXX_Tests class uncluttered. For example:

    conn.prepareStatement("DROP SCHEMA public CASCADE;");
  5. Running a DDL file – This is a blog entry in and of itself. Which is good because I made it on

Conclusion
I’ve been using this test structure for a couple weeks now. It has served the purpose well and I expect it to live on. A nice side effect is that we find out very quickly if the DDL is incorrect in SVN!

GET vs POST and URL security

Posted on by
3

Is GET or POST more secure?  Like many things in computers, it depends!

Who are you trying to secure data against?

  1. The user in his/her browser
  2. People who legitimately see the URL
  3. Hackers

The user in his/her browser
This is the case that is usually discussed.   Some people will naively say they want to “secure” the data by using POST.  That way the user “can’t change the submitted data.”  Of course, this hooey.  Anything on the user’s machine is something the user can see/change.

People who legitimately see the URL
Many people have access to the URL such as in logs.  Having sensitive information in the URLs is a bad idea.  This actually happened recently at JavaRanch.  A user started a thread inquiring about a thread that linked to his but he couldn’t see the protected page.   At JavaRanch, as on many blogs, URLs look like “http://www.coderanch.com/t/493907/Ranch-Office/Could-anyone-enlighten-me-please”.  Luckily we had taken a precaution and used a shorter form of the URL for our private forum.  Otherwise information could leak out!

Similarly, social security numbers and other sensitive information should not be in a GET form submission because the information is then out of your control.  If at all possible, they should be kept on the server and never sent to the user’s machine in the first place.

Hackers
Hackers are a harder case because the hacking can be in multiple places.  For truly secure information, you have to use HTTPS.  For “medium” information, POST is still better than GET because URLs are easier to intercept than whole pages.

Conclusion

As a rule of thumb, POST is going to always be more secure than GET because it removes the “data in the URL” issue.  For some things, neither is secure enough.

process builder to run multi-platform postgres import

Posted on by
Reply

We saw how to clone a postgresql database for testing cleanly in an earlier blog post. The next step is to do so from Java so it can be part of an integration test.

How to run each of 4 DDLs:
Running a DDL file is easy which is good since we have four of them.

importDdlOrSqlFile("initialJForumSchema.ddl");
importDdlOrSqlFile("javaranchCustom.ddl");
importDdlOrSqlFile("users.ddl");
importDdlOrSqlFile("databaseChangesAfterMostRecentInstall.txt");

Ok. So maybe it is easy because all the functional code is in another method.

What is Process Builder?

Process Builder was introduced in Java 5.  The Runtime.exec JavaDoc says

ProcessBuilder.start() is now the preferred way to start a process with a modified environment.

Since I need to set two system properties, this is perfect.

How to create the Process Builder

On Windows, you just pass the command to run at the DOS prompt.  The actual command was shown in the closing a postgresql database blog post.  On UNIX/Mac/etc, you need to create a new shell and pass the command that way.  Luckily, it is easy to check for a Windows operating system:


private ProcessBuilder createProcessBuilder(String command) {
String os = System.getProperty("os.name").toLowerCase();
if (os.contains("win")) {
return new ProcessBuilder(command);
}
return new ProcessBuilder("sh", "-c", command);
}

How to set system properties

I need to set the user id and password as operating system properties so postgresql doesn’t prompt for them interactively.  Using process builder, I can call environment() to get a map of environmental properties for my process.  Then setting them is simply a matter of setting them in the map.


private void importDdlOrSqlFile(String fileName) throws Exception {
String command = importToPostgresCommand(fileName);
System.out.println(command);
ProcessBuilder processBuilder = createProcessBuilder(command);
Map<String, String> env = processBuilder.environment();
env.put("PGUSER",   SystemGlobals.getValue(DATABASE_CONNECTION_USERNAME));
env.put("PGPASSWORD",  SystemGlobals.getValue(DATABASE_CONNECTION_PASSWORD));
processBuilder.redirectErrorStream(true);
runProcess(fileName, processBuilder);
}

How to run the Process Builder

All ready to run it!   start() kicks it off.  I think read all the output from the commands via the process builder’s input stream and output it to the console.  (I do this since many commands are run and the developer can see what it is up to.)  Finally, I check the error code.  After all, if the database didn’t get created properly, there isn’t much point in having a whole pile of failing tests.


private void runProcess(String fileName, ProcessBuilder processBuilder) throws IOException, InterruptedException {
Process proc = processBuilder.start();
Scanner scanner = new Scanner(proc.getInputStream());
while (scanner.hasNext()) {
String line = scanner.nextLine();
System.out.println("importing data: " + line);
}
int exitVal = proc.waitFor();
System.out.println("Completed loading " + fileName + " Exit value: " + exitVal);
if (exitVal != 0) {
System.out.println("Please fix error in database script and re-run.");
System.exit(1);
}

}

Testing

This code was tested on multiple developer’s machines including Windows and Mac.