Live from TSSJS – Testing in the Cloud with Andrew

Posted on by
1

This afternoon I’m live blogging from TheServerSide Symposium, attending “Breaking all the Rules:  The Myth of Testing and Deployment in the Cloud” presented by fellow CodeRanch Andrew Monkhouse.

1.  CodeRanch History
Andrew opens his talk with a discussion of CodeRanch performance, traffic, and memory usage.  He mentions there have been serious issues in the past that haven’t temporarily brought the performance of the website down to a crawl.  CodeRanch uses a fork of JForum (2008) and staffs around 20 developers among 150 moderators, all volunteer participants, to build a custom community-based forum.

CodeRanch developers use a number of test strategies including smoke tests, load tests, and stress test.

2. Redeploying in the Cloud
The focus of Andrew’s presentation will be to demonstrate how CodeRanch would function in the cloud.  He has allocated a “small EC2 instance” (1.7GB memory, 1 compute unit, moderate I/O performance) within Amazon’s cloud that is useful for quick set up, but extremely small for practical use and reliable testing.

3. Benefits of the Cloud
The cloud provides a relatively clean environment in which it is ‘cheap’ to add extra servers, at least compared to running a local test server which may be allocated for additional processes.  Also, tests can run by anyone, anywhere, which is great for a distributed developer team, since Amazon allows you to assign rights to additional users.

You can easily create, set up, and desotry Amazon Machine Images (AMI), so you do not need to get it right the first time.  Lots of backup storage options, depending on your needs.

4.  Dangers of the Cloud
Testing in the Cloud can be difficult.  First off, it is hard to get absolute numbers because of the nature of the underlying change in hardware happening within the cloud.  Second, calibrating results is difficult.  Also, developers have to remember to shut down servers after they are down with them else other developers will run out of resources.  Finally, network traffic does cost money so testing can be expensive, although hardware is cheaper in cloud.

5.  Test Setup
The performance tests were set up into 3 servers, in part because it turned out to be cost effective to set it up this way based on the individual server options.

  • Application Server running CodeRanch
  • Database Server
  • JMeter Test Server

JMeter was used at the testing suite software given its large variety of features, options, and plugins.

6.  Test Analysis
Andrew discussed some items to keep in mind when testing the server and understanding the results:

  • It is quite possible for the Test Server to have greater load than the Application server, so be sure to monitor all your servers!
  • Try different graphs, they may tell you different things
  • JMeter plugins are your friend, they will help you identify problems with your application server and/or test set up.
  • Don’t forget about ramp up times

7.  Where do we go from here?
Automated weekly performance tests using the latest code base.  Add more stress tests.  Andrew comments there are no plans to move the CodeRanch to the cloud at this time given that we require full-time administrators since we don’t have a dedicated, paid team of administrators.

Conclusion
Andrew presents a compelling argument why you should be testing your applications within the cloud.  It provides a sanitary environment in which to perform a large variety of tests across multiuple servers.  I’ve heard arguments of moving application servers to the cloud, but this is the first time I’ve heard it suggested to migrate all automated testing to the cloud.

Testing java ee applications – the server side java symposium

Posted on by
3

This session was also fast moving and information filled, but it didn’t feel as overwhelming as the Spring session. The presenter included demos of some of the technologies.  He did use the term “unit test” for things that test through the web and database. I would rather these be called integration/functional tests. I wish the testing industry would use one set of definitions. I subscribe to the one where a unit test doesn’t involve the database or the server

Servlet layer

  • Junit – need to build on
  • httpunit – higher level api on http level – the demo shows creating requests and responses using the java api. This technology is preferred over htmlunit because you are usually not generating raw html if you are using a servlet.  [you should be using MVC]
  • htmlunit – higher level api on html level
  • Cactus – in container tests, good for functional testing
  • Selenium – can be used for acceptance testing by users, no coding required
  • Servletunit gives you a simulated container or can mock the request/response objects for simple cases

JSF

  • Jsf 2 project stages help for debugging and testing. This helps because you can easily specify different web.xml settings.
  • Selenium – for simple cases
  • jsfunit – uses cactus, junit, httpnit and htmlunit behind the scenes

Tip: define ids in your jsf tags so not randomly generated. That way you can refer to them

CDI/EJB 3.1

  • Arquillian – lets you say you do not want your whole aoo to be deployed. You use @RunAs(Arquillian.class). Then you can createDeployment() with the classes you want deployed. the ShrinkWrap API lets you add specific classes which lets you test one EJB or injected element.  The demo used embedded glassfish as specified by a Maven profile.  Also lets you inject into the unit test
  • Apace myfaces codi testing is a competing choice
  • For simple cases mock objects are sufficient, not testing production environment [that is why you test logic in unit tests and environment in integration tests which are in container]
  • @RunWith(ResinBeanContainerRunner) starts a very light container. Only supports CDI and EJB 3.1 lite

JPA

  • Embedded database – derby, hsql, java db
  • Embedded container like arquillian
  • Dbunit – helps with data consistency [and setup]
  • ejb3junit, mocks or pure junit again works for simple cases

Web Services

  • Soapui – good for complex cases with either SOAP or REST. Same level as Selenium
  • Can use junit and arquillian if want test to be self contained
  • Can use jersey test framework

Live from TSSJS – Web Frameworks with Matt

Posted on by
3

Live blogging from TheServerSide Java Symposium again, this time with Matt Raible, who is giving a presentation entitled “Comparing JVM Web Frameworks”.  He began his talk taking feedback from the audience on what frameworks people are using, with one humerous dialogue that proceeded as follows:

“How many people are using Struts 1?”.
One gentleman raises his hand.
“Poor bastard.  Are you using Struts with WebSphere?”
The gentleman nodded.
“Why haven’t you quit your job yet?”

1.  Web Frameworks – Too many?
In the non-Java market, jQuery has risen to the top but this hasn’t happened in the Java world.  Some people have criticized Java as having too many frameworks to choose from.

Matt is personally very much against Struts 1, as well as JSF.  He talks about how the founder of both, Craig McClanahan has essentially ditched Struts/JSF for Rails in around 2007.  He also quotes James Gosling as saying he hates JSF with a link to the YouTube video, although indicates he may have been referring to JSP.

2. Choosing a framework
Matt often talks about being asked which framework to use.  He has constructed a list of 20 criteria to determine which framework to use such as:

  1. Developer Productivity
  2. Developer Perception
  3. Learning Curve – Should be able to learn in under a week
  4. Project Health
  5. Developer Availability – Will you be able to find developers to hire who know the material?
  6. Job Trends
  7. Templating
  8. Components
  9. Ajax
  10. Plugins or Add-Ons
  11. Scalability
  12. Testing Support
  13. i18n and testing
  14. Validation
  15. Multi-language Support
  16. Quality of Documentation/Tutorials
  17. Books Published
  18. REST Support (client and server)
  19. Mobile / iPhone Support
  20. Degree of Risk

Matt reviewed a number of platforms and posted the resulting matrix online.  The top 5 ones based on these scores are Grails, Tapestry, Lift, Spring, and Struts2.  Often the 5, 6, and 7 providers change every few months.

3.  Business Decisions
Often companies hire consultants to reinforce web framework decisions that they have already made.  A lot of companies have a proprietary web framework that can be successful for their product.

4.  Controversary
Matt has been criticized for his scoring by members of the community.  He has posted a detailed explanation of how he calculated these ratings online.  As a well-read GWT developer, I disagree with some of his scores for GWT, especially templating which only scored a 0.5.

5. Matt’s favorite web frameworks

  1. Grails and Groovy:  Eeasy transition for Java developers.  It is also a gateway to Groovy.  He mentions some companies and developers resist Groovy because it is targeted only for Java developers and there isn’t a high availability of jobs.
  2. GWT:Great for Java developers who hate JavaScript.  The community is open and Google has done a good job growing a vibrant developer base.  The down sides, though, are that it is slow to test and requires Java knowledge to use.
  3. Ruby on Rails: Good for web developers, rather than hard-core Java developers.  The community is excellent.  The down sides is performance, testing, and type safety are limited.
  4. Spring MVC:  Easy configuration and usage
  5. VAADIN:  Excellent theme and layout support.  Uses the GWT API for developing view.
  6. WICKET:  Great for Java Developers.  Active community with creators providing a lot of support.  One downside is very little jobs available for WICKET developers.

Active Mailing Lists
One good measure of the health of a web framework is the number of developers asking questions and involved in the community, such as mailing lists and forum websites.  Some frameworks have hundreds of questions posted a month while others have 1-2 and are clearly not being used much.

Conclusion
Matt finished asking the question “What if there is no ‘best’ web framework?”, to which he spent time presenting competing argument about how people ‘shop’ for frameworks.  He compared shopping for frameworks with shopping for food and discussed an article with a researcher who did trending for how people choose food products.  “When we pursue universal principles in food, aka web services, we are doing ourselves a massive disservice”.  In other words, pick the frameworks that work for you, not neccesssarily the “best” framework.

Matt gave one of the best presentations I’ve attended at TheServerSide Java Symposium, and covered a large variety of web frameworks, while demonstrating expertise in each.  As a fellow consultant, I appreciate that he emphasized how quickly it is to pick up these frameworks as well as how easy it is to find a job with these skills.  At one point, he was one of only 3 active GWT developers in the Denver area and would get a call anytime there was a GWT job available.

Side note:  Matt recommends all Java developers learn about and know JRebel given its power, saying if you take away only one thing from this presentation it is that you should know JRebel.