I’m currently integrating JMS (Java Messaging Service) into a project. Being a true enterprise component it has a very complex nomenclature and consists itself of many modules to cover all kinds of topologies. To aid other newcomers here is an overview of what I’ve learned so far.

Introduction

The Java Messaging Service is an official part of Java EE and defined in JSR 914. In its own words it provides “loosely coupled, reliable, and asynchronous” messaging services for Java applications. Essentially this means that the service is fault-tolerant towards network and application failures and messages will reach their intended target eventually. However, in contrast to other means such as RPC it is not designed for time-critical communication.

Components

The major building blocks of JMS are brokers, transports, queues / topics, producers and consumers.

Consumers and producers

Producers and consumers, commonly referred to as clients, are producing and consuming messages respectively, just like the Consumer-Producer pattern. They are the main point where your application interacts with the JMS infrastructure by sending and receiving messages.

Brokers

Brokers are responsible for storing and forwarding messages to their intended targets, that’s why one common JMS topology is referred to as as store-and-forward network. What this means is that brokers, of which there can and most likely will be several in a given JMS topology, have an inbuilt persistence layer (store). This ensures that messages are not lost due to the application crashing. Furthermore, brokers can send messages to other brokers (forward).

Depending on the JMS implementation used brokers can usually be run in stand-alone more or embedded in your application

Transports

JMS provides, depending on the implementation used, many different means of transportation for the messages. Transports can vary from TCP sockets to HTTP-based to intra-VM communiation (thus avoiding network latency). The VM transport makes it possible for components inside a single VM to communicate with each other, therefore making it unnecessary to access objects directly.

Transports are used both for communication between clients and brokers, as well as between brokers.

Queues / Topics

When a consumer sends a message in a JMS network it usually does not have to know exactly what its intended target is. While the brokers make sure that the eventual target can receive the message, queues and topics are used to discern the recipient. With Queues a producer can send a message to an arbitrarily named queue. The consumer can then fetch messages from this queue, after which the message is removed from the queue (although there are exceptions to this rule). This is mostly equivalent to point-to-point messaging. With Topics a publish-subscribe mechanism can be realized. The consumer sends a message to a topic (publish) and all interested producers can receive this message (subscribe) thus making it a broadcast.

By default communication is uni-directional although bi-directional communication is possible. Several different message types are supported such as simple text messages, (serialized) object messages and byte streams.

Example

For my example application I implemented a system where several servers receive and process certain queries. The queries shall be logged on a central server. While there are many JMS implementations I have used Apache ActiveMQ due to the usual high quality and maturity of Apache projects and their open nature.

Topology

The topology is essentially comprised of one broker per query server (the producers), and one broker on the central server (the consumer). The brokers are embedded into the applications and the applications can run on different physical machines.

The clients use the VM transport to communicate with their local broker while the communication between the query brokers and log broker uses a TCP transport. This way no messages are lost in case of a network failure as the query brokers make sure to resume the connection to the log broker once the network connection is reestablished and resend any undelivered messages.

JMS example topology

Once the consumer has received the message it can act on it depending on the business requirements, for example log it into a database for further analysis.

For the working example see JmsTest.java. To make things easier both brokers in the example reside in the same VM but still use TCP for communication. With this setup hey could just as easily run on different physical machines.

Moving on

As mentioned above JMS is a highly sophisticated API which should fulfill most requirements at the price of a steep learning curve. It integrates well with frameworks such as Spring and can be configured using XML files. Further topics include features fail-over / high availability setups, hub-and-spoke topologies, JMX support and discovery networks. Furthermore, many different implementations are available, each of which certainly has its own perks and focus.

I just successfully installed CyanogenMod 7 on my Samsung Galaxy Ace. Finding all the right instructions was a bit tricky so here’s my personal recollection on how to get you up and running with CM7.

If you follow any of these steps you do so at your own risk. You might void your warranty, your phone could get irreversibly bricked.

That being said, apparently it worked for me, so just be careful and you should be fine.

Starting position

My Ace came installed with 2.2 Froyo. After finding out about a leaked 2.3 Gingerbread firmware I flashed that one (in the meantime it has officially been released by Samsung). You can find it at SamFirmware.com and flash it with the usual Samsung procedure of Odin (see SamMobile.com for more info). It worked fine for the most part but there is a well known problem with WLAN that causes random hang-ups. I could have tried the official 2.3 firmware which is supposed to partially fix the problem. Since I would have to flash anyway I decided to try out CyanogenMod 7 which had just been ported to the Ace and will become an official port. Part of the reason was of course that my brother had just installed it so I had to have it too

So, to start out you will be running some 2.3 firmware, otherwise your mileage may vary.

Rooting your Ace

The first step will be to root your Ace which is the prerequisite for all further parts.

I first tried SuperOneClick which is supposed to be an easy-to-use GUI program. I couldn’t get it to work in either Windows or Linux so I quickly dismissed that way. It turned out there were some root exploits just for Galaxy Aces running 2.3. All you need to do is upload a zip archive, reboot in recovery mode, apply that update and you’re done. It worked flawlessly for me. You can find all the info on how to root your Galaxy Ace at the xda-developers forum, and you just need to follow step D.

Installing ClockWork Recovery

Next up is installing ClockWork Recovery. ClockWorkMod (CWM) is an advanced recovery mode and ROM manager which allows you to easily pull backups of your current ROM and install new ones. Luckily it was just ported to the Ace. Why CyanogenMod cannot be installed with the stock recovery mode is beyond me but I’ll just go with it. It also seems like a good idea to get backup of your current ROM (do this!).

Same as the root exploit you just need to upload a zip archive to your sdcard, reboot, apply the update from the zip and you’re done. After this step you will have a new recovery mode so to check if CWM was successfully installed just reboot into the recovery mode and it should look different and have a lot more options available. You can get ClockWork Mod Recovery (CWM) at androidflip.com

Installing CyanogenMod 7

Finally, the moment you have been waiting for. The port for the Ace is currently in an unofficial state and deemed an RC1 – it’s working, the features are there, but there might still be some (smaller) bugs. It was originally posted to the xda-developers forum and the original poster keeps the thread in shape and collects all updates in his original post. So head on over to the CyanogendMod 7 for Samsung Galaxy Ace post for all the required files and instructions. Just be sure to grab the ROM for flashing from stock or other custom ROMs, not the one for updates. Again, upload the zip to your sdcard, reboot into recovery mode, apply the update and you’re done!

Epilogue

I haven been running CM7 for a few days now, and so far it has been rock solid – not a single crash and the uptime is approaching 2 days! I didn’t really notice much in the way of added features. My very subjective feeling is that the battery life is slightly shorter, and while the performance is mostly good sometimes it gets very sluggish, mostly after the phone has been idle for a while. Opening a menu might take a few seconds in that state but it will get better the longer you use it, all the way up to the menus opening instantly as is expected.

So, to summarize, theres’s no great advantage so far but no big disadvantage either. But seeing as the first release was less than two months ago, and Samsung is notoriously lazy about firmware updates, CyanogenMod for the Ace should hopefully get ahead of the stock firmware over time, fingers crossed. The more people use it, the better, so go ahead and flash the hell out of your Ace

I just released a small extension to integrate Zendesk with Thunderbird.

At a a project I am heavily relying on PostgreSQL in the unit tests. This works fine but creating and dropping databases takes quite a while which goes against the purpose of unit tests. One simple solution to this is to run PostgreSQL in a RAM disk which will make IO-heavy operations a snap.

Setup

The way I suggest works simply by

1. Creating a RAM disk
2. Copying the contents of the PostgreSQL directory to the RAM disk
3. Mounting the RAM disk over the PostgreSQL directory

This has the clear drawback that there is no write-back of any changed data. However, as I need the data only during testing this is acceptable.

Alternatives

Unfortunately PostgreSQL does not support RAM-based tables otherwise this would be a much cleaner solution. Another solution would be to create a new cluster directly in the RAM disk (using pg_createcluster). The drawback with this method is that the configuration and possibly already existing DB structures are not preserved.

Mount script

This was hacked together in a few minutes but “it works for me”™ using Ubuntu 10.10.

sudo /etc/init.d/postgresql stop
TMPDIR=/tmp/tmpfs; # the actual mount point for the tmpfs
MOUNTPOINT=/var/lib/postgresql/;
[ -d $TMPDIR ] || mkdir $TMPDIR;
sudo mount -t tmpfs -o size=512M,nr_inodes=10k,mode=0777 tmpfs $TMPDIR;
sudo rsync --archive $MOUNTPOINT/ $TMPDIR/;
sudo mount -o bind $TMPDIR $MOUNTPOINT;
sudo /etc/init.d/postgresql start

This is sort of a continuation of a previous post about how to integrate your own Java classes into the Maven build lifecycle. This entry described how to generate files for inclusion in the final project artifact (ie. .jar, .war etc.). It took me some time to figure out how to access these files using Java’s resource mechanism so this post will summarize my findings.

Resource locations

If you follow the Maven convention your source code is located in src/main/java/ and the source code for your tests in src/main/test/. Resource files will be in src/main/resources/ and src/test/resources/ respectively. In the final artifact the resouce files will be located in target/classes/ or target/test-classes/. Sub-directories will be preserved.

If you want to be able to access a file during runtime drop them in either of these resource folders depending on whether you want to access them during tests or in your actual application.

Packaging

Copying resources to target/ will be done automatically by Maven. After packaging to .jar etc. the contents of target/test-classes/ will not be included while the contents of target/classes/ will be found in the root directory of the artifact (for jar packaging) or in WEB-INF/classes/ (for war packaging)

Accessing resources

During runtime of the artifact you can access resources with

InputStream resource = SomeClass.class.getResourceAsStream("<file or directory name"))

or

URL resource = SomeClass.class.getResource("<file or directory name"))

where SomeClass is a class in the artifact. The first way will give you an InputStream that you can use to read the contents of the file, the second way will return a URL that you can use to find out the path of the file or directory or move along the directory tree.

Accessing a resource will be delegated to the classloader of the class you use to load the resource. So when SomeClass is a class in a dependency of your project the resource will be loaded from that dependency’s artifact.

Resource paths

Resources have a notion of absolute and relative paths. The resource /filename will be resolved to target/classes/filename whereas filename will be resolved using the package of the class ie. with org.example.SomeClass it will be resolved to target/classes/org/example/filename. Again, for test classes the base resource directory will be target/test-classes/.

Note that you do not have to give a file name at all. / for example will be resolved to just target/classes/.

I love Maven for its flexibility, but this flexibility at times makes it hard to figure out how to achieve certain tasks. In my case I wanted to execute a certain Java class that is included in the project at some point during the build lifecycle. While I can see many uses in my case the class itself compiles a SQL dump out of several source files so it can be included in the final artifact.

Integrating with Maven’s build lifecycle

Long story short, here is what you need to add to your pom.xml:

<build>
  <plugins>
    <plugin>
      <groupId>org.codehaus.mojo</groupId>
      <artifactId>exec-maven-plugin</artifactId>
      <version>1.2</version>
      <executions>
        <execution>
          <id>build-dump</id>
          <phase>process-classes</phase>
          <goals>
            <goal>java</goal>
          </goals>
        </execution>
      </executions>
      <configuration>
        <mainClass>com.example.BuildDump</mainClass>
      </configuration>
    </plugin>
  </plugins>
</build>
 <pluginManagement>

The org.codehaus.mojo.exec-maven-plugin is used to execute the class com.example.BuildDump. The execution is done in the process-classes lifecycle using the java goal of the plugin. The id is arbitrary but has to be unique.

The whole idea of this snippet is to hook certain plugin’s goal into maven’s lifecycle at a certain phase. In that sense, the same effect can be achieved by calling (on the command line)

mvn exec:java -DmainClass=com.example.BuildDump

As you can see the goal in the execution of the pom.xml corresponds to the part after the colon, the part before the colon references the plugin (plugin:goal). If I recall correctly org.codehaus.mojo.*-maven-plugins have a shorthand name which is the part before -maven-plugin.

The execution doesn’t have to take place in the process-classes phase but it has to take place after the compile phase so the class has already been compiled. It also has to take place before the test phase if you need the files during testing. I do not know if the process-classes phase is correct (it is used to post-process the generated files from compilation, for example to do bytecode enhancement on Java classes.) but it fulfills these two requirements.

I am currently working on a Spring-based Neo4j (a graph database) application and needed to add user authentication. Since Spring Security allows integrating custom authentication providers it felt only natural to implement a provider based on Neo4j. A data structure to support users and groups is already provided in the Neo4j Wiki. I used it as the basis for my implementation with some wrapping so it could be integrated with Spring Security and provides administrative methods.

When you take a look at the data structure you will see how suitable a graph is to store user data. It allows you to easily create a hierarchical group structure and in order to aggregate all of the user’s groups just need to traverse (find one or more paths) from the user node to the start node.

Implementation

Only three pieces are really needed once you have the required components such as Neo4j and Spring Security included in your project:

1. Define an interface that defines ways to manipulate the users and groups: AuthenticationService.java (actually I needed this only because of some JDK proxying issues, but it never hurts to have an interface)
2. An implementation of the user service interface for Neo4j: NeoBasedUserService.java
3. Integration of the authentication provider in your context: app-security.xml

NB: The “GrantedAuthority” objects in the Spring nomenclature are implemented here simply as groups.

The interface and implementation provide the bare minimum of what I needed for the application. Still, the interface is well defined and reasonably well tested, so don’t be afraid to give it a try.

For some reason the Nagios package in Debian / Ubuntu does not provide a plugin to check the memory consumption so I built a simple script to do that. It ignores swap space (since if your machine starts swapping you’re lost anyway) and caches (it’s nice to have memory available for caches but this isn’t really required). That way only the real memory consumption is checked and if that passes a certain threshold you can rest assured that you’re in for trouble.

Get the file here: check_memory (it’s got an .txt ending due to WordPress restrictions; rename to .sh or just remove the ending altogether).

The default warning and critical thresholds are triggered when more than 90% / 95% of the memory are in use. You can change these thresholds by passing in the well known -w and -c arguments.

The plugin was developed in cooperation with FINDOLOGIC GmbH.

Spring 3 allows for wonderfully simply forms and automatic binding of form values back to the command object. However if your command object has further nested objects that you want to make editable from the form as well things get a little tricky. In that case you can’t just use a standard JSTL forEach loop as this will not allow you to build the path to the nested object correctly. The trick is to use the varStatus attribute to keep track of the index.

Sets and Lists

Assuming that your command object is called target the following will do just fine:

<ul>
  <c:forEach items="${target.nestedObjects}" varStatus="index">
    <li>
      <form:input path="nestedObjects[${index.count - 1}].property" />
      <form:errors path="nestedObjects[${index.count - 1}].property" cssClass="errors" />
    </li>
  </c:forEach>
</ul>

Keep in mind that the varStatus counter is one-based while most collections’ indices are zero-based; hence the “itemIndex.count – 1″ expression.

Another way is to use the <spring:bind> tag. This will allow you to use standard HTML-Tags at the expense of some Spring form tricks such as checkboxes that are always submitted.

<ul>
  <c:forEach items="${target.nestedObjects}" varStatus="index">
    <li>
      <spring:bind path="nestedObjects[${index.count - 1}].property" />
      <input type="text" name="${status.expression}" value="${status.value}" />
      <p class="errors">${status.errorMessages}</p>
    </li>
  </c:forEach>
</ul>

Of course you can also mix and match both approaches as you see fit.

Maps

To iterate over a map the approach is slightly different:

<ul>
  <c:forEach items="${target.nestedMap}" varStatus="index">
    <li>
      <form:input path="nestedObjects[${index.current.key}].property" />
      <form:errors path="nestedObjects[${index.current.key}].property" cssClass="errors" />
    </li>
  </c:forEach>
</ul>

As you can see the varStatus variable will be set to each respective map entry as they are iterated. Therefore “index.current.value” is also available if needed.

Thanks to Spring’s new MVC Annotations (@Controller, @RequestMapping, @RequestParam etc.) unit testing a Spring controller is now easier than ever before. Still, I had a hard time figuring out how to do this since most of your controller methods will not take a ServletRequest as input. Instead they use a form-backing / command object, a BindingResult and maybe a ModelMap. In a web environment Spring takes care of filling the command object and validation results however in a unit test you will have to do this yourself.

Following is the basic outline for writing such a test.

This StackOverflow post got me started originally.

Configuration

Make sure there is a validator configured somewhere in your test context, the corresponding bean could look like this:

<bean id="validator" class="org.springframework.validation.beanvalidation.LocalValidatorFactoryBean" />

In order for your controller to be picked up and injected into your test you will also need this in your config:

<context:component-scan base-package="the.package.your.controllers.are.in" />

I am using the Hibernate validators which provide a reference implementation of JSR-303 (Bean Validation) so you will need these dependencies in your pom.xml (if you use Maven):

<dependency>
  <groupId>javax.validation</groupId>
  <artifactId>validation-api</artifactId>
  <version>1.0.0.GA</version>
  <type>jar</type>
  <scope>compile</scope>
</dependency>
<dependency>
  <groupId>org.hibernate</groupId>
  <artifactId>hibernate-validator</artifactId>
  <version>4.0.2.GA</version>
</dependency>
<dependency>
  <groupId>org.springframework</groupId>
  <artifactId>spring-context</artifactId>
  <version>3.0.0.RELEASE</version>
</dependency>

Test

Let’s get down to business, here is what your unit test could look like:

public class ControllerTest {

  @Autowired
  private org.springframework.validation.Validator validator;
  @Autowired
  private YourController yourController;

  @Test
  public void testController() {

    MockHttpServletRequest request = new MockHttpServletRequest("POST", "/edit");

    // populate the request, use the same names as you would for the form elements in the corresponding .jsp
    request.setParameter("text", "Some text");

    // create a new instance of your form object
    YourFormObject formObject = new YourFormObject();

    WebDataBinder binder = new WebDataBinder(formObject);
    binder.setValidator(validator); // use the validator from the context
    binder.bind(new MutablePropertyValues(request.getParameterMap()));

    // validation must be triggered manually
    binder.getValidator().validate(binder.getTarget(), binder.getBindingResult());

    // if you want to test if the validation works as expected you can check the binding result
    // eg. if no errors are expected:
    assertEquals(0, binder.getBindingResult().getErrorCount());

    // now call your controller method and store the return type if it matters
    // f. ex. a different view might be returned depending on whether there are validation errros
    String view = yourController.edit(binder.getTarget(), binder.getBindingResult());

    assertEquals("error", view);

    // now you can test the target object, f. ex.:
    assertEquals("Some text", binder.getTarget().getText());
  }
}

Conclusion

Spring rocks

As you can see there is a lot of boilerplate code (creating the binder, binding, validation) that could easily be moved to a private method. More importantly there are two things which I have not really figured out yet:

1. I need to explicitly call a method on the controller instead of having it figure out by itself which method to use depending on the request method and URL. Of course this would make building and passing in the command object impossible so I guess a viable option is to find which Spring component is responsible for mapping URLs to controller methods and test these mappings explicitly.
2. An @ExceptionHandler has to be called manually. This stems from the fact that if the controller method throws an exception it is passed on to the test which will then fail if it does not catch the exception. Therefore the exception never reaches the exception handler. If you think about it that makes sense – the automatic calling of the @ExceptionHandler would be useful in an integration test. However in a unit test catching the (expected) exception from the controller method is sufficient as you only want to test that unit.