JMX java.lang naming between Windows and Linux (and a bit of OpenNMS)

I’ve been banging my head against something here at work, so thought I’d post the solution up on the Interwebs to hopefully prevent cranial trauma for others.

First, if you don’t know what JMX is, you can read about it on the lovely Wikipedias here: JMX. Basically, it’s a technology that lets you monitor the internals of a running Java process, which is very useful for troubleshooting.

I setup JMX monitoring on the Java app and in OpenNMS, which was pretty straight forward, then sat back and waited to see what data was collected. And that’s when the headaches started, because different data was collected from the Java processes running on Linux than on Windows. Specifically, OpenNMS was not collecting any data from the Linux boxes about different types of memory usage in the Java process, which is one of the key things I wanted to get.

After a bunch of poking around, I found the lovely command line jmxsh utility, which allowed me to do some quick lists of JMX mbeans exposed on each machine. And it showed me this…

Linux - Sun java 1.6.0_24 running as Tomcat 7 using normal startup script

       1. java.lang:type=Memory
       2. java.lang:type=MemoryPool,name=PS Eden Space
       3. java.lang:type=MemoryPool,name=PS Survivor Space
       4. java.lang:type=MemoryPool,name=Code Cache
       5. java.lang:type=GarbageCollector,name=PS MarkSweep
       6. java.lang:type=Runtime
       7. java.lang:type=ClassLoading
       8. java.lang:type=Threading
       9. java.lang:type=Compilation
      10. java.lang:type=MemoryPool,name=PS Perm Gen
      11. java.lang:type=GarbageCollector,name=PS Scavenge
      12. java.lang:type=OperatingSystem
      13. java.lang:type=MemoryPool,name=PS Old Gen
      14. java.lang:type=MemoryManager,name=CodeCacheManager

Windows - Sun java 1.6.0_25 running as Tomcat 7 service using jvm.dll

       1. java.lang:type=Memory
       2. java.lang:type=GarbageCollector,name=Copy
       3. java.lang:type=MemoryPool,name=Code Cache
       4. java.lang:type=Runtime
       5. java.lang:type=ClassLoading
       6. java.lang:type=MemoryPool,name=Perm Gen [shared-rw]
       7. java.lang:type=Threading
       8. java.lang:type=MemoryPool,name=Perm Gen [shared-ro]
       9. java.lang:type=Compilation
      10. java.lang:type=MemoryPool,name=Eden Space
      11. java.lang:type=MemoryPool,name=Survivor Space
      12. java.lang:type=GarbageCollector,name=MarkSweepCompact
      13. java.lang:type=OperatingSystem
      14. java.lang:type=MemoryPool,name=Tenured Gen
      15. java.lang:type=MemoryPool,name=Perm Gen
      16. java.lang:type=MemoryManager,name=CodeCacheManager

As you may notice, there’s a PS prepended to a bunch of those names on Linux but not on Windows, and the OpenNMS JMX config (jmx-datacollection-config.xml) didn’t have the PS entries for MemoryPool items. To fix the problem I edited the config and duplicated the affected entries, and updating only the name and objectname to add the PS but not changing the rest of the definitions. In this way (I hope) it will still see separate entries in the JMX config, but collect them into the same back-end databases for display.

A quick grep for Memory in my jmx-datacollection-config.xml now shows:

[...]
      <mbean name="JVM MemoryPool:Eden Space" objectname="java.lang:type=MemoryPool,name=Eden Space">
      <mbean name="JVM MemoryPool:PS Eden Space" objectname="java.lang:type=MemoryPool,name=PS Eden Space">
      <mbean name="JVM MemoryPool:Survivor Space" objectname="java.lang:type=MemoryPool,name=Survivor Space">
      <mbean name="JVM MemoryPool:PS Survivor Space" objectname="java.lang:type=MemoryPool,name=PS Survivor Space">
      <mbean name="JVM MemoryPool:Perm Gen" objectname="java.lang:type=MemoryPool,name=Perm Gen">
      <mbean name="JVM MemoryPool:PS Perm Gen" objectname="java.lang:type=MemoryPool,name=PS Perm Gen">
      <mbean name="JVM MemoryPool:Old Gen" objectname="java.lang:type=MemoryPool,name=Old Gen">
      <mbean name="JVM MemoryPool:PS Old Gen" objectname="java.lang:type=MemoryPool,name=PS Old Gen">

This seems to be collecting the expected memory stats regardless of platform. Yay!