Tuesday, November 4, 2014

Netty decoder for the collectd binary protocol

EDIT: February, 12 2015

The netty-collectd repository has been deprecated and the code is now maintained in the Hawkular metrics repository.

The first snapshot of netty-collectd, our Netty decoder for the collectd binary protocol, has been pushed to the JBoss.org Maven repository!

What is it?

netty-collectd decodes collectd UDP datagrams. The decoding process results in the creation of so-called Event object instances.

Each Event instance wraps:
  • the sender host (see Hostname in the collectd.conf Global Options)
  • a metric timestamp (when it was collected)
  • the plugin data (plugin and plugin instance names)
  • the type data (type and instance names)
  • the values
  • the interval (how often this metric is collected)

Why was it started?

The RHQ project team is working on turning some of  RHQ's core features into re-usable services. One of them is the RHQ Metrics project, a time-series database with charting.

While RHQ was historically dependent on its agent plugins to gather metrics data, we'd like RHQ Metrics to support some popular collectors/transmitters as well.

Its is planned to re-use netty-collectd in ptrans, our protocol translator (from collect/statsd/... etc to RHQ-Metrics).

How to use it?

The snapshot binaries are hosted in the JBoss Snapshots repository. Refer to the JBoss.org Maven repository guide for setup instructions. Releases will be pushed to Central.

First, add the dependency to your project. For Maven projects:

Then configure your Netty server :

Your handler will look like:


Development is in its early stage, so we're still missing important features, a good test suite... and community feedback!


The decoder is not able to read type files yet. Consequently:
  • sample types cannot be verified
  • it’s not possible to give the sample’s name


The decoder does not yet understand notification messages. They are ignored.

Signed or encrypted traffic

The decoder does not yet support signed or encrypted data.


Like the idea and want to give a hand?
Join us in #rhq.next on Freenode
Twitter: https://twitter.com/RHQ_Project
Forum: https://developer.jboss.org/en/rhq/rhq-metrics

Tuesday, October 28, 2014

Maven plugin for RHQ Agent Plugins goes 1.0

The Maven plugin for building RHQ Agent plugins is now at version 1.0.

The bits are already available on JBoss public repositories and on Maven Central.

The version bump is mostly an indication that it is now considered stable.

The only feature added is an enhancement to the remote deployment mojo: you can now trigger a plugin deployment on all the agents (works with RHQ 4.12 servers and above).

This allows to easily apply continuous integration and continuous delivery principles to the development of RHQ Agent plugins.

Feel free to reach us on IRC in #rhq room @ freenode if you have any questions, or post a comment here.


Thursday, June 5, 2014

RHQ: monitoring JMX beans of a Java EE application deployed on Wildfly 8

You may already have read my post on monitoring JMX beans of a Java EE application deployed on EAP 6 ? And maybe wondered how to do the same with Wildfly 8?

Well it's possible now with a master branch build of RHQ (the feature will be released with RHQ 4.12). A couple of problems had to be addressed.

Firstly, we needed proper support for Wildfly monitoring. RHQ was able to discover Wildfly servers as such, but failed to monitor them as soon as they were added to inventory. This was due to a wrong product name check in AS7 / Wildfly8 plugin.

Secondly, the ApplicationMBeansDiscoveryComponent class needed  an update to take Wildfly management and port changes into account:
  • single port for management and remoting on a standalone server (HTTP upgrade)
  • single port for applications and remoting on a managed server in domain mode (HTTP upgrade again)
  • new JMX service transport ("http-remoting-jmx")
Also, the Wildfly version of jboss-client.jar (required to support "http-remoting-jmx" transport) has class loading issues with RHQ's AS7 / Wildfly8 plugin. So we had to update our EMS[1] library connection descriptor to isolate the connection classes loader.

The new feature has been tested with Widfly 8.1.0.CR2 in standalone and domain mode but since WildFly 8.1.0.Final was released recently, we'd be glad if you give it a try and come with feedback on the RHQ forum!

[1] RHQ uses the EMS library to simplify its JMX connectivity code.

Friday, May 16, 2014

A new version of rhq-agent-plugin-plugin is out

I just released a new version of the Maven plugin for building RHQ Agent plugins.

v0.6 is a small release which:
  • fixes a bug in the validation mojo (JAR without plugin descriptor used to pass validation)
  • lets you use the failOnError plugin configuration attribute with the RHQ CLI Command and RHQ CLI Script mojos (before RHQ 4.11 the CLI returned exit code 0 even after an error occured)
The bits are already available on JBoss public repositories and should be very soon on Maven Central.

Special thanks to Blandine Bourgeois, Victor Grousset and Eliesse Hadjem for their contributions (Marseille JUG Hackergarten power!)


Tuesday, March 18, 2014

RHQ: monitoring JMX beans of a Java EE application deployed on EAP 6

 Everything in this article applies to both EAP 6 and AS 7

On RHQ forum, we have been asked quite a few times how to monitor MBeans of a Java EE application deployed on EAP 6.

Users are often confused because on one hand, all the goodies from the JMX plugin need a JMX connection while, on the other hand, EAP 6 plugin communicates with managed servers over an HTTP management connection.

JMX connection to EAP 6

A JMX connector is accessible over the management-native port (9999 by default) on standalone servers and over the remoting port (4447 + server port offset) on managed servers (domain mode).

A custom protocol is used ("remoting-jmx") so the JMX service url will start with "service:jmx:remoting-jmx://" and EAP6_HOME/bin/client/jboss-client.jar has to be in the client code's classpath.

For authentication, standalone servers rely on the ManagementRealm, and Managed Servers on the ApplicationRealm.

If you correctly configured your server, you should be able to see your MBeans in JConsole at this point (see Using jconsole to connect to JMX on AS7).

Custom plugin requirements

So we want RHQ Agent to auto-discover our application MBeans and the corresponding resources to show up under the parent standalone server or the parent managed server.

For that, we need to create a custom plugin extending the EAP 6 plugin (see Plugin Injection Extension model). The plugin descriptor will contain a line like:

We also need component and discovery component classes from the JMX plugin, but  RHQ prevents plugin developers to use classes from more than one dependency. So there's no other choice here, the custom plugin will have to embed the JMX plugin JAR.

Eventually, we must create a component class (and its discovery counterpart) that will group our MBean resources and provide them with a JMX Connection.

AS7 JMX Util project

In RHQ's repository, the AS7 JMX Util project gives the root component classes for grouping MBeans resources: ApplicationMBeansDiscoveryComponent and ApplicationMBeansComponent. Here is how they work.

ApplicationMBeansDiscoveryComponent will be the discovery class of the custom "grouping" resource type. Plugins authors are required to provide a resource key and name and may provide a resource description and version by declaring plugin configuration properties of the following names:
  • ApplicationMBeansDiscoveryComponent.PluginConfigProps.NEW_RESOURCE_KEY
  • ApplicationMBeansDiscoveryComponent.PluginConfigProps.NEW_RESOURCE_NAME
  • ApplicationMBeansDiscoveryComponent.PluginConfigProps.NEW_RESOURCE_DESCRIPTION
  • ApplicationMBeansDiscoveryComponent.PluginConfigProps.NEW_RESOURCE_VERSION
Alternatively, they can create a subclass of ApplicationMBeansDiscoveryComponent and override one/some/all of the following methods:
  • getNewResourceKey(Configuration)
  • getNewResourceName(Configuration)
  • getNewResourceDescription(Configuration)
  • getNewResourceVersion(Configuration)
By default, application MBeans will be searched with an EMS query defined in the plugin descriptor by the ApplicationMBeansDiscoveryComponent.PluginConfigProps.BEANS_QUERY_STRING plugin-config property. It is also possible to define the query string in a subclass overriding the getBeansQueryString(Configuration) method.

Plugin developers can implement their custom MBeans lookup method in a subclass overriding the hasApplicationMBeans(Configuration, EmsConnection) method.

JMX host and port discovery

The JMX server host will be detected by looking at the top level server resource plugin configuration (StandaloneASComponent or HostControllerComponent). The JMX port detection mechanism depends on the parent resource:
  • on standalone servers, the discovery component will look at the management port defined in the parent StandaloneASComponent plugin configuration and add the value STANDALONE_REMOTING_PORT_OFFSET
  • on managed servers, the discovery component will use '4447' plus the port offset of the managed server.


JMX connectivity on EAP6 requires authentication and standalone and managed servers behaviors differ:
  • on standalone servers, the EAP 6 server will use the ManagementRealm, just as for HTTP management interface authentication. Consequently, the ApplicationMBeansDiscoveryComponent will pick up the credentials of the management user defined in the plugin configuration of the parent server resource (StandaloneASComponent)
  •  on managed servers, the EAP 6 server will use the ApplicationRealm. Consequently, there is (currently) no way for the discovery component to discover credentials automatically and it will use "rhqadmin:rhqadmin" by default.
When working with managed servers, plugin developers should subclass the discovery component and override the getCredentialsForManagedAS() method. For example, an implementation could lookup the credentials in a text file.

The custom agent plugin

The custom agent plugin must:
  • contain a plugin descriptor following the requirements described above
  • embed the JMX plugin JAR
  • embed the AS7 JMX Util project JAR
  • optionally, contain a subclass of ApplicationMBeansDiscoveryComponent 
This is very straightforward when the custom plugin is built with the Maven plugin for RHQ agent plugins: you need a POM file and a plugin descriptor (there is a sample project in RHQ repository). Let me show you what the POM file and the plugin descriptor look like:

That's it! Here are some screenshots of an example application in inventory:

"Myapp Services" grouping resource plugin configuration
"Myapp Services" grouping resource plugin configuration

Creating an "Hello Service" operation, on a managed server (domain mode)
Creating an "Hello Service" operation, on a managed server (domain mode)

Enjoy and let us now if you like it or need any help (as always, on RHQ forum or the users mailing list).

Monday, March 17, 2014

Announcing Simone project, a Java library for system monitoring

In English, pronounce sə-mōn'
Today, I pushed to Github the initial commit for Simone, a Java library for system monitoring, with no JNI code. Native invocation, when needed, is provided by JNR and friends (JNR Posix, … etc)

This work is experimental.

Currently, you can get information on CPUs, Memory and Processes on Linux systems. Next step is filesystems and networks statistics. Windows and Mac OS X are planned as target platforms.

Give it a try!

It's not yet available in any Maven repository so you’ll have to build it yourself.

Any issues? Questions? Feel free to report with GitHub issues.

EDIT: I forgot to point to the Simone project on GitHub

Wednesday, December 4, 2013

Announcing RHQ Agent Plugin Plugin

I'm happy to announce the release of RHQ Agent Plugin Plugin!

Plugin Plugin? Yes. Plugin Plugin: a Maven plugin to help you build RHQ Agent plugins.

It provides a handful of very convenient mojos for:
  • Packaging
  • Validation
  • Deployment to a local RHQ Server (dev-container)
  • Uploading to to a remote RHQ Server
  • RHQ CLI script or command execution
  • Standalone Plugin Container setup for integration tests

Dependent agent plugins are supported (many plugins depend on the parent JMX plugin, for example).

The RHQ Agent Plugin Plugin documentation is hosted on Github Pages and the bits are available on JBoss Releases and Maven Central repositories.

Need help to get started?