A Brief History of Apache Karaf

Everyone once in a while a question comes up in discussion of Apache Karaf regarding its origins. Well to help answer this question I've written up this brief history of Karaf so that I may reference it.

In the beginning...

Apache Karaf can trace it's origins back to the Apache Servicemix project's Kernel. As early as 2008 the first blog posts regarding its development can be observed. The first GA release was on September 18th, 2008. The original release notes can be viewed here. Subsequent releases soon followed, introducing new features.

The Servicemix Kernel was off to a great start, eventually however by spring 2009 the project grew to the point that it was thought best to leave Servicemix and join it to the Apache Felix community. It was felt that this would allow awareness to grow about the project, and to expand the Kernel community. At this time the Servicemix Kernel project was renamed to "Karaf". For those of you interested about how the project became called 'Karaf' here is the relevant excerpt from the discussion:

"A carafe is a small container used for serving wine and other drinks (http://en.wikipedia.org/wiki/Carafe). In similarity to the name the Kernel allows applications to be more easily handled, and improves their characteristics (much like a bottle of wine left to breath in a decanter) :)"

So why the respelling? It was felt that 'Karaf' would be an easier search target, and it would fill the relatively empty 'K' project name listings at Apache ;)

Once established in the Apache Felix community Karaf again started to grow and mature, once again coming to the point that it was felt that another growth spurt was required - becoming a top level Apache project. This occurred mid 2010. Since then Apache Karaf has released it's first GA as 2.0, and several point releases and patches. The community is now busy readying the next major point release, and preparing for its future 3.0.

Karaf is now a top level Apache project!

It was barely a little more than a year ago that I was announcing that the Servicemix Kernel sub-project had moved to Felix and renamed to "Karaf". Time has moved on and development has been proceeding at an amazing pace, this growth has been so pronounced that we have requested to have Karaf recognized as its own top level project - this request was successfully granted :)

Now that the process has begun to establish Apache Karaf as a top level project, I thought it would be good to start publishing a few links so that Karaf developers & users could find its new home.

Karaf home page:
http://karaf.apache.org/

Karaf issue tracker:
https://issues.apache.org/jira/browse/KARAF

Karaf source code (anonymous check out from Karaf trunk):
http://svn.apache.org/repos/asf/karaf/trunk/

Karaf hudson build status:
http://hudson.zones.apache.org/hudson/job/Karaf/

Karaf mailing lists:
http://karaf.apache.org/mailing-lists.html

Karaf contributing guide:
http://karaf.apache.org/contributing.html

As a small personal note, with the establishment of Karaf as a top level project I have been included as a committer and the Karaf PMC :) 

I'm an Apache Felix committer!

 This morning I received an invitation to join the Apache Felix project as a committer, which I happily accepted! :)

I've mostly been involved with the Felix Karaf subproject since it was donated from the ServiceMix project in April of 2009. Over the last year I've had the pleasure of getting to work closely with members of the Felix community, and hope to only enjoy more fun times going forward :)

Cheers!

Setting up SMX4 failover with Oracle

In the tradition of posting articles regarding Servicemix 4 failover here is yet another blog entry. The functionality described below is currently available as a patch to Karaf, a future release should include this feature.

The following system.properties configuration will allow your Karaf based SMX4 instance to use an Oracle JDBC lock:

karaf.lock=true
karaf.lock.class=org.apache.felix.karaf.main.OracleJDBCLock
karaf.lock.jdbc.url=jdbc:oracle:thin:@hostname:1521:XE
karaf.lock.jdbc.driver=oracle.jdbc.OracleDriver
karaf.lock.jdbc.user=user
karaf.lock.jdbc.password=password
karaf.lock.jdbc.table=KARAF_LOCK
karaf.lock.jdbc.clustername=karaf
karaf.lock.jdbc.timeout=30

Please note that the Oracle JDBC driver will have to be added to your classpath, this can be easily done by coping the Oracle JDBC jar into the SMX/lib folder. Also note that the karaf.lock.jdbc.url requires an active SID, this means you must manually create a database instance before using this particular lock.

Setting up SMX4 failover with MySQL

This post is an update to my previous post on setting up SMX4 with high availability in mind.

Since the transition to Apache Felix Karaf the support for more database locking table back ends has expanded. Specifically you can now use Apache Derby, MySQL, or a generic JDBC connector. The Derby and Generic JDBC locks make use of the 'FOR UPDATE' feature common to many databases. The MySQL lock however makes use of the ' LOCK TABLES' directive, this is due to the 'FOR UPDATE' command not returning a ResultSet on more recent versions of MySQL (see some discussions on this here).

The following system.properties configuration will allow your Karaf based SMX4 instance to use a MySQL JDBC lock:

karaf.lock=true
karaf.lock.level=20
karaf.lock.delay=1000
karaf.lock.class=org.apache.felix.karaf.main.MySQLJDBCLock
karaf.lock.jdbc.url=jdbc:mysql://address:port/dbname
karaf.lock.jdbc.driver=com.mysql.jdbc.Driver
karaf.lock.jdbc.user=root
karaf.lock.jdbc.password=karaf
karaf.lock.jdbc.table=KARAF_LOCK
karaf.lock.jdbc.clustername=mycluster
karaf.lock.jdbc.timeout=30

Please note that the MySQL JDBC driver will have to be made available on the classpath.

Finding Bugs with FindBugs

FindBugs is a static analysis tool developed and maintained by the University of Maryland. The tool can find bugs in Java code, and present a listing of issues detected to developers. The project maintains a list of bug patterns they detect when performing an analysis of a code base here.  

As an experiment to begin learning about FindBugs, I ran the tool against a recent build of Apache Felix Karaf. In figure 1 you can see the listing of issues flagged.

Figure 1: FindBugs analysis screen.

Reviewing the flagged issues, not all of them are bugs but suggestions on performance or alternative implementation practice. The remainder of issues are actual problems, luckily many of these can often be remedied in a few minutes. One such issue was a minor file descriptor leak discovered in Karaf's Main class. Resolving the issue only required closing an IO stream after use. After modifying the code, and testing to ensure nothing accidentally broke, I re-ran FindBugs; the issue list was reduced by one :) Having made a small improvement to the code base I took the time to open a minor issue under Felix Karaf issue tracker and submitted a patch. 

Using tools such as FindBugs is only one part of the process of developing software, many issues will not be caught with out proper unit and system testing, and feed back from users in the field. As a second set of eyes to help catch programming omissions and in maintaining good coding practices I feel that the time spent with FindBugs is well worth the effort.

Getting to know VisualVM

Recently I had to refactor a feature that I had introduced to the old Servicemix Kernel, now Felix Karaf project. The issue was relatively minor to resolve, however testing it would require profiling memory and CPU usage over a period of time (to avoid an eventual out of memory error).

There are multiple tools out there that would allow me to do this sort of monitoring, for this particular task I chose to try out VisualVM as it was already bundled with my JDK 1.6 install. VisualVM is a visual tool that integrates several commandline JDK tools and lightweight profiling capabilities for the Java SE platform.

To begin my refactoring task I first setup Karaf with the configuration required to reproduce the out of memory condition. No special configuration or instrumentation of Karaf itself was required for using the profiler. Starting Karaf, using JDK 1.6, I was able to attach to the running process via the VisualVM interface. Once connected to the process I could view the live heap usage, threading, permgen, and loaded classes, further I could inspect the parameters fed to the JVM at start up to confirm the environment was as I configured. To view memory and CPU usage of Karaf, I selected the profiler display tab. Adjusting the profiler settings to only monitor org.apache.felix classes, I could quickly identify the offending set of method calls that contained the source of my error (high CPU usage, and multiple instances accumulating in memory).

VisualVM displaying monitor view of Felix Karaf

Resolving the error didn't require much work except to ensure a few resources were being properly freed, and removal of an unneeded for-loop. To test out my modifications I redeployed Karaf, and began reviewing its run time characteristics via VisualVM. This time the trouble methods were not being called as often, and not creating memory leaks, while still providing the same functionality the original code was intended to provide.

As a simple profiler I can recommend VisualVM as a quick, convenient way to observe your Java applications. It will be interesting to see what features they integrate into the tool in future releases.