I've been experimenting with the Garbage Collection (GC) policies in IBM Java 6 on AIX 7.1 while using Apache Karaf 2.2.5, and thought I'd share some tips and links for anyone else following in my footsteps.
If you're wondering why one would play with the GC policy of the underlying jvm Karaf runs upon it stems from wanting to get a better idea of the throughput of applications and the overall system, and the pause times that are caused by garbage collection (wondering what "pause time" is? It's the time spent by the jvm cleaning up, or freeing, heap memory when an object can't be created using the available heap memory).
The version of Java I was testing has three options for garbage collection; gencon, optthruput, and subpool. Gencon is a combination of concurrent and generational GC, it's goal is to minimize the amount of time spent in any GC pause. Optthruput is the default GC policy, and is designed to deliver high through put at the cost of occasional GC pauses. Subpool is aimed at large SMP systems, and attempts to deliver better performance on those systems. Unfortunately the system I have access too is too small to try out subpool with any meaningful results, as such I kept to gencon and optthruput policies.
To setup Karaf to use specified garbage collection policies edit the KARAF_HOME/bin/karaf file as follows:
# grep for the setup defaults routine:
setupDefaults(){
# in the aix section try adding the highlighted text:
DEFAULT_JAVA_OPTS="-verbose:gc
-Xverbosegclog:$KARAF_DATA/gc-log.txt
-Xgcpolicy:gencon
-Xverify:none -Xlp $DEFAULT_JAVA_OPTS"
When you start Karaf using the above Java options you will enable verbose garbage collection logging, sending generated data to KARAF_DATA/gc-log.txt. You can use this log file to review GC statistics as you tweak jvm options. The -Xgcpolicy flag is where you select the GC policy. When you change this value do not be surprised if you do not see this changed reflected in the Karaf info command's memory section. Use the generated verbose GC logs to perform your analysis.
Trying out gencon and the default optthruput policies I can see very different runtime behaviors in regards to frequency of GC and durations. Gencon ran more often, but for short periods of time, while optthruput ran infrequently but took longer when called. Given different application loads the characterization of GC runs could become important in providing optimal performance, the determination of "optimal" however is very subjective and will vary depending on the goals of your deployment.
Of course, changing the GC policy isn't the end of the story. There are a lot of parameters that can be tweaked which affect the size of the jvm heap in use, and the thresholds different GC policies use. I would recommend when testing to tweak one value at a time, then assessing it's impact. The online documentation (see below) will steer you in the right direction.
IBM JDK 6 Garbage Collection Links:
Overview of Java 6 Memory Management. Read this to learn about the garbage collector and allocator.
Detailed description of garbage collection. Dive deep into garbage collection.
Garbage Collector diagnostics. How to diagnose garbage collection. A must read.
Specifying garbage collection policy. Part of the running Java applications section in the IBM User Guide for Java v6 on 64-bit AIX.
Garbage Collector command-line options. List of all the garbage collector options (here there be dragons, careful tweaking these values).
Verbose garbage collection logging. Examples of how to analyze garbage collection logs.
Showing posts with label AIX. Show all posts
Showing posts with label AIX. Show all posts
Tuesday, January 24, 2012
Friday, January 20, 2012
Apache Karaf on IBM AIX 7.1!
Apache Karaf has supported runtimes on IBM AIX for quite some time, this however has been my first opportunity to personally try out Karaf on AIX 7.1.
I've been using IBM JDKs on Linux systems for the last few months, it's nice to have a full AIX environment to verify that the source and binaries are compatible. As I take time to test out tools, tweaks, and demo projects I'll be sure to post here my observations of Karaf on this platform and any best practices.
![]() |
| Apache Maven and IBM Java versions. |
I'm happy to report that Apache Karaf trunk (currently 3.0.0-snapshot) builds from source using IBM Java, and using GNU Tar I was able to easily extract a snapshot kit, and start Karaf.
![]() |
| Apache Karaf console start up screen, and contents of info command output. |
Labels:
AIX,
Apache Karaf
Wednesday, September 21, 2011
Apache Karaf on IBM JDK 7!
As of Apache Karaf 2.2.3 JDK 7 is supported as a runtime JVM, it has only been recently though that IBM's JDK 7 implementation has been available so I took some time to test it out.
![]() |
| Apache Karaf console startup screen. |
I'm happy to report that the Apache Karaf 2.2.3 tagged source tree fully compiles and runs on the new JVM! :)
![]() |
| Karaf info command output on IBM JDK7 for Linux. |
There are a few IBM centric modifications related to JDK 7 that have been noted, mostly centered around the memory management, class data sharing, and diagnostics components amongst other changes (see IBM's JDK 7 information center for more details).
The memory management changes I find particularly interesting, new GC options are outlined here, and the new Balanced Garbage Collection Policy is outlined here. The new GC policy being of particular interest to users whose heaps are regularly larger than 4GB (not too uncommon these days).
Labels:
AIX,
Apache Karaf
Monday, October 4, 2010
Resources for System P (AIX) learners
Something I've encountered only in my working career has been System P machines running AIX. Their not the easiest platform to develop, especially if you don't have local resources to learn upon. I earned my experience the hard way by attempting to resolve issues on production systems with just google searches, luck, and determination on my side. So with this in mind I'd like to share some links to resources that System P learners may find helpful.
Power Systems Community - lots of links here:
http://www-03.ibm.com/systems/power/community/index.html
PSeriesTech - A System P support forum:
http://www.pseriestech.org/
AIX Articles:
http://www.ibm.com/developerworks/views/aix/libraryview.jsp?type_by=Articles
AIX Tutorials:
http://www.ibm.com/developerworks/views/aix/libraryview.jsp?type_by=Tutorials
IBM Rational Cafe:
https://www.ibm.com/developerworks/rational/community/cafe/index.html
Power Systems Community - lots of links here:
http://www-03.ibm.com/systems/power/community/index.html
PSeriesTech - A System P support forum:
http://www.pseriestech.org/
AIX Articles:
http://www.ibm.com/developerworks/views/aix/libraryview.jsp?type_by=Articles
AIX Tutorials:
http://www.ibm.com/developerworks/views/aix/libraryview.jsp?type_by=Tutorials
IBM Rational Cafe:
https://www.ibm.com/developerworks/rational/community/cafe/index.html
Labels:
AIX
Wednesday, September 15, 2010
Developing Open Source on AIX
I was recently pricing some AIX gear for supporting my work on Apache Karaf and quickly became depressed on the totals -- no surprise of course, UNIX is never cheap (to be completely fare HP-UX and Sun/Oracle non-x86 systems are also pricey). Looking around the net for alternatives turned up very few leads. I guess one is left with having to look for used systems from a reputable reseller or on ebay.
Since my projects are mostly Java based I have spent some time crafting a Ubuntu Linux VM loaded with IBM Java, and tweaking the shell & environment to be more AIX like. This has served my needs well for the most part, unfortunately there is no substitute for a real working AIX system.
This process has left me wondering how have other open source developers approached supporting this platform? Please leave your insights and experiences in the comments below.
Since my projects are mostly Java based I have spent some time crafting a Ubuntu Linux VM loaded with IBM Java, and tweaking the shell & environment to be more AIX like. This has served my needs well for the most part, unfortunately there is no substitute for a real working AIX system.
This process has left me wondering how have other open source developers approached supporting this platform? Please leave your insights and experiences in the comments below.
Labels:
AIX,
Open Source
Monday, September 13, 2010
Happy Programmers' Day!
Happy Programmers' Day!
Programmers' day is an official holiday in Russia, which is celebrated on the 256th day of the year. This year it fell on September 13th which is today :)
I celebrated this year by spending some time working on an AIX issue which I had reproduced with IBM Java on a Ubuntu Linux VM, running on my Macbook - then spending some time planning the up coming Apache Karaf 2.1.0 release, finally turning towards some background tasks for the programming competition I'm volunteering at and helping to sponsor at Memorial University later this week.
To help celebrate this day with my fellow programmers I'd like to share some pics from my other blog that I think you may all enjoy.
Programmers' day is an official holiday in Russia, which is celebrated on the 256th day of the year. This year it fell on September 13th which is today :)
I celebrated this year by spending some time working on an AIX issue which I had reproduced with IBM Java on a Ubuntu Linux VM, running on my Macbook - then spending some time planning the up coming Apache Karaf 2.1.0 release, finally turning towards some background tasks for the programming competition I'm volunteering at and helping to sponsor at Memorial University later this week.
To help celebrate this day with my fellow programmers I'd like to share some pics from my other blog that I think you may all enjoy.
Labels:
AIX,
Apache Karaf,
Local
Wednesday, December 31, 2008
Happy New Year!
Happy New Year!
This past year has been very busy, and 2009 looks like it will be more so. I'm going to continue posting about AIX development issues however I'm going to focus more upon running ServiceMix on the AIX platform. You can also expect to see more Mac related posts and my observations on open source.
Cheers,
This past year has been very busy, and 2009 looks like it will be more so. I'm going to continue posting about AIX development issues however I'm going to focus more upon running ServiceMix on the AIX platform. You can also expect to see more Mac related posts and my observations on open source.
Cheers,
Tuesday, November 11, 2008
Using cxf-codegen-plugin on AIX
Have you ever seen this message while using cxf-codegen-plugin on AIX?
[ERROR] BUILD ERROR[INFO] ------------------------------------------------------------------------[INFO] com.sun.org.apache.xerces.internal.impl.dv.DVFactoryException:DTD factory class com.sun.org.apache.xerces.internal.impl.dv.dtd.DTDDVFactoryImpldoes not extend from DTDDVFactory.
The solution to this problem is easy, just add the
following to your project's pom file where you declare
use of the cxf-codegen-plugin:
<plugin> <groupId>org.apache.cxf</groupId> <artifactId>cxf-codegen-plugin</artifactId> <version>${cxf.version}</version> <dependencies> <!-- need to add this to get around the following error on
IBM's JDK org.apache.xerces.impl.dv.DVFactoryException: DTD factory
class org.apache.xerces.impl.dv.dtd.DTDDVFactoryImpl does not extend
from DTDDVFactory. --> <dependency> <groupId>xerces</groupId> <artifactId>xercesImpl</artifactId> <version>2.8.1</version> </dependency> </dependencies></plugin>
This is another case of the small differences in the IBM
JDK and Sun SDK. In code that is expected to run on
multiple target platforms I would place this dependency
in a profile set to activate on detection of an IBM JDK.
Cheers,
Labels:
AIX,
Open Source
Friday, October 31, 2008
MiR: October 2008
I've settled into a routine this month that consists of daily building the latest SMX4 Kernel, NMR, and Features code from SVN and checking for any AIX specific errors. While nightly continuous integration testing works great for automating the discovery & identification of issues I find that this manual approach has been quite fruitful (When no issues are found I just move on to other JIRAs in the SMX4 projects).
Common AIX issues I've seen this month include updates to the classpath and JVM security settings to work around IBM JDK differences in comparison to Sun, and some defensive coding around annotations.
Cheers,
Labels:
AIX,
Apache Servicemix
Sunday, October 12, 2008
More fun with IBM JDK on AIX
An issue that tends to turn up with various projects using Java's security mechanism is that the IBM JDK uses a different set of security providers than the reference Sun SDK. You'll know that your project is hitting this issue when a security related test fails due to a provider not found error (or similar message), even though the same test case passes on other platforms.
The manual solution is to edit the java.security file in $JAVA_HOME/jre/lib/security/java.security to contain the providers list in the order expected by Sun, however this may not be an option on your system (perhaps all java configuration files are locked down to read only permissions). In the case that you can't modify the java.security file directly you'll have to specify an alternate properties file for the JVM to use. This is as simple as passing the following JVM parameter to your test case launcher "-Djava.security.properties==
${path.to.ibm.jdk.security}", where ibm.jdk.security is your updated java.security file.
The below maven entry sets up a profile for IBM JDK and instructs the surefire test harness to use a provided java.security properties file. Please note that only when an IBM JDK is detected will the new java.security file be used.
You may have noticed something a little strange about the JVM parameter, the use of the double equality is intentional - if you use a single equals (=) then it appends your specified property file to the default property file, the double equality forces your specified configuration to be used.
Cheers,
The manual solution is to edit the java.security file in $JAVA_HOME/jre/lib/security/java.security to contain the providers list in the order expected by Sun, however this may not be an option on your system (perhaps all java configuration files are locked down to read only permissions). In the case that you can't modify the java.security file directly you'll have to specify an alternate properties file for the JVM to use. This is as simple as passing the following JVM parameter to your test case launcher "-Djava.security.properties==
${path.to.ibm.jdk.security}", where ibm.jdk.security is your updated java.security file.
So how would one incorporate this into a maven based testing harness?
The below maven entry sets up a profile for IBM JDK and instructs the surefire test harness to use a provided java.security properties file. Please note that only when an IBM JDK is detected will the new java.security file be used.
<profiles>
<profile>
<id>ibmjdk</id>
<activation>
<property>
<name>java.vendor</name>
<value>IBM Corporation</value>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<includes>
<include>**/*Test*</include>
</includes>
<forkMode>pertest</forkMode>
<argLine>
-Djava.security.properties==
${path.to.test.resources}/ibm.jdk.security
</argLine>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
You may have noticed something a little strange about the JVM parameter, the use of the double equality is intentional - if you use a single equals (=) then it appends your specified property file to the default property file, the double equality forces your specified configuration to be used.
Cheers,
Labels:
AIX
Sunday, October 5, 2008
Relaxing Sunday afternoon with the IBM JDK and Servicemix 4.
I may have strange habits, nothing says 'relaxing Sunday afternoon' like diving into a project on an AIX system while hanging out in a cafe.
Unlike many developers I actually enjoy my time with AIX, its a different environment that works well with my development style. Currently I'm attempting to resolve some testing issues that pop up when using IBM JDK to build & run unit tests of a Servicemix 4 Features SNAPSHOT - debugging this is a relatively simple process of identifying small differences in what comes packaged with Sun SDK vs the IBM JDK.
How does one know how to identify this kind of issue when working with Servicemix 4 on AIX?
Well the simplest way to determine this issue is reviewing the surefire test reports that are generated during a Servicemix 4 build. All test output is redirected to log files, inspecting these you'll most likely find that there will be a "<classname> not found error". If you know the jar that the particular class comes from then you can simply add the jar dependency to the pom file controlling the build of that component and retest. Regardless of if you know the class's parent jar or not it would be a good idea to open a report in SMX4's Jira to bring the issue to light.
What if the class/jar should already be available on the classpath?
In this case run the build with "mvn -X" to see exactly what libraries are being loaded. I tend to redirect the output from running maven with this flag to a file, then searching its contents. Sometimes you may need to reorder the loading order of jars due to naming conflicts, this however is rare (I've only seen that when working on Websphere based projects). A useful exercise when investigating this sort of issue is to build/test on another *nix platform to have a baseline to compare logs (personally I use Linux or Mac builds as a reference point).
Once you've determined the solution to getting builds/unit tests running again then you'll want to try out your changes on a non-AIX system to verify a platform safe change, then submit a patch :)
Cheers
Labels:
AIX,
Apache Servicemix
Subscribe to:
Posts (Atom)





