Happy New Year!

Happy New Year!

This past year has been very busy, and 2009 looks like it will be more so. I'm going to continue posting about AIX development issues however I'm going to focus more upon running ServiceMix on the AIX platform. You can also expect to see more Mac related posts and my observations on open source.

Cheers,

VirtualBox on Mac

I'm a full time Mac user these days however some times I need to use a Windows client to connect to certain Windows only applications (websites that are IE only for instance). After searching around for suitable VMs for Mac I tried out VirtualBox, an Open Source virtualizer for x86 hardware.

After reading the website I downloaded the installer for Mac and gave the VM a try. The installation process was straight forward, following the regular Mac process. Setting up my first VM was quite easy; I selected to create a WinXP VM, allocated to it up to 2GB of my system's RAM, and made a static 20GB virtual partition. After creating the VM in VirtualBox I started the WinXP install process as I would on any other PC (this went as per normal, with many restarts after each windows update run). 

WinXP running as an application on my MacBook

WinXP recognized my processor as Intel Core 2 Duo T7500 @ 2.2Ghz however it operates the VM at 955Mhz. This clock speed was achieved after enabling a few features that VirtualBox leaves off by default, before enabling the following settings my clock speed was 505Mhz:

• ACPI
  
• IO APIC
  
• VT-x/AMD-V and,
  
• PAE/NX. 

So far I've been impressed by the simplicity of setting up the Windows VM on my Mac using VirtualBox. It's not going to become a gaming VM, however I expect it will work quite well as a client for those few non-mac friendly apps I need to access.

MiR: November 2008

This month my routine has been changed up a little by refocusing upon ServiceMix 3 issues and internal infrastructure projects (more CPI work, again this always benefits ServiceMix as it frees up developer time to focus on features and improvements).

On a completely different topic, this past month has been fun as a Mac user/fan - the new Macbooks are sweet, convincing more of my friends to finally make the switch. We don't have a local Apple store here in Newfoundland so I get to help friends in their transition to Mac OS X - a favour I happily oblige :)

Meeting the new Macbook.

Cheers,

Using cxf-codegen-plugin on AIX

Have you ever seen this message while using cxf-codegen-plugin on AIX?

[ERROR] BUILD ERROR

[INFO] ------------------------------------------------------------------------

[INFO] com.sun.org.apache.xerces.internal.impl.dv.DVFactoryException: 

DTD factory class com.sun.org.apache.xerces.internal.impl.dv.dtd.DTDDVFactoryImpl 

does not extend from DTDDVFactory.

The solution to this problem is easy, just add the
following to your project's pom file where you declare
use of the cxf-codegen-plugin:

<plugin>
 <groupId>org.apache.cxf</groupId>
  <artifactId>cxf-codegen-plugin</artifactId>
   <version>${cxf.version}</version>
    <dependencies>
     <!-- need to add this to get around the following error on
IBM's JDK org.apache.xerces.impl.dv.DVFactoryException: DTD factory
class org.apache.xerces.impl.dv.dtd.DTDDVFactoryImpl does not extend
from DTDDVFactory.
     -->
     <dependency>
      <groupId>xerces</groupId>
      <artifactId>xercesImpl</artifactId>    
      <version>2.8.1</version>
     </dependency>
    </dependencies>
</plugin> 

This is another case of the small differences in the IBM
JDK and Sun SDK. In code that is expected to run on
multiple target platforms I would place this dependency
in a profile set to activate on detection of an IBM JDK.
Cheers,

MiR: October 2008

I've settled into a routine this month that consists of daily building the latest SMX4 Kernel, NMR, and Features code from SVN and checking for any AIX specific errors. While nightly continuous integration testing works great for automating the discovery & identification of issues I find that this manual approach has been quite fruitful (When no issues are found I just move on to other JIRAs in the SMX4 projects). 

Common AIX issues I've seen this month include updates to the classpath and JVM security settings to work around IBM JDK differences in comparison to Sun, and some defensive coding around annotations.

Cheers,

More fun with IBM JDK on AIX

An issue that tends to turn up with various projects using Java's security mechanism is that the IBM JDK uses a different set of security providers than the reference Sun SDK. You'll know that your project is hitting this issue when a security related test fails due to a provider not found error (or similar message), even though the same test case passes on other platforms.

The manual solution is to edit the java.security file in $JAVA_HOME/jre/lib/security/java.security to contain the providers list in the order expected by Sun, however this may not be an option on your system (perhaps all java configuration files are locked down to read only permissions). In the case that you can't modify the java.security file directly you'll have to specify an alternate properties file for the JVM to use. This is as simple as passing the following JVM parameter to your test case launcher "-Djava.security.properties==
${path.to.ibm.jdk.security}", where ibm.jdk.security is your updated java.security file.

So how would one incorporate this into a maven based testing harness?

The below maven entry sets up a profile for IBM JDK and instructs the surefire test harness to use a provided java.security properties file. Please note that only when an IBM JDK is detected will the new java.security file be used.

<profiles>
<profile>
<id>ibmjdk</id>
<activation>
   <property>
       <name>java.vendor</name>
       <value>IBM Corporation</value>
   </property>
</activation>
<build>
 <plugins>
  <plugin>
   <groupId>org.apache.maven.plugins</groupId>
   <artifactId>maven-surefire-plugin</artifactId>
   <configuration>
    <includes>
     <include>**/*Test*</include>
    </includes>
    <forkMode>pertest</forkMode>
    <argLine>
     -Djava.security.properties==
      ${path.to.test.resources}/ibm.jdk.security
    </argLine>
   </configuration>
  </plugin>
 </plugins>
</build>
</profile>
</profiles>

You may have noticed something a little strange about the JVM parameter, the use of the double equality is intentional - if you use a single equals (=) then it appends your specified property file to the default property file, the double equality forces your specified configuration to be used.

Cheers,

Relaxing Sunday afternoon with the IBM JDK and Servicemix 4.

I may have strange habits, nothing says 'relaxing Sunday afternoon' like diving into a project on an AIX system while hanging out in a cafe. 

Unlike many developers I actually enjoy my time with AIX, its a different environment that works well with my development style. Currently I'm attempting to resolve some testing issues that pop up when using IBM JDK to build & run unit tests of a Servicemix 4 Features SNAPSHOT - debugging this is a relatively simple process of identifying small differences in what comes packaged with Sun SDK vs the IBM JDK.

How does one know how to identify this kind of issue when working with Servicemix 4 on AIX?

Well the simplest way to determine this issue is reviewing the surefire test reports that are generated during a Servicemix 4 build. All test output is redirected to log files, inspecting these you'll most likely find that there will be a "<classname> not found error". If you know the jar that the particular class comes from then you can simply add the jar dependency to the pom file controlling the build of that component and retest. Regardless of if you know the class's parent jar or not it would be a good idea to open a report in SMX4's Jira to bring the issue to light.

What if the class/jar should already be available on the classpath?

In this case run the build with "mvn -X" to see exactly what libraries are being loaded. I tend to redirect the output from running maven with this flag to a file, then searching its contents. Sometimes you may need to reorder the loading order of jars due to naming conflicts, this however is rare (I've only seen that when working on Websphere based projects). A useful exercise when investigating this sort of issue is to build/test on another *nix platform to have a baseline to compare logs (personally I use Linux or Mac builds as a reference point). 

Once you've determined the solution to getting builds/unit tests running again then you'll want to try out your changes on a non-AIX system to verify a platform safe change, then submit a patch :)

Cheers

MiR: September 2008

September was another busy month, so busy that I haven't had time to update my blog as much as I'd like too. I've been involved in a lot of the behind the scenes work that goes into project testing infrastructure, the dividends of this work will however trickle back as more developer time can now be devoted to handling issues as their discovered (the return on investment on xharness CPIs are always worth the time).

Cheers.

Maven integration for Eclipse

I've been getting into using Eclipse more this month, moving away from my normal vi/vim/nano way of doing things. Its not that I'm particularly against IDEs, its  just that I  spend most of my time remotely connected to AIX, HP-UX, Solaris, and other *nix machines quickly editing files and hence find light weight (resource wise) editors more sensible. When it comes to working with large code bases however my use of these editors slows my progress down considerably.

The only thing that was holding me back from setting up Eclipse with the projects I like to work on has been getting all the code to load correctly and not end up with loads of errors due to not having all the jars loaded. Now however I've started to use m2eclipse to automatically setup my projects.

The project is by all means not new, many of you are probably already quite familiar with it. In case your not however here is a quick introduction to setting up a maven based project in Eclipse.

• Goto the Maven Integration for Eclipse site (http://m2eclipse.codehaus.org) and install the components required for your Eclipse installation. Alternatively use Eclipse's find & install feature to acquire the plugin.
  
•  Once installed follow the next three steps to load your project:

1. Click File -> New -> Other...
2. Select "Maven -> Checkout Maven Projects from SCM." from the list.
   
   
3. Select the SCM option for your project, then provide the URL to the SCM, click finish.

Your project resources will now be downloaded and setup :)

Hope this little demo was useful, cheers.

Responses: How to contribute to Apache?

In response to a question left on my last post I'll present my quick introduction to contributing to Apache projects (using Apache Servicemix as an example):

Contributing to an Apache project:

The Apache Software Foundation (ASF) is a volunteer based organization. They provide the infrastructure for each ASF project (mailing lists, code repositories, bug tracking systems, etc). The main way for people to become involved with ASF project is to contribute time to their favorite project.

So what kind of contributions can I make as a person with (next) to no knowledge of the inner workings of Apache?

Taking Apache Servicemix as my favored example, there are many ways to begin contributing:

1. First off you should become familiar with the project by downloading it, installing the application and trying out it's examples. Following the project's online documentation will assist you in understanding what's happening behind the scenes.

Servicemix can be downloaded here:

Servicemix documentation can be found here:

2. The next step is to start following the project's mailing list & forums. People tend to be quite friendly and helpful, this is an invaluable part of the community support you always hear about open source projects.

Servicemix mailing lists link here:

Servicemix forums list link here:

3. Now that you've tried out the project, and have listened to some of the discussions involving the project you should visit the project's issue tracker. As a contributor this is the place to report issues (Note: you'll have to setup an account to make submissions - this is a quick & easy process however). Reporting issues is one of the fastest ways to begin contributing.

Servicemix issue tracker here:

4. If you feel comfortable with source code then I would suggest checking out the code branch corresponding to the release you've installed. Follow the build instructions for your project, typically this will consist of typing "mvn install" in the root directory of the project.

Servicemix source here:

Servicemix build information here:

5. Now that you've become familiar with the project, spent some time reviewing reported issues, and have learned to build the application, you should now be ready to start contributing code :)

So how do I contribute code?

• Select an open issue from the project's issue tracker. At first it may be best to select an issue described as 'minor'. If you check out the bottom of the Servicemix contributing page you will see that the team keeps a running list of issues they'd like to see resolved.
• After reviewing the issue begin working on a solution (the fun part to many of us).
  
• Once you believe that you have created a suitable solution create a matching unit test (if applicable).
  
• Use the project's source control tool to produce a diff of the files involved.
  
• Submit the diff file to the issue tracker, then email the project's developers mail list to alert them of the submission.
  

Once you've submitted a patch you will have to wait for an project member with committer status to review, then apply your patch to the source tree. Over a period of time if you continue to contribute high quality patches to the project you may get approached to become a committer yourself (see ASF project specific details for more information). One important note on submitting patches that you should be aware of is that for inclusion of patch one will have to approve ASF license grant, that is to say that your contributing this code under the Apache license. You can learn more about the Apache license here.

I hope the above guide is helpful, each project has its own variations so please read their online documentation carefully.

Cheers,

MiR: August 2008

Month in Review: August 2008

August started out with quite a bit of excitement as the FUSE team gathered in Dublin Ireland to discuss project status and future development. This was my first time to see Dublin and meet the core group of Open Source developers working on the various components of FUSE.

Upon return to Newfoundland I dived into research and development involving Apache Servicemix Kernel and XHarness. The first phase of this work was to gain a deeper understanding of Apache Geronimo GShell, Plexus, and JLine. The second phase has been to improve XHarness to inter-operate smoothly with these technologies.

The first fruits of these efforts were released in XHarness 1.0-beta2, you can expect to see more improvements in the upcoming XHarness 1.0-beta3.

Open Source headline in Quebec

A recent headline carried on the CBC is reporting that "Quebec's open-source software association is suing the provincial government, saying it is giving preferential treatment to Microsoft Corp. by buying the company's products rather than using free alternatives." The original story posting can be found here, a press release by FACIL regarding this story can be found on their website here.

This will be an interesting story to follow.

Cheers

SNAPSHOTs are my Kryptonite

I've been playing around with building my own SNAPSHOT of Apache Geronimo GShell . For those of you not familiar with Maven SNAPSHOTs they are the artifacts generated during a Maven build on a project's SNAPSHOT branch (this is defined in the project's pom file).

SNAPSHOTs are a necessary step in the development of project code. Not every build of a project's source code can be considered a point release. To add new features, fix reported bugs, refactor code, or just improve internal documentation, among other things, developers need to build the source and test the modified modules. The process of iterating through successive build/test cycles allow for a higher quality finished release.

So far so good, how could this cause development pains?

Allow me to elaborate. When building larger applications developers will rely on features from other projects, sometimes these features only exist in SNAPSHOT form as the next point release containing the required feature have not yet been made (and may be days, weeks, or months away from release). So to continue onwards with their own project developers will include dependencies on SNAPSHOT builds. Something to keep in mind however is that SNAPSHOT builds may or may not be available for a given project's repository or mirrors at all times (resulting in an Artifact not found message, and failed build)...

So back to my above statement, why are SNAPSHOTs my Kryptonite?

By depending on a SNAPSHOT one becomes dependent upon the continual successful build of an unstable code branch. These dependencies deepen as relied upon projects also include their own SNAPSHOT requirements. The possible non-availability of particular SNAPSHOTs can result in you not being able to build the project your currently interested in. Sure you can checkout the source code to your dependencies and build your own SNAPSHOTs, and repeat the process until you can build your entire project... this process normally works, it just takes time :S

Is there any solution to this problem?

Considering the nature of SNAPSHOTs there probably isn't much that we can do to ensure that we don't spend too much time building dependencies, forward progress in development can not always be with a stable code base. In practice we have to assume that SNAPSHOTs will be available from some repository, those that are not will hopefully be a quick svn co <branch>, mvn install away.

It's taken several hours now to build up the dependencies I required for my SNAPSHOT of GShell. Now that I have these artifacts in my .m2/repository I can continue on developing as per normal. I just wanted to share a little of my adventure in the process, and relieve some of the sudden headache I develop every time I build a project and get artifact unavailable in all defined repositories - then see the word <ProjectName>-SNAPSHOT.jar.

Cheers

XHarness How To: Using ignore ANSI.

With the release of XHarness 1.0-beta2 I have decided to write a quick how-to for using the new ignore ANSI feature. The reason for this feature addition was to resolve the issue separation of content and presentation.

When testing an application we are generally concerned with the correct content of output, presentation details such as bold text or text colour markups may not be essential to asserting a correct response.

Take for example an application that takes in a username and reports back "Hello username, welcome to example1 program". The general format of the XHarness definition to test this program would be:

<xh-execbg executable="${path.to.app}/bin/example1"
processname="example1"
dir="${path.to.app}"/>
<arg line="foobar"/>
</xh-execbg>
<assert timeout="10" erroronfail="true">
<and>
<outputcontains stream="stdout"

task="xh-execbg">Hello foobar, welcome to example1 program.</outputcontains>
</and>
</assert>

Given the input "foobar" we expect the output to be "Hello foobar, welcome to example1 program." In this case XHarness will run the demo application, assert the correct output was received, then report success.

Now what happens if we modify example1 program to print out username in bold? The above XHarness definition would report a failed assertion. The reason for this is that the output contains method will be looking for plain text, the markup to present bold text would violate the matching specified.

To simplify working with output that may include ANSI markups the ignoreansi attribute was added. The default behavior is to not ignore ANSI text markup control characters, thus allowing developers to consider these markups part of a correct response. If the developer however would like to ignore such presentation details and just focus on the content of a response they may alter their output contains fragments to the following:

<assert timeout="10" failonerror="true">

<and>

<outputcontains stream="stdout"

task="xh-execbg"

ignoreansi="true" >Hello foobar, welcome to example1 program.</outputcontains>

</and>

</assert>

This time when XHarness processes the output from example1 program it will assert a match successfully.

For more information on using XHarness please see the XHarness web site: http://xharness.codehaus.org

XHarness 1.0-beta2 Released

"XHarness is a system test harness for Apache Ant 1.6.x. It allows people to describe their product/system tests in the form of XML as part of an ant build file, describing the tasks and processes that comprise the test cases and what their expected behaviour is."

This latest release includes fixes to the Unix async launcher and includes a new ignore ANSI control character mode for output matching.

You can download XHarness 1.0-beta2 from the download page.

Hello World!

Another blog is born... This one will focus on my adventures in open source, events, and anything else occupying my mind.

Cheers