Wednesday, July 22, 2015

Using OpenCL on OpenDaylight

GPU processing on OpenDaylight!
I've been preparing for OpenDaylight Summit 2015, the yearly gathering of the OpenDaylight SDN community for tutorials, keynotes, sessions, and design forum. I wanted to build a little demo or two that I could run while on the conference floor. The first one is KTop: the Karaf Top processes command, the second is a small OpenCL on OpenDaylight demo.

OpenCL allows developers to construct kernel functions that execute on OpenCL capable devices - such as your GPU. Given there are a few use cases where massively parallel computing on a GPU could drastically reduce processing time, having OpenCL made available to applications running on OpenDaylight could be interesting.

My demo is just a proof of concept at this stage, however I'd enjoy discussing what could be made of the underlying concepts. That being said, I'd also love to chat about how OpenDaylight can make more use of its Apache Karaf container, and how we at Savoirtech can help organizations develop and maintain their Karaf-based applications.

Tuesday, July 7, 2015

Full Stack, Toronto 2015

This Fall, I'll be speaking at Full Stack, Toronto 2015! The conference focuses on the full stack used by developers to power the web. My talk will present how Apache Karaf provides an ecosystem (stack) for building micro service architectures for SOA and SDN.

Tittle:
"Empowering MicroServices with Apache Karaf"

Description:
"Deploying modular Java applications requires a container capable of supporting their special needs out of the box - Apache Karaf provides this and more.

Apache Karaf has become the defacto modular application container for Service Oriented Architectures and Software Defined Networking. These Karaf-based projects are powering web services and network infrastructure world wide.

In this session we'll present how Apache Karaf provides an ecosystem for supporting modular development and deployments of modern micro service architectures found in today's largest SOA and SDN infrastructures."

Tuesday, June 2, 2015

Attending OpenDaylight Summit 2015

I've booked my tickets to OpenDaylight Summit 2015, in Santa Clara, California!

This will be my first time to attend an OpenDaylight conference, and I can't wait to meet all the contributors.

The summit organizers have posted the week long event schedule. I'm not sure which tutorials and talks I'm going to attend yet, so many interesting ones overlap with each others time slot.

While I'm on site, I'm hoping to meet up with developers/maintainers interested in maximizing their use of Apache Karaf (the base runtime container the OpenDaylight Controller operates upon). I believe there are many best practices to be shared between our communities as SDN deployments build more field experiences. If you're attending the conference, and want to meet up to chat OpenDaylight-Karaf please let me know.

Tuesday, April 14, 2015

Building my new release server

Over the weekend I embarked upon building a new server for performing the Apache Karaf release candidates I build. I thought I'd share some specs, setup, and photos of the completed project.

The Gear:



Configuring The System:


The Apache Karaf Release Guide provides the blueprint for this build, providing the community with a reproducible set of steps to help us avoid relying on a 'magic build' machine. The guide outlined the various tools, their versions, and configurations required to produce a build candidate.


No more spinning disks for my build machine :)

System Layout:


The 120GB disk was chosen as the primary boot disk, hosting Ubuntu 14.04 LTS and a swap partition (/dev/sda). The 240GB disk was formatted as a single partition using an ext3 filesystem (/dev/sdb).

I prefer to keep my Operating System and 'task' drives separate on a server.

Build Drive Organization:  


The build drive hosts three directories:

  • tools - All build tools.
  • repo - local m2 repository.
  • source - SVN and Git repos.

The tools directory contains various versions of JDKs, Ant, Maven, PrinceXML, and my environment scripts.

Sunday, January 11, 2015

OpenDaylight Helium on Intel Edison!

Edison mounted on mini breakout board.
Canadian Quarter for Scale.
How small a physical device can you get OpenDaylight running upon? Well so far the answer is the Intel Edison.

If your not familiar with the Edison, its a system on module device containing a dual core Intel Atom CPU (32bit @ 500 Mhz), 1 GB of LPDDR3 RAM, 4 GB of eMMC flash storage, and built-in wifi. At 35.5 x 25 x 3.9 mm it is arguably one of the smallest machines I've seen run an Apache Karaf based system.

My build utilizes a Yocto Linux OS build for Edison, and a standard 32 bit Oracle JDK 1.7.0 installation. Downing loading utilities onto the Edison was a small challenge in that the version of wget that ships with Yocto does not support https, to get around this I used SCP to copy files over to the device.

Out of the box I only had to make one customization to the OpenDaylight setenv script:
  • The set environment script (setenv) had JVM parameters for PermGen and Max Memory set higher than the Edison could support. I set these values to 340MB for PermGen, and 756MB for Max Memory. These are not optimal settings, but will work well enough to get things going.
After OpenDaylight had displayed the console banner, I executed the console info command to see system environment information:
OpenDaylight Banner and info command output
Sweet! OpenDaylight has booted, and the CLI is responsive!

Now to verify the installation is running correctly, I started up the OpenDaylight Toaster demo.
Log Display Output
I used jmxterm to manipulate the makeToast and clearToastsMade operations exposed via JMX on the controller to verify their correct operation - using log:display I could view each operation's logging events.
Edison CPU Info

I've posted to my github page my notes and files for getting OpenDaylight Helium running on Intel Edison. I'm looking forward to seeing what the community makes on the platform.

Links for further reading:

Intel Edison - One Tiny Platform, Endless Possibility:
http://www.intel.com/content/www/us/en/do-it-yourself/edison.html

Edison IoT module ships with Atom plus Quark Combo SOC:
http://linuxgizmos.com/edison-iot-module-ships-with-atom-plus-quark-combo-soc/

OpenDaylight:
http://www.opendaylight.org

Apache Karaf:
http://karaf.apache.org

Tuesday, November 18, 2014

KTop update and OpenDaylight MDSAL Status command!

KTop Enhancements!
We're happy to announce our latest Milestone release of the Aetos KTop command for Apache Karaf 3 based containers.

This latest revision includes:

  • Improved CPU time reporting,
  • Pressing q to quit,
  • Sorting column change via left and right arrow keys, and
  • Reverse sort by pressing r key.

Under the hood we've also made several bug fixes, and runtime performance improvements.

Sweet! How do I get this latest version?


We've published our new MileStone release to Maven Central: http://search.maven.org/#search%7Cga%7C1%7Cctop

You can also grab it from GitHub:
https://github.com/savoirtech/ctop/tags

Source tag link:
https://github.com/savoirtech/ktop/tree/ctop-0.2.0.M1

If you're using an Apache Karaf 3.0.x based system (such as Aetos 3.0.2), you can install this MileStone release using the following Karaf console command:
install -s mvn:com.savoirtech.karaf.commands/ctop/0.2.0.M1

Feedback is welcome! Please submit any ideas, enhancements, bugs to the project issue tracker: https://github.com/savoirtech/ctop/issues

So, tell us about this MDSAL Status command!


MDSAL:Status - a live feed of how MDSAL is operating.
The OpenDaylight community recently ported their project to live on top of Apache Karaf 3 with their Helium release. To help spur development of new OpenDaylight Karaf commands we've built a simple Model Driven Service Abstraction Layer status command to demonstrate how Karaf's console can empower developers and operators to create their own monitoring tools.

So what does MDSAL Status do?


The MDSAL Status commands provides an updated view of MDSAL metrics, including:

  • ConfigRegistry version and health,
  • DOMDataBroker statistics, and
  • Metrics for CommitExecuter (CE), CommitFutureExecutor (CFE), InMemoryConfigDataStore (IMCDS), and InMemoryOperationalDataStore (IMODS).

The metrics table is of particular interest, providing live updates of ten tracked attributes, including:

  • ActiveThreadCount,
  • CompletedTaskCount,
  • CurrentQueueSize,
  • CurrentThreadPoolSize,
  • LargestQueueSize,
  • LargestThreadPoolSize,
  • MaxQueueSize,
  • MaxThreadPoolSize,
  • RejectedTaskCount, and
  • TotalTaskCount.

These values are obtained from MBeans provided by the OpenDaylight controller. The Karaf console provides the mechanisms to allow users to view these metrics without having to use additional external tooling.

Sweet! How do I get this latest version?


We've published a MileStone release to Maven Central:
http://search.maven.org/#search%7Cga%7C1%7Cmdsal-status

You can also grab it from GitHub:
https://github.com/savoirtech/mdsal-status/releases

Source tag link:
https://github.com/savoirtech/mdsal-status/tree/mdsal-status-0.1.0.M1

On OpenDaylight Helium based distributions, you can install this MileStone release using the following Karaf console command:
install -s mvn:com.savoirtech.karaf.commands/mdsal-status/0.1.0.M1

How do I use MDSAL Status once installed? 


The status command requires users to have the MDSAL feature installed in their container at runtime. Once MDSAL is available, the MDSAL Status command will become functional.

To execute command on Helium, invoke:
mdsal:status

To exit status, press control + c or q to quit.

Feedback is welcome! Please submit any ideas, enhancements, bugs to the project issue tracker:
https://github.com/savoirtech/mdsal-status/issues

Wednesday, November 12, 2014

Want to see your top Apache Camel routes in a CamelContext? Try the CTop command!

Aetos Integration Platform
I've been working on a utility command for Apache Karaf based containers that will display Apache Camel Context Route metrics in a manner similar to the Linux Top command. The result has been the Aetos ctop command.

A Top like command for Apache Camel Routes, awesome!
Aetos is Savoir Technologies' Integration Platform - essentially a custom stack of raw Apache projects that makes using Apache Karaf easy for developing and running production large scale enterprise solutions.

The Aetos ctop command provides a Top like display of vital Camel Route metrics, including:

  • Total Exchanges,
  • Completed Exchanges
  • Failed Exchanges,
  • Minimun Processing Time,
  • Maximum Processing Time,
  • Mean Processing Time, and
  • Last Processing Time

The ctop command allows users to specify which column they'd like to rank routes, and the information update interval. Use the --help option to read the command's usage information.

Sounds cool, I'd like to try it out on my Karaf system!


We've published a MileStone 1 release to Maven Central: http://search.maven.org/#search%7Cga%7C1%7Cctop

You can also grab it from GitHub:
https://github.com/savoirtech/ctop/tags

Source tag link:
https://github.com/savoirtech/ktop/tree/ctop-0.1.0.M1

If you're using an Apache Karaf 3.0.x based system (such as Aetos 3.0.2), you can install MileStone 1 using the following Karaf console command:
install -s mvn:com.savoirtech.karaf.commands/ctop/0.1.0.M1

Feedback is welcome! Please submit any ideas, enhancements, bugs to the project issue tracker: https://github.com/savoirtech/ctop/issues

How do I get the code?


git clone https://github.com/savoirtech/ctop.git

Currently, the command is aimed at Apache Karaf 3 & 4 containers. If community interest exists, we'll port it back to Karaf 2.3 & 2.4.

How do I build and install ctop into my container?


To build, invoke:

mvn install

To install in Karaf, invoke from Karaf console:

install -s mvn:com.savoirtech.karaf.commands/ctop/version-SNAPSHOT

How do I use ctop once installed?

The CTop command requires users to have the Apache Camel feature installed in their container at runtime. Once Camel is available, the CTop command will become functional.

To execute command on Karaf, invoke:

aetos:ctop CamelContextName

To exit ctop, press control + c


The code is under the GNU license at the moment, as per a project requirement -- we will have it under the ASL in the future. In the mean time if you're looking for a way to monitor your Camel Routes from your console window, give Aetos ctop a try!