OpenDaylight Cookbook available for early access!

OpenDaylight Cookbook is available for early access!

Currently, you can obtain the book via Packt Publishing or Amazon:

• Packt Publishing: https://www.packtpub.com/virtualization-and-cloud/opendaylight-cookbook
• Amazon: https://www.amazon.com/OpenDaylight-Cookbook-Mathieu-Lemay-ebook/dp/B01JS2KVTK

Why do I want this book?

Within each chapter of the cookbook you'll find simple, no-nonsense instructions for how to accomplish tasks; we go beyond the introductory documentation and delve into how to get things done.

What is Early Access?

Packt Publishing offers early access to select titles to allow readers the most up to date revisions of books while under development. For more information please see https://www.packtpub.com/books/info/packt/alpha.

OpenDaylight Summit 2016: Apache Karaf Report

My talk submission to OpenDaylight Summit 2016 was accepted! I'm very excited to have this opportunity to address the OpenDaylight community directly this Fall in Seattle. If you're attending the conference, please feel free to sign up for my talk on Sched (http://sched.co/7RBf).

Abstract: OpenDaylight's Apache Karaf Report:

Apache Karaf is at the core of any OpenDaylight system; the architecture and capabilities of Apache Karaf will, in the end, place an upper bound on what OpenDaylight can do as a whole. In this presentation, Jamie Goodyear will review recent events in the Apache Karaf development community, discuss the current state of Apache Karaf and its sub-projects, the challenges it faces, and look forward to how Apache Karaf may address those challenges.

What does OpenDaylight SDN Platform need from the Apache Karaf container?

Just wanted to share my talk from OpenDaylight Summit 2015.

I'd like to thank the OpenDaylight community again for allowing me speaking time during the developer forum.

Decanting OpenDaylight

The Monitoring and Alerting Solution for Apache Karaf

During OpenDaylight Summit 2015 I was given the opportunity to talk about Apache Karaf's road map, and Karaf's new subproject Apache Karaf Decanter.

Decanter is a complete monitoring and alerting solution for Apache Karaf.  It provides a web based collection of dash boards, which contain nicely graphed metrics from your applications' runtime, and lets you explore logged events.

Architecturally, Decanter is composed of:

• collectors retrieving monitored data (JMX metrics, log messages, etc)
• dispatchers powered by EventAdmin, and
• appenders storing monitored data.

Decanter provides a Service Level Agreement layer, which allows you to check values of harvested data and send alerts when data in not in the expected state.

User and Developer guides are available from Karaf's website.

OpenDaylight Lithium, Decanted

Perhaps the most exciting news for OpenDaylight users is that the core Decanter framework works on Lithium! All the OpenDaylight community needs to do now is develop the various collectors, dispatchers, appenders, and alters to make OpenDaylight based services appear in one of Decanter's dashboards.

Thank you ODSummit

George Zhao, OpenDaylight Release Manager
receiving a signed copy of Learning Apache Karaf

I recently attended OpenDaylight Summit 2015 in Santa Clara, California - a community gathering of the developers, and users of OpenDaylight SDN Platform. The energy and enthusiasm of the crowd for building this new open source community was invigorating. I met many of the people behind the IRC nicknames I've come to know over the last year & half of working with the community. It was truly a great event, and I look forward to seeing how OpenDaylight's next release, Be, evolves.

The various keynotes, sessions, and tutorials are available for reviewing from the OpenDaylight projects' youtube channel. I'd like the share the below two keynotes, as it helps show how deeply OpenDaylight is reaching into today's networks and how integral OpenDaylight (and it's Apache Karaf core) is becoming in tomorrow's infrastructure:

Using OpenCL on OpenDaylight

GPU processing on OpenDaylight!

I've been preparing for OpenDaylight Summit 2015, the yearly gathering of the OpenDaylight SDN community for tutorials, keynotes, sessions, and design forum. I wanted to build a little demo or two that I could run while on the conference floor. The first one is KTop: the Karaf Top processes command, the second is a small OpenCL on OpenDaylight demo.

OpenCL allows developers to construct kernel functions that execute on OpenCL capable devices - such as your GPU. Given there are a few use cases where massively parallel computing on a GPU could drastically reduce processing time, having OpenCL made available to applications running on OpenDaylight could be interesting.

My demo is just a proof of concept at this stage, however I'd enjoy discussing what could be made of the underlying concepts. That being said, I'd also love to chat about how OpenDaylight can make more use of its Apache Karaf container, and how we at Savoirtech can help organizations develop and maintain their Karaf-based applications.

Attending OpenDaylight Summit 2015

I've booked my tickets to OpenDaylight Summit 2015, in Santa Clara, California!

This will be my first time to attend an OpenDaylight conference, and I can't wait to meet all the contributors.

The summit organizers have posted the week long event schedule. I'm not sure which tutorials and talks I'm going to attend yet, so many interesting ones overlap with each others time slot.

While I'm on site, I'm hoping to meet up with developers/maintainers interested in maximizing their use of Apache Karaf (the base runtime container the OpenDaylight Controller operates upon). I believe there are many best practices to be shared between our communities as SDN deployments build more field experiences. If you're attending the conference, and want to meet up to chat OpenDaylight-Karaf please let me know.

OpenDaylight Helium on Intel Edison!

Edison mounted on mini breakout board.
Canadian Quarter for Scale.

How small a physical device can you get OpenDaylight running upon? Well so far the answer is the Intel Edison.

If your not familiar with the Edison, its a system on module device containing a dual core Intel Atom CPU (32bit @ 500 Mhz), 1 GB of LPDDR3 RAM, 4 GB of eMMC flash storage, and built-in wifi. At 35.5 x 25 x 3.9 mm it is arguably one of the smallest machines I've seen run an Apache Karaf based system.

My build utilizes a Yocto Linux OS build for Edison, and a standard 32 bit Oracle JDK 1.7.0 installation. Downing loading utilities onto the Edison was a small challenge in that the version of wget that ships with Yocto does not support https, to get around this I used SCP to copy files over to the device.

Out of the box I only had to make one customization to the OpenDaylight setenv script:

• The set environment script (setenv) had JVM parameters for PermGen and Max Memory set higher than the Edison could support. I set these values to 340MB for PermGen, and 756MB for Max Memory. These are not optimal settings, but will work well enough to get things going.

After OpenDaylight had displayed the console banner, I executed the console info command to see system environment information:

OpenDaylight Banner and info command output

Sweet! OpenDaylight has booted, and the CLI is responsive!

Now to verify the installation is running correctly, I started up the OpenDaylight Toaster demo.

Log Display Output

I used jmxterm to manipulate the makeToast and clearToastsMade operations exposed via JMX on the controller to verify their correct operation - using log:display I could view each operation's logging events.

Edison CPU Info

I've posted to my github page my notes and files for getting OpenDaylight Helium running on Intel Edison. I'm looking forward to seeing what the community makes on the platform.

Links for further reading:

Intel Edison - One Tiny Platform, Endless Possibility:
http://www.intel.com/content/www/us/en/do-it-yourself/edison.html

Edison IoT module ships with Atom plus Quark Combo SOC:
http://linuxgizmos.com/edison-iot-module-ships-with-atom-plus-quark-combo-soc/

OpenDaylight:
http://www.opendaylight.org

Apache Karaf:
http://karaf.apache.org

KTop update and OpenDaylight MDSAL Status command!

KTop Enhancements!

We're happy to announce our latest Milestone release of the Aetos KTop command for Apache Karaf 3 based containers.

This latest revision includes:

• Improved CPU time reporting,
• Pressing q to quit,
• Sorting column change via left and right arrow keys, and
• Reverse sort by pressing r key.

Under the hood we've also made several bug fixes, and runtime performance improvements.

Sweet! How do I get this latest version?

We've published our new MileStone release to Maven Central: http://search.maven.org/#search%7Cga%7C1%7Cctop

You can also grab it from GitHub:
https://github.com/savoirtech/ctop/tags

Source tag link:
https://github.com/savoirtech/ktop/tree/ctop-0.2.0.M1

If you're using an Apache Karaf 3.0.x based system (such as Aetos 3.0.2), you can install this MileStone release using the following Karaf console command:
install -s mvn:com.savoirtech.karaf.commands/ctop/0.2.0.M1

Feedback is welcome! Please submit any ideas, enhancements, bugs to the project issue tracker: https://github.com/savoirtech/ctop/issues

So, tell us about this MDSAL Status command!

MDSAL:Status - a live feed of how MDSAL is operating.

The OpenDaylight community recently ported their project to live on top of Apache Karaf 3 with their Helium release. To help spur development of new OpenDaylight Karaf commands we've built a simple Model Driven Service Abstraction Layer status command to demonstrate how Karaf's console can empower developers and operators to create their own monitoring tools.

So what does MDSAL Status do?

The MDSAL Status commands provides an updated view of MDSAL metrics, including:

• ConfigRegistry version and health,
• DOMDataBroker statistics, and
• Metrics for CommitExecuter (CE), CommitFutureExecutor (CFE), InMemoryConfigDataStore (IMCDS), and InMemoryOperationalDataStore (IMODS).

The metrics table is of particular interest, providing live updates of ten tracked attributes, including:

• ActiveThreadCount,
• CompletedTaskCount,
• CurrentQueueSize,
• CurrentThreadPoolSize,
• LargestQueueSize,
• LargestThreadPoolSize,
• MaxQueueSize,
• MaxThreadPoolSize,
• RejectedTaskCount, and
• TotalTaskCount.

These values are obtained from MBeans provided by the OpenDaylight controller. The Karaf console provides the mechanisms to allow users to view these metrics without having to use additional external tooling.

Sweet! How do I get this latest version?

We've published a MileStone release to Maven Central:
http://search.maven.org/#search%7Cga%7C1%7Cmdsal-status

You can also grab it from GitHub:
https://github.com/savoirtech/mdsal-status/releases

Source tag link:
https://github.com/savoirtech/mdsal-status/tree/mdsal-status-0.1.0.M1

On OpenDaylight Helium based distributions, you can install this MileStone release using the following Karaf console command:
install -s mvn:com.savoirtech.karaf.commands/mdsal-status/0.1.0.M1

How do I use MDSAL Status once installed? 

The status command requires users to have the MDSAL feature installed in their container at runtime. Once MDSAL is available, the MDSAL Status command will become functional.

To execute command on Helium, invoke:
mdsal:status

To exit status, press control + c or q to quit.

Feedback is welcome! Please submit any ideas, enhancements, bugs to the project issue tracker:
https://github.com/savoirtech/mdsal-status/issues

OpenDaylight Helium on Raspberry Pi first performance benchmark results!

Last week I posted about getting OpenDaylight Helium release to run on my Raspberry Pi. This week I've spent some time setting up the platform for some simple benchmarks.

The TL; DR version is that I was able to observe approximately 170 flows/second! **

Now for the longer story...

To benchmark the system three things had to come together: Helium running on Karaf, the wcbench utility running on a separate host, and the overall platform being stable enough to execute enough tests to produce statistically relevant data sets.

In my previous post, I outlined how I was able to tweak Helium into running on RPi. To these alterations I added one more tuning parameter: Xss200k. This reduces the startup size of Java threads, making it easier on the JVM and RPi to provide resources. See my github repo for sample Helium scripts.

Setting up the wcbench tool required standing up a host machine, in my case a Fedora Core 20 VM, and configuring its scripts to point to the RPi. Given the RPi's limit resources I tuned the number of switches, mac addresses, and time to run each test down. See my github repo for sample wcbench configuration.

Ready for testing!

Using the test tool would also require installing a few packages onto the Helium deployment. Through trial and error I discovered it was easiest on the RPi if I installed as many smaller features first before attempting top level targets. See 'Setting up Helium for testing' on my github page main read me file.

Unfortunately this initial setup resulted in a failure to collect useful output :(

The trouble was that even though OpenDaylight Helium could boot up and install packages, it couldn't handle the heavy traffic. The system would freeze up, becoming unresponsive. Memory was completely consumed.

Looking at the base system's 700MHz CPU, and memory specs, something would need to change:

This is where the above '**' comes into play. Up to now I've been using a relatively default vanilla installation of Rasbian on the RPi. To obtain useful benchmark results I would have to provide OpenDaylight with more resources to do its thing. Hence I proceeded to the net, and found performance tweaks for RPi. See my github repo for a full list of the configuration changes I made to my RPi.

The final result was a base system with a 850MHz CPU and the following memory specs:

An extra 40MB free ram, and 255MB of swap space!

With my platform tweaks applied, the system booted up much quicker. I executed the loop_wcbench script with the -l and -t5 option for 25 data sets. See results.csv for full results. Unfortunately, the scripts were not able to dial into the instance to grab detailed system information, so I made the adjacent screen capture of the top command during the test.

After running the benchmark tool with 10 simulated switches and 10 MAC addresses, I attempted increasing the load to 100 switches and 100 MAC addresses - the system promptly froze.

There are probably more system tweaks that could be applied to the RPi to free up more memory, and  additional JVM tuning to make operating Helium more performant. I've published all of my tunings, and configurations to a git repo so that others may use my experiments as a starting point.

If you do decide to dive in and try out Helium on RPi, please let us all know about your results in the comments below.

OpenDaylight Helium Release on Raspberry Pi!

The question we've all been asking has finally been answered - yes, OpenDaylight Helium Release can run on Raspberry Pi!

If you're not familiar with Raspberry Pi it's a small, inexpensive computer which at its heart runs a 700 MHz Low Power ARM1176JZ-F Applications ProcessorCPU with 512MB ram.

My build utilizes a Debian based Linux distro, Raspbian, and an Arm compatible Oracle JDK build.

Out of the box I had to make three small modifications to the scripts found in the OpenDaylight distributions' bin folder.

• The set environment script (setenv) had JVM parameters for PermGen and Max Memory set higher than the RPi could support. I set these values to 340MB for PermGen, and 400MB for Max Memory. These are not optimal settings, but will work well enough to get things going.
• The karaf script used to start the container also required a memory setting to be adjusted from 512m down to 340m (just grep the script code for 512 and replace). 
• Finally, in the karaf script the JVM "server" flag was not recognized by the Java installation on my RPi. Editing this out of the script allowed Karaf to start up happily.

w00t!

Once I had the OpenDaylight console banner displayed, I executed the console info command to see system environment information:

Now we're cooking with fire! Time to start making some toast...

Awesome! The Apache Karaf container environment OpenDaylight Helium uses appears to be stable. Next step was to try to deploy the OpenDaylight Toaster demo... which failed on not being able to find SSHD 0.12.0 :(

This turned out to be a minor inconvenience. A simple invocation of install -s mvn:org.apache.sshd/apache-sshd/0.12.0, and an edit to ODL_HOME/etc/config.properties to include a JDK 1.8 entry for osgi.ee and the ODL Toaster was installable, and running!

Helium on RPi can make toast!

I used jconsole to manipulate the makeToast and clearToastsMade operations exposed via JMX on the controller to verify their correct operation - using log:display I could view each operation's logging events.

RPi CPU Stats

This is an very impressive improvement over the previous OpenDaylight release which, despite my best efforts, I could not get to boot on RPi to a stable console out of the box. It's amazing how well OpenDaylight has taken to scaling from small PCs like the RPi to serious big iron servers we find in datacenters.

Its important to note that for my ODL on RPi test I've only installed the base ODL Helium distribution, SSHD 0.12.0, and performed a feature install on odl-toaster target. The Karaf based features mechanism took care of downloading, installing, and configuring all the plumbing required to get the demo running.

So, now that we know we can get OpenDaylight Helium running on an RPi, what should we program it to do? I'm looking forward to reading your ideas in the comments below :)

Please vote for our talks to be accepted to ConFoo 2015

Heath Kesler and I have submitted talks to ConFoo 2015 for consideration, and we need your votes to help make our proposals get noticed!

Easy RESTful services using Apache Camel and CXF

"In this session attendees will learn how simple RESTful services are to implement and interact with using Camel Integration Engine and the CXF Framework. In as little as a few lines of code, a developer can implement a fully functioning, testable CXFRS endpoint. Learn how to implement thread pooling and asynchronous processing on the service while maintaining a flexible and scalable architecture that can grow with demand."

How OpenDaylight learned to breathe in Apache Karaf.

How OpenDaylight learned to breathe in Apache Karaf. OpenDaylight is an open platform for network programming to enable SDN and NFV. Apache Karaf is a small, light weight server side OSGi runtime environment. In this session I will share our experiences in introducing OpenDaylight to Apache Karaf, and the subsequent large scale, multi-project migration to Apache Karaf’s OSGi environment.

Apache Karaf in the Trenches

Apache Karaf is a small, light weight server side OSGi runtime environment, that has been gaining in popularity in the Open Source community. The shift towards using OSGi has introduced new challenges to architects, developers, and operators in building and deploying these systems. In this session I will share some of my experiences in developing, deploying, and maintaining Karaf based software systems among Global 2000 companies.

Please register and vote on the ConFoo 2015 website.

Apache Karaf 2.3.6 Released!

The Apache Karaf team are pleased to announce the availability of Apache Karaf 2.3.6!

This is an update patch for Apache Karaf 2.3.5, containing many bug fixes, dependency updates, and improvements. New feature highlights include: Automatically create data sources from configuration files, add backing engine serivce for PublickeyLoginModule, add a flag to bin/client so that it can read private key from a specified location, and Provide shell:env command.

It is recommended that you review our release notes before updating your deployments.

The Karaf 2.3.6 RC process was enjoyed with Placido Chianti 2011.

As always, I'd like to thank the community for coming together to help make the Karaf release process one of the best in all of open source software.


OpenDaylight Update:

The OpenDaylight port to Apache Karaf has been proceeding steadily towards a stable ecosystem wide build. Next week's Techicial Work Stream conference call will include an End-To-End Hands-on walk-through of ODL Karaf Features. I've copied in below the TWS meeting objectives as outlined by Madhu Venugopal:

"Due to multiple queries from various projects, we are organizing an end-to-end hands on walk-through on Karaf. This will cover all the way from

1. Writing the Karaf feature file on any project
2. Building the .kar
3. Loading the Karaf Container
4. Installing the built feature and Validation.

Time permitting, we will also be covering the Docker based Integration Testing & Sonar coverage. Mathieu Lemay & myself will be leading these topics.
We request at-least 1 representative from each of the projects participating in Helium to attend this session."

I strongly recommend all OpenDaylight developers and users attend this meeting, also any interested Apache Karaf community members as this is a great opportunity to see Karaf's many benefits being realized in a large scale project. Please see the meeting page for conference call details.

Up coming Apache Karaf releases, OpenDaylight update, and my acceptance into the ASF.

We have two release candidates in progress: Apache Karaf 2.3.6 and 3.0.2.

Our first target is the Karaf 2.3.x line. 

Highlights in this RC include; Better management of RMI/JMX ports for pax-exam integration tests, more robust JDBC against broken data sources,  several improvements to JAAS, among many other refinements and bug fixes.

Wine pairing for this RC is Placido Chianti 2011

Our second release focus is the Karaf 3.0.x line.

Highlights in this RC include; RBAC recognize group configuration when use Publickey to login, more robust JDBC against broken data sources, add support for role based JMX authorization, Mbean Server RBAC guard should support wildcard role, among many other refinements and bug fixes.

Wine pairing for this RC will be Gabbiano Chianti 2010.

We can't wait to get these release candidates out to the community for validation and voting. Please keep an eye on our dev list for when the votes go live.

Any updates on OpenDaylight on Apache Karaf?

Lots of initial features sets have been defined, and commands adapted to work in their new environment. It's still a work in progress, but coming along nicely :)

There will be an open demo on Monday June 2nd, 2014 on the OpenDaylight Technical Work Stream call: https://wiki.opendaylight.org/view/Tech_Work_Stream:Main

Topics on the call include:

• Karaf in OpenDaylight & Hands-On demo on developing ODL components for Karaf Runtime - Mathieu Lemay (40 mins) 
• Multi-Tenancy Discussion - David Lenrow (20 mins)

Membership with the Apache Software Foundation!

On a personal note, I've recently accepted an invitation to become a member of the Apache Software Foundation. I can not express how honoured I am to be given the opportunity to participate with the Apache Community on this level. I'm looking forward to years to come, helping to grow our communities, and contribute more to their success.

OpenDaylight on Apache Karaf!

I'm placing aside a bottle of
Colores Del Sol, Malbec, Mendoza, Argentina, 2010
for when the first OpenDaylight Apache Karaf
Release becomes available.

The OpenDaylight Apache Karaf distribution is coming soon!

What is OpenDaylight?

OpenDaylight is an open platform for network programmability to enable Software-Defined Networks and create a solid foundation for Network Functions Virtualization for networks at any size and scale.

What is the OpenDaylight Apache Karaf Distribution?

To put it succinctly, it's an OpenDaylight distribution in which key ODL components are deployed on top of Apache Karaf. The base Karaf container has been configured to support OpenDaylight's specific needs, including Equinox core, key component features available by default, custom commands, and branding!

Why Apache Karaf?

OpenDaylight has been an OSGi based project from the start, now it's ready to improve its base environment experience via Karaf. Specifically, some of the key reasons for the distribution are:

• Ease of deployment.
• Friendly console CLI.
• Superior Pax Exam integration, and
• Awesome, supportive community ;)

Stay tuned for updates as the OpenDaylight community moves towards the debuting of the new distribution.