The Nuclear Bunny Blog

In Part 1 of this series we discussed getting an initial Zenoss environment checked out and running on a Mac OS X or Ubuntu system. In Part 2 we discussed how to configure Eclipse to use the Zenoss source. In this part, we’ll discuss how to handle day-to-day operations such as branch management and working with multiple versions.

Branch-Based Development

At Zenoss we do all development, including bug fixes and small maintenance tasks, in a private development branch within the subversion repository. This allows us to independently work-on changes, check-them into the repository for safe-keeping, and then perform code reviews with team members without having to share files or using pastebin style tools (even though we do both at times).

1. Create a branch within your user’s sandbox. In this example, I’ve decided to name the sandbox new-widget to identify what I’m working on. If I were fixing a defect, I’d use the defect number from the defect tracking system. Create the branch by copying the trunk folder to the sandbox branch. In Subversion this is a low-overhead operation and doesn’t actually copy files.
svn copy http://dev.zenoss.org/svn/trunk http://dev.zenoss.org/svn/sandboxen/cgibbons/new-widget -m " * Copying trunk to sandbox branch."

2. Switch your working directory to use the new sandbox branch. You can do this either from the command-line or using Eclipse. From the command line, you’d do the following:
cd $HOME/zenoss/core
svn switch http://dev.zenoss.org/svn/sandboxen/cgibbons/new-widget
From within Eclipse, secondary-click on the core project and choose Switch to Another Branch/Tag/Revision… option from the Team menu. On the dialog that appears, enter in the sandbox URL. After switching, your Eclipse will show the new location next to the core project item.
   

Once your development environment has been switched, you can make changes and commit to the Subversion repository as desired. If you’re unsure if you are in the right branch, you can always use the svn info command to see which directory is being used.

Merging Branches

Once you have completed changes in a branch and have had them reviewed by a peer, it is time to merge them into trunk (or another branch, if using a maintenance release). Merging can be tricky, but a consistent process can make it much easier to handle.

1. Change your working directory to a checked-out and clean version of the branch you want to merge into. For example, I keep a $HOME/zenoss/clean-trunk directory that I never make changes to, except for merging.
2. Determine the base working revision of your working branch. There are a variety of ways to do this, but one of the best is to view the revision log graph within the Trac system directly. For example, for the branch discussed above we can browse to http://dev.zenoss.org/trac/log/sandboxen/cgibbons/new-widget/ and see that revision 15513 is the base.
3. Perform a dry-run on the merge to get a general idea of what the changes into the branch will be. You should see your expected changes, plus any conflicts from changes to the other branch while you have been working in the sandbox branch.
svn merge --dry-run --revision 15513:HEAD http://dev.zenoss.org/svn/sandboxen/cgibbons/new-widget .

4. If the merge results look satisfactory, rerun the command without the dry-run argument.
5. Look at the final merge results using svn status and svn diff, and once you’re ready, issue an svn commit.

Multiple Branches and Zenoss Configuration

As you switch between branches you will often render your Zenoss configuration useless.  Resetting your database after each branch switch is usually a good practice, and being able to quickly recreate any test data you may need makes this process less painful.

After switching a branch, my process is usually the following:

1. Shutdown zenoss and restart only zeo.
zenoss stop
zeoctl start
2. Run the zenwipe script from the inst source directory.
$HOME/zenoss/inst/zenwipe.sh --no-prompt
3. Run zenmigrate to install any database changes available within the current branch.

Depending upon the task at hand, I may install additional ZenPacks and add new devices through the command-line if those are needed. Helper scripts, such as install-windows.sh to install of the Windows ZenPacks and create several local test devices in the instance, are useful tools to have for your typical configurations.

In Part 1 of this series we discussed getting an initial Zenoss environment checked out and running on a Mac OS X or Ubuntu system. In this part, we’ll discuss configuring Eclipse to use this environment.

Mac OS X Prerequisites

1. Install Eclipse Classic 3.5.x Mac Cocoa 32-bit.

Ubuntu Prerequisites

1. Install the Sun Java JDK. Why? Eclipse is a Java application and requires a full-fledged Java environment to run properly.
sudo apt-get -y install sun-java6-jdk

2. Install Eclipse Classic 3.5.x Linux 32-bit.

Eclipse Configuration

1. In part one, we created a Zenoss root project directory of $HOME/zenoss – use that directory, or the one you created, for the rest of these steps.
2. Launch Eclipse and configure it to use the Zenoss root project directory. Why? Eclipse needs a workspace directory to keep track of configuration settings for a group of related projects. By placing the workspace directory inside of the Zenoss root workspace, we can separate the requirements for a Zenoss workspace from other projects you may be using.
3. Install the PyDev plug-in for Eclipse. Why? PyDev provides Python language support to Eclipse.
   1. Go to the Help Menu and choose Install New Software.
   2. In the Available Software dialog, choose Add and enter in http://pydev.org/updates/ as the Location and close the dialog. Eclipse will return to the Available Software dialog.
   3. Software matching PyDev will appear in the dialog; pick the PyDev option but do not pick PyDev Mylyn Integration. Click Next and install the plug-in.

   

4. Install the Subclipse plug-in. Why? Subclipse allows you to work with the Subversion version control system directly from within Eclipse.
   1. In the Available Software dialog add the Subclipse plug-in update site:

      http://subclipse.tigris.org/update_1.6.x

   2. In the Available Software dialog, choose the Subclipse, Subversion Client Adapter, Subversion JavaHL Native Library Adapter and the Subversion Revision Graph items.
5. Go to the Window menu, choose Open Perspective and then SVN Repository Exploring (you may have to choose Other… to see this option).
6. Choose the New Repository Location button in the SVN Repositories panel. Add the Zenoss SVN site at http://dev.zenoss.org/svn
7. Open the SVN repository and select the trunk folder. Secondary-click the folder and choose the Checkout… option. Be sure to change the Depth option to be Only this item so that we don’t check out any of the sub-folders of trunk just yet (many of the folders within trunk are not needed for development, but we want to keep the directory structure). In the next dialog you will be asked to choose a Project Wizard. Open the Pydev tree item and select the Pydev Project option.
8. Create the project in the Pydev Project dialog:
   1. Use core as the project name and use the default location. This should create a core project at $HOME/zenoss/core.
   2. Make sure the Python project type is selected.
   3. Select 2.4 as the Python Grammar version.
   4. Uncheck the Create default ‘src’ folder... option.
   5. Click the Click here to configure an interpreter not listed… option in order to add the python interpreter built by the Zenoss installation scripts.
      1. In the Preferences Dialog, choose the Interpreter – Python item underneath Pydev and select the New… button to add a new Python interpreter.
      2. Browse to the $ZENHOME/bin directory and choose the python2.4 executable from that directory.
      3. Name the interpreter python-2.4, zenoss-python or some other variant.

      After the new Python interpreter has been added, return to the Pydev Project dialog and choose that interpreter from the list and then click Finish to create the project.

   6. Update the core folder from the command-line SVN client so you can selectively pull the sub-folders of the core project and not all of them:
cd $HOME/zenoss/core
svn update Products

   7. Move the Products directory the installation checked out and create a symbolic link to the one updated above. Why? This allows the Products source tree to be worked on from Eclipse and then also used by the Zenoss run-time.
cd $ZENHOME
mv Products Products.old
ln -s $HOME/zenoss/core/Products Products
   8. Return to Eclipse and choose the Refresh option from the File menu so that Eclipse notices the updated directory and builds necessary dependencies.
   9. Secondary-click on the core project folder in Eclipse and choose Properties. Choose the PyDev – PYTHONPATH item and add source folders so PyDev can reference the project from within itself.
      1. In the Source Folders tab choose the Add source folder button and pick the root core folder from the provided list.
      2. In the External Libraries tab choose the Add source folder button and choose the $ZENHOME/lib/python directory.

   At this point, your Eclipse project should allow you to navigate between dependencies within the Zenoss project. You can also simultaneously switch between using the Team feature within Eclipse to update and manage Subversion or do so using the command-line svn client.

   Next…

   In Part 3 of this series, we’ll discuss how to manage day-to-day activities such as creating sandbox branches for changes and dealing with multiple versions are done from within the same environment.

Over the past 18 months the developers at Zenoss have used a variety of development environments and methods to productively work with Zenoss, but there are a lot of best practices that have emerged out of this diversity.

I develop Zenoss primarily on Mac OS X, with some work done on Ubuntu when necessary. I normally use Eclipse as my development editor, although I’ll often just use vim when doing quick changes or bug fixes. In either case, careful setup of the Zenoss environment is key to being able to productively work with both the new development version and with the shipping versions that require maintenance.

The environment you get by default when you check out the source version of Zenoss from the source repository, or from a source tarball, is not necessarily setup in the best way to develop productively using tools like Eclipse.

The rest of this post will document how both my Mac OS X and Ubuntu environments are initially configured so that a working source Zenoss installation is realized.

Mac OS X Prerequisites

These prerequisite instructions assume Mac OS X 10.5 Leopard; 10.6 Snow Leopard will not be able to compile Zenoss’s third-party dependencies, so an additional work-around is required for that platform until Zenoss moves to Python 2.6.

1. Install Xcode. Why? Installs the GNU C/C++ compiler and other command-line development tools needed to build dependencies used by Zenoss.
2. Install Universal Subversion 1.6.x from CollabNet Community.Why? Leopard only comes with Subversion 1.4.x. This version is not compatible with the Subclipse plug-in for Eclipse that will be used later. In order to be able to use both the command-line and Eclipse Subversion versions simultaneously on the same checked our source, the release of subversion should match. This installation will place the Subversion binaries in /opt/subversion and should automatically add it to your PATH.
3. Install MySQL Community Edition 5.1 32-bit. Why? Zenoss needs MySQL for storage of event data.

Ubuntu Prerequisites

These prerequisite instructions assume Ubuntu 9.04 32-bit Desktop Edition. Installing the server edition or one with different options may require additional prerequisites to be installed.

1. Install Subversion. Why? Working with the Zenoss product in source form requires Subversion to access the source repositories (it is also possible to build directly from a source tarball, but this is not discussed here).
   sudo apt-get -y install subversion

2. Install MySQL. Why? Zenoss needs MySQL for storage of event data.
   sudo apt-get -y install mysql-client mysql-server libmysqlclient15-dev

3. Install additional development environment tools. Why? Zenoss third-party dependencies require several binaries to be built from source.
   sudo apt-get -y install patch make vim gcc g++ autoconf
4. Install SNMP support. Why? Zenoss requires SNMP libraries for monitoring, and having a local SNMP agent is useful for testing.
sudo apt-get -y install libsnmp-base snmp snmpd
5. Install Liberation TrueType fonts. Why? Graphs generated by RRDtool will not contain the correct glyphs without this font package.
sudo apt-get -y install ttf-liberation

Environment Configuration

Configuring Eclipse will require determining where you want to work with your Zenoss installation, and installing Eclipse plug-ins to provide the features required for Python and Subversion support.

1. Create your Zenoss root directory:
mkdir $HOME/zenoss

2. Create and run a setup.sh script that will configure needed environment variables for zenoss. This script can be started from your log in profile if desired.
cd $HOME/zenoss
cat > zenoss-config.sh <<EOF
ZENHOME=$HOME/zenoss/zenhome
PYTHONPATH=$ZENHOME/lib/python
PATH=\$ZENHOME/bin:\$PATH
export ZENHOME PYTHONPATH PATH
EOF
chmod +x zenoss-config.sh
. zenoss-config.sh
mkdir $ZENHOME

3. Checkout the Zenoss source installation tree. Why? This tree is used to bootstrap the installation from the source repository and create a running Zenoss installation. We’ll need this before we modify it to fit the needs of our development environment.
svn co http://dev.zenoss.org/svn/trunk/inst inst

4. Run the installation script to checkout, compile, and configure a zenoss environment. If you need to customize your MySQL configuration at all do not use the --no-prompt argument.
cd inst
./install.sh --no-prompt

5. Modify the zensocket file to be setuid root. Why? Some of the Zenoss daemons use a privileged port and making the file owned by root and accessible by the Zenoss user allows the daemons to be run as a non-priviledged user but still use the privileged port.
sudo chown root:`id -g` $ZENHOME/bin/zensocket
sudo chmod 04750 $ZENHOME/bin/zensocket


At this point, you should have running zeo and zope processes and be able to log on to the local Zenoss instance. Your Zenoss root directory should look similar to the following:

Next…

In Part 2 of this series, we’ll download and configure the Eclipse IDE.

I have two different late model (a 2006 and a 2007) GM vehicles with satellite navigation systems. Both are manufactured for GM by Denso, although they clearly have different implementations and features. The one thing they both share, however, is blatant stupidity when calculating routes.

I travel quite a bit between Houston and Austin and there are really only three reasonable ways to go: US-290; I-10 & TX-71; or I-10 & TX-183. Over the years my preference has moved from US-290 to I-10 & TX-71 as traffic along US-290 has increased on the eastern fridges of the Austin area, but the distance and time between the two is very similar and really it depends upon where in Austin you might be headed.

The GPS systems in my cars have three route options when you pick a destination: fastest, shortest and other. In both cars, the route displayed is none of the above three, but instead a long, out-of-the-way route of I-10 to Seguin before heading north on TX-123 to San Marcos, and then heading into Austin on I-35. This option is a whopping 222 miles with an estimated travel time of 3 hours and 39 minutes.

Now, the fastest route is usually not the shortest. Route calculations are supposed to take into account the actual speeds of the road segments involved, and since interstate highways are almost always faster than anything else, there is a natural preference to these roads. In this case, however, the calculation is way off. Taking I-10 to US-71 (via US-59 & TX-8 inside of Houston for those familiar with the area) is only 166 miles and estimated to take 3 hours 29 minutes according to Google. Likewise, taking US-290 is only 168 miles and estimated to take 3 hours 22 minutes.

Of course these time estimates are usually worst case scenario. Driving a few miles per hour over the speed limit I often make my trip in 2 hours and 30 minutes; worst case has been 3 hours and 10 minutes when driving 55 as an experiment (a painful one at that).

What about the shortest route option in the GPS system? That does actually route you onto I-10 & US-71 in this example, but the shortest route calculations are always incredibly stupid, too. They will often take you onto side streets for half a mile or less just to maintain the shortest possible distance. Once, when driving through Memphis, TN, the GM GPS took us on and off the interstate highway three times when driving through town, often for just a couple of blocks, just to maintain the shortest distance. Granted this is a clear example of a GPS system not being there to replace your own intelligence, but if you don’t know the town…

In the early days of consumer-level GPS systems, one of the common problems was having accurate speed data for different road segments. In western states especially, secondary highways often have the same speed limits as interstate highways. In Texas, the past 20 years has seen many of these highways improved so they route around the smaller towns, rather than going through them and stopping at every light. Consequently it is often better to take these roads than the interstates whenever possible. If the GM GPS systems do not have accurate speed data then it can account for some of this behavior, but not all.

On my normal route, once you finally hit TX-71 and the GPS decides you mean it, suddenly the estimated arrival time drops dramatically along with the distance. Is it computing this arrival time based upon your actual speed so far, or based upon the speed data of the road?

Over the years I’ve gone from absolutely wanting a satnav system in the car to wishing mine didn’t. I get much better results using a smart phone with Google Maps to get a general idea of the route to take and then consulting with it now and then if things get tricky. But for the most part, they seem poorly done enough that they just get in the way rather than help. Will the newer generations out now that provide real-time traffic data and have hard-disk based data improve their logic and performance enough that these problems are solved?

One final note for embedded systems developers: you don’t get to break the interactive response time rule of 250ms just because you are writing code for an embedded system.

The IcyBee project is a client for the ICB chat network with a graphical user interface and written in the Java language for cross-platform portability. The current version of the software has several years of stability behind it and few user requests, but there is always more work to do. The project is currently hosted on SourceForge and all of the source is already available.

I’m looking for a maintainer that will take the reins of the project from me and give the project some much needed love as time and platforms march on. If interested, drop a comment on this post and we can discuss the possibility.