This article is part of a small series describing the use of several new open source components in my company’s new product, Kizby.
Kizby is built using Tycho. There has been quite a bit of buzz about Tycho recently, with several recent blog posts providing worked (and hopefully maintained) examples of using Tycho. And it’s well deserved: our experiences with Tycho have been very positive: the community is responsive, Tycho’s functionality is very capable, and (more importantly) diagnosing build issues is straightforward.
But getting started was a bit bewildering due to having to learn Maven coupled with the vagaries of building Eclipse-based products. And from the questions from newcomers to the Tycho mailing lists, I can see that I was not the only bewildered newcomer. Hence this post.
[For those at EclipseCon, I recommend you attend Monday’s tutorial on building with Tycho.]
When we began work in earnest on Kizby, we quickly reached the point where we needed to have a build and deployment process. As PDE/Build, the granddaddy of the various Eclipse build systems, is highlighted in many Eclipse reference books, it seemed a logical choice. The experience of coaxing forth a build from PDE/Build is perhaps the closest that I will ever get to understanding the pain and joy of giving birth.
But after a month or so, we hit a wall with PDE/Build: we wanted to build a set of four related products, whereas PDE/Build was designed to produce a single product. Although I could have jerry-rigged something with make(1) and PDE/Build to repeatedly invoke the build for each of the products, I knew there must be a better way.
And so I surveyed the field and discovered a bewildering set of technologies bearing bewildering acronyms or names. Even restricting consideration to just those tools for building Eclipse-based products still left a headache. p2, I learned, is not a build system. Athena Dash is a set of scripts built around PDE/Build, but is no longer a recommended approach. Buckminister has a whack of documentation but seems oriented towards build-from-the-IDE, whereas I wanted a headless build technology to future-proof myself for continuous integration [aside: Buckminster, I discovered later, can be used headless]. b3 is under development, and currently seems more geared towards manipulating p2 repositories. And there was a (then) newer contender, the Maven-based Tycho.
I had actually come across Tycho before I first embarked on PDE/Build. But my few exposures to Maven (shortly after the new millenium) had left me scarred. Maven projects had strange directory schemes, the seemingly essential pom.xml had little content, and the documentation was opaque (I think you had to understand Maven to understand the documentation). For a tool that was positioned as a do-everything solution that would build, test, assemble, and deploy a system, I couldn’t even figure out how to get started. After some desperate web searches, I figured out the minimum to make a project usable within Eclipse (with the magical mvn eclipse:eclipse), I ran away.
But I was now smarter — and desperate. And fortunately Maven has matured, and there are now some great tutorials and free e-books available. Building using Tycho turned out to be a bit simpler than I expected. And Maven actually does build, test, assemble, and deploy too.
Maven and Tycho
Tycho is actually a set of extensions (unfortunately called “plugins” too) for Maven to build Eclipse/OSGi-based components. Key to using Tycho is to understand Maven. And hence the reason for this blog entry: the existing Tycho docs (which are admittedly a bit minimal) assume prior knowledge of Maven, of which I had little.
A Little Bit About Maven
Maven is often described as being opinionated. You provide a declarative specification of what you want built, and Maven decides how it will happen. For those used to procedural build systems, it requires a bit of an adjustment.
The declarative specification is placed in a file named pom.xml, called the Project Object Model or POM. In Maven, a directory with a pom.xml is called a project. The pom.xml describes the purpose of the project, called the packaging type (e.g., to produce a jar file, or a war file, or a bundle), and includes other information such as the project’s dependencies, etc.
What’s nifty is that a project inherits information, both from its parent project (usually in the parent directory, in which case this project is a subproject), and from Maven’s own super POM. So providing your project conforms to Maven’s opinionated directory layouts, you rarely have to actually write anything beyond some minimal XML boilerplate: the pom.xml only serves to inform Maven of changes from the defaults.
For a project with a jar packaging, Maven automatically knows how to (1) generate any necessary resources, (2) compile the java files in src/main/java, (3) test using the unit tests in src/test/java, (4) jar up the class files, etc. Maven only needs to know a few details such as the project’s name and any dependencies. And hence my confusion from years ago upon seeing a seemingly near-empty pom.xml file.
More on Projects
With Maven, each project builds some artifact. A project may have many subprojects, typically organized as subdirectories, and each of which contributes to the creation of the project’s artefact. The type of the artifact is described by the POM’s packaging directive (e.g., a Java project is generally a jar or war; an Eclipse plugin with Tycho is eclipse-plugin).
An artifact is identified by a :: tuple, called the coordinates, which are provided in the pom.xml. The artifactId is typically taken to correspond to the directory name, and the groupId the logical purpose. In building Kizby, most projects have groupId=ca.mt.kizby and the artifactId is the bundle symbolic id like ca.mt.kizby.core. (Note: there’s actually 2 other coordinates, the packaging and classifier, but they don’t seem to be talked about much.)
Maven actually doesn’t do much on its own, but instead delegates to various plugins (similar in concept, but different from Eclipse plug-ins). These plugins actually do the stuff that you would stick in a make(1) directive or a sequence of Ant tasks. Maven uses the packaging type to determine a set of phases, called a lifecycle, for building and deploying a project of that type, and then calls out to various plugins as it progresses through the different phases. The lifecycle is similar to typical all target found in most Makefiles.
all: clean build zip deploy
And in Maven/Tycho, a make all corresponds to:
$ mvn clean deploy
The compilation and zipping is done as part of the deploy.
For more information on Maven, I highly recommend you look at Sonatype’s free book, Maven By Example.
Back to Tycho
Tycho is actually a set of Maven plugins for compiling, resolving, and provisioning using OSGi. It provides a different set of packaging types for building bundles, features, and tests (e.g., eclipse-plugin, eclipse-feature, eclipse-test-plugin). Whereas the traditional Maven Java plugins pull configuration settings such as dependencies, compiler versions, etc. from the pom.xml, Tycho’s plugins use the information encoded in the OSGi/Eclipse manifests (e.g., META-INF/MANIFEST.MF); the traditional approach is often called pom-first vs Tycho’s manifest-first approach.
Tycho also provides a bridge between Maven coordinates and OSGi identifiers. Tycho usually assumes that the artifactId should be the bundle’s symbolic name. The groupId doesn’t really have an analogue in the OSGi world, so I use it as a logical grouping. The version should match the bundle or feature version, though Maven uses a -SNAPSHOT suffix instead of .qualifier.
So a complete bundle’s pom.xml looks like:
That’s it! Seriously! Remember that a project inherits from its parent, so this bundle will have the same groupId. Features are very similar. Of course you need to provide the parent’s definition, which has some other stuff to define the Tycho plugin versions to be used and repository definitions for resolving bundles (e.g., the p2 repository at http://download.eclipse.org/releases/helios). But that’s pretty straight forward and covered in other tutorials.
Go Forth and Build
At this point, you should hopefully have a very general overview of Maven and should be ready to read a bit from Maven by Example and work through Chris Aniszczyk’s example.
Tricks or Gotchas
There are many questions that come up on the Tycho mailing list, but most are PDE- or p2-related rather than Tycho-related. Unfortunately getting to the position where you know the difference is painful. Here are the things I’ve found out the hard way:
- The OSGi resolver and p2 resolver are not the same.
You may receive resolution errors in building that do not occur when running in Eclipse.
- Startups are different.
When run from the debugger, plugins are automatically started. They aren’t in a product.
- Creating .product.
There are some bugs with the p2 publisher that prevent publishing products on different platforms. There are workarounds.
I don’t yet use Maven to control deployment to servers, and instead use some shell scripts. But there are some nice examples out there though.
I’ve started documenting these issues on the PDE FAQ and the Tycho FAQ; feel free to add your own.