I was in Santa Clara last week at SD West where I attended the Jolt award ceremonies. Continuous Integration walked away a winner of the best technical book of 2007. I was in shock to the point where I didn’t know we were supposed to say anything or pose for a picture when accepting the award. I made up a few things off the top of my head, but speaking extemporaneously is not a specialty of mine, so I was a man of few words in my “acceptance speech”. In any case, it was a great honor to win the Jolt award. As I said in the acknowledgments of the book, any effort like this is never done alone. There were contributors, reviewers, a production team, editors, marketing and probably more than I even realize to put together a book like this. The book seems to be out of stock in many places right now, which is unfortunate. However, there are two places where I know the book is currently available:

• BookPool
• InformIT

In my next post, I’ll upload some pictures from the Jolt awards. For now, see TestEarly.com

As of December 1st, the Microsoft Store in Redmond will for the first time start carrying books from publishers other than Microsoft Press. My book, Continuous Integration: Improving Software Quality and Reducing Risk, is one of those books. For the time being, the book will only be available at the physical store, not at the online Microsoft store.

One of my goals from the beginning in writing the book was to provide examples in many different languages/platforms. In the book, we cover C#/NAnt examples, .NET static analysis tools along with Java and Ruby examples. There’s something for everyone. CI, as a practice, is independent of any particular tool. That said, tools greatly support the practice, given the vast array of code inspection, build, and CI servers on the market. So, if you’re a Microsoftie and are curious about CI or are currently using the daily build practice, purchase a copy, or three, for yourself at the Redmond store.

Clint Shank describes an interesting game for keeping the integration build “in the green”. It’s inspired by a story in Alistair Cockburn’s Agile Software Development book. This is a very interesting way to use positive and negative reinforcement to ensuring a healthy build. Clint has some other articles on similar topics here.

I’ve added a new video that demonstrates an example of the Automate Database Integration practice described in chapter 5 of the Continuous Integration book.

The examples use Ant with HSQLDB. The first example shows a script that creates tables using an in-memory HSQLDB database as show below. By executing the create-tables.sql script, you can drop database/tables and recreate the database (DDL) through the build process.

<target name=”db:create-tables”>
<sql driver=”${db.driver}”
url=”${db.url}”
userid=”${db.username}”
password=”${db.password}”
classpathref=”db.lib.path” mce_href=”db.lib.path”
delimiter=”;”>
<fileset file=”${database.dir}/create-tables.sql”/>
</sql>
</target>

The next example shows the insertion of test data (DML) that is necessary to run automated component-level and functional tests.

<target name=”db:insert-data” depends=”filter-sql-files”>
<sql
driver=”${db.driver}”
url=”${db.url}”
userid=”${db.username}”
password=”${db.password}”
classpathref=”db.lib.path” mce_href=”db.lib.path”
xsrc=”${filtered.sql.dir}/insert-data.sql” mce_src=”${filtered.sql.dir}/insert-data.sql” />
</target>

What the…?! There, I said it. Whew…now I feel better. Maybe this was a provocative blog title and is better said “Effective CI has little to do with your CI server”. Better?

Last year, in an Automation for the people article, I wrote:

“CruiseControl is the granddaddy of CI servers. It’s been available for over five years, and in many ways, the CruiseControl server has become synonymous with the practice of Continuous Integration.”

That a tool has become synonymous with a practice is actually a good and a bad thing. It’s good because there’s a name and an actual tangible thing that we can attach to this practice called “Continuous Integration”.

To be clear, a CI server…ummm, lemme see, it…it…builds your software and then notifies you when it’s done.

However, having a tool synonymous with a practice can be bad thing because people can get carried away with the various bells and whistles different CI servers provide. Don’t get me wrong, I’m a big fan of CruiseControl and other CI servers, but let’s not get lulled into thinking that the tool is what provides the practice. Vendors may want us to think this, but we’re smarter than this, right?

I will have done a disservice to readers of Continuous Integration if you were to believe that the most important facet of CI is the CI server itself. It just might be the most unimportant part of the practice of CI. In fact, as I indicate in the book, some high-performing teams don’t even use an automated CI server for their integration builds (they perform a manual synchronous integration build - which uses an automated build). If someone thinks that the difference between practicing effective and ineffective CI is tweaking the automated CI server to be “just right”, beware…you may be listening to either a theorist or a misguided practitioner. The most important part of the effective practice of CI is the development team’s daily practices (committing code often, keeping integration builds in the green and so on) and the fully automated build from a single source repository. Of course, there are a number of other practices (that we discuss in the book), but “Using a CI server ” is far down the list of importance.

Having actually implemented fully-functioning CI systems (not just the server) on many projects and platforms, I’ve found that installing and configuring the CI server (thanks to many of the great CI servers on the market) is typically one of the more trivial parts of the implementation.

Interestingly, I have found that when teams hear that a CI “server” is being installed, it can help instill or even encourage people to begin employing beneficial development practices that work well in a CI environment, but unless the team culture changes, it’s usually a short-lived gain.

I’ve included a full example of a CruiseControl configuration and a fully automated build (using Ant) with a small working Web application at IntegrateButton.com. Every week, I will add additional videos and examples to the site using different tools (e.g. Hudson in the upcoming weeks) and environments as this is a better and more timely medium than a book to share this information.

Just remember, don’t be the fool with the tool. You can employ CI with or without a CI server.

Practice 5 of the book (which is the fourth practice in chapter 2) is Write Automated Developer Tests. It’s a high-level practice that covers:

• Writing an automated test using an xUnit-based framework such as JUnit or NUnit
• Committing these tests to a version control repository
• Running these automated tests as part of an integration build with every checkin (using Continuous Integration)

This video example demonstrates using the JUnit test framework with Eclipse.

Videocast demonstrating the first of the 46 practices in the book, Build Software with Every Change. This five-minute video uses CruiseControl, Ant and Subversion to demonstrate the processes involved in this important practice. The video has sound, so “turn them headphones up! Turn ‘em up. Turn them s#$%’s up!” (I will send a free copy of the Continuous Integration book to the first person to name that reference.)

In a recent post of the future of Continuous Integration , I pondered how the broken build can affect your development team and offered potential solutions. My next topic is what many consider to be the bane of CI, Long-Running Builds. Again, I don’t think there’s a silver bullet answer to this conundrum. But, depending on how the problem manifests, there are some solutions. I spend quite a bit of time in chapter 4 discussing long-running builds, so this is a quick synopsis.

The Problem

• In the context of CI, long-running builds can slow developers from moving onto their next task. For instance, on Agile projects, many employ the rule that you don’t work on your next development task until the integration build is successful

How Long is too Long?

• Well, it depends! Kent Beck suggests in Extreme Programming Explained, 2nd edition, that an (integration build) shouldn’t take any more than ten minutes.

Symptoms

• The obvious: developers complaining about how long it takes for the builds to run
• Developers check in their code less often (making integrations more painful when they do happen)
• People sitting around waiting for the integration build to complete

Possible Solutions

First, as I describe in the book , it’s useful to know what the actual problem is. From my experience, I’ve found that long-running tests are often the cause for a “long-running build”.

• Add some horsepower - Increase your CPU speed, disk speed, and/or your RAM. This is often a cost-effective and quick solution to the problem.
• Stage your Builds - Run a quick commit build that consists of compilation, fast-running automated tests (i.e. unit tests without heavy dependencies), and deployment. After this commit build succeeds, run a slower-running tests, full database integration, inspections, and installation (depending on your build)
• Categorize your Tests - Segment your unit, component, system, functional, etc. tests so that you can run them at different frequencies
• Run Tasks in Parallel - Run certain build tasks in parallel - such as tests. This can only work if the result status of one task is mutually exclusive to another

As I mentioned, this is only a high-level view on a difficult problem. There’s a lot more detail in the book addressing this issue. Depending on the situation, there are simpler and more difficult solutions than others.

The problem with using an IDE as the sole means to build your software (i.e. on your integration build machine) is that it’s unlikely your IDE will be deployed into production (or at least it shouldn’t be). One of the themes I visit in chapter 2 of the book is that of reducing assumptions (risks or whatever you want to call them). Many of the IDEs wrap everything but the kitchen sink into them (I won’t name any names), so if you assume that the particular files and configuration provided by the IDE are there, you are setting yourself up for problems. If you can’t script your build, be ready to debug problems right before you’re ready to release into production…or worse, when your users are using the software. As a developer, use your IDE to execute your automated build. However, your automated build should have no dependencies with your IDE.

A client I worked with recently had a very astute characterization of Continuous Integration. I will paraphrase it as “CI is a reflection of reality”. He was referring to the immediate feedback you receive from a successful or failed integration build. Using a single source (your version control repository), you’re able to quickly determine if any of your recent changes “played well with others”. Not tomorrow, not two weeks from now, but immediately after a change is made. It reminds me of something Bill Cosby used to say in one of his stand-up routines when someone mentioned they liked using cocaine because it “…intensifies your personality”. Cosby’s response was “Yes, but what if you’re an a**hole?”. CI provides you with a mirror of your software under development. It’s not the fault of CI if it comes back and tells you your code is not working as intended. If fact, you should embrace it before it tells you it’s ugly.