|
 |
| Wed, Jan 07th | home | browse | articles | contact | chat | submit | faq | newsletter | about | stats | scoop | 00:23 UTC |
|
|
login « register « recover password « |
|
| [Article] | add comment | [Article] |
|
|
|
So, you want to write software? Don't forget that you'll need to build or package it, test it, fix some stuff, test it again, and ultimately release it... somehow. The "somehow" is the art and science of Build and Release Management. Copyright notice: All reader-contributed material on freshmeat.net is the property and responsibility of its author; for reprint rights, please contact the author directly. Let's get some terms straight. Building software, as any 15-year-old geek knows, is the generic term applied to the compilation or aggregation of sources into a usable utility or application. Sources needn't be compiled (built) in the classic sense, which relies upon a compiler linking files together; modern systems may consist entirely of interpreted source files (such as those for Perl or PHP). Still, putting everything together in a usable format may be considered "building". Releasing software refers to the process of providing some named (or otherwise uniquely identified) files to others for use. The others may be your department at work, your classmates, or The World. Managing the release means you know:
Why is any of this important? I mean, if it compiles, SHIP IT! Right? Wrong. The larger and more complex the software project becomes, the greater the need for an extremely well-managed Build and Release function. What characterizes a well-managed Build and Release function? Several features are present, some quite basic. You may even be doing some of these things on your project. If you're only doing one -- or none at all -- walk away from the keyboard now before you hurt yourself.
Overnight buildsThis sounds so basic, but I've seen many really big shops that weren't doing this. Building the software overnight accomplishes two primary objectives:
These may sound like the same thing, but they aren't. You expect to see code-wise integration as the project matures; the overnight build confirms you have it. You do NOT expect to see code-wise disintegration (that old "it built yesterday, but not today" problem), and thus want to know immediately when it is encountered. Engineers should be the only ones who see legitimate code-wise disintegration. They should catch it during their unit tests and fix it prior to committing to the project. By running overnight builds, you will never have more than 24 hours of source code changes to review and/or backout if the build breaks. Overnight builds also give your QA department something to throw test cases at, and to target for specific bug regressions. If you don't establish overnight builds very early on in your project, your QA team will suffer schedule slippage late in the project. This is almost guaranteed. Competent Build NumberingIf you're going to build the project every night (and more often, some days), you will need to establish a build numbering system before you do much of anything else, along with some rules for build publication. Build publication rules refer to what builds get published for QA consumption and where. This usually just means a special directory structure on a specific machine. You should determine this up front, start using it, and not change it. This way, throughout the entire project cycle, any interested party can quickly look at any build. Use a sequential numbering scheme that is flexible enough to provide specific, built-in meaning in the number itself. For example:
This build "number" allows for visible identification of its contents. While the exact numbering scheme I use for Freepository isn't important, the fact that I can look at this number and know the build contains:
is very useful. Ideally, come up with a numbering scheme that is easily incremented via automated build scripts. The last thing you want to do is remember what build number to use, and then screw things up by accidentally reusing a number or skipping a number. (But hey, I've never done that...) Automate, automate, automateHuman error suchs. I mean sucjs. Oh shti. See? Human error is an acceptable risk on trivial, performed-once operations such as writing an email message. Human error is NOT an acceptable risk when building and releasing your software. If you can type the steps into the keyboard, you can automate the process. Trust me, it is worth the trouble. I automate virtually everything. In fact, an extremely complex Perl script is writing this article for me. I'm really asleep... Which brings me to Perl. I began using it in 1996 and am still learning some neat tricks. This is really the Magic Belt. Using Perl, I have automated tasks that at first seemed completely unapproachable. If you don't like Perl, try Python or even one of the older scripting languages, like sed or awk. Heck, some of my earliest automation (1985) was accomplished using DOS batch files. Just use something. Auto-Build NumbersGot a build number embedded in your source? This is pretty common. Open up virtually any application and click "About". You'll see something like "Version 12.3 Build 1234". Those strings came from the source code, and someone probably had to hand-edit them prior to that build, commit them, tag the source, and then re-edit the strings to free the build number. Yuch! Automate that whole nasty process. Write a little script that reads your master build number file, increments and caches it, then opens the source file with the build and version string in it, greps or searches for the string in the file, does a replace, closes the file, commits it, and tags it. SchedulingI have encountered numerous problems over the years with cranky AT and WinAT schedulers, so I gave up and wrote my own. I use Perl; use whatever you like, but remember that Perl is extremely portable. It's very likely that the problem you resolve in Perl today will be a problem you'll face later on another project. By doing all your work in Perl, it is likely that you'll be able to reuse your work. ReuseModularize your automation so it can be used over and over and over. Perl modules are an obvious Perl-centric solution. Create and export all your globals in a few well-thought-out modules. I have one module (now about 4 years old) that does NOTHING but figure out what today is and what yesterday was. Sound worthless? Not if you have to automatically roll back a release that begins to exhibit problems once in production. The concept of a build system-wide "Yesterday" is very useful. NotificationI have my build system email me when it starts and finishes. I log everything, and, at the end, cull out the important stuff and format it into a message. This gets sent to the project team, and the text has a high-level "Build OK" message. There's a link for the details, but keeping the eye-catching message brief and to the point is extremely useful. Since I know how long the build takes (to within a couple of minutes), I can be reasonably sure something is wrong if I don't receive the "completion" email on time. This allows me to proactively investigate and deal with whatever may have happened. This reduces schedule impact and, ultimately, project costs. HTMLWeb servers are everywhere. Use one as your presentation manager for builds and releases. All of my logs have the capacity to be displayed in HTML. We're not talking about flaming logos here, but rather the newest lowest common denominator: the browser. There's nothing fancy in my build logs, just lots of data that I must present in the most meaningful, cost-effective fashion possible. Database storageOnce you discover how much data a build generates, you'll begin to consider all the really cool things you can do with that data, or, more properly, metadata. Examples of build metadata that I capture and key on the build number and date are:
I use PostgreSQL and MySQL. Each is freely available and can be quickly set up on a Linux box, and you can import your existing logs, after you massage them a bit. There's a practical limit to the amount of metadata you store. For example, even though you could capture and store things like "Number of binary files included in the build", you have to ask yourself whether the data is real. Just because a piece of information is available, it doesn't automatically become data. Implementing one or more of the features mentioned above could have an immediate positive impact on your project. Start with the biggest ones, like the overnight build, and move on from there. Getting the framework in place isn't easy, but it only has to be done once. So get to it. Author's bio: John Minnihan has been providing business computing services or support from more than 20 years. With a background in hardware repairs, network implementation, and application support, he settled into software configuration management in 1995. His work in that field led to the founding of freepository in 1999, which itself led him down the wild and crazy startup path. Today, he works at another startup and commutes between his home in Denver and his office in San Francisco. T-Shirts and Fame! We're eager to find people interested in writing articles on software-related topics. We're flexible on length, style, and topic, so long as you know what you're talking about and back up your opinions with facts. Anyone who writes an article gets a t-shirt from ThinkGeek in addition to 15 minutes of fame. If you think you'd like to try your hand at it, let jeff.covey@freshmeat.net know what you'd like to write about. [Comments are disabled]
[»]
Another tool concentrate on build automation Just try luntbuild, an open source project.
[»]
Perl and Release cycle We are developing an web application in perl, and due to an expansion of
clients, we are having to look at a more complex release cycle. What tools
would you recommend to use to manage releases? We currently us CVS on a
linux box for vc, but looking at ant and anthill and their ilk, they don't
exactly seem to fit us as we do a flat (ie no compile) distribution.
[»]
I like to buid and release software with... ...ABSOLUTE NO WARRANTY.
--
[»]
Some other key parts Make sure your builds are reproducable -- this
[»]
Great comments - thanks! Thanks for reading with such a critical eye. It is nice to see that there
are others out there who share my passion for building better software.
--
[»]
The process of deploying... I am the guy that is in charge of deploying our software I have broken it
down into a set of scripts. I look at the whole process as deploying and
yes if should be fully automatic.
[»]
Missing concepts and tools At least two important concepts are missing with regards to build management: Completeness; That the correct versions of all components are built into a release and Consistence; that the whole system is build with all components. That the author forget to mention the make tool but instead focus on Perl is strange. Make provides a dependency graph of components which is important for consistent builds and make also provide a much better tool for completeness and automatic build than Perl. I also find it strange that the author’s descriptions of release management don’t mention Version control and Change control. For version control, CVS is probably the common single most used tool in the OSS community and should have been mentioned. And Bugzilla is a good example for change control.
[»]
Re: Missing concepts and tools
Make does only a fraction of this, and it isn't especially good for situations where you have mulitple architectures operating in parallel. This is where tools like autoconf and imake can come in handy. But the point is that perl is a convenient, flexible, portable, and powerful tool to do all the associated fiddling around that is necessary in any non-trivial build environment. In my own discussions, I tend to break everything out: Source Code Control is different from Change Control, which is different from Build Management, which again is different from Release Management. And don't forget Environmental Control and Tools Management. All wrapped up together we can refer to the whole mess as Configuration Management. Since the author was talking about Build and Release Management, I don't think he was out of line to promote perl as a important tool. To further split hairs, the most important goal of build management is neither completeness or consistancy (not that these are unimportant) -- the number one goal of build management is repeatability. In other words, you have the ability to duplicate any arbitrary build from any arbitrary date in the past -- so that bugs can be duplicated and then later be confirmed to be fixed.
[»]
Ant For java projects, ant (http://jakarta.apache.org/ant)
[»]
Re: Ant + AntHill Yesterday i just stumbled over AntHill at www.urbancode.com/projects/anthill/ - it is a Tomcat based web-thing that makes the releases built with Ant available on a web server.... and its OSS :) --
[»]
Re: Ant + AntHill Here is the Anthill link: http://www.urbancode.com/projects/anthill/default.jsp
|
|
