Tuesday March 30, 2004

Here's my list:

source control with commit notifications

It might seem obvious but when changes are happening all engineers who have any interaction (direct or indirect) with the altered component may need to know about it. Thankfully, getting emails out of CVS is a piece of cake. Other source control systems usually have hooks for this as well.

bug tracking

Identifying issues and keeping tracking track of them so that narrative descriptions, conversations and related media (screen shots, log files, etc) can be associated with the issues is crucial. Bugzilla is E-Z 'nuf.

reproducable builds

If a lot of steps that have to be manually triggered is required to build a software project then development is going to be intrinsically slow. The build cycles should generate reproducable results and do so in a scripted framework that can ultimately kicked off with one command.

unit testing

I'm a big fan an keeping as much business logic as possible out of the web runtime environment. If a web server,servlet container, EJB container and/or database is required to be running with your code deployed to run the tests, then you've been writing integration tests, not unit tests.

integration tests

Of course, your code has to work in the runtime environment of your application, so "in-container" tests such as Cactus are crucial.

functional tests

Ultimately, an end-to-end test that goes from HTML and HTTP, through the business logic and frameworks and down to the database is important to validate the system as a whole. This is where things like HTTPUnit come in.

automation

If you use something like Cruise Control to take care of updating from source control, kicking off builds, running unit tests, integration tests and functional tests with as high a frequency as possible, you've got a pretty robust development environment.

test coverage calculation

Not everything requires a test but there whenever there are semantics, the business logic should be tested and whenever there are environments to deploy to, there should be a level of integration testing -- I don't know what tools will effectively give you that stuff (send it to me if you know!).

a wiki

It's useful to have a central repository of facts about the project. Best practices, HOWTO do weird things type documents and stuff like that should be kept as authoritative documents.

a blog

A lot of software design decisions are made in face to face conversations and email threads. It's difficult to retrieve the original intentions and the pro's/con's discussed in that way. I've seen email threads pasted into bugzilla reports and wiki pages as a work around. However it seems to me that if the wiki pages and bugzilla reports instead linked to the blog entries where these conversations took place, they would be easier to keep track of, search for and so forth. Of course, a lot of decisions are still going to be the outcome of face to face and email conversations but they can still be synopsized and made into blog entries.

• How many software commits has each engineer made?
• How many different files were they commiting changes to?
• How many bugs were created?
• How many bugs were closed?
• How complete (percentage wise) is our test coverage?
• How many broken code artifacts were checked in (i.e. put the code base in a state when it wouldn't even compile)?
• Who did it (are there any grossly flagrant offenders)?
• How many builds with all tests run have completed successfully?
• How many failed?
• For the failures, what was the average time to resolution?

In the end, I think getting all of these parts working together will render a whole greater than the sum its parts. Getting a collaborative environment functioning isn't easy but a lot of these tools are readily available off the shelf to help it along. ( Mar 30 2004, 03:04:12 PM PST ) Permalink