What's That Noise?! [Ian Kallen's Weblog]

All | LAMP | Music | Java | Ruby | The Agilist | Musings | Commute | Ball

20041221 Tuesday December 21, 2004

Rigorous Operations, Agile Development When it comes to software development, I generally eschew rigorous planning. I prefer to take a defined set of use cases, ascertain that they are the important ones and develop tests and code to fullfill them -- no design documents, no architecture documents and no UML diagrams unless they're helping me think through the data model and call stacks (i.e. sequence diagrams).

The recent colo move that I worked on all last weekend with my cohorts at Technorati was a good demonstration where rigor pays off: operations. Physically moving an entire network requires shared knowledge of complex network systems, detailed resource allocation and troubleshooting contingencies amidst a maze of dependencies requires rigorous documentation, planning and coordination. My hats off to Matt, David, Aaron, Bill and Adam for rising to the occasion and kicking butt. It's been an honor and a pleasure, gentlemen.

I'm in New Mexico for a spell, hoping to take a side trip up to Ski Santa Fe to get a lil bit o' frozen bliss.

( Dec 21 2004, 04:35:58 PM PST ) Permalink

20040908 Wednesday September 08, 2004

Work Environments: Didn't you get the memo?

Software engineers engage in a collaborative game of intellectual creation that can only be won if they can easily converse with and overhear each other. That doesn't mean disrupt each other or sit on top of each other. That simply means that the barriers to communication need to be as low as possible.

The topic has come up a lot recently as a thread spun from Joel-on-software's Bionic Office fetish for private offices. Hey, I dislike interruptions as much as anybody else but in my experience, fixed, permanent private offices for individual team members are communication barriers. Shared workspaces where team members can't turn to face each other for impromptu meetings are also a problem. A JoS discussant Brad Hill gets it

...offices are not always the way to go - sometimes the nature of the product means that a much more collaborative and shared workspace is best.
The standard-issue cube farm is the worst of both worlds, offering neither the privacy and freedom from distractions of a real office or the collaborative flow and facilities of a "war room".
Sure, cube farms invoke nightmares of pointy-haired-boss reminders about TPS reports. However, the alternative isn't necessarily private offices. Private offices implies doors. Doors close. Closed doors hinder communication. Joel's cite of Philip Greenspun is almost laughable
Your business success will depend on the extent to which programmers essentially live at your office. For this to be a common choice, your office had better be nicer than the average programmer's home. There are two ways to achieve this result. One is to hire programmers who live in extremely shabby apartments. The other is to create a nice office.
OK... nothing there about private offices for everybody. Let's see what a real expert says. Kent Beck (Extreme Programming Explained: Embrace Change) describes the an effective programming environment as one where
We will create an open workspace for our team, with small private spaces around the periphery and a common programming area in the middle.

Maybe Joel had a bad experience with The Bobs and their cube farm but his big kick for putting everybody in sequestered little spaces is a recipe for incommunicado.

( Sep 08 2004, 07:33:09 AM PDT ) Permalink
Comments [1]

20040405 Monday April 05, 2004

Building software could be like layering a cake, which could be good I did a little write up on Aspect Oriented Programming (AOP), titled "Aspect Oriented Programming: An Introduction," I'm more than just a tad stoked that it's the lead story currently on CMP/TechWeb's DeveloperPipeline (ya know, I'm just having a great day today, there's lots of good stuff happening).

Anyway, I believe we are entering an age of non-intrusive frameworks (unlike traditional J2EE) that serve as lightweight application servers, or lightweight containers, if you will. AOP isn't a piece of software or a container, it's a different and complementary approach to ye olde Object Oriented Programming. AOP is about layers. It's about being able to layer on supporting functionality and weave it into your core functionality. AFAIC, Sun is put on notice: your big overdone API's for J2EE aren't necessarily the best way to develop all of the components for a scalable OO architecture.

A lot of the things that J2EE provides (such as security and persistence) might be better served as application aspects, layered onto the core OO model. And technologies such as Hibernate are here to help as well.

Layers involve complexity, but that's not a bad thing. Why, there are even bits of wisdom on the matter to be found in everybody's favorite DreamWorks feature film:

SHREK: For your information, there's a lot more to ogres than people think.
DONKEY: Example?
SHREK: Example? Okay, um, ogres are like onions.
DONKEY: [Sniffs] They stink?
SHREK: Yes. No!
DONKEY: They make you cry?
DONKEY: You leave them out in the sun, they get all brown, start sprouting' little white hairs.
SHREK: No! Layers! Onions have layers! Ogres have layers! Onions have layers. You get it? We both have layers. [Sighs]
DONKEY: Oh, you both have layers. Oh. [Sniffs] You know, not everybody likes onions. Cake! Everybody loves cakes! Cakes have layers.
SHREK: I don't care... what everyone likes. Ogres are not like cakes.
Who can argue with that? ( Apr 05 2004, 08:55:55 PM PDT ) Permalink

20040330 Tuesday March 30, 2004

Software Development Team In A Box I was musing yesterday about off-the-shelf tools that, when put together, provide the basic needs to make a software development team productive.

Here's my list:

source control with commit notifications
It might seem obvious but when changes are happening all engineers who have any interaction (direct or indirect) with the altered component may need to know about it. Thankfully, getting emails out of CVS is a piece of cake. Other source control systems usually have hooks for this as well.
bug tracking
Identifying issues and keeping tracking track of them so that narrative descriptions, conversations and related media (screen shots, log files, etc) can be associated with the issues is crucial. Bugzilla is E-Z 'nuf.
reproducable builds
If a lot of steps that have to be manually triggered is required to build a software project then development is going to be intrinsically slow. The build cycles should generate reproducable results and do so in a scripted framework that can ultimately kicked off with one command.
unit testing
I'm a big fan an keeping as much business logic as possible out of the web runtime environment. If a web server,servlet container, EJB container and/or database is required to be running with your code deployed to run the tests, then you've been writing integration tests, not unit tests.
integration tests
Of course, your code has to work in the runtime environment of your application, so "in-container" tests such as Cactus are crucial.
functional tests
Ultimately, an end-to-end test that goes from HTML and HTTP, through the business logic and frameworks and down to the database is important to validate the system as a whole. This is where things like HTTPUnit come in.
If you use something like Cruise Control to take care of updating from source control, kicking off builds, running unit tests, integration tests and functional tests with as high a frequency as possible, you've got a pretty robust development environment.
test coverage calculation
Not everything requires a test but there whenever there are semantics, the business logic should be tested and whenever there are environments to deploy to, there should be a level of integration testing -- I don't know what tools will effectively give you that stuff (send it to me if you know!).
a wiki
It's useful to have a central repository of facts about the project. Best practices, HOWTO do weird things type documents and stuff like that should be kept as authoritative documents.
a blog
A lot of software design decisions are made in face to face conversations and email threads. It's difficult to retrieve the original intentions and the pro's/con's discussed in that way. I've seen email threads pasted into bugzilla reports and wiki pages as a work around. However it seems to me that if the wiki pages and bugzilla reports instead linked to the blog entries where these conversations took place, they would be easier to keep track of, search for and so forth. Of course, a lot of decisions are still going to be the outcome of face to face and email conversations but they can still be synopsized and made into blog entries.
So here's the money shot: if I were the VP of Engineering or the Director of Software Development or the Grand Pooh Bah of Code Artifact Creation at SoftwareCo Incorporated, I'd want easy insight into all of these things. A kind of portal that contained summaries of software development metrics: ...and so on. A lot of pointy-haired-boss metrics like lines of code and code to comment ratios are nice, I suppose for amusement value, but likely uninformative about anything important.

In the end, I think getting all of these parts working together will render a whole greater than the sum its parts. Getting a collaborative environment functioning isn't easy but a lot of these tools are readily available off the shelf to help it along. ( Mar 30 2004, 03:04:12 PM PST ) Permalink

20040324 Wednesday March 24, 2004

Refactoring: making things better in little increments I had a nice conversation with some folks who weren't looking to for help building The Next Great Thing (that was yesterday's nice conversation). They just had a mish mash of code, all Perl, old and new (but mostly old), that needed to be revamped. They didn't say that they needed a refactorer but that's about what it boils down to: their code needs some serious refactoring.

The funny thing was, the code in question sound like the kind of stuff that you'd typically think of as refactoring candidates. A lot of it is apparently old CGI's (run as Plain Old CGI's or Apache::Registry scripts) that atrophied into speghetti of gobbledy gook. So the problem there isn't one of taking software objects and restructuring them for better design, the first refactoring, if you will, is to take a bunch of procedural code and re-compose it as an object system.

Frankly, this could sound really boring. But on the other hand, there can be something very satisfying in something like this. Have you ever set aside an afternoon or a day to clean out the garage? You sort through a bunch of junk, shelve things that were set down on the floor, throw out a bunch of garbage, fire up the shop-vac and vacuum out all of the cob webs and organize the toolchest. In the end, you haven't produced anything tangible with any value. But you've created something that allows for creating value. I would never go into the garage and build a planter box if it's all cluttered and dusty. It's just too grotty.

Code gets the same way. It gets cluttered and dusty. Refactoring is explicitly not about creating tangible value (i.e. you don't add new features in a refactoring), it's about creating an environment within which tangible value can be created. Fowler sez that refactoring is more than just "cleaning up code" - it's a more methodical, tested and controlled fashion of cleaning up. That's all good. Unfortunately, it seemed as though I drew a blank stare from these folks when I talked about testing. I mean, I understand that test harnesses in Perl such as plain old Test, Test::Unit and Test::Harness are hardly standardized or well entrenched in Perl culture but it was kinda disheartening to see only vague recognition of the value of writing tests. When you're done cleaning the garage, there's that satisfying feeling knowing that you've brought a little order to a chaotic corner of the world. I get that same feeling when I'm working on code, breaking it and fixing it and when it's all said and done, running the test harness; see "100% passed" is another way of bringing order to the chaos.

Anyway, so the basic starting points for refactoring were absent: a system of objects and a test harness but if the goal of refactoring is to "make the software easier to understand and modify" (quoting Fowler, again) then perhaps just getting things to that starting point is the first important refactoring to undertake. Somehow, I don't think the connection between the "fire fighting" mode that these folks were perpetually in and absence of tests was immediately clear to them. They know that "things" need to be fixed (which is a good start) but IMO those fixes need to come in little increments of refactoring.

Now, I've never unit tested a garage cleaning effort but today's conversation made me consider how that'd be done. ( Mar 24 2004, 10:43:06 PM PST ) Permalink

20040322 Monday March 22, 2004

Offshore Agility Two software industry trends, offshore development and agile methodologies, seem to be squarely at odds with each other. This has increasingly been a concern for me because I want to work where functional and unit testing, refactoring and short iterations are core development practices. But as I've been thinking about this, I've come to conclude that offshore and agile might work together just fine.

Martin Fowler recently wrote an article about this. In discussing the use of offshore programmers with his company, one of the things he mentions is using Cruise Control to aid continuous integration of multiply located collaborators. This sounds good. I've seen lots of problems with Cruise Control grinding down with OutOfMemory errors but I'm more suspicious of running in-container tests like Cactus than I am of Cruise Control per se.

An article I read in the paper the other week mentioned that the help-line calls for the welfare and food stamp programs in California are routed to offshore call centers in Mexico (for Spanish speakers) and India (for English speakers). What would be grimly humorous is if the State of California's Employment Development Department started offshoring its labor. ( Mar 22 2004, 09:35:16 PM PST ) Permalink

20040312 Friday March 12, 2004

Agile Software Development Ecosystems I've had a lot of recent conversations about agile software development. Unless the practices have meaning to you (i.e. you're a software engineer, you manage them or interact with their outputs), talking about "extreme programming", "scrum" and other agile practices probably sounds like a lot buzzword bouncing.

There are a lot of different flavors of agile development. Some of them work best, ideally anyway, if all of the practices in that particular flavor's agile "recipe" are adhered to. It's been my experience that most situations are less than ideal. People come to work with a lot of preconceptions about the right way to do things or with their own ego issues that get in the way of buying into an agile methodology lock, stock and barrel. However, I found the book Agile Software Development Ecosystems (by Jim Highsmith) very thought provoking and led me to conclude that depending on the context of the workgroup, there may be no perfect fit for a particular methodology but not to despair. If you can get people to agree upon principles and values about what they want to accomplish, values that are open to new ideas and not doctrinaire about waterfall processes, it is possible (and in fact likely) to create an agile environment borrowing what is needed from one or many of the established agile development flavors. A properly established environment should be generative of agile practices and that's what I'm really looking for in my work place - instead of rigid rules, agree upon principles and communication norms and the appropriate practices will become clear as everyone's work styles come into play.

At least, that's my current thinking on the matter; I'm open to new ideas. ( Mar 12 2004, 10:53:22 AM PST ) Permalink

20040225 Wednesday February 25, 2004

Much Ado About Outsourcing Vivek Paul seems to be the annointed spokesman for India's outsourcing industry. Last week he was the lead protagonist in an eWeek piece The Offshore Proposition. In the current issue, eWeek has him again in a two page interview. The most interesting thing is an excerpt in the response to the last question ("Your advice for the 40-year-old engineer here in Silicon Valley whose job has been outsourced?"). Paul is optimistic about the prospects of US engineers who are, "...pushing the edge of innovation" but then goes on to say:
"If your job involves sitting in a cubicle and not talking to anyone else, you are at risk. Get yourself into a role that is more attached to the customers or suppliers and is on the cutting edge of innovation."
So maybe you don't think you're Milton but according to Paul's point, don't content yourself with being relegated to the basement. Recall Peter and the Bobs:
Peter: You’re gonna layoff Samir and Michael!
Bob Porter: We’re gonna bring in some entry level graduates for us to work in Singapore, that’s the usual deal.
Bob Slydell: Well, it’s standard operating procedure.
Yea, and Milton and his stapler will be outsourced. So stay close to the customers, dammit!

Milton is Nervous
( Feb 25 2004, 10:48:03 AM PST ) Permalink