Sunday July 30, 2006

Since the universally understood (at least among the intelligentsia) descriptor user generated content continues to nag at people (Tim raised it again during his OSCON session) and the alternatives have been difficult to pin down (was Tim suggesting people contributed experiences?), it's my caffeinated Sunday morning aspiration to consider the alternatives.

Having a label is important, we're making a distinction between published artifacts that are developed by editors and/or paid staff and the stuff created by Normals who are contributing the artifacts of their creative process to the web. Yes, the term user is definitely sterile, generated too mechanical and content seems so... vacuous. Does participant created artifacts work as a descriptor for all of the photos we're uploading, blog posts we're posting and so forth?

Saturday July 29, 2006

Had a great time at OSCON! Besides the previously noted keynotes and sessions, my faves were Perrin Harkins' Low-Maintenance Perl (a good discussion of best practices in Perl as the simple practices, Perl as the sole domain of wizards is so old school), Moazam Raja's Troubleshooting the JVM and the Applications That Run Within It (a good survey of the built in runtime diagnostics available for java), Tim Bray's The Atom Publishing Protocol as Universal Web Glue (a good example of using vi and curl for bare metal wire protocol demos as well as how slow JRuby's start-up time is!) and Damian Conway's Friday keynote was suitably humorous! If there was anything that I wish I coulda rearranged it was the time slots when there were more than one session I wanted to be in. But the timeslots with nothing interesting going on were good opportunities for hallway conversations; which are often the most important activities at these events, so I won't complain vigorously.

Enjoyed hanging out Friday afternoon for "OSCON decompression" at Urban Grind ("Coffee should be black as night, hot as hell, and strong as love.") with James, David, Josh, David and Scott. Heh, I got PostGIS running on my powerbook, which gave me something to play with on the flight home!

Portland is a really nice town, the Disneyland-like lightrail system (complete with automaton announcements in english and espanol), the neighborhood ambiance, the surrounding greenery... I dig it. For next year's OSCON trip, I'll be bringing the family along!

Thursday July 27, 2006

At Greg Stein's talk, A Google Service for the Open Source Community, he outlined how Google's following up on it's Summer of Code project with a new contribution to open source. No, it's not a dating service or personal trainer service for geeks (that'll be Google's 2007 contribution). And no, it's not source code search (does Krugle have that covered?). This is project hosting on Google Code Project Hosting.

Yes, there is nothing new about project hosting; there's long been things like Source Forge, Tigris, java.net and so forth. Things like Sourceforge are doing a great job, but there are strengthes that Google has that could be brought to bear on the project hosting space. Some of the unique features of Google's project hosting that Greg cited are

simplicity, scalability, reliability

The system is built with Google's trademark minimalist approach to user interfaces as well as leveraging Google's horizontal scaling and robust the data center sauce.

Subversion on Bigtable

They've built a new backend for Subversion that, instead of using flat files or BerkeleyDB, uses Bigtable. Bigtable data is fast, highly available and replicated across data centers so that all Subversion stored resources will benefit from the Bigtable backend.

Complete re-think of issue tracking

Bugzilla, Trac, Jira and most other issue tracking systems are workflow heavy and laden with permissions and security provisions. The heavy stuff is often unnecessary, it just gets in the way of issue tracking for open source projects instead of facilitating the core use cases that people need. They've replaced all of the highly structured data model and query environment with labels (tags). Labels get attached to projects and issues to tags and provide a minimalist structure to query by using full text queries.

Creating a new hosted project is simple enough, you fill out a form with the project name, a summary description, a full description, select a license (Apache, Artistic + GPL, GPL v2, LGPL, BSD, MIT and Mozilla are choices ... dual licensing is not permitted) and apply some labels (tags). If you don't have one yet, a subversion password is created for you (it's *not* your gmail password). Your project will have a tabbed interface for the main page, issues, browsing the soruce and an administrative page. Project creators and administrators must use a GMail account. If not using GMail, bug reporters must have some Google account (Picasa, Groups, etc). The "Issues" screen provides a tabular view of bugs, the columns are ajax enabled for parameterization. The neat thing is that instead of using a big form with tons of check boxes and selectors, the issue tracking uses query expressions to refine issue search results. Status field for a bug can be free text; while a static vocabulary is defined and selectable in an ajax drop down the vocabulary is unconstrained. Status isn't the only metadata that's open-ended, instead of having "release version", "milestone", "component", etc the system uses labels. The issue list column repertoire is adaptable so that you can select labels you've defined as listing criteria. All of the open endedness may be an invitation to pandemoneum but the focus is on having the user interface make it easy for the user to do the right thing.

Some of the administratively defined aspects of a project include the issue creation template (defines the prompts that issue creators will see), project links, project discussion groups (using Google Groups), project blogs and activity notification email addresses. The system will support issue tracking feeds. Most of the metadata that will be visible on the project summary page that newcomers to the project will see.

There's currently no "tarball download" service and integration with other Google services is in the works. For the time being, any downloads made available must be done within the limit of the quotas on your subversion repository (100 MB). Plans for importing and exporting, creating APIs and so forth are underway (the issue tracking seems like a natural fit for Atom and Atom Publishing Protocol).

Congrats to Greg and the Google Code team on a great launch!

Wednesday July 26, 2006

I missed the first keynotes (I just arrived in time for Tim O'Reilly's "what technologies are hot according to these slices on the data" bit that he does) but enjoyed Greenplum's Scott Yara talk, School of Rock. He highlighted the parallels of open source development and rock and roll. I'll paraphrase his points.

Open source, like rock and roll, has flourished simply because people enjoyed it. Like rock and roll, money has jumped into open source and an industry has swelled around it. Like rock and roll, open source threatens the establishment but also mutually coopts and becomes the establishment. Yara showed a funny "twins separated at birth?" photo pairing of Rick Rubin and Richard Stallman! What will sustain open source's integrity (like rock and roll's) are the intangibles, the real emotions and inspirations the drive innovation. The popularity game isn't a measure of quality... just because it's widely downloaded doesn't mean it's good just as Britney Spears' and N'Sync's sales success aren't validations of "good" music. So, beware of the vogue of open source, people are starting to believe that open source is better but don't let that undermine what's important. For those who are building their business on open source, go for the $$$ but keep your integrity. At that point Yara ran a little excert of Metallica goofing on a radio promo production (from Some Kind of Monster?), the ironies of choosing them as illustrations of how money changes everything, given how they coopted and have become the music establishment, were high humor for me. Nonetheless, Metallica like a lot of successful open source software projects have succeeded by being a little dangerous, by being genuine and not bothering with the constraints of the legacy establishment.

Anil Dash gave a talk about Trying to Suck Less: Making Web 2.0 Mean Something basically outlining that beyond the technology stack (i.e. LAMP), there are higher level tools that developers can employ to suck less (yep, I confess, at Technorati when we can't quite kick the butt that we aspire to, we focus on sucking less). Citing the technologies that have grown out of SixApart's software plumbing, he highlighted that all successful Web 2.0 compnaies are using load balancing, messaging, caching, filesystems and other scalability and performance platform components. In SixApart's case, perlbal, memcached, mogilefs and djabberd are the core technologies that they build on ... and, so the pitch goes, should you if you want to suck less.

Those the high points of the morning (so far).

Tuesday July 25, 2006

In case you hadn't heard, we've had a lot of things cooking at Technorati. Besides the engaging new look, the new features and the complete overhaul of URL search and link counts, we've been making great strides in our blog spam mitigation (you wouldn't believe the stuff we catch ... and the shear quantity of it!), our internal caching and messaging infrastructure and our data center network. Of course, there's still much to do but we've been heads down on it; if you haven't checked us out lately I think you'll find that our efforts to improve the front end, the back end and all of the cogs and pullies in between have been moving forward.

I'm really proud of the team I work with at Technorati! If you'd like to join the team, we have a lot of innovation ahead. Grab me this week at OSCON and tell me about how you'd like to materialize the real time web! I'll also be moderating a Microformats BOF, this will be a good opportunity to talk about the implementations for producing and consuming microformats. See ya in Portland!