What's That Noise?! [Ian Kallen's Weblog]

All | LAMP | Music | Java | Ruby | The Agilist | Musings | Commute | Ball
« Previous page | Main | Next page »

20050215 Tuesday February 15, 2005

A Java i18n Checklist I've worked on Java projects with the human language aspects abstracted out but now that I've had to get down and dirty with real localization problems, I'm starting to take mental notes in the "if I had known that earlier it would've saved me a lot of grief" folder.

It seemed pretty straight forward going into the project that I'm working on:

  1. Text is stored as UTF-8 character data
  2. Use ResourceBundle property files to manage display strings
  3. Maintain the set of property keys in the properties file
  4. Let the browser's Accept-Language request headers drive what property file to prefer
See, it's easy! Well, for simple proofs of concept with Western characters, it's just about that easy. When dealing with multibyte strings for asian languages, there's a whole lot more to consider.
  1. Make sure the servlet container is handling UTF-8 appropriately
    For instance, if Tomcat is serving on the HTTP tier edit server.xml and make sure the URIEncoding attribute (absent by default) is set for the connector. 
        <Connector port="8080"
               maxThreads="150" minSpareThreads="25" maxSpareThreads="75"
               enableLookups="false" redirectPort="8443" acceptCount="100"
               debug="0" connectionTimeout="20000"
               disableUploadTimeout="true"
                URIEncoding="UTF-8"
    />
    The same holds true for letting Apache do the HTTP dirty work and connecting with mod_jk 
        <Connector port="8009"
               enableLookups="false" redirectPort="8443" debug="0"
               URIEncoding="UTF-8"
               protocol="AJP/1.3" 
    />
    And, by the way, if you have static content served by an Apache server, you probably want this as well 
    AddDefaultCharset utf-8
  2. Wire up the native2ascii ant task into the build system early in the project.
    Manually dealing with the ASCII escaping is a nuisance. If the conversion can't be transparent, at least automate it.
  3. Make sure the database connection drivers are being gentle with their data handling.
    In the case of MySQL, changing the JDBC URLs from this 
    jdbc:mysql://localhost/fubar
    to this 
    jdbc:mysql://localhost/fubar?useUnicode=true&characterEncoding=UTF-8
    made a world of difference.
  4. Check the HTTP response headers to assure that the Content-type header value is appropriate
    If the charset isn't set to UTF-8 when it really is, you could be confusing the client. This can set in a servlet, in a JSP and IIRC the struts-config.xml allows you to set it declaritively. You want to set the Content-type before writing to the response object's PrintWriter. Apparently if you have multibyte characters in your JSP page components, you need to set the pageEncoding i.e. in the JSP file itself, something like this: 
    <%@ page language="java" contentType="text/html; charset=UTF-8" pageEncoding="UTF-8 %>
    Though my whole motivation for using Java on this project was to have page components have only markup and display code; all of the lanugage is abstracted. Anyway, I'm preferring Velocity over JSP these days.
  5. Be prepared to convert request parameter values.
    In my experience, doing this 
    request.setCharacterEncoding("UTF-8");
    before getting the parameter values is not reliable (could be Tomcat bugs though). However, this appears to be a fairly standard idiom 
    String formValue = new String(request.getParameter("formParam").getBytes("ISO8859_1") /* bytes */, "UTF-8");
There are other i18n traps to beware of; seems like every place data is passed from one subsystem to another there's an opportunity for the encoding to get mangled.

( Feb 15 2005, 10:17:29 PM PST ) Permalink


20050214 Monday February 14, 2005

Time Slips Away Another day of end-to-end frenzy and the day is still not done. Short meetings turn long. Interruptions and blocking issues. Who has time for anything? Who has time to blog?

I just took my trusty old swiss army watch out of my pocket, I shoved it in there on the way out the door this morning and have had nary a chance to adorn my wrist with it all day. And now it's too late. Where goes the time?

( Feb 14 2005, 09:24:56 PM PST ) Permalink


20050127 Thursday January 27, 2005

Speed Isn't Everything and It's Crap Without Automated Testing The software project management triangle has these three things at the corner:
  1. scope
  2. speed
  3. cost

The third item refers to allocating more resources (typically, people) to projects. The idea is that if you're resource starved (short handed) you need to reduce scope and/or the schedule (sacrifice speed) to compensate. If you increase scope, either the schedule slips or costs increase (or both). Of course, throwing more resources at the problem is often counter-productive.

There's another triangle associated with software development:

  1. rigor
  2. automated testing
  3. quality
Most software projects involving technology innovation can ill-afford to lean to close to the rigor corner; problems that haven't been identifiably solved before are intrinsically riskier. Rigorous development practices such as old-school waterfall processes are usually a bad fit where the emphasis has to be on speed of execution and technical creativity. Culturally, automated testing can't just be a function for quality assurance staff; unit testing, integration testing and functional testing have to built into the DNA of a software project's build system or else the engineering costs for writing and running tests grows too high to sustain. So the third option is winging it in an ad-hoc fashion and live with the associated reduction in quality. Obviously for things like flight control and medical monitoring where lives are at stake, you want rigor and automated testing otherwise bad things happen. Space crafts blowing up and/or becoming astro-debris, missile guidance systems failing or heart transplant patients dying due to software defects are tragic. But most software projects don't have those kinds of consequences attached. Thus the incentive is high for making test driven development part of the software engineering culture wherever speed of execution and breadth of innovation are among the primary motivations.

So if low quality is crap, perhaps there's a mathematic expression here 

  SPEED - AUTOMATED TESTING = CRAP
...and perhaps it's even transitive: high quality + automated testing = speed.

Speaking of which, it must be time for more coffee.

( Jan 27 2005, 10:11:18 AM PST ) Permalink


20050123 Sunday January 23, 2005

Technorati + Firefox Search tools in browsers are usually pre-populated with all of the usual suspects: Google, Yahoo, Amazon and so on. That's all well and good but what about when you want to know about what's being said on the real-time web?

There's now a Firefox plugin that adds a "Technorati Engine" to the pulldown list. Sweet!

( Jan 23 2005, 09:45:08 AM PST ) Permalink


20050122 Saturday January 22, 2005

Start Up Pains I've posted here about Technorati's misadventures in recent months: power outages, disk upgrades, data center moves and so on. In the life of a startup, these kinds of foibles seem to be just a part of growing up.

So I've empathized and enjoyed recent readings of other's mishaps. Not in celebration perhaps in feeling the bonds of shared trauma.

FlickrBlog's Growing Pains
  • unbalanced load/capacity distributions
  • bottlenecked queues
  • non-specific system instabilities
Word to your mutha, I know that.
LiveJournal's Power-loss post-mortem
  • hardware configuration/system problems that exhibited themselves as hosts that didn't came back up correctly
  • MySQL databases getting hosed
  • Databases tuned for speed over safety
Been there, done that.
So is it the destiny of all web service start-ups to have fabulous disasters? Probably. A lot of times, these kinds of things are predicated on having Innovative architectures, on doing some things that aren't widely known to have been done that particular way. Can you really guard against finding yourself with arrows in your back when you're out on the frontier? I don't think so. Count on the topsy turvy. It's not always fun when you're in the thick of it but if you adapt, you'll be better for it.

Of course, you could just laugh about it. Or post to your blog about it. Or both. So far, from what I've stumbled across, this is the funniest of the bunch.

10:11 pm: So far so good. Things are checking out, but we're wearing tinfoil hats. A few annoying LJ users, but nothing that's not fixable. We're going to be buying a bunch of weed on Monday so that, if this happens again, we'll just be too baked to care.
This weekend's disastrophe for me is relatively mild: sore throat and congestion. So I'm drinking tea. And laughing about it. And posting to my blog.

( Jan 22 2005, 08:42:19 PM PST ) Permalink


20050112 Wednesday January 12, 2005

Jack of All Trades I thought I was pretty broadly skilled with my work spanning software engineering, network and systems operations, some DBA-duties and variety of other do-it-cause-it-needs-to-be-done endeavors.

Well here's a something funny: Chef and Missile Treaty Compliance Inspector. Yea, this is an actual job that is posted right now:

Job Description: Responsible (sharing duties with one other chef) for the preparation of three daily meals (brunch & dinner only on Sundays) for approximately 20-25 personnel at remote, self-contained site in Russia. Duties include menu planning, food ordering & resupply, inventory management, kitchen maintenance and sanitation and catering support to infrequent special functions on and offsite. As treaty monitor, may occasionally operate the On-Site Continuous Monitoring System in accordance with official START treaty missile inspections at Votkinsk.
Where can you sign up? HotJobs (what do you mean by "hot"?). I took the liberty of mirroring the job ad, since I'm not really familiar with the lifecycle of job ads posted there. I thought it was good enough to keep!

If that's not your cup-of-tea, Technorati has some great jobs open right now. Most of them don't even require you to

walk, cross-country ski, or walk with snowshoes, a distance of 2.5 - 3 miles without rest in all seasons, including winter temperatures as low as -20 degrees Fahrenheit
nor
shoveling sidewalks and building entrances and digging out snow around satellite dish area and other equipment
Not that there's anything wrong with that.

( Jan 12 2005, 12:58:23 AM PST ) Permalink


20050108 Saturday January 08, 2005

Managing Multi-Timezone Conference Calls I'm ringmastering a project that requires coordination with parties near and far. Most of the coordination for these kinds of things functions well with email, IM, wiki and CVS. But sometimes a plain-old conference call is in order.

Getting all of the parties lined with

It turns out that FreeConference.com offers tools for all of this for, well as the name implies, free. They make money up-selling premium services but I've got my first conference call with parties in three timezones setup through them and haven't paid a dime for it (which BTW, is their cost-per-minute-per-connection to have the dial-in number be an 800 number); I'll find out how it goes next week!

( Jan 08 2005, 10:26:54 AM PST ) Permalink


20050102 Sunday January 02, 2005

In Every Season: Turn, Turn, Turn I still feel a small sense of propriety for things I've worked on in the past. Even way in the past. Even when not an inkling of anything I'd worked on back then is likely to have survived any of the incarnations since then. On the other hand, I'm probably doing better than my friends whose efforts over the years completely dot-bombed-in-flames. I guess this is the time of year when I mark the time by life's events (including gigs) and reflect.

Hearing talk of Gamespot's decline (for me, Gamespot.com was a gig that started nine years ago and ended three later), like "Gamespot going down hill" is kinduva bummer. I was concerned when I read of Scott Rosenberg's book break but a lot of the fine folks I have fond memories of at Salon (four years ago) are still there and doing a great job! And like myself, a lot of folks that I worked with over the years have moved on to work on and accomplish great and interesting things, so it's all good.

OK, reflection time is over now.

Thankfully, I really dig what I'm doing now! Happy 2005 to all of my friends and colleagues past and present!

( Jan 02 2005, 11:02:38 AM PST ) Permalink


20050101 Saturday January 01, 2005

The Top Tsunami Sites I did a little digging around and found some of the top sites for relief information, reactions, pictures and videos of Sunday's Tsunami.

The South-East Asia Earthquake and Tsunami Blog
On going coverage of relief news, publications and other linkable resources
Tsunami Relief
Google's thumnail list of relief organizations
QuickBird Images of Tsunami Sites
Stunning satellite photos with before and after shots of Sri Lanka and Indonesia
Cheese and Crackers: Tsunami Video
Tons of links to amateur video's from the region that capture the power and immensity of the tsunami
Blog
I can't read Japanese but people that do are reading this a lot
Raw Print | Westboro Baptist Tsunami Statement
Those Kansas right-wing nutjobs (ya know, the "god hates fags" zealots) are at it again with their Sodom and Gomorrah crap. "Thank God for Tsunami. Thank God for 3,000 dead Americans!" Wow, you'd think the pastor at Westboro Baptist Church is really Osama Bin Laden. More and more, I think that seceding from the union sounds like a good idea.
Support UNICEF's Tsunami Relief Efforts
Ya know, it's for the kids. $50, $100, $250, $500 or any amount you can give will make a difference.
Network for Good :: Help Support Disaster Relief in Southeast Asia
A good collection of emergency response and on-going relief organizations. Certainly there must be one for you.
CNN.com - Aid groups accepting donations for victims
Who says that big media has no soul? They care too!
MoveOn.org: Tsunami Relief
The Bush administration stepped forward first with $15M, then $35M and today brought it up to $350M. MoveOn's petition says
President Bush and Congress must offer whatever assistance is necessary to prevent further human suffering in the wake of the tsunami disaster.
I arrived at this list with some simple pokes at the data coming through at Technorati, I might try to spiff this up a little bit and turn it into a feature.

( Jan 01 2005, 03:51:00 AM PST ) Permalink


20041231 Friday December 31, 2004

Pictorial Fallout Seems like there are no events without photostreams anymore.

Here are some from last night's fiesta and after party that followed Scobleizer's Technorati visit.

( Dec 31 2004, 12:49:48 PM PST ) Permalink


Hey 2004, don't let the door hit you on the butt on your way out! I have a lot of things to be thankful for this past year. However, 2004 brought on a sufficient number of painful kicks-in-the-teeth, I say good riddance!

The bad stuff has been damn bad:

But there was good stuff too:

Wishing for a good 2005!

( Dec 31 2004, 10:56:36 AM PST ) Permalink


20041230 Thursday December 30, 2004

New Years Resolutions I have a number of personal, professional, creative and spiritual objectives, but I won't chunk them into specific intervals or milestones. At least not here and now. So for next year, I'll keep it simple: I resolve to help make the world a better place than it was last year.

That's my new years resolution.

( Dec 30 2004, 02:13:30 PM PST ) Permalink


20041224 Friday December 24, 2004

Learning to say kaddish Recent reading: Living a Year of Kaddish

One of the things rattling around my mind these days is grief. I recently listened to my dad, aunts and uncles eulogize my recently departed grandfather at his burial service in New Jersey. It's given me plenty to think about as far as what I knew of him on both a first and second hand basis. Growing up on a coast opposite of his, my knowledge of him has been the product of the fleeting visits and the lore passed on by my parents. But I'll always be fond of the interest he took in my goings about, the twinkle in his eye that sparked when he engaged in conversation with me and some of his funny little habits like cutting an article out of the newspaper for some anticipated future reference that would never take place.

The traditional grieving process has a number of rituals and practices that are vaguely familiar but only by hearing or reading of them. I've not before been proximate to these traditions but my grandfather's passing has produced an interest in them. So I picked up Living a Year of Kaddish by Ari Goldman to learn a little more about it. The book consists of a succession of short thought recordings (even blog-like, as it doesn't read like a diary) of the year that followed the death of Goldman's father. The traditional purpose of kaddish, a daily prayer for the deceased (preferably three times a day), is to help the loved one get closer to and eventually arrive at gan eden (paradise). Saying kaddish for eleven months and then on the death anniversary (yahrtzeit) is an obligation of the children but is also a prayer for all who grieve. At least, that's my understanding of it and my knowledge is nominal with these things. But I have to say that Goldman's take on it, that the purpose of kaddish is more inward looking, resonates with me more.

To me, kaddish is more for the living than for the dead. I believe that in my daily recitation of the prayer, I was coming to terms with who my father was and who I am. If I missed a day of kaddish, I suffered, not my father.
When I die, I want my children to say kaddish for me, but for themselves, too.
Indeed, I've been thinking a lot about who my grandfather was, who his eldest son, my father is and who I am. And what will my children know of my father and myself in the years ahead. There is much to consider. I previously didn't know the kaddish prayer but I'm taking the time to learn it now.

( Dec 24 2004, 03:31:40 PM PST ) Permalink


20041218 Saturday December 18, 2004

Rollin' rollin' rollin', keep them servers rollin' Technorati's entire infrastructure was brought down this morning in order to move it.

Here's the flickr tag to follow it.

No ETA on service restoration but I'll post updates where possible.

( Dec 18 2004, 03:01:57 PM PST ) Permalink


20041214 Tuesday December 14, 2004

Greener Pastures

This weekend Technorati's network and server infrastructure is going to move. In one big fell swoop. Well, hopefully nothing will fall.

The home page sez: "Movin' on up" cause Technorati is substituting the Jefferson's theme song for the old ops/facilities anthem, the Talking Heads' "Burning Down The House"

( Dec 14 2004, 12:47:43 AM PST ) Permalink