Sunday November 30, 2008

I usually don't rant about economics but I wasn't shopping on "Black Friday" (nor will I be tomorrow on "Cyber Monday") - I'm trying to figure out how to tighten my belt. How it is that I, someone outside of the real estate, finance and auto industries that are so problem plagued, am getting caught in our economy's downdraft? Well, let's see.

Last January, Business Week raised the question "When is an institution too big too fail?" Until September of this year, the financial industry's downward spiral meandered along, like a persistent flu. There were bank failures but the conventional wisdom seemed to be that this was the market at work, winnowing the weak. The bad news ebbed and flowed: mortgage failures, rising oil prices and the weak dollar were countered by stimulus package checks, housing sales leveling off or even rising (where prices crossed their local tipping points) and vibrant web 2.0 and green enterprises. There had been bank failures this year but it took the evaporation of really Big institutions, Lehman Brothers and Merrill Lynch to put Business Week's question on everyones lips. To free market purists, the answer is obvious: whatever may come, let the failures fail. But the reality is that when an enterprise is so big that its failure disrupts significant portions of the overall national and global economy, whatever may come of its failure won't be good. Everyone suffers and bigness is the problem. When these companies become indispensable institutions, we should be afraid.

It seems for years there's been a breakdown in accountability. Loan originators could resell their loans and write new ones, no harm no foul. Right? But one of the key problems with that system is that the originators don't have any skin in the game. The have a money merry-go-round and whoever is left holding the paper (big institutions and their investors) draws the short straw. It's total madness. To date, all of the bank failures have resulted in consolidation in some form or another. Lehman is absorbed by Barclays. Merrill by BoA (which already absorbed Countrywide). The big are getting bigger as the competitive field shrinks. Ironically, this perpetuates the problem: bigness. What happens when Barclays or BoA start wobbling next? Now we have yet bigger institutions that are again too big to fail.

Among the remedies dismissed by free-market adherents is one of the Federal Government investing, taking an ownership stake in the banking, insurance and auto giants who have exposed themselves to risk that has subsequently blown up in their faces. "The government won't know any better how these companies should be run" goes the admonishment. But as if it isn't clear by now, the executives paid the big bucks to know how they should be run apparently don't either. As Newsweek explains in The Monster That Ate Wall Street - How Credit Default Swaps Became a Timebomb, the financial industry had no shortage of creativity when it came skirting the liquidity requirements imposed on them in the years following the S&L crisis. Is it really such a surprise? Michael Lewis (Liar's Poker, Moneyball) recounts in The End of Wall Streets Boom (Portfolio Magazine / December 2008), there were those calling Bullshit but things were just going too damned well for those alarms to be heeded.

It's unescapably clear now that the old adage applies, "if it's too good to be true, it probably is." Until recently, I thought this was only impacting me with the difficulty I had getting my mortgage. But no, the cavalier rating agencies ("the fox was guarding the hen house"), excessively leveraged financial arrangements and detached accountability have led us down this financial rabbit hole into what some now describe as a death spiral. It's not just a Wall Street problem, it's spillover to Main street has cacaded down Sandhill Road. Here's the ominous and infamous slide deck from Doug Leone and friends at Sequoia Capital:

As bummed as I am about seeing colleagues depart and seeing my paycheck shrink, I'm actually optimistic about the future. Valuations on real estate seem to be reaching reality: they're hitting thresholds that people can afford with conventional financing. Technology continues to fuel innovation and innovation holds the potential to re-shape markets. Come Inauguration Day, it looks like Obama is coming into office surrounding himself with a team of economic advisors who are committed to preserving free markets but are also not so steeped in ideology that they're paralyzed about how to intervene.

I'm looking forward to this cloud lifting. That's my rant.

economy   banking   layoffs   finance   economics   obama   death spiral   venture capital  

Saturday November 29, 2008

Back in the 1990's I used FreeBSD fairly extensively. One of my favorite things about the FreeBSD project was the "ports and packages" system for installing libraries and application software. Since Mac OS X is, essentially, BSD with a lot of updated chrome, it's not surprising that there's a well functioning "ports and packages" system for it, MacPorts. While it's not perfect, MacPorts seems to function and dovetail nicely with everything I use my Mac for, more so than Fink. Sure, dpkg/apt-get seems to work OK on Debian, every effort I've encountered to apply that model elsewhere has left me disappointed... anyway, yum seems to work well enough, I don't expect to use Debian again.

Recently I found myself with a port that would not install,

port install postgis

"Target org.macports.checksum returned: Unable to verify file checksums" postgis

port -d selfupdate
port clean --all postgis
port install postgis

macports   freebsd   postgresql   postgis   macosx   fink   debian   yum   package management  

Friday November 28, 2008

Since Technorati announced pay cuts for the staff earlier this week, I've been a little worried. The mortgage, an upcoming bat mitzvah (nothing opulent, really), doctor bills... the world won't wait for the economy's doldrums to turn around. I think I'll find ways to to tighten our belts (bag lunches, cancel the gym membership, etc) but if you're currently more fortunate than I am and so inclined, this PayPal Donate button is a way you can help.

If I end up with more than needed, I'll simply donate the excess to a worthy charity.
Thanks!

Thursday November 27, 2008

Public service announcement: your low-tech dowdy public libraries have slicked up high-tech. The old days of long searches through card catalogs and filling out forms in triplicate are gone. Since moving to the east bay several years ago, I've been impressed with the Contra Costa County Library's online catalog that searches all of the branches in the country, online reservations and inter-branch transfers. One of my favorite features is the visual topic clustering.

When searching for "django", a hub-and-spoke is displayed with related nodes such as "reinhardt" and "guitar" as well as misspell candidates. The search results are pretty good too, the first result is for a Gypsy jazz guitar (Django Reinhardt's signature style) instructional video by the main guy from Hot Club San Francisco (Paul Mehling can often be found gigging here in the east bay at the Left Bank in Pleasant Hill, good stuff). Overall, the selection of books, CD's and videos matching "django" was what I expected.

As fond as I am of Gypsy jazz, I'm also interested in the web application framework written in the Python programming language. Changing my query to "python django" brings up a different visual cluster with some of the same cluster terms ("reinhardt" and "guitar") but adds some new ones "monty", "boa" and "computer". The search results were exactly what I wanted: The Definitive Guide to Django: Web Development Done Right by Adrian Holovaty and Jacob Kaplan-Moss and Sams Teach Yourself Django in 24 Hours by Brad Dayley. I'm planning on using django (the python web app framework) for a project (not work related) and, while the online docs are pretty good, having a book (or two) to refer to is definitely welcomed.

All said, I'm a fan of the search and clustering technology enabled by AquaBrowser that the CCC library is using, it's had me wondering how well it would perform against the more volatile data set flowing through Technorati.

django   python   programming   gypsy jazz   search clustering   library   technorati  

Wednesday November 26, 2008

The incidence of Wordpress compromises I wrote of in the spring is still high but the rate of new infections has dropped considerably. A lot of people learned of their blogs' affliction because they were not getting indexed by Technorati. Props to the folks from Google and the Wordpress team for getting the message out too.

Yesterday's release of Wordpress 2.6.5 doesn't target SQL injection or XML-RPC vulnerabilities, this time it's a cross site scripting vulnerability.

The security issue is an XSS exploit discovered by Jeremias Reith that fortunately only affects IP-based virtual servers running on Apache 2.x. If you are interested only in the security fix, copy wp-includes/feed.php and wp-includes/version.php from the 2.6.5 release package.
2.6.5 contains three other small fixes in addition to the XSS fix. The first prevents accidentally saving post meta information to a revision. The second prevents XML-RPC from fetching incorrect post types. The third adds some user ID sanitization during bulk delete requests. For a list of changed files, consult the full changeset between 2.6.3 and 2.6.5.
read the full post

wordpress   security   technorati   blogging  

Tuesday November 25, 2008

It's been a long time since I've felt hopeful about the outcome of an election. I remember well the civil rights anti-war marches of my childhood. The recent years have felt like a return of profound delusional corruption and polarization that marked the Nixon era. The most peculiar aspect of it is how it came to a head with desperate gasp of Sarah Palin's VP nomination. That the questions about her qualifications were questioned at all demonstrates the height of delusion. While the republican fringe ran to embrace her, those with a brain could only ask "WTF?" and cross party lines. If she had been equipped otherwise between those legs, it would never have happened. It's like reverse sexism, if it had been a man with that background and view points, he would have been laughed out the door as a naive hillbilly. (Yep, I am a reasonably educated urban elitist, so?) Instead, we were treated to tragic comedy in slow mo. The graphic included here is a chart of the blogosphere's mention trajectory for Sarah Palin over the prior 100 days.

The timeline starts off with the igloo phase: practically nobody has heard of her and nobody is talking about her. Then there's the nomination at the republican convention. Followed by interviews, the Tina Fey phenom, the election and ...back to the igloo. So long, Caribou Barbie.

All of the talk about her PR representation, book deals, etc are for naught; she's proven at every opportunity that she has nothing to say that is meaningful and contributing to moving our society forward. Nonetheless, it's amusing to read the conservative bloggers who talk about Sarah Palin as "the future of the republican party." As long as they stick to that meme, they're assuring themselves falling further adrift of where this country is going. Bon voyage, don't let that iceberg hit you in the butt on the way out!

Sarah Palin   Election   Politics  

Sunday November 23, 2008

At long last, I'm reviving this blog from dormancy. A lot has happened since my prior posting here. In no particular order:

• The arachna.com server is hosted on a new machine -- nice upgrade.
• I gave in: bought a Wii, fun has ensued.
• The blog software is upgraded (as of today, it's running on Roller 4.0).
• I've been leading a complete ground-up rewrite of Technorati's crawler (still rolling that out though).
• On the home front, we bought a new house and moved across town.
• Coached my kid's soccer team and ended up with a strong second place.
• I've purged a ton of spam from Technorati's index ("splogs"), work to keep the index coverage high while keeping the spam out is ongoing.
• I've been working fairly extensively with Amazon's infrastructure services.
• I did not get married but when you fill-in previously empty parts of your facebook profile, it looks like a a "new" status. Thanks for all the congratulations though :)
• A lot of new software investigations and implementations: thrift, hadoop, solr and so on.

But they'll wait for another post. It's nice to be back!