It’s Monday afternoon, the day before JavaOne 2003 begins. I’m here early because I’ve been house hunting in the Bay Area over the weekend. I just got a call from my real estate agent that our offer has been accepted on the house we’ve selected. We’re all set to move from a large, executive home in Orange County to a cramped townhouse in Fremont, a suburb in the East Bay.

I mention these personal facts to point out that my family and I are gambling on the future of the tech industry in the Bay Area. We lived here during the craziness of the dotcom boom, when recruiters seemed to be stalking me and insane amounts of money were flying in all directions.

But things are different now. Since we moved away a couple of years ago, the tech landscape has become more of a barren wasteland. And recruiters, once so friendly, no longer even bother to reply to inquiries. The ones that are still employed as recruiters are inundated with resumes for every job opening they post.

Since the heady days of the boom, industry expectations have settled back to earth and ROI considerations are once again important to those who hold the corporate purse strings. One of the most significant effects of this has been the inexorable, and accelerating, trend toward offshore IT outsourcing.

To someone like myself, who’s been in the computer business for more than a quarter of a century (and always in demand), the prospect of becoming unemployable at 40-something is downright frightening. In the current IT labor market, I’m too expensive to hire. A company can get five people offshore for the cost of one of me.

Now, the standard reaction here would be to call for action to block this trend, legislate against it, restore things to the way they used to be. But, pragmatist that I am, I think that’s an exercise in futility. The market does what the market does - it’s that simple. The only way to influence the market is to offer a better value proposition.

Some percentage of software development is going to be lost to offshore permanently. We just have to accept that as a given - erecting barriers always ultimately fails.

But will all software development go offshore? Is this the end of the software industry as we know it? I think not. In fact, I think the best stuff will remain here. The fun stuff, the stuff we got into the software business for in the first place.

The offshore operations, of necessity to keep costs down, will become rigidly structured, assembly line operations (if they aren’t already). You need an accounting system? Fine they can do that and do it very well. Inventory? Financials? No sweat, done it a million times. But if you need highly customized, innovative, ground-breaking stuff; software that’s integral and vital to a new hardware device or business process or biological model or whatever, you need a tightly focused team onsite that can adapt and react and make it work right now.

It strikes me that this pattern echoes the Agile vs. BDUF debate. Offshore by its very nature has a high communication latency and the customer, almost by definition, is disconnected from the development process. This favors the Big Design UpFront approach. For well-understood business processes, like accounting, inventory, etc, this can and surely will work out fine.

But what of the cross-disciplinary innovations that are the lifeforce that keeps high tech so vital? What about the unique challenges that apply only to narrow vertical markets and cannot be easily commoditized? What do you do when the only way to get the software working is to sit down with the guy who invented the widget or the dispatcher who’s been scheduling deliveries manually for 35 years? How do you achieve the Ah-hah! moment across 12 time zones?

It’s my belief that Agile Development, in one or more of its various forms, will take hold as a way to boost productivity. If an agile methodology can make five local people as productive as 25 offshore, then it once again becomes cost effective to hire people locally. The challenge is to demonstrate the ROI. Agile has to prove itself a superior way of producing results which can’t be duplicated offshore.

The transition will be tough and many of us will have to unlearn things we’ve always taken for granted. I think this may be the beginning of a revitalization of the software industry. Hopefully, it will lead to a better understanding among programmers of the value their work has to a company and how their design choices and develpment methodologies can affect the bottom line.

One thing is certain: we’ve all got to keep learning and growing as professionals. For me, that’s why I got into this business in the first place. I’m optimistic that things will improve and that the tech market in general, the Bay Area in particular, will recover and shake the world once again. This is where the great ideas are born and nourished. This the place where the magic happens. That’s why I’m moving back here - I want to be in position to surf the next big wave. I can feel the swells beginning to rise already.

If anyone mentions H1B I’ll just scream.