I just got myself an iPhone and I’m extremely pleased with it. I think it’s the best cell phone on the market - a sheer pleasure to use.

The purpose of this post is to alert new iPhone customers about a security vulnerability in AT&T/Cingular’s Voicemail system that has not been fixed for more than a year. I first wrote about this on February 1, 2006: Exploit Cingular Voicemail Vulnerability via Caller ID Spoofing. As soon as I got my new AT&T/Cingular number, I tested for this vulnerability and I can confirm that it still exists for new AT&T/Cingular accounts (atleast for iPhone customers). I can’t force AT&T / Cingular to fix this issue, but I can tell you about it so you know what to do to protect yourself from this vulnerability.

My involvement with the Wild West side of Python came somewhat accidently. I am helping organize PyAtl and on June 14th we had an incredible meeting! My company Racemi gave a mind boggling demo of our datacenter management tool that is written in all python. Our FlagShip Product Dynacenter allows any OS, including Windows to move around to different hardware in the time it takes to warm reboot…go Python! Finally, Google gave two presentations, one on Cross Site Scripting Attacks and one on Twisted. We also officially launched the PyAtl website that night which is running the bleeding edge Turbogears stack of Sqlalchemy,Genshi, and Toscawidgets. My friend Alberto Valverde is in charge of Toscawidgets and the concept is really awesome! If you haven’t met Alberto yet, you should, he is one of those rare exceptionally helpful, yet insanely smart people.

Here is where the the fun started…

I invited Mark Ramm and Jonathan Lacour to come to our meeting and talk about Turbogears. Mark and Jonathan mentioned that on the way up to the meeting they had a crazy idea. How about building Turbogears on top of Pylons? They announced an experimental sprint the next weekend and this is where things got wacky!

Rick Copeland, Jonathan, Mark,Mike Schinkel, and myself met at Jonathan’s house and started to experiment. We ran into an initial snag with understanding the pylons controller and I called up Shannon Behrens, another friend, who is insanely smart and incredibly helpful. Shannon works on the Pylons trunk and asked him how we would mount Turbogears on top of Pylons. After he got over the “you want to do what!”, he helped us with some good advice. At some point we all went to get some Pizza, then came back to watch Jonathan and Rick go into the “Zone”. After they came up for air, a controller was working and Frankenstein was born..mu ha, ha, ha, ha!

It was 1 in the morning by the time we all quit, but Mark, Jonathan, Rick and I decided to meet at Panera the next day at 1PM to finish it off. A little more work was done the next day, but part of the day was spent just hanging out and talking shop which was pretty cool as I hadn’t met Mark or Jonathan before. It turns out Mark and I have a bit in common as we both grew up on a “Ranch type compound” for parts of our lives, we both have been SysAdmins, and we are both writing a Python book right now. Mark is a really fun guy to hang out with for anyone who hasn’t met him yet!

So, after the weekend was over with I started to hear about some of the excitement. I emailed my most educated friend Mr. Phd from Caltech Titus and mentioned maybe he could contribute with some Twill stuff for TG2. I talked via email a little with Kevin Dangoor and noticed his big announcement.

Apparently, people were really fired up about the collaboration between Pylons and Turbogears. Lets face it, I am very excited that all of these smart people are working together! It now seems that some momentum in the battle for the perfect Python Web Application has shifted, as Pylons and Turbogears have the 800 lb Gorilla of ORM’s in SQLAlchemy, and they have Toscawidgets which is about to come into its own.

I have written several small web applications in Turbogears and Django and I like both. Currently Turbogears and Pylons don’t have a way to graphically manage the database like Django’s admin tool and the API isn’t as stable, but from what I hear this is about to change…..

I do get the impression that many people in the Turbogears/Pylons world feel left out and a common heard rallying cry is that Django has a “Not invented here attitude”. Whether this is true or not, I learned this past week that if smart python programmers feel they aren’t apart of the fold, they are capable of creating an uprising and doing just about anything!

I will close with this comment, Ian Bicking, who wrote paster which I think is pretty sweet, mentioned in a fairly famous post that it would be great, but unlikely that Pylons and Turbogears would merge, yet the impossible happened and the two frameworks are closely working together. May I suggest an equally implausible scenario? What if Django, Pylons and Turbogears worked on developing an interchangeable API? Is this impossible…you tell me!

Version 3 of the GPL, years in the making, will be released in its final form on Friday, June 29. Visit the Free Software Foundation web site at noon Friday East Coast US time, to see Richard Stalllman talk live about this historic occasion. You can find some commentary about the current state of feelings in the computer industry about v3 in a recent blog of mine, and Allison Randal, who has participated heavily in the development of v3, has written about it extensively on the O’Reilly site.

Mark Ramm, author of the TurboGears book, just posted his experience with working on the next generation of TurboGears. All the details aren’t totally in yet, but it looks like they are working with Pylons code somehow. I hope the work that is being done will help promote cross-framework cooperation and strengthen Python’s (already strong) position in the web arena. Personally, I like the diversity in the Python web realm, but I think some more sharing and cooperation could only be a good thing. I’ll continue to watch the progress of the new TurboGears and post back here with my findings.

PJE posted today regarding the status of PEAK, the Python Enterprise Application Kit. In the above referenced post, he linked to an email he sent to the PEAK mailing list with a subject of “PEAK Status Report” which goes in depth on where PEAK is and where it is going. I’d highly suggest reading the status report. However, the nutshell is that PEAK as a unified entity is pretty much dead, but some of the individual projects inside of PEAK such as setuptools, RuleDispatch, and wsgiref are doing rather well. Even in its “dead” state, PEAK still has plenty of promising and usable pieces.

Untangle could turn out to be a poster child for free software. The company started out considering both free and proprietary software for its platform, but settled on a flat-out, pure-play open source approach. In return, they demonstrate the kinds of enhancements a commercial firm can make to free software in a range of areas commonly known as productizing.

The Summer 2007 issue of The
Perl Review is out, and it’s a special edition for the href="http://perlworkshop.dk/">YAPC::NA sponsored by href="http://www.livetext.com">LiveText (who are looking for good
Perl programmers in the Chicago area). href="http://www.theperlreview.com/Images/covers/v3i3-cover-large.png"
>This issue’s cover is some of the conference detritus I’ve
collected over the years.

The >Summer 2007 issue of The Perl Review
is online and ready for download. Subscribers should have already received an email
telling them all about it.

In this issue:

• Managing Modules Without Going Crazy — brian d foy
• Carp & Friends — Alberto Simões
• The Perl Debugger — Richard Foley
• Tk Mega-widgets — Charles Colbourn
• and other stuff

We’re always looking for people with good Perl stories to tell to, and you can
submit an article idea.

I’m registering for OSCON, and Suzanne Axtell asked me to say some kind words about the conference. I always have trouble deciding which tutorials to take.

If you read up on the Model-View-Controller (MVC) design pattern, you might find yourself a bit confused. In fact, I found myself confused by it when I first started reading about it, because there are plenty of resources out there to describe it, but so many of them seem to have different flavors of MVC and different diagrams explaining how data flows that it’s no wonder that programmers are bewildered about it. Fully believing that I don’t want perfect to be the enemy of the good, I’ll show a few practical implementation details of one way of looking at MVC, primarily focused on the Web needs.

On Thursday, June 14, 2007, the Python Atlanta Group met and had an eventful time. There were two presentations by Google and one by a company named Racemi.

The first presentation was an introduction to the Twisted network application framework by Cary Hull of Google. It was a very informative introduction. He used Twisted itself as the presentation engine for the talk. Nice touch. I’m hoping for Mr. Hull to present in the future either a more in-depth presentation on Twisted or an overview of Zope interfaces (hint hint, Cary :-). Or both.

The next presentation was by Luis Caamano of Racemi. I’ll only gloss over this presentation now because I plan to get more detailed with them soon. But their product is pretty amazing. Basically, you can move operating systems from machine to machine in a data center. Yes, it sort of sounds like what you can do with VmWare, but it really is very different. And it’s written in Python. Something over 200,000 lines of it (and something over 100,000 of test code) if memory serves me correctly.

The final presentation was on cross site scripting by Dan Morrill of Google. It was an interesting talk emphasizing the necessity of sanitizing user input. This was an interesting talk for a few reasons. First (and in increasing order of interestingness), he used the BaseHTTPServer and CGIServer from the Python standard library. Second, several of us are perpetually working on projects using Django or TurboGears, so topics of this sort are always interesting. (As a side-note, Mark Ramm was there and mentioned that Genshi and Kid automatically escape data that you pass into a template, so should be nearly non-susceptible to javascript-injection-type attacks.) Third, he pointed out projects which Google is working on such as Gears and the Google Web Toolkit. Again, these types of projects are always of interest as some of us are constantly working on things which could benefit from Google code goodness.

Interestingly, and as already mentioned, Mark Ramm showed up for the meeting. He mentioned some collaborative effort which was beginning with TurboGears and Pylons. I’m still waiting on details on this, but it sounds promising. I’ll post back here as I learn more.

The meeting was packed (not surprisingly given Google’s presence). Tons of new faces were there. I hope there was enough interesting material to bring some of those folks back. It’s always a good time to hang out with folks with similar interests.

Somehow I missed James Carr’s TDD Anti-Patterns late last year. I’ve perpetuated almost every one at least once. If you’re new to testing, browse the list, think about each entry, and watch for it in your own code.

I actually like fixing bugs and optimizing code. It’s satisfying to simplify a piece of code while making it perform better and use less memory.

I’m a terrible guesser, though. I guess about where bottlenecks are correctly perhaps half of the time, and likely less often than that. To make the most of my available time, I need a good profiler.

I’ve tried to use GNU gprof, but the requirement to recompile all of my software specifically to use gprof was too much. Instead, I use Callgrind.

Callgrind works like Valgrind (and it’s part of the Valgrind tools now). Run your program through Callgrind as normal (valgrind --tool=callgrind program_name opt1 ... optn); it will collect statistics about the run. Then run callgrind_annotate on the output and see a nice report about where your program spends its time. Though this is only the most basic of Callgrind’s features, it’s often sufficed for me to find and fix true bottlenecks.

If you need more features, I hear that KCachegrind is a useful visualizer of Callgrind’s statistics. I haven’t used it enough to discuss its value.

Because of Callgrind, I spend more time optimizing bottlenecks than I do finding them. Thanks to its developers and all contributors!

Somewhere among the readers of the O’Reilly Network are people who know something relevant to pending software patents. For instance, you might have seen papers, conference presentations, or actual working code similar to a “Cooperative mechanism for efficient application memory allocation” or a “User selectable management alert format.”

The US Patent and Trademark Office wants your help. Through the Peer to Patent project you can look for prior art, discuss its relevance with other people in your field, and tell the patent office why they should take it into consideration–and you’ll be listened to.

At noon Pacific time today (June 18), members of the Peer-to-Patent project team will discuss the project on the New York Law School’s Democracy Island in Second Life. This is sure to be informative for anyone interested in public policy regarding inventions, and perhaps a memorable occasion in a project that could change how government interacts with citizens.

Head over to the O’Reilly Network OnLAMP site for my article Why Do People Write Free Documentation? Results of a Survey, which analyzes the 354 responses to a survey on the O’Reilly Network.

Via John Lilly (COO of Mozilla), Steve Jobs misses the gold ol’ Browser Wars.

Now I’m not the chief of a successful hardware/software/consumer products company, but the goal of being #2 by swallowing #3, #4, #5… seems somewhat wrong to me.

The Linux Foundation–the new organization formed this year from the merger of Open Source Development Labs and the Free Standards Group–is holding a summit this week at the main Google campus.

I think we can already call the summit a success, and an indication of the Linux Foundation’s acceptance by its community, just on the basis of the many busy, well-known people who chose to show up. It was a great place to make connections among both speakers and attendees.

The first day was a pretty public affair, with an audience of 200 that included journalists, so the discussions were basically polite and stuck to acceptable debates such as how to recruit more developers. I could not, unfortunately, attend the sessions later in the week where (to use Executive Director Jim Zemlin’s metaphor) the sausage would be made. But some interesting points came up anyway.

Thanks to Simon Morris’s A Rose by Any Other Name, I can now identify myself as a Neo-Desktopian. I’d say more, but I’m too busy right now trying to figure out why JSVim won’t launch on my iPhone; I’m trying to debug a compilation problem in the Monopolight GCC front-end running on EC2.

I think it’s a CSS problem.

One of the sub-projects of Pugs is a series of Perl 6 sanity tests which define a minimal set of useful Perl 6 features. The idea behind those tests is that a Perl 6 implementation which can pass the sanity tests supports enough features so that it’s possible to bootstrap the rest of Perl 6 in that minimal implementation.

The Parrot project recently borrowed those sanity tests for the Perl 6 on Parrot implementation. (I work on Parrot in part because I believe that Parrot’s compiler tools are much more suitable for building compilers and languages than anything else I’ve ever used.)

Though I spend more of my Parrot time these days applying submitted patches, fixing bugs, and refactoring code, I try to make time for new development. I heard that we almost had all of the first suite of sanity tests passing and decided to see if I could improve the situation.

I’m a Vim bigot, but I reluctantly leave my comfortable off-white-on-black terminal windows for a web browser once in a while. Sometimes I even have to type more than a word or two in a textarea.

I used to grumble every time a site provided a tiny little text box for entry, or when I wanted to make an edit and left a row of j and k characters splattered through my text.

Now I use the It’s All Text Firefox extension and, before I let annoyance creep up on me, flick my mouse to a little blue box at the bottom right corner of the text area. With one click, up pops Gvim.

Suddenly, it’s worth typing more than a sentence into a web form again.

Thank you to all contributors to It’s All Text for making textareas usable.

One of the dimensions we watch at O’Reilly is the “platform” category. At a high-level in our taxonomy, the Platform category can be split into two groups — Open Source and Proprietary. The following charts represent the “platform” world for the first five months of each year [January thru May of 2003 - 2007].

This first Chart shows Dollars for the first 5 months of each year.

The next Chart shows Units for the first 5 months of each year.

Here is a little context for these charts. The data here is for the whole computer/technology publishing market, and not just O’Reilly. The data is actual cash-register sales in bookstores, as measured by Neilsen Bookscan, throughout the United States. Typically about 85% of this market is Amazon, Borders and Barnes and Noble.

The market as a whole is down about 10% compared to 2006. The open source and proprietary trends on the unit chart have closely mirrored each other except for this year. On the revenue chart, open source has produced more dollars for the past 4 years and looks like it will again this year. I thought you would find this data interesting.

Technorati Tags: Analysis, Books, Data, linux, Markets, mysql, open source, perl, php, python, Trends

Two interesting conferences are coming up in Lowell: Penguin_day (Friday, June 22) and Grassroots Use of Technology (Saturday, June 23). They’re aimed at non-profit organizations and are very inexpensive. Both are hosted by the Organizer’s Collaborative, a small, dedicated non-profit that helps other non-profits by teaching them to use open source technology. So naturally, there’s a good deal of overlap between the conferences.

Presentations range from the imminently practical (e.g., Digital Advocacy on a Small Budget) to big-name (prize-winning author Allison Fine) and geeky (Open Standards: Why the grassroots should care).

Lowell is a fascinating place, well suited to these events. The setting of the early days of the Industrial Revolution in North America, it has its up-and-coming neighborhoods as well as areas mired in poverty. Its ethnic diversity is invigorating and also challenging. And it’s home to a U.Mass. campus with strong computer-related offerings. If anything I’ve mentioned touches you, check out the conferences.

But the idea of the Mozilla Foundation de-emphasizing applications in order to transform ourselves into a general purpose “platform” organization — giving up the fundamental focus on the human being a application focus provides, reducing our ability to help individuals directly — seems an absolute non-starter to me.

— Mitchell Baker, Application vs. Platform Focus

“Mozilla” is the people who understand that telling Mozilla what it should be doing is like saying “nobody in my neighbourhood cares about the litter” but not picking up a piece.

— Mike Shaver, now don’t take this the wrong way.

You can substitute just about any F/OSS organization for “Mozilla Foundation” in the first quote and any F/OSS project for “Mozilla” in the second.

If you’re still writing your own authentication for your websites, you may want to get with this program. Have a look at the bottom right of this page:
www.buxfer.com/index.php
Yep, almost everybody has one of those accounts these days, and more and more of those users are getting tired of endlessly multiplying username/password combinations.

I realize that Microsoft tried to do some Passport service in the past, but you can base your website’s auth on any site or combination of sites, even if they don’t have an API. As Tony Stubblebine has put it: “A login form is an API.”

Suppose you’re the author of a software project. You spend your time developing new versions and prefer to add new features in new versions.

Suppose you’re the user of a software project. You prefer not to upgrade to new versions. You might want some new features.

Suppose you’re the distributor of a software project. How do you reconcile these desires?

I have some thoughts on backporting to publish in the near future, but I would like to get other perspectives first.

Advocates of software standards and open source–including such major backers of the Open Document Format as Sun Microsystems and IBM–have been arguing a year against the standardization of Microsoft’s proprietary Office formats in the form of OOXML. We’ve heard lots of arguments about what OOXML doesn’t come up to standards of what makes a real standard. Now Sam Hiser, an O’Reilly author and a director of the OpenDocument Foundation, has written a sleek 14-page summary of the major arguments against making OOXML a standard.

I know Piers Cawley hates the word “metaprogramming” as much as I hate any of my pet peeves, but Tomasz Węgrzanowski just made a fantastic point:

Every use of text-based runtime code generation is a failure of language’s reflection model.

(See Ruby and methods with weird names.)

Language–and library–designers should keep this principle in mind. One of the explicit design goals of Perl 6 is to enable all of the metaprogramming of Perl 5 without exposing as much of the machinery. (For some reason, assignment to dynamically-scoped typeglobs frightened people.)

If you loosely follow the US presidential party nominations like I do, you know the second republican debate was last night. I heard there were surprisingly strong results for Ron Paul in the MSNBC online survey, so I thought I’d have a look myself at http://www.msnbc.msn.com/id/18963731/.

Six questions, a vote button, then a little note instructing me to “Vote to see results.” Well, I didn’t watch the debate and can’t even name all the candidates, so I’d really like to just see the results. If I were to make up answers, it would serve only to inflate the size of their sample with garbage.

Well… who knows, maybe they thought of this and I’m just supposed to press the Vote button with no selections to get through. Nope, a pop-up informs me to “Please make a choice before submitting your vote.”

So I did.

I had been fighting with some silly old-fashioned servlet code. No need to say, I was not getting the upper hand. Never having been a web developer, I was baffled with why my POST request never got the contents I expected it to.

I spend a lot of time searching text files. More accurately, I spend a lot of time searching nested directories of text files. For source code, I know I should use Ctags, but I’ve never quite made the switch.

For plain text files (books, articles, stories, weblog entries, notes, contracts, et cetera), I’m still a GNU grep fan.

I spent a few hours in the past week editing a book manuscript and producing well-formed and valid DocBook XML. (I wrote two books in DocBook XML. While it’s a great file format for producing a book, it’s a face-stabbingly hateful format for actually writing a book.) Unfortunately, the conversion process to DocBook revealed some problems in the source material. In specific, certain links from one part of the manuscript to others were invalid.

I needed to find and fix the dangling links in all fifteen book chapters, spread out in several dozen individual files. Grep and a little bit of command-line magic made the task much, much easier. I ended up with the pattern:

vi $( grep -l 'L<refactoring_strategies>' ?_*/*.pod)

That is, search all of the .pod files in directories whose names start with one character and an underscore. For all of those files which contain a link to an anchor named refactoring_strategies, print their names. Open that list of files in Vim.

I still had to edit plenty of text, but finding only the files I needed saved me a tremendous amount of time. Throw in grep’s -r (recurse into subdirectories) and -i (use a case-insensitive match) switches, and I’m very happily productive.

Thank you to everyone who’s contributed to grep and GNU grep through the years. Your work helps me work, every day.

Yesterday was the Perl Teach-In at the BBC. People have been saying for months (years probably) that Perl is dying. If that’s true, then I’d like to know why the fifty places on the course were fully booked in less than two days and why another forty people signed up to be on the waiting list.

Perl certainly isn’t dead. On the contrary, the demand for Perl programmers in London is greater than I’ve seen it for many years - just take a look at the archives for London.pm’s jobs mailing list.

So that’s why fifty or so Perl programmers were willing to spend one of the hottest Saturdays of the year hidden away in a BBC conference room listening to me talking about Perl. It was particularly gratifying to see that most of them were people that weren’t already involved in London.pm or any part of the Perl community. It’s always been my belief that the majority of people who use Perl regularly aren’t part of the Perl community - so it was good to be able to reach out to some of these people and encourage them to join us.

The day seemed to be successful. Pretty much everyone told me that they enjoyed themselves and that they found it useful. That makes it very likely that something similar will happen again in the future. But no firm plans have been made yet.

I’ve put the slides online and they’re under a Creative Commons licence so that anyone else can use them to run a similar course in their city. The presentation was also recorded (well, until the microphone batteries ran out twenty minutes before the end) and those recordings will go online at some point in the next week or so.

All in all, I’m very pleased with how it all went. It was an interesting experiment and I’m glad that it all worked out so well. Thanks to all the attendees for turning up and to London.pm and BBC Backstage for their help in organising the event.

Google Gears, as you may have heard, is a browser extension that lets you develop applications that can run offline. If you haven’t already, try out the sample applications to get a feel for the functionality Google Gears has to offer. You can even use it to read Google Reader offline.

It’s a good idea to brain-storm in the possible security implications of Google Gears because it facilitates web code to act upon the user’s local disk (sand-boxed with the browser’s same origin policy). I’ve spent a few minutes looking at the architecture, and here are my initial thoughts:

The last amusing discussion in perl6-language@perl.org mailing list was a proposal on renaming Hash to Dict. Perl and, after it, Ruby use the word “hash” for their associative arrays while Python and Smalltalk use “dictionaries” and Java uses “maps”. As so many things are changing while going from Perl 5 to Perl 6, the suggestion was attempting to introduce a “better name” while it is still possible.

Corey Goldberg has begun work on a Python-based open source performance testing tool. This topic is near and dear to my heart since I spent a few years doing quite a bit of performance testing work. Good luck, Corey. I’m interested to hear of your progress.