| Weblog: | The Python comunity has too many deceptive XML benchmarks | |
| Subject: | So effbot goes from crybaby to libelous cretin... | |
| Date: | 2005-01-25 18:49:06 | |
| From: | uche | |
|
I really hate to follow this ugly race all the way to the bottom, but a couple of Fredrik's posts here have gone far beyond the pale of civility, and I'm not going to put up with it.
|
||
Showing messages 1 through 3 of 3.
-
So effbot goes from crybaby to libelous cretin...
2005-01-26 03:29:44 faassen [Reply | View]
Let's all calm down a bit. I think both of you make valid points, and both of you feel your egos are being trodden on by the other at this stage.
Fredrik has gone rather overboard in the flaming and insinuations, I agree, but Fredrik is in my opinion correct that this article harms far more than it helps. It is rather surpremely ironic that your article is called "The Python comunity has too many deceptive XML benchmarks" and that then the benchmarks here fall apart miserably at minimal prodding.
You claim you want to improve the benchmarking procedure of XML libraries. Let's do that. What *is* up with the following:
* the absurdly low Pystone rating on a 1700 mhz centrino?
* why is Amara such a speed-outlier? Why couldn't Fredrik reproduce this?
* did you send output of the programs to the terminal or a file? If you did send it to the terminal, won't that skew the best performing libraries down unfairly?
* Isn't measuring whole-program running time for a sub-second measurement rather absurd? If you do that, won't it skew the best performing libraries down unfairly?
Let's figure out what's going on with this. We need to do this in order to get sane benchmarks.
Let's also put to rest the notion that parsing speed doesn't matter anyway and is not a useful benchmark. If what you are parsing produces a Pythonic datastructure, which happens in the case of cElementTree, then it *does* matter. How much is up for debate, but it does matter and it's a useful benchmark.
That said, obviously your test to find all verses that say begat is *also* a useful benchmark. It just happens that ElementTree is good at this too, because of this forementioned parsing into Pythonic datastructures. For instance, lxml.etree, which does not parse into Pythonic datastructures but is written mostly in C, can only beat it if it goes down to C completely and expresses it as an xpath expression.
Concerning benchmarks, even though Fredrik didn't publish his benchmarking procedure widely at the time I started to try to replicate them, and I think that's a mistake, it wasn't that hard to replicate them either. Later on he posted what he was doing to a mailing list. For what it's worth, I *can* replicate Fredrik's benchmarks concerning memory almost perfectly, and fairly well for performance figures (the difference there is likely due to platform differences).
And Fredrik, can you please shut up with the insinuations? Let's not blame on evil what we can blame on stupidity (sorry, Uche :) I guess Fredrik cannot believe some mistakes were made as he thinks you're too smart for it. That harms even more than Uche's article. It doesn't help your believability either, and distracts the debate from what it should be about.
-
So effbot goes from crybaby to libelous cretin...
2005-01-26 04:32:07 faassen [Reply | View]
My last paragraph was rather badly written, so I'll rewrite:
And Fredrik, can you please shut up with the insinuations? Let's not blame on evil what we can blame on stupidity (sorry, Uche :) I guess Fredrik cannot believe some mistakes were made as he thinks Uche's too smart for it. The insinuations harm even more than Uche's article. It doesn't help Fredrik's believability either, and distracts the debate from what it should be about.
| Showing messages 1 through 3 of 3. |





Rerun the darn tests in the proper way. Don't include the python startup time and don't write to the standard output. Measure the actual XML processing time, even better ask a few people to do it and average their results. It is pointless to claim impartiality when one of the frameworks is yours.