September 2004 Archives

Preston Gralla

AddThis Social Bookmark Button

To own a PC is to be a potential victim. A potential victim of phishers, trying to steal your credit card information through online scams. A potential victim of malware writers, trying to hijack your browser or record your every keystroke. A potential victim of Trojan writers, virus writers and more.

I’m researching a book about malware, spyware, and other pests, and if you’ve been victimized, I’d like to hear your war story. If you’ve found your PC filled with spyware, if you’ve inadvertently fallen for a phishing scam, or if you’ve been attacked in any other way, let me know. I’d like all the gory details - how you got the pest, what it did to your PC, and how you killed the beast.

Post some basic details here for the world to see, but send me email at preston@gralla.com, and I’ll use the most relevant war stories in my book.

So you may have been a victim, but here’s your chance to help others avoid what you went through.

Tell me your war story. Get it off your chest — you’ll feel better.

Jesse Liberty

AddThis Social Bookmark Button

Related link: http://www.ondotnet.com/pub/a/dotnet/2004/09/27/libertyonwhidbey.html

On many web sites, it is important to achieve a consistent “look and feel” as the user moves from page to page. While this was possible with .NET 1.1, it was difficult and required both programmer and designer discipline. ASP.NET 2.0 makes this far easier with the creation of master pages.

Glenn Bisignani

AddThis Social Bookmark Button

Marc-Henri Poget, software project manager at the University Hospital of Lausanne, Switzerland, shares his experiences creating a billing application utilizing Perl.

If you have a Perl, Python, or PHP success story, I’d like to feature your work in my Weblog. Please send your success story to Glenn Bisignani (glennb@oreilly.com).

Easy Healthcare Billing with Perl

Billing applications usually contain complex rules that transform raw activity into billable activity. This is especially true in healthcare where fixed prices are applied for repeated medical services.

At the University Hospital of Lausanne, Switzerland, there are many applications that send billing data based on the medical and technical activities, to the hospital information system (HIS). The HIS then combine those billing data to produce the invoices.

The introduction of a new medical tariff at the beginning of 2004 was the unique opportunity to improve existing applications billing modules. As the responsible for the pathology information system, I faced the choice of either configuring the available vendor provided billing module or rewriting it. Since the existing module exhibited bad performances, was not so well integrated within our infrastructure and was difficult to configure to cope with the new medical tariff, I thought about rewriting it. The new module had not only to apply the transformation rules that produce billable data, it also generates ASCII files to be sent to the HIS and handles the ASCII files with error messages returned by the HIS.

I wrote the whole billing application in Perl 5.8.0, using DBI 1.37 and DBD Oracle 1.14. In addition, I wrote Korn Shell scripts responsible to setup the environment variables, launch the Perl scripts, manage the ASCII files and send e-mails containing billing errors using the Unix mailx command. Being a data centered application, the new billing application consists mostly in several steps that either perform SQL requests or process the data. The first step gets the exams eligible for billing and the associated raw activity data. Then, the billing rules are applied to produce billable data. The final step updates the database and generates the ASCII file for the HIS.

In order to make changes in rules easier, I chose to group all the rules in a dedicated Perl module which embodies the domain specific knowledge. Billing rules handle a structure in memory that contains both the raw and billable data. The structure itself is manipulated through primitives that ease rules writing (this could be rewritten in true OO Perl as well, but the main point is to ensure both encapsulation and abstraction). As an example, consider the following extract from the rules module:

$nNbAct = NbActByExam($aActes, ‘TS1′);
if ($nNbAct == 2)
{
NewVal($aActes, ‘TS1′, 0);
NewVal($aActes, ‘TS2′, 1);
Trace(4, “2 slides read by the pathologist”);
}
elsif ($nNbAct > 2)
{
NewVal($aActes, ‘TS1′, 0);
NewVal($aActes, ‘TSN’, 1);
Trace(4, “More than 2 slides read by the pathologist”);
} # if

The variable $aActes is a reference to the structure that contains both the raw and billable data for an exam. In this case, the NbActByExam primitive retrieves the total number of acts of type TS1, which corresponds to the number of microscope slides read by the physician. There are 3 billing codes for this case:
• TS1, price X, exactly 1 slide has been read by the pathologist,
• TS2, price Y, exactly 2 slides have been read by the pathologist,
• TSN, price Z, 3 or more slides have been read by the pathologist.

Thus, the above code is a straightforward translation of the rules. When $nNbAct equals 1, no changes are made. When $nNbAct equals 2, the number of TS1 acts is set to 0 through the NewVal primitive and the number of TS2 acts is set to 1. The same principle applies when $nNbAct is greater than 2. In this case, rules application order can be easily defined, therefore, I didn’t design a complex rules engine to choose which rules to fire. I simply coded the rules one after the other as in the above example.

While the HIS processes the ASCII file contained the billable data, it generates another ASCII file with the errors such as bad patient number, wrong billing date and some others. Since these errors require actions from an office clerk, the billing application needs to update the exam status so that it can be billed again after correction. Moreover, the application needs to notify the errors to the office clerk, it performs that by automatically sending HTML formatted mails.

To ease development and testing, I implemented 2 features. First, a logging module able to write traces to a file (more convenient for batch applications) has been written. Trace levels allow selecting what is written in the logfile. In the above code sample, there are 2 calls to the Trace function with a level of 4. A trace level of 4 is used to display very detailed information, such as which rule has been applied, as opposed to a trace level of 1 used to display more general information, such as a successful connection to the database. In addition to the level of each trace, there is a current trace level at runtime, used to filter out all the traces with levels greater than the current level. The default current level is kept low to avoid filling the logfile with large amounts of data. When debugging is required, the current level is increased so that more traces are written.

Second, when testing database applications, it is convenient to avoid updates so that tests can be easily repeated with the same data. I implemented a test mode that disables the commit and issues a rollback at the end of execution. Since the update, insert and delete operations are performed in the Oracle rollback segment, which is visible within the database session held by the Perl program, it remains possible to check out what has been changed in the database by issuing DBI calls from the Perl debugger prompt.

Perl and the DBI module have brought many benefits to this project. The expressive power of Perl and the ease of issuing database requests through DBI allowed to quickly craft this application. Moreover Perl is used for many tasks in our computer department including: system administration, web development and integration. I’m myself using Perl since 1997, therefore the language and its environment were a natural choice for this project. Perl’s string handling capabilities made the task of handling ASCII files a breeze and its data structure and memory management facilities helped to map the data to the application domain very easily. And last, but not least, a tenfold improvement over the previous vendor provided implementation in a compiled language has been measured.

Marc-Henri Poget is a project manager in the computer department of the University Hospital of Lausanne, Switzerland. He holds a degree in software engineering from the Swiss Institute of Technology (www.epfl.ch) as well as an MBA. He is involved in medical packages deployment and his interests include project management, troubleshooting and open source software.
He can be reached at Marc-Henri.Poget@hospvd.ch.

comments welcome

Preston Gralla

AddThis Social Bookmark Button

Among the many reasons why Firefox is superior to Internet Explorer is its use of extensions, add-ins that give it all kinds of cool new features.

Wei-Meng Lee tells you how to use extensions and picks some of his favorites in his article Using Extensions in Firefox. But there are more great extensions than can be fit in a single article, and so I thought I’d tell you about some of my favorites as well.

If, like me, you hate having to register at Web sites before you can do something as simple as read an article, then you should get BugMeNot. When you get to a page that requires a user name and password, right-click on the page, and the extension will give you a pair you can use.

One of the slimier and increasingly popular scams on the Internet are phishing attacks, in which a site looks like real ecommerce site like PayPal or CitiBank, but in fact is a forgery, including what appears to be the real URL in the address bar. Type in your name and password, and a scammer has access to your account. The great SpoofStick extension fixes the problem by showing you the true URL you’re visiting, no matter what the address bar says.

Like it or not, some sites are built specifically for Internet Explorer - they don’t display properly in Firefox. So get the ieview extension and when you come to a page like that, right-click and choose View This Page in IE, and Internet Explorer launches, with the page displayed.

Single Window helps you get the most out of tabs. When a page launches that would normally launch in a new browser window, the extension instead puts it on a new tab. And Close Tab on Double Click does exactly what it says - it lets you close any tab by double-clicking it.

There are a lot more great extensions you should check out as well. Get them here .

What are your favorite Firefox extensions? Let me know.

AddThis Social Bookmark Button

Related link: http://internet.conveyor.com/RESTwiki/moin.cgi/FrontPage

With names like SOAP and REST, you’d expect a couple cute jokes here. Perish the thought; I promised code in my last post and code you shall have.
The VB/VBA tools and samples from Microsoft demonstrate calling web services through SOAP interfaces. For example, the Office Web Services Toolkit generates proxy classes for web services using SOAP. You then use those proxy classes in your local VBA code to call the remote web service.
That works fine as long as you don’t try to modify the proxy classes to, say…call the web service asynchonously. Modifying generated code is often more difficult than writing (and thus understanding) the code yourself.
The Web Services Toolkit also generates proxy classes for every item described by the web service’s WSDL — in the case of the Amazon web service that’s dozens of classes. It’s hard to figure out which ones you need and cumbersome to use this overly object-oriented approach.
Good news! Some web services, like Amazon, provide an alternate, string-based interface. This interface looks like query strings (heck, that’s what it is really) but it also has a formal name: REST.
To see how it works, click on the following web address:
http://xml.amazon.com/onca/xml2?t=webservices-20&dev-t=D1UCR04XBIF4A6&page=1&f=xml&mode=books&type=lite&KeywordSearch=’wombat’
OK, you got a bunch of XML about wombats. That’s useful from a programming perspective because the interface is so much simpler than using the Web Services Toolkit and SOAP.
Lookee here:

Sub VBCode()
' Requires reference to Microsoft XML object library.
Dim xdoc As New DOMDocument, search As String
' Create the search request
search = "http://xml.amazon.com/onca/xml2" & _
"?t=" & "webservices-20" & _
"&dev-t=" & "D1UCR04XBIF4A6" & _
"&page=1" & _
"&f=xml" & _
"&mode=books" & _
"&type=lite" & _
"&KeywordSearch=wombat"
xdoc.Load search
' Display the results
Debug.Print xdoc.XML
End Sub

That code just displays the results in the Immediate window, but you can just as easily display those results in a document through an XML Map you set up previously. For instance:

Set wb = ThisWorkbook
wb.XmlImportXml xdoc.XML, wb.XmlMaps("AMZN_Map"), True

Voila! You’ve gotten your results in six lines or less.
The other advantage of this interface is that it’s easy to use asynchronously. If you set the DOMDocument object’s Async property to True, you can then hook into the ondataavailable event as seen here:

' Move the xdoc declaration to the module level.
Dim WithEvents xdoc As DOMDocument

Private Sub xdoc_ondataavailable()
Set wb = ThisWorkbook
wb.XmlImportXml xdoc.XML, wb.XmlMaps("AMZN_Map"), True
End Sub

So why doesn’t Microsoft show REST examples? I don’t know, but you should be aware that not all web services have REST interfaces. Google doesn’t, and that’s a complaint among some web programmers. On the other hand, SQL Server’s XML support through IIS looks like REST. So you decide.

Opinions?

Jean Hollis Weber

AddThis Social Bookmark Button

Related link: http://www.newsforge.com/article.pl?sid=04/09/17/171220

In case you haven’t seen this…

On NewsForge on Friday, Ryan Singer countered speculation that Sun’s recent no-sue agreement with Microsoft might have adverse effects on the continued development of OpenOffice.org:

“Since Sun Microsystems’ 10k filings, I have been asked many times by press and pundits to clarify what this agreement means for OpenOffice.org. There is much misunderstanding and confusion among the Free Software and Open Source communities as to how this affects OpenOffice.org. I have, with the help of community volunteers Daniel Carrera and Colin Chalres, developed this FAQ as an aid to the FS/OSS community, to better understand the situation. I hope that this will ease the fears of the FS/OSS community.”

See article http://www.newsforge.com/article.pl?sid=04/09/17/171220 for details.



Jean Hollis Weber

AddThis Social Bookmark Button

Recently I conducted a survey of members of a professional group, using an online survey service. The data from the survey was provided in a spreadsheet, along with simple breakdowns of the results (for example, numbers of males and females, or numbers of people in each age group).

In addition, I needed to make tables or graphs showing combinations of the data (for example, number of males in each age group or number of females in each salary group).

I’ve had very little experience setting up formulas in spreadsheets. But I wasn’t concerned, because lots of people do this sort of basic analysis, so I assumed that information on how to do it would be readily available. Surely I could find an appropriate spreadsheet function or formula, or even better a template, and apply it to my data.

No such luck. I’m convinced that the information is out there on the Web somewhere, but I haven’t been able to find it. I wasn’t surprised to not find much detail about OpenOffice.org Calc usage, but I did expect to find a clue in Excel literature. Despite some differences in notation, Excel and Calc use essentially the same function names.

I waded through the descriptions of functions in the OpenOffice.org online help and in some Excel books, but many of the descriptions didn’t make much sense to me and even when they did make sense, the examples didn’t seem to fit my situation.

The closest match I could find was DCOUNT. I could get it to work on any one combination of cells (for example, females in age group 3), but not in a way that would generate a table of results without doing a lot of manual work. Perhaps I’ve missed some trick to make this function do what I want.

COUNTIF also seemed like a good candidate, but I couldn’t find a way to get that to work on two columns of data instead of just one.


SUMPRODUCT does the job

Finally I stumbled upon a great article by John Walkenbach titled “Count and Sum Your Data in Excel 2002″ (http://www.microsoft.com/office/previous/xp/columns/column10.asp), which gave me the answer. Scroll down in the article until you get to “Conditional Counting and Summing Using Multiple Conditions.” This does exactly what I want.

Here’s the basic formula for the Gender-Age Group table:

=SUMPRODUCT((A2:A141=1)*(B2:B141=1))

The parameters vary depending on the number of rows of data and the value of the variable I’m analyzing. To make this work on an array of data, press CTRL+SHIFT+ENTER after entering the formula.


Replicating the formula into other cells

Now that I had the basic formula to do the calculation for one cell of my results table, I needed to replicate the formula in the other results cells. Here are the formulas for the other cells in the first row of the Gender-Age Group table:
=SUMPRODUCT((A2:A141=1)*(B2:B141=2))
=SUMPRODUCT((A2:A141=1)*(B2:B141=3))
=SUMPRODUCT((A2:A141=1)*(B2:B141=4))

And here’s the formulas for the second row of that table:

=SUMPRODUCT((A2:A141=2)*(B2:B141=1))
=SUMPRODUCT((A2:A141=2)*(B2:B141=2))
=SUMPRODUCT((A2:A141=2)*(B2:B141=3))
=SUMPRODUCT((A2:A141=2)*(B2:B141=4))

These were easy to input by copying and pasting the basic formula and changing the parameters as needed (remembering to press CTRL+SHIFT+ENTER), but it’s still a tedious and time-consuming process when you have lots of survey questions.

I’m still hoping to find a trick to doing it with less manual fiddling about.

I’ve put on my website a longer version of this article, including some sample data and results and a more detailed explanation. http://www.taming-openoffice-org.com/calc/surveydata.htm


Do you have a better way to accomplish this job using a spreadsheet? I’d love to hear what it is!

Preston Gralla

AddThis Social Bookmark Button

The world’s best browser just got better.

FireFox is finally out of beta, and is in its “preview” release, which means…well, it’s a little hard to understand exactly what it means, other than that it’s been improved, and is one step closer to its final form.

If you’re an Internet Explorer user, you owe it to yourself to download FireFox and see how a real browser works. If you’re already a FireFox user, download the latest version, for a slew of new features.

My favorite is its new RSS support – when you’re on a page that has an RSS feed using the <link> tag, an RSS icon appears in the status bar. Click it, and you’ll see a list of all feeds on the page. Then click on any to subscribe, and you’ll be able to read the feed from directly within FireFox.

There are other goodies as well. There’s a nice new Find toolbar across the bottom of the page, better bookmarking, more flexible popup blocking, and more as well.

Unfortunately, there’s also a downside to the update. Many extensions won’t work with it, including the Googlebar. In time, expect the extensions to be updated, but for now, you’ll have to forgo some of your favorites.

The newest version of FireFox shows just how tired and old IE has become. In time, perhaps, it’ll join the modern era and add features such as tabbed browsing. But for now, if you’re looking for the smartest, slimmest, and most feature-rich browser, you should head straight to FireFox.

What do you think of the newest version of FireFox? Let me know.

Jean Hollis Weber

AddThis Social Bookmark Button

This report was posted to the OpenOffice.org [discuss] list on 8 September by Hirano Kazunari.

On September 4th, 10:00-17:00, OpenSource Conferece 2004 was held at the No.7 building of Japan Electronics College (http://www.jec.ac.jp/en/index.html) in Tokyo, Japan.
The 8-floor building was full of OpenSouce people all day on the Saturday.

26 OpenSource communities attended the conference and exhibited OpenSource softwares and tools they love to use and play with, such as Fedora, Firebird, Jabber, OpenOffice.org, Plamo Linux, Plone, Squeak, Xoops, Apache, MySQL, NetBSD, PHP, PostgreSQL, Ruby, Samba, vi, Webmin, Zope, Mozilla and so on.

The conference was jammed as a total of roughly 1,500 people visit. The 2nd floor was the exhibition floor where the OpenOffice.org Japan Users Group (whose core is the Japanese Native-lang project) also distributed about 200 Japanese OOo CD-ROMs.

Hideya Kawahara was invited as a key note speaker for a morning session of the conference. He is a Senior Staff Geek, Project Looking Glass, Advanced Development Group, Sun Microsystems, as read on his name card. The media calls him an “Ichiro (baseball player of the Seattle Mariners)” in the IT industry. His demonstration of the 3D desktop system with the Looking Glass fascinated the audiences packed in the main track room. He emphasized the importance of world-wide community development and said that the current forcus is Japanese community. He called, “Go get it! You are invited!” to the community (http://lg3d.dev.java.net, http://www.sun.com/software/project-looking-glass).

In the afternoon there were more than 30 sessions separated into several tracks for each floor. In the main track they had 3 sessions; “OpenSource and Agriculture,” “OpenSource Business (panel discussion)” and “Nature of Community and its Future (panel discussion).” Other tracks included Community, Server, DataBase, Web, Plone, Desktop and Hands-on. One of the sessions in the Hands-on track was on “OpenOffice.org DataSource Function - Challenging Microsoft Office+Access.” Computers were set on every desk in the Hands-on track room. Masahisa Kamataki from the OpenOffice.org Japan Users Group used Knoppix+OpenOffice.org and showed 25 attendees how to create tables, how to use DataSource, how to print labels and so on with dBASE. A session “Marketing OpenOffice.org in Japan” in the Desktop track was conducted by Yutaka Kachi, the marketing lead of the Japanese Native-lang project.

200 OpenSource people from various communities, who worked for the conference very hard, gathered at a party in the main track room after the conference. At the party the organizer Toru Miyahara, Begi.net Co. president, announced that he will organize OSPN (OpenSource People Network) which will support OpenSource people and hold OpenSource Conferences. His plan is to have a series of OpenSource Conference 2005 in Hokkaido, Tokyo, Osaka and Okinawa next year.

Sponsors included Platinum sponsors (Apple Computer, NTT Comware and Sun Microsystems), two Gold sponsors and nine Silver
sponsors. Supporters included AIST (http://www.aist.go.jp/), IPA (http://www.ipa.go.jp/about/english/index.html), Japan Linux Association (http://jla.linux.or.jp/) and Japan Unix Society (http://www.jus.or.jp/index-e.html).

The very first OpenSource Conference in Japan has got a success with many OpenSource people and supporters.



Preston Gralla

AddThis Social Bookmark Button

No matter how good a job you do, you can always do better. So we’re asking your help to make WindowsDevCenter and its sister site OnDotNet as good as they can possibly be.

It won’t take much of your time - just a few minutes. We’d like you to take a survey telling us, among other things, when you think Longhorn will be released, your primary operating system and hardware, and a bit more. We’ve clocked it, and it should take you under six minutes.

Fill it out and you get a chance to win free O’Reilly books, including my bookWindows XP Hacks.

Your responses will help shape not only WindowsDevCenter and OnDotNet, but also the kinds of books that O’Reilly will publish as well.

So help us, and help yourself in the bargain. Take the WindowsDevCenter survey, or the OnDotNet survey.

What do you think of WindowsDevCenter? Let us know.

AddThis Social Bookmark Button

I was canoeing with my wife and infant son on the Snoqualmie river near Redmond years ago. That river rips, even in the summer and we were trying to get back to the launch site upstream. The swiftest current came rounding a bend in the river and we really had to dig in. I felt we were virtually flying and hollered out “We’re getting it.” but she corrected “No, we’re not.” I looked at the rocks on shore and saw she was right.
I kind of feel the same way about all the different tagging strategies that have evolved over time. There’s a bunch of them now and an imperfect evolution is something like SGML>HTML>XML>RSS>OWL. Periodically I have to dig in and learn a bunch of new stuff but there’s always some new version or standard just around the bend.
The problem addressed by all these is that humans interpret what they read by inference and computers can’t do that (or don’t do it well).
For us, those inferences are established by formatting, word choice, and personal experience. For computers, associations must be coded into the content explicitly. That sounds doable until you attack “personal experience”. I know there are approaches to that, but the blocking problem is easy to see in the real world: reasonable people differ.
So what am I getting at? Choosing a modest scope may yield more benefit than rigorous exercise. In other words, beach the canoe and portage across because there’s always more river ahead.
Applying this philosophy to tagging systems means using schemas (or the like) that are minimally or moderately complex. In my opinion, it is better to lose nuance than increase complexity.
OK, that’s an editorial. I promise next time to talk code.

Offer observations on the semantic web.