Data Persistence with Waba

Last month we took a look at some of the GUI components within Waba. There aren't many useful Palm OS programs that don't have a GUI interface. Likewise, there aren't many programs without data. Whether writing business apps or games you need some mechanism to store data.

If you intend on using Java (or a Java-like language) as your language of choice, you face a challenge in Palm OS data persistence.

The Palm OS is composed of featureful C APIs, which give you a lot of programming power, including the DataManager and Palm Database (pdb) files. (Before you raise an objection -- I know that the Palm has no filesystem, but I'm using the term generically.)

The Sun KVM release comes with classes for creating simple interfaces on the Palm, but these are mainly a reference implementation, not intended to be used for commercial applications. Among these classes is the com.sun.kjava.Database, which gives a programmer the ability to read and write pdb files. However, if the kjava Database class gave you full access to the pdb API, it would cease to be portable across other resource-constrained devices that do not run the Palm OS. Sun's intention is to release a PDA Profile, which would in theory be usable for Palm Pilots, Windows CE devices, etc. This has been delayed, and if tradition holds, there will probably be an announcement related to device profiles at JavaOne next month.

How does all this relate to data persistence with Waba? If you'll recall, Waba is also a cross-platform solution, and its creators did not want to provide a data mechanism which would make it a Palm-only solution. Like the kjava.Database class, persistence in Waba understands only byte arrays. This is quite primitive, but Waba gives you some help that the KVM doesn't.

Creating a Catalog and accessing data.

Waba provides data persistence primarily through the Catalog class.

The Catalog class extends waba.io.Stream, an abstract superclass of stream-based I/O operations. To create a Catalog in Waba, you write

Catalog c = new Catalog("MyCatalog.MYxx.DATA", Catalog.CREATE);

The constructor takes two arguments: the catalog name and the mode in which to open the Catalog. (Other modes are READ_ONLY, WRITE_ONLY, and READ_WRITE.)

The Catalog name is actually a concatenation of three things. The first part is the name proper. This would show up in your Palm pilot if you selected the menu, then selected info, and scrolled through all of your installed applications.

The second part is the creator ID, which is a four-byte identifier for the Palm OS to tie together application resources on the device. This has to have at least one uppercase letter, and it must be unique on the device. If you intend on releasing your application to a wider audience, you should register your creator id on the Palm OS site. This will ensure that no one else has chosen the same name. The third is the type of database. This should be hardcoded to DATA as this is the only kind you can create through the Catalog class.

Once you have opened the catalog, you have access to it. Before you actually call any of the write methods, you need to add a record by the addRecord(int recordSize) method. This will allocate a new record of the given size for you to write data to.

Most application programmers are accustomed to relational databases, which provide a wealth of ways to manipulate your data, but ways that resource-constrained devices can't handle. In other words, a Catalog is in no way a relational database.

Catalogs support the concept of a key field other than the record id itself. There isn't a method like Catalog.findRecord("Smith") that will return a record id. Access to the data is sequential only; to find a particular record, you need to search the entire set, looking for a match. There is a utility method, inspectRecord(byte[] buf, int recPosition), that is designed to offer a quick look at the data to see if it contains what you want. The API documentation cautions that using this method can be unreliable as none of the parameters are checked for validity. You must avoid running past the end of the Catalog.

To return to the matter of putting data into a Catalog, everything needs to be in the form of a byte array before it is written. That can be quite cumbersome when dealing with Strings, especially since some of the methods of the java.lang.String class are not present in the waba.lang.String class. In particular String.getBytes is not there.

Waba provides some extra help for dealing with such a primitive persistence facility. Waba allows you to wrap a Catalog in a DataStream class, which eases the burden. (In fact, a DataStream can be wrapped around any other stream-oriented class such as a SerialPort). When you create the Catalog that we looked at previously, you could wrap this in a DataStream as follows:

int j = 0;
Catalog c = new Catalog("MyCatalog.MYxx.DATA", Catalog.CREATE);
 if (!c.isOpen())
        return;

DataStream ds = new DataStream(c);
j = c.addRecord(6);
ds.writeString("Fred");
j = c.addRecord(6);
ds.writeString("Lucy");
ds.close();
c.close();

We open the Catalog, then after ensuring that we have a valid handle, we create a new DataStream wrapped around the Catalog instance. After this, we can add records, then call DataStream's writeString method, which will convert the String to a byte array so the Catalog can accept it.

Other methods in the DataStream class to deal with various data types include writeBoolean, writeBytes, writeFloat, writeInt, writeShort, and writeStringArray. These have matching read methods as well. Armed with these tools, let's look at another example of reading data from a Catalog, and populating an Edit control.

c = new Catalog("MyCatalog.MYxx.DATA", Catalog.READ_ONLY);
ds = new DataStream(c);
if (!c.isOpen())
{
edtEmail.setText("Not open!");
        return;
}
else
{
count = c.getRecordCount();
}

for (int i = 0; i < count; i++)
        {
                c.setRecordPos(i);
                edtNames.setText(edtNames.getText() + " " +
ds.readString());
        }
ds.close();
c.close();

We open the Catalog as before (but now in READ_ONLY mode) and wrap it in a DataStream. After ensuring it is open we get the count using the getRecordCount() method. This will be used when we loop through the record set, getting each name, populating the Edit control. Notice also that we position the pointer at each record by using the setRecordPos(record) method. This is necessary to step through the Catalog.

Then we use the DataStream to get the data by calling the readString() method.

This demonstrates the basic functionality of Catalogs in Waba. However, these examples are not what you would likely encounter in real programming in that they deal with a very rudimentary set of data (first names only).

If you were writing the contact manager that we began to construct last month, you'd need to store first name, last name, address phone, fax, email, and several other fields as well.

Since the Catalog only understands byte arrays, how can you determine where one field ends and another begins? Using DataStream can again make your life easier. If you call the methods described above to write the data to the Catalog, then getting it out will be equally easy. For example, if you write the data like this,

ds.writeString(edtFName.getText());
ds.writeString(edtLName.getText());
ds.writeString(edtPhone.getText());
ds.writeString(edtFax.getText());

then getting it out of the Catalog would be done with

edtFName.setText(ds.readString());
edtLName.setText(ds.readString());
edtPhone.setText(ds.readString());
edtFax.setText(ds.readString());

As long as the proper order is maintained, the data will be properly delimited by the DataStream class. This is a major advantage over the kjava.Database class where delimiting the data is entirely up to you, either by offset or with some character.

Performance Considerations and Conduits

When coding for a resource-constrained device such as the Palm Pilot, Visor, etc., you need to bear in mind that memory is at a premium. Treating small devices like they are PCs with 256MB RAM can definitely get you into trouble. For example, our contact manager application may have 150 records in it. Reading these into a Hashtable or some other collection object could quickly max out the allocated heap space. Instead, you should consider working with the record Ids only. Then when you need a particular record you can use the id to look it up, and you don't need to risk the overhead of dealing with a large collection of objects.

The final thing to note is that to write an app that goes the distance, you will likely need a conduit which will synchronize you Palm Database with the desktop using the HotSync technology. Palm has a couple of Conduit Development Kits available, one in C, the other in Java. The kjava.Database class is not synchronizable since it does not conform to the pdb format, which a conduit expects -- at least, not with the Conduit Kit for Java. It is possible that one could do it using the Conduit Kit for C, but if you know C, why write your Palm app in Java at all?

Guilherme Hazan, who's added several of the SuperWaba classes to the API, has written conduits in Java to synchronize a Waba Catalog, and he assures me that it can indeed be done. Writing a conduit is a big subject in itself, and if your application calls for it, I would encourage you to download the Conduit Development Kit from the Palm OS site. It includes samples and a tutorial.

Given the arsenal of Waba tools you now have, you can certainly begin experimenting with a very useful and well-performing platform for Palm OS application development.

Matthew E. Ferris is President, Chicagoland Java User Group, a Sun Certified Java Programmer, and a contributing Author for Professional Java Server Programming, Wrox Press.

Return to ONJava.com.