The Peer to Patent project (which I’ve reported on before) just pointed me to a particularly broad patent that could encumber user interfaces on the web and desktops for years to come.
The idea in this patent, submitted by Yahoo!, is a clever little idea: if someone is starting to drag an icon or mail message somewhere, why not bring the mountain to Mohammed, so to speak? If she is dragging a photo, for instance, the browser or operating system can guess that she wants to open it with PhotoShop or the Gimp, and present that choice right next to the icon for the photo.
Yes, an idea worth experimenting with, and one that (as the patent application amply emphasizes) could turn up soon in desktop window managers, in applications such as mail programs, in Ajax-enabled web pages, on mobile devices, and all sorts of places. And Yahoo! wants a cut for supposedly thinking it up.
Discussed by a lawyer in his blog and presented in full on the Peer to Patent site, this application seems to be a tough one to crack, despite its simplicity. Only two comments are posted, and no actual instances of prior art (although one comment tentatively suggests on instance). If no prior art is found in 35 days, the comment period closes on Peer to Patent and little stands in the way of granting the patent.
I have nothing against acknowledging the brilliance of the interface designers at Yahoo!. If GUI users overcome their initial surprise at having targets pop up at them, smart drag-and-drop may prove to be a convenience. Amusingly, the author of the patent application presents it as much more–a radical improvement to user interaction. But this has no impact on the granting of the patent. A novel idea can get a patent no matter how small an increment over current practice it represents.
What’s more disturbing to me is the cavalier way in which a Yahoo! designer has picked off the easiest layer of a complex design and tries to set up a toll gate to profit from it. (More likely, exercising the patent on such a small advance would just hold back its adoption, so I imagine Yahoo! is just trying to beef up its cross-licensing patent portfolio.) A consideration of the work that would go into implementing smart drag-and-drop quickly reveals that it would require serious research into user interaction and extensive changes to many parts of an application or desktop to deal with design issues such as:
- How does the system choose the targets to present to the user for each type of icon or other user interface element?
- How should the system subdivide its elements so that relevant targets are grouped with elements that can be dragged?
- Once heuristics for these choices are developed, what data structures store these targets, how are targets linked to the objects being dragged, and how can an association be made quickly so as to present targets almost instantly to preserve a comfortable user experience?
- What hooks does the desktop have to present to applications so they can create seamless experiences such as the hypothetical mail program suggested in the patent application? How do applications need to be redesigned in order to categorize user interface elements that are presented to the desktop for this service?
If you’re a web developer or other person with experience in graphical interfaces, maybe you can prevent this. Check out the claims and see what prior art you can find.
The first UI technique it reminds me of is context-specific circular "pie" menus that pop up around the mouse cursor after doing some kind of special-click on an on-screen object. (The Firefox extension for this was called RadialContext, I think). Wherever the mouse is, the set of choices for the chosen object pop up right there. But I guess this is still different from normal DnD.
What I find vexing about this particular innovation is that it sounds a lot like a standard context menu (e.g. Open With...), except that it is triggered by a click-and-drag rather than a right-click.
While it may be difficult to find prior art that destroys the novelty of this invention it should be rather easy to argue against an inventive step (context menus seem like a good candidate). Moreover, as you explain, there is much more involved than just bringing the mountain to the prophet. How is this achieved? This means that there is clearly insufficiency of disclosure of essential features. This argument would prevent a patent from being granted. And maybe the invention as described even lacks a technical effect at all.