This may have been said in many ways before; the edge/distributed/p2p discussion has spoken extensively about the transition of services to the edge of the network
There is another way to look at this, however: specifically, there are two factors:
- The transition of HARDWARE services into software services at the edge
- The transition of MANUALLY CONFIGURED services into automatic services
Let’s see how P2P stacks up against this: Name service is traditionally handled by DNS. DNS could be considered a HARDWARE service, MANUALLY configured. It’s a bunch of servers somewhere; you have to buy a name, wait several days, then manually enter the IP of your machine into the DNS service and manually enter the name into the host machine, etc. This lengthy exercise, in many P2P apps has been replaced with an automatic system (the names and thus addressability are registered with little or no human intervention). Note that the name registration is still client/server in Napster’s case, but has been fully distributed in the newest filesharing apps.
Thus, Clay Shirky’s assertion of “DNS alternative” as a criterion for P2P systems does not, inherently, require a decentralized implementation. I am arguing that a key part of the “new” phenomenon that is occuring is the automation of traditionally manual tasks.
Certainly, the motion of services to the edge is the most major part of this technology evolution. But even there, the automation of centralized services is another phenonmena at work. There was much excitement (and still is) about “software agents”- sort of AI for services, pieces of software that ‘learn’ or at least, know how to do some complex data recombination to perform more sophisticated data handling for humans. Many people have begun to make software agents synonymous with the edge networking theme, but once again, agents can be completely centralized. And at the user interface level, it is not necessary for the user to know “where” their agent is executing (local host or on yahoo’s server farms?).
Concretely, agents might comprehensively search web based data stores for particular information, or coordinate synchronization between address books or schedules, etc, or track down a user and forward important information to them in the real world (on their cell phone, pager, office phone, etc.) The point is, this is certainly an “edgy” phenomenon- the process is going to various machines on the edge of the network to search, or it is coordinating between two edge computers with different, overlapping views of a person’s data, or it is reaching outside of the internet to edge resources such as pagers or cellphones to get ahold of a human. BUT, it is doing it from a central server…
Another example is web caching: this is a mature, proven technology, used over and over. The basic theory behind various caching techniques resonates throughout computer science, from chips up to networks. What, then, is novel about caching in an emerging technology sense? The automated deployment of edge caches is what is new. With edge caching (or “outer edge” or “far edge” “over the edge” caching, when it includes end user machines), we have potentially tens of thousands to millions of machines participating in the caching for their networks, instead of the mere thousands on a CDN. Or for instance, you have the automated deployment of server farms in distributed computation. In order to make this work, you need automation, because you can manually configure 100’s of machines in an in-house server farm, but you can’t expect to manually (using IT labor) configure tens of thousands of remote computing resources.
Another further example is firewalls- you see increased automation in both hardware NAT’s (linksys et al., mostly automatic) at the edge of the network, and fully automated firewall systems like ZoneAlarm that run on the host and pretty much configure themselves.
There are many areas where traditional centralized servers are doing the job, and often these centralized servers are bound to the hardware platform they were deployed on.
Remember, many central, hub-and-spoke systems actually suffer a tremendous efficiency reduction when they are decentralized. There are some systems that would never work in a pure decentralized setup without fundamentally changing what they do; And, decentralizing these systems is far more complicated and error-prone than the straightforward central implementation ever was.
Part of what enables us to now look at decentralizing these servers is the automation. The user doesn’t have to tinker with settings, they system just goes when it is executed. And the automation allows the network effect, because the system starts working without configuration roadblocks (’configure your IP address’) that would stop it from going up instantly.
Look over some traditionally succesful technologies, and imagine a world where this technology was 1) implemented in software 2) available on every single machine in the network. You see many technologies that could work this way, and do not already.
Note that in order to achive total penetration in the network, the technology has to be auto-configuring- most of the users on today’s Internet can’t cope with ‘too many screwdrivers’, configuration complexity. This is, in my opinion, a key enabler of distributed systems, and a way of analyzing old systems to see if they can be distributed on today’s commercial Internet.