Women in Technology

Hear us Roar



Weblog:   The Greatest Test of Open Source: Beating Google
Subject:   More then one thing powers Google greatness.
Date:   2005-02-12 18:59:26
From:   tima

Google. One thing powers its greatness. Software.


I would disagree. The infrastructure -- particularly its scale -- is as much a part of their greatness (perhaps even more so) as the software that powers it. This is implied, though not acknowledged:


...Nutch hasn't prospered to where I would like it to be because of the costs of running the operation. To truly test the system you need a big index. You need to spend a lot of money crawling. To test it against Google anyway.


So, I really don't think this is a challenge open source can entirely address since there is no such thing as "open source hardware" or "open source bandwidth" that is necessary to match Google's feat.

Full Threads Oldest First

Showing messages 1 through 2 of 2.

  • Open source needs to do it different
    2005-02-14 16:05:33  babelex [View]

    The resources of hardware and bandwidth are quite prohibitive.

    Thus Open source attempts should tackle it from a different angle. Rather than attempting the google/yahoo/MSN route an alternative route like SETI should be used.

    That is leverage the unused bandwidth and cycles of the community and beyond to do the trawling and analysis. Just centralise the summaries maybe.

    I'm no search guru beyond standard inverted tress and ratings but I bet it is possible to do it in a distributed fashjion like this and it would remove the resource issues.

    worth thinking about anyhow...
  • Steve Mallett photo More then one thing powers Google greatness.
    2005-02-14 07:15:48  Steve Mallett | O'Reilly Blogger [View]

    "I would disagree. The infrastructure -- particularly its scale -- is as much a part of their greatness (perhaps even more so) as the software that powers it. This is implied, though not acknowledged:"

    Jeez, I dunno about that. Yahoo certainly has a ton of infrastructure & they don't touch Google.

    "So, I really don't think this is a challenge open source can entirely address since there is no such thing as "open source hardware" or "open source bandwidth" that is necessary to match Google's feat."

    That's true. I was alluding (or thought I was) to this with the link to a commercial entity to ramp up that part of the equation.

Showing messages 1 through 2 of 2.