this is sippey.com

An obvious note: several of the companies I write about this week actually have exclamation points as part of their names. In the spirit of reasoned discourse, I had to remove them. Forgive me. Otherwise we’d be yelling every third word.

This is not going to be news to anyone. But Yahoo is overvalued. So are Lycos and Excite. But they’ve gone public, socked some money in the bank, and their founders must be laughing all the way home.

All of these sites are operating on similar business plans. Create a large database of web sites for cataloging, subset some of those into a hierarchical directory, and provide reviews of even fewer “selected” sites. Create a brand name for yourself (preferably using an exclamation point in your name), boost the hit counts by “partnering” with other high-traffic sites, and crank up the ad revenue.

Of course, behind the scenes, each of them struggles to maintain a technological edge – whether it be in their database structure, their surfing bots, their search algorithm, or their ability to customize experiences for advertisers.

But technology will not win or lose it for any of these folks. Because all of them (or most of them) are doomed. It’s fairly obvious that they’ll each face one or more of the following roadblocks…

• Traffic Patterns. It’s a cutthroat world out there. Yahoo, Lycos, Excite, Infoseek, Inktomi – they’re all playing a dangerous game, and are all after larger pieces of the same traffic “pie.” How many of you have allegiance to your search engine? Is there one you hit more than others? Or does it depend on what shows up first on Netscape’s Net Search page? And what does search engine loyalty really mean when you can use search.com, a [c

  net](http://www.cnet.com/) branded front end to all the major search engines on the web?

• Ad Revenue. I’ve harped on this enough over the past few weeks. But the search engines are stuck in the same boat as every other “free” content provider out there – they’re beholden to ad dollars. Now, I’ll admit that the directories and search engines have a nice advantage over a Pathfinder or [c

  net](http://www.cnet.com/). And that is depth; or at least the illusion of depth. Try this little experiment. Go to Yahoo. Do a search on the word “wedding.” See the ad banner that comes up? You’ve just discovered the real value of a search engine.

• Exploding Web Sites. By some estimates, there are currently some 50,000,000 individual pages on the web. Lycos is currently indexing around 37 million; Alta Vista around 33 million. So what happened to the other 13 to 17 million pages? And what happens when there are literally billions of web sites, and those search engines run out of processing power to keep up with the growth? Inktomi is reportedly using some interesting technology to scale their database/search engine with the growth of the web.

• Evolving Content. One obvious problem the search engines are going to face is the devolution of text. More and more of the web’s content is not ASCII – it’s multimedia, be it JPEG images or Quicktime movies or RealAudio clips. How will Alta Vista index the full text of Netscape’s corporate profile from their recently released annual report? It’s an enormous GIF file, after all. (When I recently asked a partner in the Inktomi project how they’ll deal with this problem, he answered that they’ll be able to index Shockwave presentations on the Web based on the textual description that an author attaches to the file. Uh huh…and I thought the point was full text indexing…)

Don’t get me wrong. I find search engines useful, to a point. If I’m looking for a very particular piece of information, like all the references on the web to a long lost friend, then they’ll do me just fine. But try to find anything useful related to broad topic, like snowboarding, and they’re a mess. How in the world am I going to find my way through 4,325 finds?

What I think we need is more localized search engines and directories. Leave the full text, full web searching to the recently IPO’d. There is still a tremendous opportunity to do regional or topical directories, where depth and experience really matter. Do I trust the 20 or 30 people Yahoo has employeed doing site categorization to categorize literary fiction as well as they do freeware HTML editors? I would rather see smaller niche directories where particular topics can be categorized in-depth by people who care about the subject.

Suppose this happens. Suppose we do see a flurry of small, localized, specialized directories and search engines. This doesn’t mean that the major search engines get left out in the cold. It could generate new opportunities for them. Strategic directory partnerships – Yahoo could farm out the responsibility of categorizing art on the web, for example, to a small startup company founded by art history majors. Magellan could have exclusive “linking rights” to a new, comprehensive, subscription-based directory of Silicon Valley software job openings.

Or, if all else fails, and the ad revenue dies, the big boys could just sell their search algorithms and CGI scripts for next to nothing. Or for nothing at all.

Now there’s a model we’re all familiar with.