Submitting to the search engines

Written by Dave Collins, SharewarePromotions Ltd.

http://www.sharewarepromotions.com/

Please note that due to the content of this article, much of the information may now be out of date. For historical purposes, I'll leave it here... but follow the advice at your own risk!

It already seems like a long time ago since I began this series of articles on the search engines, and by now we've covered a fair amount of ground. We began with the HEAD and META tags of your site pages, and moved onto some of the basic rules, tips and techniques. We then moved up a gear and looked at more advanced ground, dealing with hallways, doorway pages, the robots.txt file and frames.

This means that we've more or less covered the necessary preparation work, and are now ready to jump in at the deep end, and submit our site to the search engines. This article aims to cover all the remaining issues; what to submit, where to submit it, the issues of time, and some of the many solutions that are out there to make life easier.

The obvious starting point is where to submit. The most commonly voiced view is some variation of the "95% of my site traffic comes from ten (or so) search engines, so there's little point in aiming at the rest". Many authors feel the same way when it comes to submitting their software to the shareware sites too. A listing on CNET's Download.com may generate many times the number of visitors than Quality-Shareware produces… as Google will doubtless generate in the region of 500+ times the amount of traffic than ListYourLink.com will.

However, traffic alone probably isn't the name of the game here. You're not an excited teenager watching the counter on their homepage hit double digits, and ultimately you're after one main thing - sales. My own view is that while a significant amount of time should be spent working with the main sites and search engines, neglecting all the smaller sites is a wasted opportunity. How long does it take you to fill out a form at these sites? Twenty seconds? A minute? If one copy of your software is sold as a result of doing so, doesn't that make it worthwhile?

If 95% of your site traffic comes from ten search engines, then it makes sense that you should probably spend around 95% of your "search engine" time working on these listings. But don't forget the other 5% - they can and will produce visitors, and visitors will hopefully lead to sales.

The next obvious question is which engines are the main ones; while ten people will probably produce ten different lists, most would probably more or less agree with the following. Please note that this information may now be a little out of date: AltaVista, Yahoo, Excite, Google, HotBot, GO, Lycos, Northern Light and WebCrawler.

Oh, and in case you're puzzled, some of the search engines do use the same databases; submit to one of them and you should appear in the other. Excite and Magellan are one combination, as are Inktomi, HotBot, AOL, Goto and MSN. InfoSeek and the Go Network are another.

Now that we know where to submit, the next question is what to submit. Some engines will "spider" your pages, following the links from page to page, while others will only look at certain pages. While there isn't the space to go into all the sites here, the main ones are as follows.

Yahoo can potentially generate more traffic than most of the other search engines put together. When submitting to Yahoo, you only need to send your root page. Yahoo use live personnel only (yes, real people!) to decide whether to list your site or not, and the data you fill out on their forms is critical, second only in importance to your site itself. Bear in mind that sites are listed alphabetically; other than that one element of control, you're entirely at their mercy! The ultimate search engine - where content, and only content, is king. Two important points for Yahoo. Firstly that NO software should ever be used for their submissions. None. Many claim to be capable of this task, but none will actually work; the only way to be sure that the submission is carried out correctly is to do it by hand. The other point is that this is the one place where you should never use any tricks or techniques to get listed. No multiple submissions, no doorways, and no hallways. They won't work, and will only damage (if not destroy) your chances. Don't say I didn't warn you!

AltaVista, on the other hand, will allow you to submit multiple pages to their site, and also spider your pages for you, so there's no need to submit every single page on your site. Their time to index and list your site varies, and over the past six months there have been massive delays in processing new requests. At the time of writing this article, most new listings appear after two to three days. META tags are important here, and bear in mind that keywords are also case sensitive. Page popularity (i.e. how many sites refer to your own) is another important factor, and doorway pages were born for this search engine!

Excite and Magellan will allow you to submit your root domain only, and usually display new additions within two weeks or so. META tags are important, as is page popularity, and the important thing to realise is that only the initial page will be spidered. InfoSeek and the Go Network however, work in a quite different way. Each page must be submitted individually, and there's no spider to scuttle across your site from link to link. A handy tip for InfoSeek is that fresh content is popular - so try to update and resubmit your main pages every month or so. New listings can take anywhere from two days to three weeks to appear.

HotBot are the single most frustrating engine to keep on top of, as their relevancy criteria seem to change with amazing frequency! The good news is that you only need to submit your main URL, and the listings may appear in a couple of days to two weeks. Well worth the effort though - our sites usually owe around 20% of their search engine visitors to this engine.

So now we know where to submit, what to look out for, and how long we may have to wait. The other commonly-asked question is how often to resubmit your site. The nutshell answer is that this varies from engine to engine, but a good rule of thumb is to make it at most a monthly event - no more than that.

The final hurdle is how to submit. Manual submissions are certainly the most effective, but are time consuming, and should be logged if you're going to develop any sort of routine. The good news is that there are a large number of applications and services out there that can do the work for you. The bad news is that many of them are garbage!

In terms of services, you don't need to look too hard too find the en-masse submitters, many of which are free. I have never and will never use one of these services, as I consider almost all of them to be little more than SPAM bait, and a total waste of time and money. However, the software that's available for this sort of thing comes in two main forms; the majority are useless, and a handful are actually useful.

I personally use AddWeb Pro by CyberSpace HQ for all my en-masse work; it's very fast, is updated regularly, and it does have a notable effect on traffic to my site. But I use it only for the link-lists and FFAs, not for the main search engines. When it comes to the main engines, there's only one piece of software that is worthwhile; WebPosition Gold.

This is one superb piece of software, and can take care of pretty much every single aspect of your search engine work. But note that it is as far from a "push-button-and-watch" solution as you can get. Using it is extremely time consuming, but the range of functions and sheer power of the software is highly impressive. You also need to subscribe to updates of the software's database if you're going to use it to it's full capabilities; in this case, it really is money well spent. Worth its weight in gold one might say…

Another consideration that you should be aware of is that almost all the search engines, lists, links-lists and FFAs require an email address of some sort. I would strongly advise you setting up a temporary address with one of the many free services that are out there for this purpose. I have more than a few setup for this very task, and routinely clear out many hundreds of "garbage mails" each month, none of which I would want in my actual inbox. In terms of the main search engines however, I've never encountered any difficulties with giving them my real email address, and most if not all have very clear privacy policies.

If you're still not sure you can handle this work on your own, there are also some highly reputable services out there that can do the work for you. Be careful though - the majority really seem to be little more than one person with their own copy of the same software, with little more experience than you yourself already have. Use common sense when choosing them - ask for referrals of past clients, follow-them up, and ask what guarantees they can offer. Throwing a few questions to test their knowledge is also a good idea - these articles should give you some good ammunition! As with anywhere else on the internet, there seem to be more sharks than water. Don't get bitten.

It goes without saying that there is an obvious correlation between the number of visitors to your site, and the number of sales. Assuming that your product is good, and your methods of capturing the attention of your visitors are honourable, you can only gain by increasing the number of site visitors. Whether you're in a market flooded with competition, or have your own specific niche, you still need to make sure that you're seen. Even taking into account the work involved in search-engine activities, it is still one of the most time and cost-effective methods of publicity available. Seize the opportunity; be seen, be sold!

What General Directories Provide Great Value?

I liked the ones I left linked above. There are a few others that are decent as well, but the broader I make the list the more likely I am to eventually promote sites that are doing lots of spammy things, like whoring out their sites to AdSense or sitewide casino links.

I see a couple of the unlinked directories listed above ranking in the SERPs for a broad range of queries, but some of them may not exercise much editorial control, and will eventually lose some of their authority.

For the sake of MSN and Yahoo! I still do submit to a number more directories than just what is listed above. The number depends on the field, but if the business is going to be a web savvy business that can afford to create strong brands and/or useful content then they will also have many links from outside the directory sphere.

Topically Relevant Directories:

It is hard for me to list quality topical directory examples because:

if you do not know a topic then it is hard to judge quality
directories change over time.

For example, I used to always use a certain directory as an example of a quality directory, but now that there are off topic airline ads on the home page and too much AdSense ads I don't put as much stock in it.

Think Local:

Some local directories are way under priced and of high quality. Quality local directories tend to drive significant hyper targeted traffic.

A few other things to consider when registering with directories:

I use Roboform to submit my sites, but mix up my link anchor text and descriptions (especially since some search engines have certainly looked at word relations outside of on-the-page content and anchor text).
If your market is competitive and your site is new you will also need to get other types of links if you want to rank in Google
mix your anchor text
if brand name is keyword rich make sure you also try to get a few variations in your listing titles outside of your brand name such that if you push the brand hard and cause significant natural linkage it won't cause your link profile to look wonky due to too much similar link text.
If your brand is not generic it may only take a couple links for you to rank at or near the top of the search results for it.
A Yahoo! listing or DMOZ listing may be worth 20 or more links from lower quality directories.
Each good link you get allows you to get many junky links without it really hurting you (say ~50 or so - depending on industry)

How Different Search Engines Count Directory Links:

Yahoo! and MSN still tend to count directory links (including low quality directory links) far more than Google does.

For a new one page flash site I got about 50 directory links in a couple days a while ago. It competes for a basket of low traffic $3 per click terms that can cost about $600 a month ranking at about #2 on the PPC ads.

In Google 2 weeks after I started link building the site ranked in the top 10 across a wide array of terms from this basket of keywords. After about 2 months without additional link building the site's rankings in Google dropped off. After that they have started to slowly improve.

Yahoo! took a bit longer than Google to react, but once it did and I went to #1 I stayed there almost every day for the last 5 months.

MSN reacted about as quickly as Google, perhaps even a few days quicker. Outside of a few fluctuations it has ranked fairly consistantly at #1.

The client ranks #1 for their brand name and related terms in all major engines. They probably would rank a bit better in Google if I got those links over time and showed consistant growth, but considering how cheap I sold those services for I am still certain they have an exceptionally strong ROI, and I am certain their ranking will rise over time if we put more effort and resources into SEO.

Why do People think Directories are Becoming Irrelevant?

As a business model directories do not work well unless you are hyper focused or have significant authority to leverage. (Unless you are selling PageRank to naive webmasters who have yet to learn much about SEO or get burned by shady directories.)
As more people write and compile information the quality of information needs to be better to be link worthy.
Most directories (especially most paid directories) do not add much context as to why a particular site is important, useful or worthwhile.
People do not give out links as freely as they once did. It is hard for a directory site to be viewed as link worthy as they were in the past, thus they do not get as much authority to pass on.
Active channels, such as topical weblogs, tend to drive far more traffic than most fairly static directory sites.
Google's algorithms are improving. They are getting better at scrubbing link quality and filtering out duplicate or near duplicate content.
Most general directories are useless spam.
Couple improving search algorithms with social bookmarking sites and they make the job of professional catalogs and archivists less relevant, except perhaps for ultra niche categories that are not well cited.

Why is the Business Case for Directories Falling Apart?

Many of the reasons listed above (market hypersaturation, lessening authority, other content types - like blogs and wikis - are fighting for the same audience, improving search quality, bottoms up social systems).
Directories create inefficiently priced marketplaces.
Most directories drive so little traffic and value that it is hard for them to make their marketplaces more efficient.

There is Still Some Value in Directory Links:

In conclusion, I still like a number of directories, but sometimes it helps to drill down to look at relevancy more than just buying any old link. I also think that even if some of the mid to low quality directories do not offer lots of value in Google they still help with the other engines. Another added bonus of building links from directories and other sources is that they can inflate your link count to help discourage competition and/or pollute your link profile to make it hard for competitors to see what all links are helping your rank where you do.

Posted in: directories seo tips

Submitting to the search engines

What General Directories Provide Great Value?

SEO articles index