How to register with the search engines

Overview

What are the Search engines?
What are the benefits of registration?
How do I register?
The Mechanics of how pages are indexed
Keywords
Target verses blanket strategy
Title tags
Keyword Density
Site URL
META Tags
Frames and CGIs

Tips

Make sure that your site is ready
Submit to your local search engines
Do not over submit
Re-submit competitors' URLs
Monitor your listings
Yahoo! submission tips

Overview

What are the Search engines?

The search engines are web sites that index web pages and allow users to perform searches for sites held on their database of sites. Some examples of popular search engines are Yahoo!, Excite, and Hotbot.

Not all search engines work in the same way. They can be classified into two categories: directories and deep search engines.

Deep Search engines

This type of engine uses a robot or spider which constantly looks for updated pages. It does this by downloading a page, then visiting all the links on that page and so on. Each page is automatically indexed, referenced and placed on the database. Here is the complete list of deep search engines that use robots to index pages:

Alta Vista, Excite, Hotbot, Infoseek, Lycos, Northern Light, Opentext, Webcrawler.

Note that there are only eight of them. Other search engines simply draw on the databases created by these main engines (NB Yahoo! is different and is discussed below).

Directories

The directories do not index pages using robots. The only information held in the database is what is input by the people registering their pages by use of input forms on the engine's web site.

Some examples of standard search engines are EINet Galaxy, new Rider's Yellow Pages and Starting Point.

Categorised listings

These are a subset of directories. Information is held in categories with short descriptions underneath. The most important point is to place the page in the most relevant category where you think people will look for your product.

There are thousands of examples of these listings often called free for all links pages. They don't get a huge amount of traffic, but if you submit to enough of them. the cumulative traffic will add up. Submitting with software is the best way to take advantage of these.

The most important search engine of all is Yahoo! This is a categorised listing. If you are selling something on the web, then approximately 70% of the business you get will originate from Yahoo! so getting a decent listing here is of great importance. Getting a listing on Yahoo! is difficult and will take several submissions over several months. The main Yahoo! differentiator is that the pages are viewed by a human rather than a web robot. The Yahoo! reviewers have seen all the tricks regarding web site submission. The reviewers can easily be put off your site, so it is necessary to try and stick to the rules.

What are the benefits of registration?

Most of our customers do not think that good positioning with the search engines would be particularly beneficial to their site. Our experience tells us that this is one of the most important method of promoting your site. Think for a moment; probably the first thing you will do when searching for a site is to query a search engine. Once your site becomes registered you will begin to see your hits rising.

Registration costs nothing except for some thought during web design and a little time to submit your pages. After that you are effectively advertising your site for free.

How do I register?

1. Using software

   This method is the quickest and most effective. High quality submission software will keep you in control of the entire process and will not only submit to the major search engines but also to the hundreds of smaller directories. The software will also perform functions such as creating META tags and html checks. Some examples of this are WebPosition Gold and TopDog.

2. By hand

   This is time consuming, although may be the best method if you only want to submit to the main engines.

The Mechanics of how pages are indexed

There is quite an art to submitting your pages effectively to a search engine. Different engines index pages in different ways, so your site has to be able to cater for all of them. Below lists some of the important components for submission.

Keywords

These are the words which you think people will type into the search engines when looking for your site. You should think of about ten keywords and then list them in order of importance. Do not use words such as "internet" or "services" as almost every internet site relates to these. Instead try to think of something individual about your site that people will still type in when searching.

Don't use sex related keywords to try and get traffic to your site. It doesn't work and the chances are your page will be penalised for trying to fool the engines.

After you have thought of your keywords then make them plural by adding an "s" if you can. The singular of the word will still be found. For instance a search for banana would still yield your site if bananas was a key word.

In a similar vein, add misspellings. It is quite possible that customers will be searching for your product using a misspelled word.

Finally think of some keyword phrases. These are two or more words that users may type together when looking for your product. For example "contact lenses" would be a keyword phrase.

Once you have thought of your keywords and phrases, test them to see what comes up. You may realise that the worlds you type in result in a listing of something totally unconnected. You will need to think of something else.

Target verses Blanket strategy

This has to be decided at an early stage of site design. If you have a large range of products then a long list of keywords would be best. However a short listing would score higher on a search for those words because the keyword density would be higher, so your site would appear higher up on the list.

Title tags

The title is the text on your page between the <title></title> tags. For instance the title of this page is "Firstserv Search engine howto". The title of your page is the most important factor for robots to compile a description of the content of your site. It is more important than anything within META tags. Title words should be chosen carefully from your keywords. Only use a few words in the title, and put a different title for each page.

Keyword density in the <body></body> of the document.

Search engines sort pages in order of the density of keywords in the document. For example if this page had only the two words "web hosting" in it and someone did a search for "web hosting", then my document would appear at the top of the list as it would have 100% density of keywords.

Your most important keywords should be placed towards the top of your document. Keywords within <!-- --> comment tags are ignored by the engines.

It is not a good idea to try and saturate your document with keywords (for instance white words on a white background) as most search engines are able to look out for this and will penalise your page should this be detected. This means that your page may not be listed at all, or may be placed at the bottom of the search results.

Site URL

It is preferable to have your own URL. For instance, motorcycles.co.uk would be listed in preference to motorcycles.yourisp.co.uk. If you have more URLs such as motorcycles.com and motorcyles.net then you have more opportunity to submit your site.

META Tags

Although META tags are necessary, they are far less important than the title of the page and the keyword density. Not all search engines recognise title tags (Alta Vista and Infoseek do). Here is an example of the use of META tags:

You should not repeat the same keyword more than three times and then you should not put the words together, but spread them through your keywords to avoid detection.

You META description should be less than 150 words. The description does not need to be a list of keywords, but an actual description that should attract visitors to click on your site.

Frames

There is presently no evidence to suggest that search engines index links to pages within frames. Therefore by using frames the majority of your site will not be indexed. This, as well as the fact that many browsers cannot read frames makes it appear sensible not to use them.

CGIs

In a similar way to frames, pages created on the fly by CGI scripts do not get indexed by most engines. Slurp is a new product used by Yahoo that can index dynamic pages.

Tips

Make sure your site is ready

Make sure that your site is ready to be submitted before doing so. Once your site has been indexed it is extremely difficult to change the listing. The robots will only visit your site every few months and Yahoo! reviewers will effectively never come back. Therefore it is important to have thought carefully about submission criteria before registering your site.

Submit to your local search pages

The search reviewers for a local domain (e.g. yahoo.co.uk) will have far fewer submissions than the international equivalent domain (e.g. yahoo.com). You stand a higher chance of being listed if you submit to the local domain.

Do not over submit

There are limits as to how many page submissions may be made each day. Alta Vista will only accept ten URLs per domain per day. If you submit more than 50 pages within one day to Infoseek, then your entire domain will be removed from its listings.

With Yahoo! the best policy is to submit, wait two to three weeks then re-submit.

Re-submit competitors' URLs

As the criteria for indexing pages has changed over time, pages that achieved a high rating a few years ago may not achieve such a high rating today. If a competitor has a better listing than you and it was last indexed some years ago, then their page should be re-submitted. It is likely that their new indexing will be substantially lower than it was before, especially if they have used keyword spamming, which was not penalised by the engines until late 1996.

Monitor your own listings

Once the initial submissions have been made, most of the work is over. However, it is necessary to constantly monitor the position of the listings. A listing may disappear for no reason or start to drift down the rankings. In such a case, it will be necessary to re-submit the page to give it a lift.

Yahoo registration tips

Approximately 90% of sites submitted to Yahoo! get rejected by the reviewers. The reasons may be one or many of the following:

• The site does not appear to be useful.
• The site has been designed poorly.
• The category chosen was inappropriate.
• You did not follow the yahoo! guidelines for submission, for instance you put your description in capitals.
• Your site is small. A single page site has much less of a chance of being listed than a multi-page site.

It is up to you to try to reduce the reasons for the reviewer to reject your site. Follow the instructions on the registration page as closely as possible. Use no more than 25 words in the description, but only use one full stop. The description will be truncated after the first full stop that the reviewer comes to.

From the Yahoo! directory support team in the UK:

"Please allow at least two to three weeks from your first submission to allow our team of editors to review your site.

If you still do not see your site listed or hear from us by then, please resubmit the site again at http://www.yahoo.co.uk/docs/info/include.html and allow another two to three weeks.

If you are still having trouble you may then email us with the URL at ukie-url-support@yahoo-inc.com and we will then open a Directory Support inquiry into the matter."

Top of Page