 Hostwrap.com |
 
|
The Myth of Manual Search Engine Submission------------------------------------------------------ by Ralph Tegtmeier ------------------------------------------------------ One of the ongoing myths abounding in the search engine world is that of manual submission: many search engine optimization agencies make a point in promoting their "manual submission services", and if you follow discussions in various forums and newsgroups you will frequently encounter people advising others to "submit manually only, or else ..." Well - or else, what? Is there something inherently evil or - at the very least - inefficient about automatic submission? And if so, where's the problem? To cut a long story short: there really isn't any, but to understand why not, it's important to get a grasp on the actual mechanics of search engine submission. This will help debunk a lot of the mystique currently surrounding the process and may save you hundreds of hours of unnecessary manual labor. All important search engines feature special submission pages endowed with HTML forms where you can enter your web site's URL or, in some cases, additional data such as your email address, your name, etc. Submission is effected by clicking on the submit button, whereupon you will typically be directed to a confirmation page. To avoid spam and oversubmissions, URLs will only be accepted if and when the data is referred by these same submission pages. Of course, when submitting manually this is a given, and the user needn't even be aware of it. The trick with automatic submission is to make believe, i.e. to fool the search engine submission setup that the URL submitted was actually entered on the submission page installed for this purpose. So how is this done? Easy: every web browser carries a variable with it which can be read by any server accessed. In technical terms: it is the "http_referer" (yes, only one "r"!) variable which tells the search engine from where the browser accessing the success or confirmation page is actually coming. If you submit a site manually, your browser will automatically convey this variable to the submission program (which, by the way, is typically a Perl or PHP script). So what every automatic submission tool has to do to see the submitted data accepted, is to pose as a web browser by emulating a browsers identification variable (the "UserAgent"), configure its http_referer variable to the search engine's submission page's - and bingo! If this is done right, and all commercial submission programs we are aware of are intelligent enough to cover this very basic routine, there's really no way a search engine can tell whether any given URL was submitted by a robot program, i.e. automatically, or manually. Of course, there are some telltale hints which may yet either give your automatic submitter away or provoke a submission denial: * If your program submits too many pages in too short an interval * If your submitter program fails to convey other required data beyond the mere URL (e.g. your email address, your name, or whatever is required by the search engine), as any diligent human submitter would * If your program is not up-to-date, referencing an obsolete submission page variable (these tend to change from time to time as search engines develop) Again, all commercial programs we know of, whether client or server based, will do all this quite nicely. Just make sure you are using a current version. To avoid infringement of submission maximums, submitting no more than 5 pages per domain and engine is generally considered a safe approach. If submitting multiple URLs to several engines (as most programs will do), make sure these are submitted to each engine consecutively: first, you would, for example, submit URL #1 to AltaVista, next to Excite, then to FAST, then to Google, next to HotBot, next to Northern Light, and so on, till the very last engine on your list. Only after this list of engines has been submitted to, would you (or, rather, your program) start submitting URL #2, etc. This ensures that you won't run afault of submission time limits like you might if you submitted all URLs first to AltaVista in one go, then to Excite, etc. Recommendations --------------- One of the most powerful and reliable client based submission tools (and certainly the best money value) is DumpTruck. We are very familiar with it because we have used it extensively before developing our own server based solution, fantomas subFrog(TM) (for which see below) - and because we (in our previous incarnation as "tsigaan softer systems") wrote the code for the German engines submission routine ourselves. Read more about it here: < http://fantomaster.com/fasmbres02.html#dumptruck > For a fully professional server based solution allowing for bulk submissions as a system background process not requiring any monitoring and offering duplicate email report generation (e.g. one copy to yourself, one to your clients) see our fantomas subFrog(TM). In contrast to most submission software available on the net, our program caters not only to US based search engines but covers many international engines as well. Moreover, if you want to see more engines added, we will upgrade the program at no extra charge, provided their submission structure allows for automatization at all. Check it out here: < http://fantomaster.com/fasubfrog0.html > This text may freely be republished or distributed in unmodified form provided the following resource box is included intact either at the beginning or the end of the article and a complimentary copy or notice (link) is sent to the author at the address specified below: Ralph Tegtmeier is the co-founder and principal of fantomaster.com GmbH (Belgium), < http://fantomaster.com/ >, a company specializing in webmasters software development, industrial-strength cloaking and search engine positioning services. He has been a web marketer since 1994 and is editor-in-chief of fantomNews, a free newsletter focusing on search engine optimization, available at: < http://fantomaster.com/fantomnews-sub.html > You can contact him at mailto:fneditor@fantomaster.com (c) copyright 2002 by fantomaster.com All rights reserved. Downloaded at: < http://fantomaster.com/ > © 2004 HostWrap.com , All rights reserved. Unauthorized duplication or publication of any materials from this Site is expressly prohibited. |
