Jump to content

Google craw.


conan318

Recommended Posts

No, Google has absolutely no direct access to your database. Why would it want to? The data in there would be a jumbled up mess in the eyes of Google, even they would never be able to meaningfully index it -- never mind somehow guess the corresponding web-page to link it to. That leads to why semantic mark-up is so crucial in SEO.

 

Yellow Pages will use a single page, but pass in dynamic URL parameters to tell the code behind it what content to retrieve from the database and display on the page. This means that each entry will have it's own URL, but still point to the same file. I imagine they also take it a step further, and use the Apache "mod_rewrite" module (or equivalent for Windows) to create rewrite rules so that they can structure the URIs how they please.

 

So for example, you may request:

 

yellowpages.com/pizza/sheffield/pizza-hut

 

Which then the rewrite rules internally re-format into:

 

yellowpages.com/index.php?category=pizza&location=sheffield&brand=pizza-hut

 

The code within "index.php" would then handle retrieving and displaying the output, based upon the parameters passed in. I highly doubt Yellow Pages is this simple however, or even written in PHP, but hopefully you get the picture.

Link to comment
Share on other sites

Basically how adam said.

 

The crawling and your links can be gotten in a few ways.

 

A crawler can rip every link from your pages, (besides ajax), then follow every of those links to any depth it wants.

Generating random words and or pages in urls can be used in crawlers to try to get results from a website, could even be in a search form.

 

Can also get information from a sitemap if you have one.

They even acquire links from your site other sites they crawled that had a link of yours.

 

There's also feed crawlers grabbing latest posts.

Link to comment
Share on other sites

This thread is more than a year old. Please don't revive it unless you have something important to add.

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.