Where Does Google Get Its List Of Urls To Spider

[Roonyroo]

Where Does Google Get Its List Of Urls To Spider, also how does it know when a new sites been created

Is it from the Url registery etc., ...

Also some more technical info on how google finds urls to spider, would be fascinating, I'm not interested in algorithhms or keywords, but where it finds urls & domains to spider

[jballi]

Hint: If you have a question about Google, just ask Google. They are usually very forthcoming. This is what I found when I asked.

https://support.google.com/webmasters/a ... 0897?hl=en

[Roonyroo]

Google's site's useless, I'm basically trying to find out where google gets its list of NEW url's to spider

How does google know when a new site is registered & launched

[joedf]

well, their "algorithm" is probably copyrighted

[jballi]

well, their "algorithm" is probably copyrighted

Google's formula for extracting, categorizing, and indexing data from a web site is a secret but how they find web sites is not.

Google's site's useless, I'm basically trying to find out where google gets its list of NEW url's to spider

How does google know when a new site is registered & launched

Like all web searching tools, Google uses a web crawling bot (aka "spider") to find new web sites. This web page from Google makes it as clear as it gets.

https://support.google.com/webmasters/answer/182072

tl;dr: They search them all, some more often than others.

I hope this answers your question.

[Roonyroo]

Thanks for the replies guy

yea everyone knows googe spiders sites, I was looking for a list of sites google uses to find newly registered url's & ip addresses

Basically where does it find brand new unspidered sites, if a person registers a bran new site

[LinearSpoon]

What part don't you get? It's all stated in the links.
When it crawls the sites it does know about, it finds links to new sites that it doesn't know about. Webmasters may also submit sitemaps to augment this process.
If you register a new site, Google won't find it until people start linking to it, or until you tell Google that it exists.

[joedf]

True +1

[tank]

So i think your asking how does it know your brandnewawesomesite.com has come to exist. Answer? it doesnt. sites it does know about and spider end up with links to it and it crawls to it. Hosts or webmasters submit the site as part of a setup. in short there is no magic sauce to it. part of googles cloud services no doubt finds things in mention like "i wanna pee inc" and does whois on versions of iwannapee i-wanna-pee etc. and obviously there is the oodles and oodles of google spyware Chrome is now considered the most tracked and yet secure as well as most popular browser.
The point is it honestly doesnt matter how they find you for a newly registered domain. your gonna get found
http://www.webmasterworld.com/forum3/25278.htm
but you can be found intentionally