How To Get Indexed
Getting into Google’s main index...
Here’s a surprising thing WP Indexer DOESN’T do...
Google's been getting better and better at indexing stuff quickly.
A few years ago they updated their algorithm to improve indexing speed. They were trying to find breaking news topics and show relevant results in the SERPS quickly.Indexing used to take days, but if you've got a reasonable blog and a good ping list Google will send a spider out to check out your blog ASAP (usually in minutes).
In fact, as posts usually get indexed quick by Google, WP Indexer doesn't even ping the main post URL - it leaves that completely to WordPress and your chosen ping list. That's one part of WP & SEO that isn't broken, so we're not trying to fix it. However WP leaves many indexing opportunities on the table. There are a bunch of different strategies it doesn’t do automatically, which is where WPI comes in.
Main vs Supplemental Indexes
So there are 2 types of indexes in Google (that I’m aware of). The main and supplemental index.
The latter is usually what people mean (although they don't realize it) when they talk about a sandbox. It's really the supplemental index.
The difference between the 2 indexes is in the "quantity" of PageRank and the other names Google gives to its similar page traversal and scoring algorithms. So basically it boils down to links. You need a certain amount and quality of links to get in the main index (hence the perceived sandbox, as new sites simply don’t have enough PageRank to make it into the main index).
The good news is that (as you probably already know) every page can give PageRank, even ones on your own site. So you can control your own link reputation and ranking to a certain extent by interlinking on your own blog. This point is very important, and I’ll go into more details when taking about the tagging feature of WPI.
Another key to this is the crawl cycle. When you view the Google cache for a particular page it shows you the "last checked at" time. If you keep an eye on the periods of time between this date updating for a particular page you get the crawl cycle. It could be hours, or for many blogs it's 2-6 weeks.
When this crawl cycle increases across your blog, Google's taking more notice of you. This is great news (and WPI helps a lot with this point) because a higher crawl rate leads to more traffic in the long run.