Question: Sitemap Creator isn’t crawling my sub-domains, why not?

Answer: By default Sitemap Creator will only crawl the pages on the same domain as the Root URL. It does not follow links that are not in the same domain as the Root URL.

For example, if the root URL was “www.example.com” the link “http://download.example.com/” will not be followed because it is not in the same domain as “www.example.com”. Sitemap Creator conforms to the XML sitemap specification outlined at www.sitemaps.org (which is accepted by Google, Yahoo and Bind).

The sitemap protocol explicitly forbids including pages from a domain that is not the same as where the sitemap is hosted. Here is an excerpt from the sitemaps.org website (http://www.sitemaps.org/protocol.html#index):

“Note: A Sitemap index file can only specify Sitemaps that are found on the same site as the Sitemap index file. For example, http://www.yoursite.com/sitemap_index.xml can include Sitemaps on http://www.yoursite.com but not on http://www.example.com or http://yourhost.yoursite.com.”

Note: Google has recently begun accepting sitemaps with multiple subdomains included in a single sitemap. If you wish to take advantage of this you must add the subdomains as Additional Root URLs in Advanced Project Settings. (Remember to include the “http://” prefix on your Additional Root URLs.) You must verify all your subdomains in Google Webmaster Tools or your sitemap may be rejected with errors.