The Best Sitemap Generators
by ComputerBob
November 18, 2006
I switched from using Windows to using Linux on my main PC this past July. As a result, I no longer run
GSiteCrawler, which was the best Windows-based, Google sitemap generator software I ever used. It will run right on your Windows PC, and it is fast and trouble-free, If you're a Windows user, I recommend that you download it, install it on your Windows computer, and use it. It does a great job.
As for me, I spent the past several days looking for a free long-term, non-Windows (or operating-sytem-independent) solution for this site's sitemap needs. This article represents what I learned by researching and trying about 20 different free, online sitemap generators, and installing (or trying to install) about 20 different free sitemap generator scripts on my remote Linux web server.
Free, Online Sitemap Generators
- Most can be run from any operating system, because you control them through your web browser and they run on their own web servers, instead of running on your computer or web server.
- The great majority of them only spider one-level deep. In other words, they find all of the main categories that appear in this site's left navigation column, but they don't see the hundreds of web pages that are in those categories. The result is a sitemap file that only contains the URL's of those main categories.
- Nearly all of them are limited in the number of web pages they will spider — the limit for most is 100-200 pages, but one or two of them will spider up to 500 pages.
- The ones that are able to spider larger sites often have long queues of web sites waiting to be spidered before yours. This can cause them to take several hours, or even a day or two, to spider your site.
- Many of them require your web browser to allow them to run JavaScript, Java or Microsoft's ActiveX. My Linux desktop doesn't run ActiveX, and as a security precaution, I never allow untrusted sites to run JavaScripts or Java applications.
Free, Server-Based Sitemap Generator Scripts
- You must install them on your web site's web server in order for them to run.
- Because they're running on the same server as your web site, they run faster and have more features than online sitemap generators.
- Nearly all of them are hideously underdocumented. Some have no documentation at all, not even in the script file itself. It only took a few minutes to upload each script's files to my web server, but I spent hours trying to get some of them to work, with no success.
- Many of them require you to install additional things on your web server, like unusual programming languages, in order for them to run.
- Most of them automatically spider your web site; however, some of them require you to manually create a list of all of your web pages' URLs; then all they do is reformat your list into a sitemap file.
- Some of them require you to have special permissions on your web server, like SSH (secure shell) access.
The Best Free, Online Sitemap Generators
It was like looking for needles in a haystack, but I finally found two great, free, online sitemap generators.
XMLEcho's Google Sitemap Generator, which I mentioned a couple of days ago, has the following features:
- It's free.
- It's fast.
- It spiders unlimited levels into a site.
- It will spider up to 5,000 web pages.
- It will obey your site's robots.txt file, if one exists.
XMLEcho also has a free sitemap generator script that reportedly removes the 5,000-page limit and provides more functionality than the online generator. I dare you to try to install it.
XML-Sitemaps' Sitemap Generator has the following features:
- It's free.
- It's fast.
- It spiders unlimited levels into a site.
- It will spider up to 500 web pages.
- It can create sitemaps in 4 different formats (XML, ROR, Text, HTML).
- It will obey your site's robots.txt file, if one exists.
I highly recommend both XMLEcho's tool and XML-Sitemap's tool. Whether your site is large or small, or whether you need just a Google sitemap or sitemaps in multiple formats, you can decide which of the two tools will better fits your needs. They are both far better than any of the many other free online sitemap creation tools that I found.
The Best Free, Server-Based Sitemap Generator Script
For all of the reasons noted above, I couldn't find even one free sitemap generator script that I can recommend.
The Best Non-Free, Server-Based Sitemap Generator Script
Fortunately, I found a great commercial sitemap generator script. XML-Sitemaps, the company that hosts one of the two free, online sitemap generators that I recommend, also sells a sitemap generator script. In the hundreds of forum messages and articles that I read about sitemap generators while researching this article, I read many positive comments about it, and didn't see any negative ones.
XML-Sitemaps' sitemap generator script has the following features:
- It currently costs only $14.99 (U.S.D).
- It is a PHP script, so it will run on most updated web servers (see its web site for detailed requirements).
- It's pretty easy to install, if you have some experience installing PHP scripts on your site's web server.
- If you don't know how to install a PHP script on your site's web server, the XML-Sitemaps.com people will install it for you for an additional $10 (U.S.D).
- It's very fast, because it runs right on your site's web server.
- It has an easy to use, point-and-click interface.
- It spiders unlimited levels into a site.
- It will spider an unlimited number web pages.
- It can create sitemaps in 4 different formats (XML, ROR, Text, HTML).
- It can automatically notify Google and Yahoo! that your sitemap has been updated.
- You can run it manually, or it can run automatically, as a scheduled cron job.
- It automatically generates a list of your site's broken hyperlinks, and tells you which web pages they appear on.
- It will obey your site's robots.txt file, if one exists.
- It has many more features (see its web site for details).
I highly recommend XML-Sitemaps' sitemap generator script. I bought it and used it to create this site's current sitemaps. And I was relieved that its "broken link" feature showed me the 7 broken hyperlinks in this site's web pages that I had been trying to find for the past few days.
Conclusions
Though there are many, many options from which to choose, it's very difficult and time-consuming to find really good sitemap generator services and scripts. The good news is that I already did all of the work and found them for you. Do yourself a favor and use the ones that I recommend.