Sitemap Creator Revision History

Version 5.1.0

  • Fixed HTML entity encoding issue for “rel=alternate” entries in sitemap
  • FTP is now compatible with FTP servers using TLS 1.2
  • Non-standard HTTP 429 (Too many requests) response is now identified and logged correctly
  • Fixed an issue that would prevent scheduled tasks from running on Windows Server 2012 R2
  • Added new “Find” feature to locate specific URLs/Text in Sitemap Creator results
  • Fixed an issue that could cause the selected UI “Theme” to revert back to the default when restarting Sitemap Creator
  • Various improvements to the HTML and JavaScript parsers

Version 5.0.2

  • Added new option to ignore specific CSS classes when crawling
  • Fixed log entry for Crawl Finished time

Version 5.0.1

  • TLS 1.1 and 1.2 are now supported if .NET 4.5 (or later) is installed on your PC
  • Crawler now auto-detects when Windows-1252 codepage is mislabeled as ISO-8859-1
  • Minor internal tweaks for Windows 10
  • Updated the YouTube video checker
  • Soft 404 detection is now case insensitive
  • JavaScript errors can now be hidden on the Login Wizard
  • Added an error message to the Scheduler Log when no output file is specified
  • Added the ability to change the default sitemap settings

Version 5.0.0

  • Brand new Windows 10 look and feel
  • .NET 4.0 is now required (.NET 2.0 no longer supported)
  • The date the sitemap was created is now included as an XML comment at the top of the sitemap
  • Added a new “#Date#” placeholder field in the HTML sitemap template
  • Updated the YouTube integration feature to use the latest YouTube API
  • “hreflang” and alternate views are now detected and included in XML sitemaps
  • Updated internal components to latest versions

Version 4.1.2

  • Updated video detection logic for sites using Schema.org embedding
  • Sitemap Creator is now compatible with sites that do not use SSLv3

Version 4.1.1

  • Fixed blank XSLT upload issue

Version 4.1.0

  • Added “Any” option to image detection settings
  • Added new Priority Rule wildcard, “!*” now matches any character except “/”
  • Added detection for HTML5 videos
  • Added right-click shortcut to copy sitemap results in CSV format
  • Updated Sitemap Creator so that it can work in a FIPS compliant environment
  • Added “#Project#” to HTML templates
  • Added the ability to upload an XSLT file along with the XML sitemap
  • Updated scheduler logic so that sitemap isn’t updated if no pages are crawled (this prevents uploading an empty sitemap if there was some sort of major crawling error)
  • Renamed “Advanced Project Settings” to “Advanced Options”, and changed “Default Project Settings” to “Project Defaults”
  • Updated 3rd party components
  • Sitemap Creator will now work in a FIPS compliant environment

Version 4.0.13

  • Updated Sitemap Creator’s internal 3rd party components

Version 4.0.12

  • Fixed an issue that could prevent video results from loading correctly from saved projects
  • Fixed sitemap generation if no file extension is provided

Version 4.0.11

  • Improved YouTube video detection (for videos using “LightBox” style embedding)
  • Improved image detection (for images using “LightBox” style embedding)
  • Changed YouTube API integration
  • Sitemap Creator now automatically pops up the “New Project” window at first run
  • Improved HTML parsing for malformed pages
  • The “Log” tab now shows the number of log entries

Version 4.0.10

  • Internal platform changes

Version 4.0.9

  • Improved cookie handling in crawler
  • Fixed various project management bugs
  • Fixed minor issue that could occur if a link used capitalized “HTTP://” protocol prefix
  • Various internal crawler improvements

Version 4.0.8

  • Improved detection of embedded videos using the Schema.org protocol
  • Changes Sitemap Creator so that .NET 4.0 is the preferred framework if multiple version of .NET are installed
  • Fixed a bug that could prevent deleting some projects
  • Added thousand separators to result tab counts

Version 4.0.7

  • Updated internal FTP component to fix SFTP compatibility issues with some servers
  • Enhanced image detection so that images from “Simple Image Gallery Pro” are automatically detected
  • Extract Video Data enable/disable setting is now saved correctly

Version 4.0.6

  • Updated scheduler logging to provide more detailed information (project name is now included)
  • Time format now uses timezone offset (instead of UTC time)
  • Fixed a localization issue with the video “rating” in video sitemaps

Version 4.0.5

  • Added the option to disabled cookies in the crawler (under Advanced Crawler Settings)
  • Added the option to include the time in the Last Modified field
  • Enhanced the video sitemap feature to detect Windows Media Player embedded videos
  • Added the ability to extract the video title and/or description from the embedded video parameters if available
  • Sitemap Creator can now use .NET 4.0 or .NET 3.5

Version 4.0.4

  • Improved Vimeo integration so that non-public video meta data can be collected
  • CSV export now uses UTF-8 encoding
  • Added Rating to the video sitemap results (if available)
  • Video parser now looks for the duration of a directly embedded video in the “dur” or “duration” parameters
  • Added “Copy” option to the result context menus
  • Enhanced image parser to search for lightbox/lytebox images
  • Multiple domains can be included in a sitemap if fully qualified Additional Root URLs are used (remember to add all your sites to Google Webmaster Tools)
  • Fixed a bug in the Sitemap Import feature that could cause an Unexpected Error if sitemap contained duplicates

Version 4.0.3

  • Internal changes to reduce the amount of temporary disk space required when crawling
  • Fixed an issue that could cause the wrong image URL to appear in image sitemaps
  • Fixed an issue with the scheduler that would prevent the default settings from loading correctly
  • Changed the default settings so that “Obey No Follow” is the default behavior

Version 4.0.2

  • Added new setting to allow a change to the log verbosity
  • Changed the trial limitations so that Video, Image and Mobile sitemaps can be generated

Version 4.0.1

  • Fixed an issues in the Projects list that could cause two projects to appear selected
  • Fixed an issue that could cause Sitemap Creator to fail to start on non-English versions of Windows
  • Improved video detection for some types of embedded videos

Version 4.0.0

  • Sitemap Creator now requires .NET 3.5 and is compatible with Windows 8
  • Added Video Sitemap support (auto detects embedded videos for YouTube, Vimeo, Dailymotion, etc.)
  • Added “Pause” capability with automatic Pause/Resume on Windows suspend/hibernate
  • New Sitemap Creator application icon
  • Projects can now be organized into folders
  • Project file location is now user configurable (can be stored on a network share)
  • Our crawler now supports parsing URLs from PDFs and XML documents so more of your site can be found automatically
  • The “Sitemap” and “Sitemap & FTP” buttons have been combined into one
  • Default Project Settings now apply to existing projects (when the project settings are unchanged from the defaults)
  • Image discovery is improved
  • Windows 8 “Metro” UI Theme
  • Plus many more internal changes and improvements…

Version 3.2.4

  • Added “Accept-Language” header to HTTP requests
  • Fixed a minor bug in JavaScript parser

Version 3.2.3

  • Fixed a bug that could cause an unhandled exception to occur when access in the scheduler on Windows Vista/7

Version 3.2.2

  • Fixed a regression bug in the task scheduler/command line interface
  • Fixed an unhandled error in the Task Scheduler (affected some Vista/Win7 PCs)
  • Removed Yahoo’s ping URL from the list of default Ping URLs (Yahoo is phasing this out as part of their search merger with Bing)

Version 3.2.1

  • Fixed a bug that would prevent Sitemap Creator from crawling on some Windows XP systems

Version 3.2.0

  • Added “Soft 404″ detection
  • Crawler enhancements for improved website compatibility
  • Added option to Image Info context menu to allow excluding image from sitemap
  • Added option to obey no-index meta tags
  • Added support for internationalized domain names (IDNA encoding)

Version 3.1.8

  • Fixed CSV sitemap import
  • Fixed various issues related to cookies when crawling
  • Windows Program Compatibility Assistant no longer appears on Windows 7

Version 3.1.7

  • Updated installer package
  • HTML “Base” tag is now used when parsing image URLs
  • Fixed bug that would cause no ping URLs to be shown for new users

Version 3.1.6

  • Updated cookie handler for improved site compatibility
  • Improved handling of HTML base tag
  • Fixed errors while pinging on some systems
  • Fixed errors while important projects on some systems
  • Fixed handling of IE conditional comments (only affects some sites)
  • Fixed handing of invalid character encoding value in Content-Type

Version 3.1.5

  • Fixed casing issue with Image Sitemap URLs
  • HTML “base” tag now used when parsing Image URLs for image sitemaps
  • Excluded URL list is now applied to image URLs for image sitemaps

Version 3.1.4

  • Minor internal changes to reduce installer file size
  • Fixed minor issues with project export/import that could cause unexpected errors

Version 3.1.3

  • Added option to obey robots “nofollow” meta tag
  • Added options to use HTTP 1.0
  • Fixed issue that could cause unexpected error to occur during crawl when creating image sitemaps

Version 3.1.2

  • Fixed issue that caused “Root URL” option not to work correctly in Priority Rules

Version 3.1.1

  • Fixed “No Priority Rules” text appearing even when Priority Rules are present

Version 3.1.0

  • New multi-threaded crawler for significantly faster crawls
  • Support for Google Mobile sitemaps
  • Support for Google Image sitemaps (with automatic image data extraction)
  • Added options to enable/disable HTTP keep-alive
  • Added options to enable/disable HTTP compression
  • Improved project Export/Import (results and data are now included)

Version 3.0.5

  • Fixed “OK” button not working on Advanced Project Settings window

Version 3.0.4

  • Fixed “Delete URL” in Excluded Pages always disabled
  • Fixed Root URL case sensitivity issue
  • Fixed scheduler error if incorrect password supplied with scheduled task
  • Added support for Google’s Ajax crawling framework (“_escaped_fragement_”)
  • Updated “Default Project Settings” page (differentiates itself from “Advanced Project Settings”)

Version 3.0.3

  • Fixed pages in sitemap not updated after editing Excluded Pages list

Version 3.0.2

  • Fixed condition where scheduled tasks could load incorrect settings

Version 3.0.1

  • Enhanced Scheduler so tasks can be scheduled to run when user is not logged on
  • File change history is only saved if crawl completes successfully. If crawl is aborted, history is not saved.
  • Excluded pages now works correctly when non standard port is used.
  • Changed Additional Root URL behavior, pages under Additional Root URL paths are now treated as internal.
  • Updated installer to automatically launch Sitemap Creator after installation

Version 3.0.0

  • Added Automatic Sitemap Priority
  • Renamed “Site Sections” to “Priority Rules”
  • Added FTP “Browse” to select upload folder
  • New crawler options
  • Added Updated Pages and crawler log
  • Added FTP upload status to email alerts
  • Added “Import Sitemap” option for modifying Priority and Update Frequency values of existing sitemaps
  • Sitemap Creator now saves sitemap information with project
  • New Windows 7 look and feel
  • Added ability Import/Export project files to/from other Inspyder applications
  • New easier scheduler interface, allows editing of scheduled tasks and uses wizard interface
  • Scheduling tasks no longer requires windows password
  • New report engine no longer requires GAC registration
  • Now runs as 64-bit processes on 64-bit versions of Windows

Version 2.0.12

  • Canonical URL detection is now case insensitive (was causing problems if casing did not match link)
  • Fixed “Create Site Section” shortcut that was effected by some Root URLs
  • If all content included in Sitemap, email notification now correctly shows all URLs that have been updated (not just HTML content)
  • Fixed Sitemap Creator application name not appeared correctly in some 3rd party firewall applications
  • Fixed trailing slash duplicate URL problem
  • Fixed Ask.com ping timeout issue

Version 2.0.11

  • Added Microsoft’s Bing search engine to Sitemap Ping URL list

Version 2.0.10

  • Added ability to exclude HTML and Style from duplicate content detection (and Last Updated calculation)

Version 2.0.9

  • Fixed minor URL decoding problem
  • Minor updates to default HTML sitemap templates
  • Fixed an issue that would keep some URLs from appearing in the HTML sitemap
  • HTML sitemap grouping is no longer case sensitive
  • Added support for Canonical URLs as defined by Google

Version 2.0.8

  • HTML sitemaps are now encoded in UTF-8
  • Fixed compatibility issue with servers using Content-Location header

Version 2.0.7

  • Fixed a bug in the trial that could cause sitemaps to contain less than the 25 page maximum

Version 2.0.6

  • Fixed Site Sections not applied correctly if site uses mix of HTTP and HTTPS
  • Updated crawler to include additional HTML attributes
  • Now impossible to create project with invalid characters in name
  • Fixed several minor problems caused by localized versions of Windows

Version 2.0.5

  • Updated schema location information so sitemaps pass 3rd party validators (does not affect Google compatibility)

Version 2.0.4

  • Fixed a minor bug that could occur on some systems when saving sitemap files

Version 2.0.3

  • Fixed performance issue when running as scheduled task
  • Minor change to installer for improved compatibility

Version 2.0.2

  • Fixed a minor UI bug that caused spacing to the left of the Sitemap Result items
  • Fixed a minor bug that could cause HTML sitemaps not to work correctly
  • Fixed a minor bug in crawler that could stop crawl if an invalid relative path occurred during an HTTP redirect

Version 2.0.1

  • Fixed unexpected exception that could occur on some PCs while crawling
  • Redirect destination URLs are now included in sitemap (rather than the actual redirect)

Version 2.0.0

  • Major Update: Sitemap Creator now requires .NET 2.0 or later
  • Significantly reduced memory utilization for large sites
  • Improved email notification, now page that have changed are marked as Added, Removed or Updated
  • FTPS and SSH (via SFTP) file transfers are now supported

Version 1.2.5

  • Added ability to include Meta Description text from crawled pages in HTML sitemaps
  • Fixed an HTML entity decoding bug when 2 digit hex values are used
  • Fixed parsing of URLs beginning with “//”
  • Added basic support for crawling JavaScript
  • Fixed GUI problem for Windows users using non-standard DPI screen values
  • Fixed scheduler ignoring page limit value

Version 1.2.4

  • Added Use SSL option to SMTP settings
  • Added ‘Include All Content Types’ option so that non-HTML/PDF content can be included in the sitemap (disabled by default)

Version 1.2.3

  • Fixed a minor bug that could stop Sitemap Creator for crawling some sites with PDF documents
  • Added “Domain Alias” feature to Advanced Project Settings
  • Improved crawler so that certain types of invalid links are now corrected and crawled on the fly
  • Fixed a minor bug in the crawler that could cause a single page to be skipped

Version 1.2.2

  • ‘Ping’ button now becomes active after ‘Export’
  • Fixed a problem exporting sitemap index file if path name contained a ‘.’

Version 1.2.1

  • Fixed broken context menu on Sitemap Info tab

Version 1.2.0

  • Changed “Additional Root URLs” to use Relative URL style
  • Removed “Options”, moved all global settings into projects and created Master Project to control default values
  • Added user adjustable User-Agent string
  • Changed text on ‘Notify Search Engines’ button to ‘Ping Search Engines’
  • Added HTML Sitemap capability
  • Fixed a bug in “Duplicate Content Detection” when checking PDF documents
  • Changed “Export Sitemap” to “Create Sitemap” in UI
  • Improved crawler speed and reduced memory usage
  • Added ability to import exclusion list from robots.txt

Version 1.1.4

  • Added Moreover.com to the sitemap ping/notify list
  • Added support for sitemap index files
  • Crawler no longer follows HTML Form ‘Action’ attributes implicitly (caused problems with some websites).

Version 1.1.3

  • Fixed bug that could cause Sitemap Creator to freeze when exporting on some PCs
  • Updated application icon
  • Updated crawler: CSS @import “filename.css” syntax now followed correctly
  • Various minor improvements

Version 1.1.2

  • Fixed minor internationalization issue with output format
  • Added Ask.com search engine ping URL

Version 1.1.1

  • Fixed a minor bug that caused the ‘Detect Duplicate Content’ option to not save correctly

Version 1.1.0

  • Replaced ‘Text’ output format with CSV (Comma Separated Values)
  • Added ‘Gzip’ export option (gzips XML output format)
  • Added referrer information to the log
  • Added Email Notification
  • Added parameter removal capability (for removing session IDs, etc.)
  • Added automatic detection (and removal) of duplicate content (such as www.example.com and www.example.com/index.html)
  • Various minor improvements

Version 1.0.5

  • The characters “.”, “-” and “_” are now allowed in project names
  • Content between tags is not included when determining if a page has changed
  • Improved speed when applying Site Section rules to results

Version 1.0.4

  • Added crawler count information to UI
  • Export settings are now saved per project
  • Crawler log now shows total elapsed time and referrer information
  • Added user adjustable crawler delay (to slow crawler) and timeout period
  • Various UI improvements

Version 1.0.3

  • Fixed error when renaming project before crawling
  • Fixed occasional error when accessing help

Version 1.0.2

  • URLs with spaces (and other special characters) are now properly URL encoded in XML sitemap output

Version 1.0.1

  • Fixed results not updating after modifying Site Sections
  • Fixed results not sorting when column header clicked

Version 1.0.0

  • Initial Release