Question: When crawling my website I get the following error:
"The CRC in GZip footer does not match the CRC calculated from the decompressed data." How can I fix it?
Answer: This error message indicates that your web server is returning incorrect HTTP data. Your server is reporting that the HTTP response is compressed using “GZip”, but our crawler is unable to decompress it for some reason.
The easiest solution is to disable HTTP Compression in Advanced Crawler Settings. To access this feature click the “Wrench” button to open the Advanced Project Settings window. Select the “Crawler” tab, and click the “Advanced” button. A new window will appear with the option to enable/disable HTTP compression. Uncheck this option and try crawling your site again.