Problem: One of your web crawling programs hangs while crawling my site. I am running Windows Vista. What’s wrong?

Solution: Depending on how Vista interacts with your web server, it may be necessary to disable “Receive Window Auto-Tuning”. You can check if this option is enabled by running the following command from the command prompt:

netsh interface tcp show global

The output should look like:

Querying active state…

TCP Global Parameters
Receive-Side Scaling State : enabled
Chimney Offload State : enabled
Receive Window Auto-Tuning Level : normal
Add-On Congestion Control Provider : none
ECN Capability : disabled
RFC 1323 Timestamps : disabled

Here we can see that the “Receive Window Auto-Tuning” level is set to “normal”. To disable it, run the following from an administrator command prompt:

netsh interface tcp set global autotuning=disabled

To revert this setting to it’s original state, run the following:

netsh interface tcp set global autotuning=normal