Error messages when it relates to data mining and the utilization of proxies can seem defeating. That feeling of despair can intensify when you’re making thousands of requests leveraging automated systems. It can also cause you to use up the bandwidth that you paid for without any results.
Fortunately, most errors can be resolved by the user in a couple of minutes.
We at ProxyEmpire have put together this ultimate proxy error checklist which functions as a cheat sheet so that you can diagnose error messages and common problems.
Go ahead and bookmark this page so that if you ever need to lookup an error code you can do so.
Before We start with how to diagnose and correct errors, we need to understand exactly how proxy servers work, to begin with. Since the bulk of you will be using rotating residential proxies, we will start with how backconnect proxies’ function.
Mobile and residential rotating proxies are sourced from peer devices ethically.
A server Is responsible for connecting you to the peer device where you use their network. That is why you will see IP addresses that are not in the location you chose for your proxy settings. The load balancing server has the job of making sure you’re connected to the fastest device possible.
Most problems with rotating proxies come down to requests timeouts because the user’s device is not responding, or you are setting up your proxy authentication incorrectly.
It may seem redundant, but you need to make sure that your authentication settings are set correctly to work with the proxy server. Almost all proxies require authentication and that can be accomplished through whitelisting your localhost IP address or using a username-password combination.
If you’re working from an office your IP address is probably static meaning that it will not change as much and whitelisting the IP address is a good option.
Residential IP addresses are dynamic, and they change every few weeks.
If your authentication relies on a whitelisted IP address and you’re using them from a residential network, you may consider that your IP address changed and that’s causing a proxy misconfiguration. You also must make sure that you update your username and password per the Geolocation you choose.
A proxy from New York City will have a different username than a proxy from San Francisco.
Now we’re going to take a look at some of the most common error codes you will find and how to resolve each issue presented. There are hundreds of different types of error codes, however, we have compiled a list of the ones you will see when you are web scraping with proxies.
Remember, if you have any questions, you can always contact us at ProxyEmpire in our live chat and we would be happy to help if you cannot find the solution here.
This error code means that the server cannot find any content for your request. It indicates that there’s no content to be returned to you. This message has nothing to do with the capability of the proxies or your settings. If you’re web scraping, you could consider choosing a new source.
You will see a 206 code message when a file or the content you are requesting is too large to send. Sometimes what the server is doing is breaking down the download size into multiple files. This error code has nothing to do with the proxies themselves.
If you’re receiving a 300 code it means that the URL is pointing to too many resources. This usually occurs from DNS errors that are out of your control as someone who is trying to access the web property. You may check to see that the URL you’re using is correct.
A 301 redirect is when a URL is changed and sends a signal to web crawlers that the change is permanent. It should have automatically redirected you to the final web asset, however, if it did not try tracking down the new URL home.
This code is similar to a 301 redirect; however, it signals that the resources were moved temporarily instead of permanently. If you’re web scraping the best thing to do is to avoid the URL and find a new target. This is a Webmaster issue and is out of your control.
A bad request is a response that indicates there’s a problem with your communication with the host. Sometimes this is a result of deceptive routing, syntax errors, or invalid formatting. Check your parameters again and give it another try but be aware of malicious intent.
Almost everyone who is browsing the Internet understands that a 404 indicates that the content does not exist in the location you’re looking for it. If you are web scraping, you can go back to your command prompt and make sure that you’re targeting the right location.
A 408 code is oftentimes an indicator that the webserver is busy. If you keep getting this same error message it could be that the node that you’re using is too slow. One quick fix would be changing ports or GEO locations. It could be an error originating from the website’s server which is out of your control.
If you’re receiving a 429 error message it’s likely that you’re sending too many requests with the same IP address at the same time. This is a leading cause for being banned from a web platform while using proxies. Try increasing the rotation time of your mobile or residential rotating proxies.
A 500 error code is received when an unexpected error has occurred server side. This is completely out of your control as someone who is gathering data using proxies. If it’s the only source for your information you could consider coming back at another time and seeing if the server works.
Unlike the 500 error code, a 502 can be an indication that a super proxy is refusing your Internet connection requests. If you’re using an automated system, it could be that you’re leaving a fingerprint that’s causing this error message which is stating that the platform is rejecting you.
Most of the time an error is not due to proxy failure but because of the host not complying with your request. Now you have a solid list of error code reasons and how to respond to them. If you need further help you can always contact us in the live chat.
Remember, proxies are not the only footprint on the internet, you need to consider setting up your automated systems to use agents that comply with platforms.