I am looking for python expert.
Budget: $50 – $100 USD
I am facing the http connection issue when I am crawling via proxy server.
This proxy server has concurrency setting, it is 5 time in a second.
When I am crawling, and the proxy has overloaded, I can't crawling anymore.
So anyone know how can resolve this issue, Please help me, it would be appreciate.
Note:
- I am using parallel processing for crawling.
- The http connection issue
"HTTPConnectionPool(host='host', port=port): Max retries exceeded with url: "
Thanks
This proxy server has concurrency setting, it is 5 time in a second.
When I am crawling, and the proxy has overloaded, I can't crawling anymore.
So anyone know how can resolve this issue, Please help me, it would be appreciate.
Note:
- I am using parallel processing for crawling.
- The http connection issue
"HTTPConnectionPool(host='host', port=port): Max retries exceeded with url: "
Thanks