I want to scrape a website and get data and make charts.
It involves following 50-200 links from one page.
QUESTION: how do I deal with Captchas?
Just wondering if someone else has done before?
Thanks.
Web scraping how to deal with Captchas
Re: Web scraping how to deal with Captchas
I think it's kinda frowned on because the purpose of Captcha is to require a person to enter the information so as to validate that a person is completing the process.
Re: Web scraping how to deal with Captchas
Circumventing captchas is usually not trivial - and probably also out of the scope of these forums, since captchas are usually used to prevent spamming, or to ensure the reasonable usage of websites. Avoiding these restrictions could be seen as malicious, in many cases, and against our forum rules.
For collecting large amounts of data, there are usually commercial or free data providers available (for example, for stocks or cryptocurrencies) which might offer some WebAPIs or similar services, without the need to circumvent captchas. I would look for those alternatives.
For collecting large amounts of data, there are usually commercial or free data providers available (for example, for stocks or cryptocurrencies) which might offer some WebAPIs or similar services, without the need to circumvent captchas. I would look for those alternatives.
Re: Web scraping how to deal with Captchas
I solved captchas with captcha solving online services.
They have api and price was OK for my client.
They have api and price was OK for my client.