Web scraping how to deal with Captchas

Get help with using AutoHotkey (v1.1 and older) and its commands and hotkeys
omar
Posts: 540
Joined: 22 Oct 2015, 17:56

Web scraping how to deal with Captchas

12 Apr 2021, 10:54

I want to scrape a website and get data and make charts.
It involves following 50-200 links from one page.

QUESTION: how do I deal with Captchas?
Just wondering if someone else has done before?

Thanks.
User avatar
mikeyww
Posts: 26885
Joined: 09 Sep 2014, 18:38

Re: Web scraping how to deal with Captchas

12 Apr 2021, 11:03

I think it's kinda frowned on because the purpose of Captcha is to require a person to enter the information so as to validate that a person is completing the process.
gregster
Posts: 9002
Joined: 30 Sep 2013, 06:48

Re: Web scraping how to deal with Captchas

12 Apr 2021, 11:05

Circumventing captchas is usually not trivial - and probably also out of the scope of these forums, since captchas are usually used to prevent spamming, or to ensure the reasonable usage of websites. Avoiding these restrictions could be seen as malicious, in many cases, and against our forum rules.

For collecting large amounts of data, there are usually commercial or free data providers available (for example, for stocks or cryptocurrencies) which might offer some WebAPIs or similar services, without the need to circumvent captchas. I would look for those alternatives.
malcev
Posts: 1769
Joined: 12 Aug 2014, 12:37

Re: Web scraping how to deal with Captchas

12 Apr 2021, 11:31

I solved captchas with captcha solving online services.
They have api and price was OK for my client.

Return to “Ask for Help (v1)”

Who is online

Users browsing this forum: Rohwedder, ruespe and 357 guests