Why Don’t CAPTCHAs work?

CAPTCHA: otherwise known as the annoying box that requires me to prove that “I am not a robot” when using the internet. This is likely everyone’s first interaction with bots, as it is a ubiquitous part of the internet.

CAPTCHAs are incredibly frustrating for users who must complete puzzles, identify trains, and decipher the unreadable text/audio. It must be highly effective for the amount of annoyance, as I can barely identify the trains.

Unfortunately, this is not the case, as bots are quickly getting around CAPTCHAs, which is a largely ineffective solution. So, yes, all of those extra clicks and games are mostly a burden on you.

Today, we will talk about why CAPTCHAs are ineffective against most bots!

Early CAPTCHAs and Brute Forcing

One of the first CAPTCHAs was a series of random letters and numbers in a photo, which looked a lot like this:

Over the years they have gotten a little more ridiculous and trickier, with my least favorite being the RapidShare cats (circa 2007).

Around 2003, CAPTCHAs were popular for forum registration and voting for your favorite websites to win an award. There was little in the way of security or protection. Realistically, all you had to do to bust a CAPTCHA in the early ’00s was to scrape every CAPTCHAs that would pop up and then write a script that would run through every potential answer until you got the right one.

There were limitations of the technology on both the defense (the generation of the CAPTCHA image at the time was a manual process) and the offense (the fuzzing process of the letters, which is the grainy bits and the line would trick the current Optical Character Recognition technology).

Most websites had never heard of website security at the time, let alone preventing a brute-forcing attack. So, this was a standard, albeit time-consuming, way to set up protection. After the initial setup, the bot would solve the CAPTCHA within seconds each time, but anywhere up to 1000 requests were being attempted before the bot would successfully solve the CAPTCHA.

CAPTCHAs and OCR

We talked about OCR (Optical Character Recognition) a little bit earlier, so let’s take our automation game to the next level and dive deeper using the same examples from above.

Most letter CAPTCHAs fear OCR the most, and it is evident by the way they warp and skew letters that they are trying to mitigate against this type of solver.

For CAPTCHAs like the above, we would need to “pre-process” the image. That means we need to eliminate the noise (i.e., the dots and lines), making it harder for OCR software to identify the letters.

There are a few solutions to this problem, first doing some computerized image cleanup with a program such as an image Magick and then running an OCR program such as Tesseract to detect the final image. We have linked a detailed explanation of how users go about breaking CAPTCHAs.

The original captcha was quite simple:

However, the multicolor noise, the warping of the letters, and the contrast make it more difficult for an OCR to solve.

After Image Magick it turned into this:

This is quite simple for a computer to read.

Unfortunately, OCR is predominantly out of date. So, if you are looking for a way to bust CAPTCHAs in 2022, OCR is not the best approach. Most corporate sites are using a more complex system built specifically to mitigate OCR, highlighted by Amazon below:

Computer Vision and Captcha

It is 2022, and we must use Machine Learning in everything. Computer Vision is one of the best tools for busting CAPTCHAs. You can use OpenCV in a variety of ways to bust captchas. On its own, or alongside Tesseract, which can break most CAPTCHAs available online and save for the Google ReCAPTCHA.

For OpenCV, there are two ways of using it to bypass CAPTCHAs.

1) The first approach is training the model by downloading CAPTCHA images and then cutting out each letter and labeling it. We have linked to a tutorial for this approach here.

2) The second method uses OpenCV as an image preprocessor to remove noise and then sends the transformed image to Tesseract for the final identification. We have linked to a tutorial for this approach here.

Ok, but I really want to use both Machine Learning and Neural Networks

That’s fine too.

First, you will need to generate an initial training set of about 50,000 or more CAPTCHAs. The best way to do this is to sign up for a service or use one of the numerous open-source libraries that will already do this.

Next, you will need to set up a Neural Network that consists of both feature extraction to get the letters out of the image and then a classification network to categorize the letters you have extracted.

Once you have trained it, there should be no clean-up involved — it is straightforward to get up and run. We have linked two examples with working code-cracking CAPTCHA with Deep Learning and another example with Mathematica.

Trust based Captchas

The CAPTCHA that you see almost every day is the Google ReCAPTCHA, which requires you to identify bicycles, trains, crosswalks, etc.

When Google launched this CAPTCHA, there were a few interesting enhancements with the “identify every image with a crosswalk” puzzle. Google wanted v3 of ReCAPTCHA to be “frictionless” and not show the challenge to every user.

This introduced the concept of a “trust” score tied to your Google account. Essentially, the more human Google thinks you are, the less likely you will have to do the crosswalk challenge, and the more likely you will be allowed to click the checkbox that says, “I am not a robot.” You can check your ReCAPTCHA score via this website.

If your Google account has a high trust score (for example, mine is .9), you will less likely need to solve a Google ReCAPTCHA when interacting with a website. For many bots, they target and farm Google accounts, not with individual site-based accounts (e.g., Best Buy accounts).

When we look at bots that are hitting commercial sites, most of them will have a Gmail site for this reason, and this reason alone. A higher trust score in Google means the likelihood of success is higher. Account generation scripts will have the ability to farm these accounts through doing normal Google-based interactions (such as looking at product pages).

Current Generation of CAPTCHAs

The current era of CAPTCHAs is a puzzle-like CAPTCHA. You might have seen these on Taobao or Coinbase, where you need to move the mouse to put the puzzle piece in place. Also, please stop rubbing salt in the wounds, telling me how many users flipped the dinosaur faster than me…

The idea behind this CAPTCHA is that an actual human will have variations in the sliding velocity that a robot will not. Unfortunately for brands, the randomization in sliding speed is easily overcome with puppeteer — a js library from Google that mimics a user’s interactions for QA testing.

A detailed walkthrough of sliding every type of slider or puzzle captcha can be found here!

I don’t want to code I just want to buy a service that does this for me.

Fortunately for some, commercial bots embed CAPTCHA solvers into their solutions. Bots are available through marketplaces, like Cop Supply, and many of these bots can put in an API code for CAPTCHA solving services.

2Captcha & Death by CAPTCHA are two highly used tools.

These pay-per-solve services can be expensive, so many heavy users create their CAPTCHA solvers. They use a combination of the CAPTCHA solvers and human beings paid like Mechanical Turk to solve CAPTCHAs.

How can I detect and stop bots?

When we try to look at bots or detect bots using a single data point, we will almost always fail. We are looking for human interaction data via a small part of the overall picture. If you could only ask one question to determine if someone was a human or a robot, would you be able to know which is which?

When we look at bot mitigation, we must start looking at the holistic picture of how a user comes to the site, interacts, and what information they are giving to figure out the truth. CAPTCHAs are just a hurdle for both users and bots, which disproportionately affects the end-user because, for a bot, it is simply a function to execute, but for humans, it’s 30 seconds of looking for boats when we want to buy sneakers.
Bots have moved beyond CAPTCHAs, and it’s time for us to move on, too.

BotNot is a modern solution that does not impact the user experience with annoying CAPTCHAs and holistically assesses every transaction to determine if the individual is a good or bad actor. We look at hundreds of features to assess the humanness and fairness to help ensure brands are prioritizing their most loyal consumers.

Thanks for reading!

Love,

The Yofi Team

‍