Akamai Technologies has released a new State of the Internet (SOTI) report that details the security and business threats that organizations face with the proliferation of web scraping bots. Scraping Away Your Bottom Line: How Web Scrapers Impact Ecommerce finds that bots compose 42% of overall web traffic, and 65% of these bots are malicious.
With its reliance on revenue-generating web applications, the ecommerce sector has been most affected by high-risk bot traffic. Although some bots are beneficial to business, web scraper bots are being used for competitive intelligence and espionage, inventory hoarding, imposter site creation, and other schemes that have a negative impact on both the bottom line and the customer experience. There are no existing laws that prohibit the use of scraper bots, and they are hard to detect due to the rise of artificial intelligence (AI) botnets, but there are some things companies can do to mitigate them.
“Every business with an online storefront relies on web scraper bots to some extent. The challenge arises when these bots are misused, as their similar functions make it difficult to distinguish between beneficial and malicious ones. It is then compounded by the rapidly evolving scraper landscape which renders traditional defenses like firewalls ineffective,” said Reuben Koh, Director of Security Technology & Strategy, APJ, Akamai Technologies. “Now, more than ever, ecommerce businesses, especially in APJ which is a key global commerce hub, must invest in solutions that are fit for purpose, capable of adapting and keeping up with the unpredictable and iterative attacks posed by malicious bots – especially if they are looking to regionalize and expand their customer base, opening them up to further threats.”
Key findings from the report include:
- AI botnets have the ability to discover and scrape unstructured data and content that is in a less consistent format or location. Additionally, they can use actual business intelligence to enhance the decision-making process through collecting, extracting, and then processing data.
- Scraper bots can be leveraged to generate more sophisticated phishing campaigns by grabbing product images, descriptions, and pricing information to create counterfeit storefronts or phishing sites aimed at stealing credentials or credit card information.
- Bots can be used to facilitate new account opening abuse — which, according to recent research, composes up to 50% of fraud losses.
- Technical impacts that organizations face as a result of being scraped, whether the scraping was done with malicious or beneficial intentions, include website performance degradation, site metric pollution, compromised credentials attacks from phishing sites, increased compute costs, and more.
