Research Outline

Business Benefits of Crawlers/Scrapers and RPA


To collect data on how the use of web crawlers/scrapers and/or RPA can benefit businesses in terms of increased revenue, productivity improvements.

Early Findings

Definition of Terms

  • Web crawler - according to Tech Target, a web crawler is "a program that visits websites and reads their pages and other information in order to create entries for a search engine index." Web crawlers are typically used to collect information from a website and the links that are related to the website.
  • Web scraper - web scrapers are applications that process the HTML of a web page so as to extract data for manipulation.
  • Robotic Process Automation (RPA) - according to Tech Target, RPA "is the use of software with artificial intelligence (AI) and machine learning capabilities to handle high-volume, repeatable tasks that previously required humans to perform." These tasks include queries, calculations and maintenance of records and transactions.

Business Benefits