Web Scraping Case Study | Scraping product information from US Top 15 E-Commerce Website

Client: A fortune 500 Company from USA

Domain: Ecommerce

Offering: Site-specific crawl and extraction

Challenge: Client’s requirement was to scrape details of the best-selling products from top 15 E-commerce platforms of USA in order to identify the potential best-sellers on respective platforms. Best seller products from the entire catalog as well as the best sellers under different categories were needed separately. The required data points were Product name, description, specifications, brand, model, , bestseller rank, reviews count and ratings.

The Solution: Client provided us with the list of specific data points to be extracted from client specific sites. This particular case comes under our Site specific crawl offering as the crawler is programmed specifically to fetch data from a particular source website. With our years of expertise in web crawling and scraping, we started delivering clean data to the client soon after the setup was successfully completed. The data delivery format opted for by the client was XML and the data was directly uploaded to client’s S3 location.

Benefits:

Every technically complicated aspect of the crawl and extraction was taken care of by our team
The setup was complete in a few days and the data was provided in record time
Client had all the bestselling products from all the specific websites, category-wise, ready to be fed into their applications. Our advanced tech stack handled millions of records each week with ease
Monitoring was set up for the source website to detect changes that need modification of our crawlers