Crawler is a C++ & Node.js application that allows you to crawl web pages, save them locally, and extract hyperlinks from the page body. It provides a simple command-line interface where you can enter the starting URL and specify the maximum number of crawls. The crawler follows the hyperlinks recursively, saves the web pages in specified directo
nodejs command-line web-crawler crawling http-requests web-scraping recursive-crawling hyperlink-extraction file-system-operations cheerio-html-parser offline-web-access
-
Updated
Jul 26, 2023 - C++