Packages and Binaries:

hakrawler

Fast golang web crawler for gathering URLs and JavaSript file locations. This is basically a simple implementation of the awesome Gocolly library.

Installed size: 8.48 MB
How to install: sudo apt install hakrawler

hakrawler

[email protected]:~# hakrawler -h
Usage of hakrawler:
  -d int
    	Depth to crawl. (default 2)
  -insecure
    	Disable TLS verification.
  -t int
    	Number of threads to utilise. (default 8)