linkmapy/README.md

linkmapy

Create a graph from the links on a website and the following sites.

Dependencies

• Python 3.9

Pip Dependencies

• networkx
• pandas
• pyvis

You can install those dependencies with

pip install -r pip-dependencies.txt

Usage

./linkmapy.py [-h] [--dump] [--max-depth N] [--max-links-per-site N] [--http-download-limit NBYTES] [--log] url

• positional arguments:

  • url   The URL of the site you want to start from

• optional arguments:

  • -h, --help   show this help message and exit
  • --dump   Only output the found connections to the console and exit
  • --max-depth N   The maximum depth at which links will be followed (default: 3)
  • --max-links-per-site N   The maximum amount of links on a page that will be included (default: 3) *
  • --http-download-limit NBYTES   The maximum length of a requested html file download (in bytes) (default: 10000000)
  • --log   Log all visited sites and links to stderr

* when there are more than max-links-per-site links on the page, the program will pick max-links-per-site random links and ignore the rest.

Example

./linkmapy.py example.org

Result: