Domain Miner 🌐 ⚒️

A comprehensive diagnostic tool that gathers information about domains, such as DNS info, HTTP status, and more. Depending on configuration, this tool can process thousands of domains in less than a second. ⚡

[16:29:36.690] DEBUG (main/67054): Writing results.json
[16:29:36.692] DEBUG (main/67054): Writing results.csv
[16:29:36.705] DEBUG (main/67054): ⚡ [Benchmark] 6,425 domains successfully processed in 1s!

Table of Contents

• Getting Started
• How It Works
• Environment Variables

Getting Started

1. Copy .config/template/.env to .config/.env
2. Refer to Environment Variables
3. npm install
4. npm run build
5. npm start

How It Works

The domain miner will process each domain from process.env.DOMAINS in-parallel. Once the test is complete, results will be delivered to .io/output/results.json or .io/output/results.csv depending on your configuration.

The results JSON will have the following structure:

type DomainTestResults = {
  domain: string;
  http: {
    protocol: 'http' | 'https';
    reason: string | null;
    status: number | null;
    redirectUrl: string | null;
  }[];
  dns: {
    reason: string | null;
    nameservers: string[];
    cname: string[];
    ips: { ip: string; reverse: string | string[] | null }[];
  };
}[];

Below is an example test result of a single domain: example.com:

[
  {
    "domain": "example.com",
    "http": [
      {
        "protocol": "http",
        "reason": null,
        "status": 301,
        "redirectUrl": "https://www.example.com/"
      },
      {
        "protocol": "https",
        "reason": "Host: example.com. is not in the cert's altnames: DNS:*.test.com, DNS:test.com",
        "status": null,
        "redirectUrl": null
      }
    ],
    "dns": {
      "reason": null,
      "nameservers": ["ns80.worldnic.com", "ns79.worldnic.com"],
      "cname": [],
      "ips": [
        {
          "ip": "109.45.114.112",
          "reverse": ["ec2-109-45-114-112.compute-1.amazonaws.com"]
        }
      ]
    }
  }
]

Environment Variables

LOG_LEVEL - Enum (optional) - The log level of the service. Defaults to info, but may be changed. One of:

• fatal
• error
• warn
• info
• debug
• trace

LOG_LEVEL=debug

---

DOMAINS - String (required) - A comma-separated list of FQDNs to process.

DOMAINS="www.example.com,google.com,www.example.co.uk"

---

WRITE_JSON - Boolean (optional) - If true, will write test results to .io/output/results.json in JSON format.

WRITE_JSON=true

---

WRITE_CSV - Boolean (optional) - If true, will write test results to .io/output/results.csv in CSV format.

WRITE_CSV=true