Open
Conversation
- Introduce `MarkDownOutput` flag and CLI option (`-markdown` / `-md`). - Implement `Result.MarkdownOutput` to generate a formatted Markdown table with optional title and CDN columns, and a response body preview. - Add markdown escaping helper. - Update runner to create `.md` output file, handle markdown generation per result, and write to file or stdout. - Adjust option parsing and related logic to include markdown handling.
- Split Markdown generation into `MarkdownHeader` and `MarkdownRow` for clearer separation. - Remove body‑preview block from Markdown rows. - Add guard in runner to write the header only once per run.
…d` flag * Consolidate output‑type checks into a single `jsonOrCsvOrMD` variable. * Adjust file‑opening logic to add the “.md” suffix only when needed. * Update related conditionals to respect the new combined flag handling. * Ensure markdown output is correctly written without creating “.md.md” files.
Add support for storing httpx scan results directly to databases with both CLI flags and YAML config file options. Supported databases: - MongoDB - PostgreSQL - MySQL Features: - Batched writes for performance (configurable batch size) - Auto-flush with configurable interval - Individual columns for each Result field (not JSON blob) - Support for environment variable HTTPX_DB_CONNECTION_STRING - Option to omit raw request/response data (-rdbor) New CLI flags under OUTPUT group: - -rdb, -result-db: enable database storage - -rdbc, -result-db-config: path to YAML config file - -rdbt, -result-db-type: database type (mongodb, postgres, mysql) - -rdbcs, -result-db-conn: connection string - -rdbn, -result-db-name: database name (default: httpx) - -rdbtb, -result-db-table: table/collection name (default: results) - -rdbbs, -result-db-batch-size: batch size (default: 100) - -rdbor, -result-db-omit-raw: omit raw data Closes #1973 Closes #2360 Closes #2361 Closes #2362
- Fix SQL injection in postgres.go using pq.QuoteIdentifier for table/index names - Fix SQL injection in mysql.go using custom quoteIdentifier function - Fix race condition in writer.go by checking closed state before channel send - Add missing idx_url index in mysql.go for parity with PostgreSQL schema - Include RawHeaders in omitRaw check for consistency
Bumps the modules group with 5 updates: | Package | From | To | | --- | --- | --- | | [github.com/projectdiscovery/cdncheck](https://github.com/projectdiscovery/cdncheck) | `1.2.19` | `1.2.20` | | [github.com/projectdiscovery/networkpolicy](https://github.com/projectdiscovery/networkpolicy) | `0.1.33` | `0.1.34` | | [github.com/projectdiscovery/retryablehttp-go](https://github.com/projectdiscovery/retryablehttp-go) | `1.3.4` | `1.3.5` | | [github.com/projectdiscovery/useragent](https://github.com/projectdiscovery/useragent) | `0.0.106` | `0.0.107` | | [github.com/projectdiscovery/wappalyzergo](https://github.com/projectdiscovery/wappalyzergo) | `0.2.64` | `0.2.65` | Updates `github.com/projectdiscovery/cdncheck` from 1.2.19 to 1.2.20 - [Release notes](https://github.com/projectdiscovery/cdncheck/releases) - [Commits](projectdiscovery/cdncheck@v1.2.19...v1.2.20) Updates `github.com/projectdiscovery/networkpolicy` from 0.1.33 to 0.1.34 - [Release notes](https://github.com/projectdiscovery/networkpolicy/releases) - [Commits](projectdiscovery/networkpolicy@v0.1.33...v0.1.34) Updates `github.com/projectdiscovery/retryablehttp-go` from 1.3.4 to 1.3.5 - [Release notes](https://github.com/projectdiscovery/retryablehttp-go/releases) - [Commits](projectdiscovery/retryablehttp-go@v1.3.4...v1.3.5) Updates `github.com/projectdiscovery/useragent` from 0.0.106 to 0.0.107 - [Release notes](https://github.com/projectdiscovery/useragent/releases) - [Commits](projectdiscovery/useragent@v0.0.106...v0.0.107) Updates `github.com/projectdiscovery/wappalyzergo` from 0.2.64 to 0.2.65 - [Release notes](https://github.com/projectdiscovery/wappalyzergo/releases) - [Commits](projectdiscovery/wappalyzergo@v0.2.64...v0.2.65) --- updated-dependencies: - dependency-name: github.com/projectdiscovery/cdncheck dependency-version: 1.2.20 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules - dependency-name: github.com/projectdiscovery/networkpolicy dependency-version: 0.1.34 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules - dependency-name: github.com/projectdiscovery/retryablehttp-go dependency-version: 1.3.5 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules - dependency-name: github.com/projectdiscovery/useragent dependency-version: 0.0.107 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules - dependency-name: github.com/projectdiscovery/wappalyzergo dependency-version: 0.2.65 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps the modules group with 7 updates: | Package | From | To | | --- | --- | --- | | [github.com/projectdiscovery/cdncheck](https://github.com/projectdiscovery/cdncheck) | `1.2.20` | `1.2.21` | | [github.com/projectdiscovery/dsl](https://github.com/projectdiscovery/dsl) | `0.8.12` | `0.8.13` | | [github.com/projectdiscovery/fastdialer](https://github.com/projectdiscovery/fastdialer) | `0.5.3` | `0.5.4` | | [github.com/projectdiscovery/gologger](https://github.com/projectdiscovery/gologger) | `1.1.67` | `1.1.68` | | [github.com/projectdiscovery/hmap](https://github.com/projectdiscovery/hmap) | `0.0.99` | `0.0.100` | | [github.com/projectdiscovery/retryablehttp-go](https://github.com/projectdiscovery/retryablehttp-go) | `1.3.5` | `1.3.6` | | [github.com/projectdiscovery/wappalyzergo](https://github.com/projectdiscovery/wappalyzergo) | `0.2.65` | `0.2.66` | Updates `github.com/projectdiscovery/cdncheck` from 1.2.20 to 1.2.21 - [Release notes](https://github.com/projectdiscovery/cdncheck/releases) - [Commits](projectdiscovery/cdncheck@v1.2.20...v1.2.21) Updates `github.com/projectdiscovery/dsl` from 0.8.12 to 0.8.13 - [Release notes](https://github.com/projectdiscovery/dsl/releases) - [Commits](projectdiscovery/dsl@v0.8.12...v0.8.13) Updates `github.com/projectdiscovery/fastdialer` from 0.5.3 to 0.5.4 - [Release notes](https://github.com/projectdiscovery/fastdialer/releases) - [Commits](projectdiscovery/fastdialer@v0.5.3...v0.5.4) Updates `github.com/projectdiscovery/gologger` from 1.1.67 to 1.1.68 - [Release notes](https://github.com/projectdiscovery/gologger/releases) - [Commits](projectdiscovery/gologger@v1.1.67...v1.1.68) Updates `github.com/projectdiscovery/hmap` from 0.0.99 to 0.0.100 - [Release notes](https://github.com/projectdiscovery/hmap/releases) - [Commits](projectdiscovery/hmap@v0.0.99...v0.0.100) Updates `github.com/projectdiscovery/retryablehttp-go` from 1.3.5 to 1.3.6 - [Release notes](https://github.com/projectdiscovery/retryablehttp-go/releases) - [Commits](projectdiscovery/retryablehttp-go@v1.3.5...v1.3.6) Updates `github.com/projectdiscovery/wappalyzergo` from 0.2.65 to 0.2.66 - [Release notes](https://github.com/projectdiscovery/wappalyzergo/releases) - [Commits](projectdiscovery/wappalyzergo@v0.2.65...v0.2.66) --- updated-dependencies: - dependency-name: github.com/projectdiscovery/cdncheck dependency-version: 1.2.21 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules - dependency-name: github.com/projectdiscovery/dsl dependency-version: 0.8.13 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules - dependency-name: github.com/projectdiscovery/fastdialer dependency-version: 0.5.4 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules - dependency-name: github.com/projectdiscovery/gologger dependency-version: 1.1.68 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules - dependency-name: github.com/projectdiscovery/hmap dependency-version: 0.0.100 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules - dependency-name: github.com/projectdiscovery/retryablehttp-go dependency-version: 1.3.6 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules - dependency-name: github.com/projectdiscovery/wappalyzergo dependency-version: 0.2.66 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules ... Signed-off-by: dependabot[bot] <support@github.com>
…dev/modules-60805faa73 chore(deps): bump the modules group with 7 updates
fix: prevent data loss on interrupt by implementing graceful shutdown
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.49.0 to 0.50.0. - [Commits](golang/net@v0.49.0...v0.50.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-version: 0.50.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [github.com/refraction-networking/utls](https://github.com/refraction-networking/utls) from 1.7.1 to 1.8.2. - [Release notes](https://github.com/refraction-networking/utls/releases) - [Commits](refraction-networking/utls@v1.7.1...v1.8.2) --- updated-dependencies: - dependency-name: github.com/refraction-networking/utls dependency-version: 1.8.2 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com>
…dev/golang.org/x/net-0.50.0 chore(deps): bump golang.org/x/net from 0.49.0 to 0.50.0
…github.com/refraction-networking/utls-1.8.2 chore(deps): bump github.com/refraction-networking/utls from 1.7.1 to 1.8.2
Bumps the modules group with 2 updates: [github.com/projectdiscovery/cdncheck](https://github.com/projectdiscovery/cdncheck) and [github.com/projectdiscovery/wappalyzergo](https://github.com/projectdiscovery/wappalyzergo). Updates `github.com/projectdiscovery/cdncheck` from 1.2.21 to 1.2.22 - [Release notes](https://github.com/projectdiscovery/cdncheck/releases) - [Commits](projectdiscovery/cdncheck@v1.2.21...v1.2.22) Updates `github.com/projectdiscovery/wappalyzergo` from 0.2.66 to 0.2.67 - [Release notes](https://github.com/projectdiscovery/wappalyzergo/releases) - [Commits](projectdiscovery/wappalyzergo@v0.2.66...v0.2.67) --- updated-dependencies: - dependency-name: github.com/projectdiscovery/cdncheck dependency-version: 1.2.22 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules - dependency-name: github.com/projectdiscovery/wappalyzergo dependency-version: 0.2.67 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: modules ... Signed-off-by: dependabot[bot] <support@github.com>
feat: add database output support for storing scan results
- Add Deprecated comment to OutputFilterErrorPage field - Coerce PageType to string for safe type assertion
Replace page classifier with dit, add -fpt flag
Bumps [filippo.io/edwards25519](https://github.com/FiloSottile/edwards25519) from 1.1.0 to 1.1.1. - [Commits](FiloSottile/edwards25519@v1.1.0...v1.1.1) --- updated-dependencies: - dependency-name: filippo.io/edwards25519 dependency-version: 1.1.1 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.50.0 to 0.51.0. - [Commits](golang/net@v0.50.0...v0.51.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-version: 0.51.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>
…filippo.io/edwards25519-1.1.1 chore(deps): bump filippo.io/edwards25519 from 1.1.0 to 1.1.1
…dev/golang.org/x/net-0.51.0 chore(deps): bump golang.org/x/net from 0.50.0 to 0.51.0
…dev/modules-6b7541c889 chore(deps): bump the modules group with 2 updates
fix resolver parsing
Neo - PR Security ReviewNo security issues found Highlights
Hardening Notes
Comment |
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Add Markdown output support
knakul853
approved these changes
Mar 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
closes #2434