feat: add ISP, organization, usage type and connection type tracking#490
feat: add ISP, organization, usage type and connection type tracking#490
Conversation
📝 WalkthroughWalkthroughThis PR extends the analytics pipeline to capture four additional network-enrichment fields—ISP, Organization, Usage Type, and Connection Type—from the DB-IP geolocation service. Changes span backend data transformation (controller, service, transformers), database schema migrations, and frontend display logic (panels, filters, localization). Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~30 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 1 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (1 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
📝 Coding Plan
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment Tip CodeRabbit can use OpenGrep to find security vulnerabilities and bugs across 17+ programming languages.OpenGrep is compatible with Semgrep configurations. Add an |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@backend/apps/cloud/src/common/utils.ts`:
- Around line 269-307: getGeoDetails currently extracts
ISP/organization/userType/connectionType traits (isp, organization, userType,
connectionType) but the deployed DB-IP City Lite (dbip-free-sync.js) does not
contain those traits so they'll always be null; fix by either switching the sync
script to dbip-commercial-sync.js and wiring the DB-IP API key so the commercial
DB with traits is used, or remove the trait extraction and related return fields
from getGeoDetails (and any upstream consumers) so the function only returns
country, city, region, and regionCode when using the free DB; update
callers/types accordingly (refer to getGeoDetails and the trait variables
isp/organization/userType/connectionType).
In `@web/public/locales/en.json`:
- Around line 1579-1583: Add the missing locale key project.network to the
English locale (web/public/locales/en.json) so ViewProject.helpers.tsx's
t('project.network') resolves; mirror the existing dashboard.network value
(e.g., "Network") to keep labels consistent and prevent fallback to the key
text.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 0a4491ee-4071-47e4-b63e-3b46abf7c20a
📒 Files selected for processing (16)
backend/apps/cloud/src/analytics/analytics.controller.tsbackend/apps/cloud/src/analytics/analytics.service.tsbackend/apps/cloud/src/analytics/utils/transformers.tsbackend/apps/cloud/src/common/constants.tsbackend/apps/cloud/src/common/utils.tsbackend/meta/dbip-commercial-sync.jsbackend/migrations/clickhouse/2026_03_19_isp_network.jsbackend/migrations/clickhouse/initialise_database.jsweb/app/lib/constants/index.tsweb/app/pages/Project/View/ViewProject.helpers.tsxweb/app/pages/Project/View/utils/filters.tsxweb/app/pages/Project/tabs/Errors/ErrorsView.tsxweb/app/pages/Project/tabs/Performance/PerformanceView.tsxweb/app/pages/Project/tabs/Traffic/TrafficView.tsxweb/app/ui/FilterValueInput.tsxweb/public/locales/en.json
| export const getGeoDetails = (ip: string, tz?: string): IPGeoDetails => { | ||
| // Stage 1: Using IP address based geo lookup | ||
| const data = lookup.get(ip) | ||
|
|
||
| const country = data?.country?.iso_code || null | ||
| // TODO: Add city overrides, for example, Colinton -> Edinburgh, etc. | ||
| const city = data?.city?.names?.en || null | ||
| // TODO: Store ISO code, not region name | ||
| const region = data?.subdivisions?.[0]?.names?.en || null | ||
| const regionCode = data?.subdivisions?.[0]?.iso_code || null | ||
|
|
||
| const isp = data?.traits?.isp || null | ||
| const organization = data?.traits?.organization || null | ||
| const userType = data?.traits?.user_type || null | ||
| const connectionType = data?.traits?.connection_type || null | ||
|
|
||
| if (country) { | ||
| return { | ||
| country, | ||
| city, | ||
| region, | ||
| regionCode, | ||
| isp, | ||
| organization, | ||
| userType, | ||
| connectionType, | ||
| } | ||
| } | ||
|
|
||
| // Stage 2: Using timezone based geo lookup as a fallback | ||
| const tzCountry = timezones.getCountryForTimezone(tz)?.id || null | ||
|
|
||
| return { | ||
| country: tzCountry, | ||
| city: null, | ||
| region: null, | ||
| regionCode: null, | ||
| isp: null, | ||
| organization: null, | ||
| userType: null, | ||
| connectionType: null, | ||
| } | ||
| } |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Check if there's documentation or configuration indicating which GeoIP database is used
rg -n -i "ip-geolocation|mmdb|maxmind|geoip" --type=md --type=txt -g '*.env*' -g 'README*' -g 'docker*' 2>/dev/null || echo "No explicit database documentation found"
# Look for any database download/update scripts
fd -e sh -e js -e ts --exec grep -l -i "mmdb\|geoip\|maxmind" {} 2>/dev/null | head -5Repository: Swetrix/swetrix
Length of output: 700
🏁 Script executed:
# Examine the utils.ts file to understand the lookup mechanism and imports
cat -n backend/apps/cloud/src/common/utils.ts | head -50Repository: Swetrix/swetrix
Length of output: 1922
🏁 Script executed:
# Check the database sync scripts to confirm which database is being used
head -30 backend/meta/dbip-commercial-sync.js
head -30 backend/meta/dbip-free-sync.jsRepository: Swetrix/swetrix
Length of output: 1930
🏁 Script executed:
# Search for the lookup library initialization and imports
rg -n "lookup\|import.*geoip\|require.*geoip" backend/apps/cloud/src/common/utils.ts -B 3 -A 3Repository: Swetrix/swetrix
Length of output: 41
🏁 Script executed:
# Check which sync script is executed for the cloud app
rg -n "dbip-commercial-sync|dbip-free-sync" --type=js --type=ts backend/Repository: Swetrix/swetrix
Length of output: 189
🏁 Script executed:
# Look for environment variables or conditions that determine which database is used
rg -n "GEOIP|DBIP|DATABASE" --type=ts -g "*.env*" backend/ 2>/dev/null || rg -n "GEOIP|DBIP|DATABASE" backend/ -g "*.env*"Repository: Swetrix/swetrix
Length of output: 161
🏁 Script executed:
# Check the lookup initialization to understand the database setup
rg -n "lookup.*=" backend/apps/cloud/src/common/utils.ts -B 5 -A 5 | head -40Repository: Swetrix/swetrix
Length of output: 659
🏁 Script executed:
# Verify the maxmind package documentation or type definitions available
rg -n "CityResponse\|traits" backend/apps/cloud/src/common/utils.ts -B 2 -A 2Repository: Swetrix/swetrix
Length of output: 41
🏁 Script executed:
# Check if there's a production configuration that specifies which sync script is used
rg -n "dbip-commercial\|dbip-free" --type=json --type=ts --type=js backend/ -g "package.json" -g "docker*" -g "*.yml" 2>/dev/null | head -20Repository: Swetrix/swetrix
Length of output: 41
🏁 Script executed:
# Check the README to understand which database cloud app uses
cat README.md | grep -i "geo\|geoip\|db-ip" -A 3 -B 3Repository: Swetrix/swetrix
Length of output: 3699
🏁 Script executed:
# Check docker or deployment configs for cloud app to see which database is used
fd docker -o docker-compose | xargs cat 2>/dev/null | grep -i "geoip\|dbip\|database" -B 2 -A 2 | head -30Repository: Swetrix/swetrix
Length of output: 206
🏁 Script executed:
# Look at what the DB-IP databases actually include
# Check if there's any documentation in the codebase about database structure
rg -n "isp\|organization\|user_type\|connection_type" backend/apps/cloud/src/common/ -B 2 -A 2 | head -50Repository: Swetrix/swetrix
Length of output: 41
🏁 Script executed:
# Check if there's deployment configuration or package.json that shows which database is used
cat backend/package.json | grep -i "maxmind\|geoip"Repository: Swetrix/swetrix
Length of output: 83
🏁 Script executed:
# Look at the dbip-commercial-sync.js to understand what database it downloads
cat backend/meta/dbip-commercial-sync.js | grep -i "download\|url\|api" | head -20Repository: Swetrix/swetrix
Length of output: 1143
🏁 Script executed:
# Check if there's any validation or warning about null traits
rg -n "isp\|organization\|user_type\|connection_type" backend/apps/cloud/src/common/utils.ts -B 5 -A 5Repository: Swetrix/swetrix
Length of output: 41
🏁 Script executed:
# Look for any comments or documentation about database requirements
rg -n "TODO\|FIXME\|NOTE\|WARN" backend/apps/cloud/src/common/utils.ts | head -20Repository: Swetrix/swetrix
Length of output: 41
🏁 Script executed:
# Check what the actual return type includes and if traits are documented
ast-grep --pattern 'interface IPGeoDetails {
$$$
}'Repository: Swetrix/swetrix
Length of output: 1137
🏁 Script executed:
# Check the full dbip-commercial-sync.js to see what exact database it downloads
cat backend/meta/dbip-commercial-sync.js | tail -40Repository: Swetrix/swetrix
Length of output: 1152
🏁 Script executed:
# Verify what API URL is being used for the commercial database
rg -n "db-ip.com" backend/meta/dbip-commercial-sync.js -B 2 -A 2Repository: Swetrix/swetrix
Length of output: 302
🏁 Script executed:
# Check if there's any environment configuration or deployment scripts that specify which sync to run
find backend -name "*.yml" -o -name "docker*" -o -name "Dockerfile*" 2>/dev/null | xargs grep -l "dbip\|geoip" 2>/dev/nullRepository: Swetrix/swetrix
Length of output: 77
🏁 Script executed:
# Check the community app's database setup for comparison
cat backend/apps/community/src/common/utils.ts | grep -n "isp\|organization" -B 10 -A 10 | head -50Repository: Swetrix/swetrix
Length of output: 41
🏁 Script executed:
# Check the Dockerfile to see which database sync script is used for cloud
cat backend/Dockerfile | grep -i "dbip\|geoip\|sync" -B 3 -A 3Repository: Swetrix/swetrix
Length of output: 201
🏁 Script executed:
# Check if there's deployment documentation that clarifies which database is used
rg -n "db-ip\|dbip\|geoip\|premium" --type=md backend/docs/ 2>/dev/null || echo "No docs directory"Repository: Swetrix/swetrix
Length of output: 76
🏁 Script executed:
# Confirm the actual behavior - check if traits are accessed elsewhere in cloud app
rg -n "\.isp\|\.organization\|\.userType\|\.connectionType" backend/apps/cloud/src/ -lRepository: Swetrix/swetrix
Length of output: 41
🏁 Script executed:
# Verify that community app doesn't use these traits since it uses free DB-IP
rg -n "\.isp\|\.organization\|\.userType\|\.connectionType" backend/apps/community/src/Repository: Swetrix/swetrix
Length of output: 41
🏁 Script executed:
# Check if there's a separate deployment config for cloud app
rg -n "deploy:cloud\|cloud.*deploy\|GEOIP" backend/package.json -A 5 -B 5Repository: Swetrix/swetrix
Length of output: 41
🏁 Script executed:
# Look for any Docker or deployment configs specific to cloud app
fd dockerfile -o docker-compose -o yml | xargs cat 2>/dev/null | grep -i "cloud\|commercial" -B 5 -A 5Repository: Swetrix/swetrix
Length of output: 206
🏁 Script executed:
# Check package.json scripts to see all deployment targets
cat backend/package.json | grep -i "deploy\|script" -A 50 | head -60Repository: Swetrix/swetrix
Length of output: 3046
🏁 Script executed:
# Check what database file path is actually used and confirm the setup
rg -n "PRODUCTION_GEOIP_DB_PATH\|DEVELOPMENT_GEOIP_DB_PATH\|ip-geolocation-db" backend/apps/cloud/src/common/utils.ts -B 3 -A 3Repository: Swetrix/swetrix
Length of output: 41
🏁 Script executed:
# Verify the difference between what community and cloud extract from the database
diff <(rg -n "export const getGeoDetails" backend/apps/community/src/common/utils.ts -A 30) <(rg -n "export const getGeoDetails" backend/apps/cloud/src/common/utils.ts -A 30)Repository: Swetrix/swetrix
Length of output: 146
🏁 Script executed:
# Check if there's any production deployment configuration
find . -name "*.env*" -o -name "docker-compose*" 2>/dev/null | xargs grep -l "dbip\|geoip\|cloud" 2>/dev/null | head -5Repository: Swetrix/swetrix
Length of output: 41
🏁 Script executed:
# Search for where PRODUCTION_GEOIP_DB_PATH is defined
rg -n "PRODUCTION_GEOIP_DB_PATH\|DEVELOPMENT_GEOIP_DB_PATH" backend/apps/cloud/src/common/utils.ts -B 5Repository: Swetrix/swetrix
Length of output: 41
🏁 Script executed:
# Compare the getGeoDetails functions between community and cloud
cat backend/apps/community/src/common/utils.ts | sed -n '1100,1200p'Repository: Swetrix/swetrix
Length of output: 2762
🏁 Script executed:
# Check the full getGeoDetails in cloud to see the complete picture
cat backend/apps/cloud/src/common/utils.ts | sed -n '269,310p'Repository: Swetrix/swetrix
Length of output: 1072
🏁 Script executed:
# Search for any initialization or documentation about database types
rg -n "CityResponse\|Reader" backend/apps/cloud/src/common/utils.ts -B 3 -A 3Repository: Swetrix/swetrix
Length of output: 41
🏁 Script executed:
# Get the exact line numbers and context for the cloud app's getGeoDetails function
cat -n backend/apps/cloud/src/common/utils.ts | sed -n '258,310p'Repository: Swetrix/swetrix
Length of output: 1679
🏁 Script executed:
# Verify the maxmind Reader type to confirm what traits it supports
rg -n "traits" backend/apps/cloud/src/common/utils.ts -B 2 -A 2Repository: Swetrix/swetrix
Length of output: 384
🏁 Script executed:
# Double-check what database the Dockerfile uses for cloud specifically
cat backend/Dockerfile | grep -n "deploy\|dbip\|node" | head -20Repository: Swetrix/swetrix
Length of output: 246
Database mismatch: traits will always be null with current DB-IP free database configuration.
The code attempts to extract isp, organization, user_type, and connection_type traits from the GeoIP database. However, the Dockerfile currently runs only dbip-free-sync.js, which downloads the free DB-IP City Lite database that does not include ISP/organization traits. These fields will always be null in production.
Either:
- Use
dbip-commercial-sync.jsinstead (requires DB-IP API key) to get a database with trait support, or - Remove the trait extraction logic (lines 277–280, 288–291, 303–305) if ISP data is not needed.
Note: The code uses the DB-IP library and database, not MaxMind.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@backend/apps/cloud/src/common/utils.ts` around lines 269 - 307, getGeoDetails
currently extracts ISP/organization/userType/connectionType traits (isp,
organization, userType, connectionType) but the deployed DB-IP City Lite
(dbip-free-sync.js) does not contain those traits so they'll always be null; fix
by either switching the sync script to dbip-commercial-sync.js and wiring the
DB-IP API key so the commercial DB with traits is used, or remove the trait
extraction and related return fields from getGeoDetails (and any upstream
consumers) so the function only returns country, city, region, and regionCode
when using the free DB; update callers/types accordingly (refer to getGeoDetails
and the trait variables isp/organization/userType/connectionType).
| "keyword": "Keyword", | ||
| "isp": "ISP", | ||
| "og": "Organisation", | ||
| "ut": "Usage type", | ||
| "ctp": "Connection type" |
There was a problem hiding this comment.
Add missing project.network locale key to match new panel mapping.
web/app/pages/Project/View/ViewProject.helpers.tsx Line 2181 now resolves t('project.network'), but this locale file only has dashboard.network. Without project.network, the network panel label will fall back to key text.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@web/public/locales/en.json` around lines 1579 - 1583, Add the missing locale
key project.network to the English locale (web/public/locales/en.json) so
ViewProject.helpers.tsx's t('project.network') resolves; mirror the existing
dashboard.network value (e.g., "Network") to keep labels consistent and prevent
fallback to the key text.
1 participant
Changes
If applicable, please describe what changes were made in this pull request.
Community Edition support
Database migrations
Documentation
Summary by CodeRabbit
Release Notes