[feature] GSoC25: WHOIS Information and IP Address-Based Geolocation

docs/developer/extending.rst

@@ -344,6 +344,7 @@ Once you have created the models, add the following to your
     CONFIG_VPNCLIENT_MODEL = "sample_config.VpnClient"
     CONFIG_ORGANIZATIONCONFIGSETTINGS_MODEL = "sample_config.OrganizationConfigSettings"
     CONFIG_ORGANIZATIONLIMITS_MODEL = "sample_config.OrganizationLimits"
+    CONFIG_WHOISINFO_MODEL = "sample_config.WHOISInfo"
     DJANGO_X509_CA_MODEL = "sample_pki.Ca"
     DJANGO_X509_CERT_MODEL = "sample_pki.Cert"
     GEO_LOCATION_MODEL = "sample_geo.Location"

docs/index.rst

@@ -47,6 +47,8 @@ the OpenWISP architecture.
     user/vxlan-wireguard.rst
     user/zerotier.rst
     user/openvpn.rst
+    user/whois.rst
+    user/estimated-location.rst
     user/subnet-division-rules.rst
     user/rest-api.rst
     user/settings.rst

docs/user/estimated-location.rst

@@ -0,0 +1,123 @@
+Estimated Location
+==================
+
+.. image:: https://raw.githubusercontent.com/openwisp/openwisp-controller/docs/docs/1.3/estimated-locations/is-estimated-flag.png
+    :target: https://raw.githubusercontent.com/openwisp/openwisp-controller/docs/docs/1.3/estimated-locations/is-estimated-flag.png
+    :alt: Estimated location flag
+
+.. important::
+
+    The **Estimated Location** feature is **disabled by default**.
+
+    Before enabling it, the :ref:`WHOIS Lookup feature
+    <controller_setup_whois_lookup>` must be enabled.
+
+    Then set :ref:`OPENWISP_CONTROLLER_ESTIMATED_LOCATION_ENABLED` to
+    ``True``.
+
+.. contents:: **Table of contents**:
+    :depth: 1
+    :local:
+
+Overview
+--------
+
+This feature automatically creates or updates a device's location based on
+latitude and longitude information retrieved from the :doc:`whois`
+feature.
+
+It is very useful for those users who have devices scattered across
+different geographic regions and would like some help to place the devices
+on the map, while being gently reminded to improve the precision of the
+location with a direct link for doing so.
+
+It also significantly reduces the effort required to assign a geographic
+location manually when many devices are deployed in large buildings like
+schools, offices, hospitals, libraries, etc. Improve the precision of the
+estimated location just once and all the other devices sharing the same
+public IP will automatically inherit the same location.
+
+The feature is not useful in the following scenarios:
+
+- Most devices are deployed in one single location.
+- Most devices are mobile (e.g. moving vehicles).
+
+Visibility of Estimated Status
+------------------------------
+
+The estimated status of a location is visible in the admin interface in
+several ways:
+
+- The location name will mention the *IP address* from which it was
+  estimated.
+- A *warning message* appears at the top of the location list page as in
+  the image below.
+
+  .. image:: https://raw.githubusercontent.com/openwisp/openwisp-controller/docs/docs/1.3/estimated-locations/estimated-warning.png
+      :target: https://raw.githubusercontent.com/openwisp/openwisp-controller/docs/docs/1.3/estimated-locations/estimated-warning.png
+      :alt: Estimated location warning
+
+- The *Is Estimated?* flag is displayed both in the location list page and
+  in the location detail page, as in the images below.
+
+  .. image:: https://raw.githubusercontent.com/openwisp/openwisp-controller/docs/docs/1.3/estimated-locations/admin-list.png
+      :target: https://raw.githubusercontent.com/openwisp/openwisp-controller/docs/docs/1.3/estimated-locations/admin-list.png
+      :alt: Estimated location admin list
+
+  .. image:: https://raw.githubusercontent.com/openwisp/openwisp-controller/docs/docs/1.3/estimated-locations/is-estimated-flag.png
+      :target: https://raw.githubusercontent.com/openwisp/openwisp-controller/docs/docs/1.3/estimated-locations/is-estimated-flag.png
+      :alt: Estimated location flag
+
+- The device list page also allows filtering devices which are associated
+  with estimated locations as shown in the image below.
+
+  .. image:: https://raw.githubusercontent.com/openwisp/openwisp-controller/docs/docs/1.3/estimated-locations/filter-devices-by-estimated-location.png
+      :target: https://raw.githubusercontent.com/openwisp/openwisp-controller/docs/docs/1.3/estimated-locations/filter-devices-by-estimated-location.png
+      :alt: Filter devices associated to estimated locations
+
+Any change to the geographic coordinates of an estimated location will set
+the ``is_estimated`` field to ``False``.
+
+When manually increasing the precision of estimated locations, it is
+highly recommended to also change the auto-generated location name.
+
+In the REST API, the ``is_estimated`` field is visible in the :ref:`Device
+Location <device_location_estimated>`, :ref:`Location list
+<location_list_estimated>`, :ref:`Location Detail
+<location_detail_estimated>` and :ref:`Location list (GeoJSON)
+<location_geojson_estimated>` endpoints if the feature is enabled. The
+field can also be used for filtering in the location list endpoints,
+including the GeoJSON endpoint, and in the :ref:`Device List
+<device_list_estimated_filters>`.
+
+Triggers and Record Management
+------------------------------
+
+The feature is triggered automatically when all the following conditions
+are met:
+
+- A WHOIS lookup is performed.
+- The last IP is a public IP address.
+- Both WHOIS lookup and Estimated Location features are enabled for the
+  device's organization.
+
+If no matching location exists, a new estimated location is created using
+coordinates from the WHOIS record. If an estimated location already
+exists, it will be updated with the new coordinates.
+
+If another device with the same IP already has a location, the system will
+assign the same location for any device having the same IP and not being
+assigned to any other location.
+
+If two devices share the same IP and are assigned to the same location,
+and one of them updates its last IP, the system will create a new
+estimated location for that device.
+
+When multiple devices with the same IP already have a location assigned
+but the locations differ, the system will send a notification to network
+administrators asking to manually resolve the ambiguity.
+
+When WHOIS records are updated as described in :ref:`the WHOIS Lookup
+section <controller_whois_auto_management>`, any related estimated
+location will also be updated, if needed and only if the estimated
+location has not been manually modified to increase precision.

docs/user/intro.rst

@@ -35,6 +35,8 @@ following features:
 - **VPN management**: automatically provision VPN tunnel configurations,
   including cryptographic keys and IP addresses, e.g.: :doc:`OpenVPN
   </user/vpn>`, :doc:`WireGuard <wireguard>`
+- :doc:`whois`: display information about the public IP address used by
+  devices to communicate with OpenWISP
 - :doc:`import-export`
 
 It exposes various :doc:`REST API endpoints <rest-api>`.
@@ -96,6 +98,9 @@ The geographic app is based on `django-loci
 geographic coordinates of the devices, as well as their indoor coordinates
 on floor plan images.
 
+It also provides an :doc:`estimated-location` feature which automatically
+creates or updates device locations based on WHOIS data.
+
 It exposes various :doc:`REST API endpoints <rest-api>`.
 
 Subnet Division App

docs/user/rest-api.rst

@@ -68,6 +68,22 @@ List Devices
 
     GET /api/v1/controller/device/
 
+.. _device_list_whois:
+
+**WHOIS Details**
+
+If :doc:`WHOIS Lookup feature <whois>` is enabled, each device in the list
+response will also include a ``whois_info`` field with related brief WHOIS
+information.
+
+.. _device_list_estimated_filters:
+
+**Estimated Location Filters**
+
+If :doc:`Estimated Location feature <estimated-location>` is enabled,
+devices can be filtered based on the estimated nature of their location
+using the ``geo_is_estimated`` filter.
+
 **Available filters**
 
 You can filter a list of devices based on their configuration status using
@@ -145,6 +161,13 @@ Get Device Detail
 
     GET /api/v1/controller/device/{id}/
 
+.. _device_detail_whois:
+
+**WHOIS Details**
+
+If :doc:`WHOIS Lookup feature <whois>` is enabled, the response will also
+include a ``whois_info`` field with related detailed WHOIS information.
+
 Download Device Configuration
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -529,13 +552,26 @@ of certificate's organization as show in the example below:
 
     GET /api/v1/controller/cert/{common_name}/group/?org={org1_slug},{org2_slug}
 
+.. |est_loc| replace:: Estimated Location feature
+
+.. _est_loc: estimated-location.html
+
+.. |estimated_details| replace:: If |est_loc|_ is enabled, the location
+    response will also include ``is_estimated`` status field.
+
 Get Device Location
 ~~~~~~~~~~~~~~~~~~~
 
 .. code-block:: text
 
     GET /api/v1/controller/device/{id}/location/
 
+.. _device_location_estimated:
+
+**Estimated Status**
+
+|estimated_details|
+
 .. _create_device_location:
 
 Create Device Location
@@ -772,6 +808,14 @@ List Locations
 
     GET /api/v1/controller/location/
 
+.. _location_list_estimated:
+
+**Estimated Status**
+
+|estimated_details|
+
+Locations can also be filtered using the ``is_estimated`` filter.
+
 **Available filters**
 
 You can filter using ``organization_id`` or ``organization_slug`` to get
@@ -853,6 +897,12 @@ Get Location Details
 
     GET /api/v1/controller/location/{pk}/
 
+.. _location_detail_estimated:
+
+**Estimated Status**
+
+|estimated_details|
+
 Change Location Details
 ~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -895,6 +945,14 @@ List Locations with Devices Deployed (in GeoJSON Format)
 
     GET /api/v1/controller/location/geojson/
 
+.. _location_geojson_estimated:
+
+**Estimated Status**
+
+|estimated_details|
+
+Locations can also be filtered using the ``is_estimated`` filter.
+
 **Available filters**
 
 You can filter using ``organization_id`` or ``organization_slug`` to get

docs/user/settings.rst

@@ -761,3 +761,92 @@ recoverable failures, improving the reliability of the system.
 For more information on these settings, you can refer to the `the celery
 documentation regarding automatic retries for known errors.
 <https://docs.celeryq.dev/en/stable/userguide/tasks.html#automatic-retry-for-known-exceptions>`_
+
+.. _openwisp_controller_whois_enabled:
+
+``OPENWISP_CONTROLLER_WHOIS_ENABLED``
+-------------------------------------
+
+============ =========
+**type**:    ``bool``
+**default**: ``False``
+============ =========
+
+Allows enabling the optional :doc:`WHOIS Lookup feature <whois>`.
+
+.. image:: https://raw.githubusercontent.com/openwisp/openwisp-controller/docs/docs/1.3/whois/admin-setting.png
+    :target: https://raw.githubusercontent.com/openwisp/openwisp-controller/docs/docs/1.3/whois/admin-setting.png
+    :alt: WHOIS admin setting
+
+After enabling this feature, you have to set
+:ref:`OPENWISP_CONTROLLER_WHOIS_GEOIP_ACCOUNT
+<OPENWISP_CONTROLLER_WHOIS_GEOIP_ACCOUNT>` and
+:ref:`OPENWISP_CONTROLLER_WHOIS_GEOIP_KEY
+<OPENWISP_CONTROLLER_WHOIS_GEOIP_KEY>`.
+
+.. warning::
+
+    If these three settings are not configured as expected, an
+    ``ImproperlyConfigured`` exception will be raised.
+
+.. _openwisp_controller_whois_geoip_account:
+
+``OPENWISP_CONTROLLER_WHOIS_GEOIP_ACCOUNT``
+-------------------------------------------
+
+============ =======
+**type**:    ``str``
+**default**: ``""``
+============ =======
+
+MaxMind Account ID required for the :doc:`WHOIS Lookup feature <whois>`.
+
+.. _openwisp_controller_whois_geoip_key:
+
+``OPENWISP_CONTROLLER_WHOIS_GEOIP_KEY``
+---------------------------------------
+
+============ =======
+**type**:    ``str``
+**default**: ``""``
+============ =======
+
+MaxMind License Key required for the :doc:`WHOIS Lookup feature <whois>`.
+
+.. _openwisp_controller_whois_refresh_threshold_days:
+
+``OPENWISP_CONTROLLER_WHOIS_REFRESH_THRESHOLD_DAYS``
+----------------------------------------------------
+
+============ =======
+**type**:    ``int``
+**default**: ``90``
+============ =======
+
+Specifies the number of days after which the WHOIS information for a
+device is considered stale and eligible for refresh.
+
+.. _openwisp_controller_estimated_location_enabled:
+
+``OPENWISP_CONTROLLER_ESTIMATED_LOCATION_ENABLED``
+--------------------------------------------------
+
+============ =========
+**type**:    ``bool``
+**default**: ``False``
+============ =========
+
+Allows enabling the optional :doc:`Estimated Location feature
+<estimated-location>`.
+
+.. warning::
+
+    :ref:`OPENWISP_CONTROLLER_WHOIS_ENABLED
+    <openwisp_controller_whois_enabled>` must be set to ``True`` before
+    enabling this feature. Enabling estimated locations while
+    ``WHOIS_ENABLED=False`` will raise ``ImproperlyConfigured`` at startup
+    time.
+
+.. image:: https://raw.githubusercontent.com/openwisp/openwisp-controller/docs/docs/1.3/estimated-locations/admin-setting.png
+    :target: https://raw.githubusercontent.com/openwisp/openwisp-controller/docs/docs/1.3/estimated-locations/admin-setting.png
+    :alt: Estimated Location setting