Skip to content

[Connectors-SDK] Handle config deprecation and migration in connectors-sdk#5563

Merged
throuxel merged 4 commits intomasterfrom
feat/0000-poc-sdk-deprecated-config
Feb 24, 2026
Merged

[Connectors-SDK] Handle config deprecation and migration in connectors-sdk#5563
throuxel merged 4 commits intomasterfrom
feat/0000-poc-sdk-deprecated-config

Conversation

@throuxel
Copy link
Copy Markdown
Member

@throuxel throuxel commented Jan 6, 2026

Proposed changes

This PR introduces a deprecation handling system for the connectors SDK, managing automatically different use-cases while maintaining backward compatibility.

  • LegacyField: Mark fields as deprecated with migration metadata
  • Automatic migration via migrate_deprecation() validator in BaseConnectorSettings
  • Support for variable renaming, namespace migration, and value transformation
  • User-friendly deprecation warnings
  • Compatible with the connector composer
  • Tests added

Exemple:

class MyConfig(BaseConfigModel):
   old_var: SkipValidation[int] = LegacyField(
       deprecated="Use new_var instead",
       new_variable_name="new_var",
       change_value=lambda x: x * 60,  # Optional transformation
       removal_date="2026-09-01" # Optional informative date
   )
   new_var: int = Field(description="New variable")

class ConnectorSettings(BaseConnectorSettings):
   old_namespace: SkipValidation[MyConfig] = LegacyField(
       deprecated="Use new_namespace instead",
       new_namespace="new_namespace"
   )
   new_namespace: MyConfig = Field(default_factory=MyConfig)

Related issues

Checklist

  • I consider the submitted work as finished
  • I have signed my commits using GPG key.
  • I tested the code for its functionality using different use cases
  • I added/update the relevant documentation (either on github or on notion)
  • Where necessary I refactored code to improve the overall quality

Further comments

The dummy connector is only for demo, and should be removed before merging.

@throuxel throuxel self-assigned this Jan 6, 2026
@throuxel throuxel added filigran team use to identify PR from the Filigran team do not merge Do not merge this PR until this tag will be removed improvement connectors-sdk labels Jan 6, 2026
@SamuelHassine SamuelHassine force-pushed the master branch 2 times, most recently from a35f198 to 34b8e57 Compare January 9, 2026 14:33
@throuxel throuxel force-pushed the feat/0000-poc-sdk-deprecated-config branch 3 times, most recently from 2c3ae0e to f036472 Compare January 21, 2026 10:21
@throuxel throuxel force-pushed the feat/0000-poc-sdk-deprecated-config branch from f036472 to 47a794e Compare February 3, 2026 11:01
@throuxel throuxel marked this pull request as ready for review February 3, 2026 11:01
@throuxel throuxel changed the title [POC] Handle config deprecation and migration in connectors-sdk Handle config deprecation and migration in connectors-sdk Feb 3, 2026
Comment thread connectors-sdk/connectors_sdk/settings/deprecations.py
Comment thread connectors-sdk/connectors_sdk/settings/base_settings.py Outdated
Comment thread connectors-sdk/connectors_sdk/settings/json_schema.py Outdated
Comment thread connectors-sdk/connectors_sdk/settings/json_schema.py Outdated
Comment thread connectors-sdk/connectors_sdk/settings/json_schema.py Outdated
Comment thread connectors-sdk/connectors_sdk/utils/deprecations.py Outdated
Comment thread connectors-sdk/connectors_sdk/utils/deprecations.py Outdated
Comment thread connectors-sdk/connectors_sdk/settings/json_schema.py Outdated
Comment thread internal-enrichment/dummy-enrichment/__metadata__/connector_manifest.json Outdated
Comment thread connectors-sdk/connectors_sdk/settings/json_schema.py Outdated
Copy link
Copy Markdown
Member

@Powlinett Powlinett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you so much for this great contribution @throuxel 🤘

I don't approve the PR right now to leave room for discussion, but as far as I tested it works as expected ✅️

Most of my comments are about naming things and it's obviously debatable.

Plus, I noticed 10 new warnings emitted while running tests, and I couldn't find the actual cause:

tests/test_settings/test_base_settings.py: 8 warnings
tests/test_settings/test_deprecation_migration.py: 2 warnings
  ~/code/filigran/connectors/connectors-sdk/venv/lib/python3.12/site-packages/pydantic_settings/main.py:447: UserWarning: Config key `yaml_file` is set in model_config but will be ignored because no YamlConfigSettingsSource source is configured. To use this config key, add a YamlConfigSettingsSource source to the settings sources via the settings_customise_sources hook.
    cls._settings_warn_unused_config_keys(sources, cls.model_config)

Do you have any idea of what's happening?

Thanks again, well done 💪

Comment thread connectors-sdk/connectors_sdk/settings/base_settings.py Outdated
Comment thread connectors-sdk/connectors_sdk/settings/base_settings.py Outdated
Comment thread connectors-sdk/connectors_sdk/settings/base_settings.py
Comment thread connectors-sdk/connectors_sdk/settings/deprecations.py
Comment thread connectors-sdk/connectors_sdk/utils/deprecations.py Outdated
Comment thread connectors-sdk/connectors_sdk/utils/deprecations.py Outdated
Comment thread connectors-sdk/connectors_sdk/settings/json_schema.py Outdated
Comment thread connectors-sdk/connectors_sdk/settings/json_schema.py Outdated
Comment thread connectors-sdk/connectors_sdk/settings/json_schema.py Outdated
Comment thread connectors-sdk/connectors_sdk/settings/json_schema.py Outdated
@throuxel throuxel force-pushed the feat/0000-poc-sdk-deprecated-config branch from 47a794e to 783796d Compare February 19, 2026 13:02
@throuxel throuxel requested a review from jabesq February 19, 2026 15:05
@Powlinett
Copy link
Copy Markdown
Member

LGTM 👏 let us know when you're ready for merge 😇

jabesq
jabesq previously approved these changes Feb 20, 2026
Copy link
Copy Markdown
Member

@jabesq jabesq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@throuxel throuxel force-pushed the feat/0000-poc-sdk-deprecated-config branch from 783796d to a947930 Compare February 20, 2026 10:30
@throuxel throuxel force-pushed the feat/0000-poc-sdk-deprecated-config branch from a947930 to 141d051 Compare February 20, 2026 13:14
@throuxel throuxel removed the do not merge Do not merge this PR until this tag will be removed label Feb 20, 2026
@Powlinett Powlinett self-requested a review February 24, 2026 09:17
@throuxel throuxel merged commit 5f7b350 into master Feb 24, 2026
9 checks passed
@throuxel throuxel deleted the feat/0000-poc-sdk-deprecated-config branch February 24, 2026 09:19
@throuxel throuxel added this to the Bugs backlog milestone Feb 24, 2026
@romain-filigran romain-filigran modified the milestone: Bugs backlog Feb 24, 2026
@helene-nguyen helene-nguyen changed the title Handle config deprecation and migration in connectors-sdk [Connectors-SDK] Handle config deprecation and migration in connectors-sdk Feb 25, 2026
@helene-nguyen helene-nguyen added this to the Release 7.260224.0 milestone Feb 25, 2026
mitchm101 pushed a commit to team-cymru/opencti-connectors that referenced this pull request Mar 26, 2026
semayellow pushed a commit to semayellow/OpenCTI-integration that referenced this pull request Apr 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

connectors-sdk filigran team use to identify PR from the Filigran team improvement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Connectors-SDK] Handle config deprecation and migration in connectors-sdk

6 participants