Skip to content

[ITEM-189] WP-1987-sort-insensitive#247

Merged
wazo-community-zuul[bot] merged 17 commits intomasterfrom
WP-1987-sort-insensitive
Mar 18, 2026
Merged

[ITEM-189] WP-1987-sort-insensitive#247
wazo-community-zuul[bot] merged 17 commits intomasterfrom
WP-1987-sort-insensitive

Conversation

@fblackburn1
Copy link
Copy Markdown
Member

@fblackburn1 fblackburn1 commented Mar 16, 2026

Depends-on: wazo-platform/xivo-lib-python#176

  • small refactor of personal dao pagination
  • improve slicing operation for pagination
  • dao: generic implementation for order_insensitive
  • personal: add test for order insensitive scenario
  • test: disable too much verbose log by default
  • typing: handle None value for sorting
  • google: sort name column by case insensitive
  • microsoft: sort specific contacts columns by case insensitive
  • phonebook: improve sort function readability
  • phonebook: sort columns by case insensitive
  • phonebook backend: sort columns by case insensitive
  • phonebook: use explicit parameters
  • personal: consolidate in-memory sorting with SelfSortingServiceMixin
  • dao: remove unused sorting logic from BaseDAO
  • phonebook: consolidate sorting with SelfSortingServiceMixin
  • refactor SelfSortingServiceMixin to be simpler
  • sorting: use unidecode instead of NFKD for accent folding

Note

Medium Risk
Changes core in-memory sorting/pagination paths for multiple backends (personal, phonebook, Google, Office365, profiles), which could alter ordering and pagination behavior in production listings. Risk is moderate because logic is centralized and covered by updated unit/integration tests, but it affects many list endpoints.

Overview
Introduces a shared sort_contacts helper (accent-folding via unidecode, optional case-insensitive ordering, and safe handling of missing/None values) and removes the old SelfSortingServiceMixin/BaseDAO in-memory sorting logic.

Updates personal contact listing to use the new sorting + simplified slicing for pagination (with explicit invalid order detection), and wires the new sorter into backend/profile services plus phonebook contact listing (also passing list params explicitly).

Enables case-insensitive sorting for selected contact list endpoints by adding sort_insensitive_columns to Google, Office365, phonebook, and phonebook-backend schemas, and adjusts/adds integration tests to validate the new ordering (including a new phonebook HTTP case-insensitive sort test). Also makes SQLAlchemy echo logging configurable in integration tests via DB_ECHO.

Written by Cursor Bugbot for commit 7d40a82. This will update automatically on new commits. Configure here.

why: improve readability and prepare next change
why: since rows is already a list, we can use slice directly
Since default implementation is provide expected behavior for personal,
to more code is required
This is only a safeguard: DB allow None value, but not API
why: to add some logic in get sort in next commits
why: avoid breaking code when ListSchema from lib-python add a new field
Sorting logic was duplicated between BaseDAO and SelfSortingServiceMixin.
Using a single implementation avoids divergence and brings NFKD
normalization to all backends using the mixin.
_apply_pagination_params and order_insensitive are no longer used
since personal contacts now sort via SelfSortingServiceMixin.
why: half of usage was done through "true" mixin, others were used
through static method
NFKD decomposes accented characters into base + combining marks
(e.g. 'Ä' → 'A' + U+0308), but the combining mark has a high
codepoint so accented names still sort after all ASCII names —
the opposite of the intended behavior.

unidecode strips diacritics entirely ('Ä' → 'A'), which correctly
groups accented names with their base letter. This matches the
behavior introduced in 2016 (commit 69bcbc2) specifically to fix
sort order with accentuated characters in the phonebook.
@wazo-community-zuul
Copy link
Copy Markdown
Contributor

Build succeeded.
https://zuul.wazo.community/zuul/t/local/buildset/be2d1948d200482d8245493c6e68bdcb

✔️ tox-linters SUCCESS in 3m 33s
✔️ wazo-tox-py311 SUCCESS in 3m 41s
✔️ debian-packaging-bookworm SUCCESS in 4m 07s
✔️ dird-tox-integration SUCCESS in 42m 09s

@DrPyser DrPyser changed the title WP-1987-sort-insensitive ITEM-189 WP-1987-sort-insensitive Mar 17, 2026
@DrPyser DrPyser changed the title ITEM-189 WP-1987-sort-insensitive [ITEM-189] WP-1987-sort-insensitive Mar 17, 2026
Copy link
Copy Markdown
Contributor

@DrPyser DrPyser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, E2E tested (WDA) on personal, phonebook, google (logic should be the same for office365).

WDA-level sorting is still accent-sensitive, to be addressed separately.

@wazo-community-zuul
Copy link
Copy Markdown
Contributor

Build succeeded (gate pipeline).
https://zuul.wazo.community/zuul/t/local/buildset/d85dac26cbda45bd9aaf21de450a0a27

✔️ tox-linters SUCCESS in 3m 18s
✔️ wazo-tox-py311 SUCCESS in 3m 21s
✔️ debian-packaging-bookworm SUCCESS in 2m 43s
✔️ dird-tox-integration SUCCESS in 48m 27s

@wazo-community-zuul wazo-community-zuul Bot merged commit 3f17879 into master Mar 18, 2026
3 checks passed
@wazo-community-zuul wazo-community-zuul Bot deleted the WP-1987-sort-insensitive branch March 18, 2026 18:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants