Skip to content

ATLAS-5194: Add pagination while fetching relationships of an entity#510

Merged
pinal-shah merged 1 commit intoapache:masterfrom
sheetalshah1007:ATLAS-5194
Mar 13, 2026
Merged

ATLAS-5194: Add pagination while fetching relationships of an entity#510
pinal-shah merged 1 commit intoapache:masterfrom
sheetalshah1007:ATLAS-5194

Conversation

@sheetalshah1007
Copy link
Copy Markdown
Contributor

@sheetalshah1007 sheetalshah1007 commented Jan 30, 2026

What changes were proposed in this pull request?

Added a new disableDefaultSorting boolean parameter to the searchRelatedEntities API across the REST layer, service interface, and implementation. When false (default), the API maintains backward compatibility by applying default "name" sorting if no explicit sortBy is provided. When true, no default sorting is applied, returning results in natural graph traversal order.

Fixed a critical bug in approximate count calculation where deleted entities were incorrectly included in the count when excludeDeletedEntities=true - the implementation now correctly filters by end vertex status instead of edge status.

Updated all existing tests to use disableDefaultSorting=false to preserve current behaviour, and added new test cases to verify sorting logic and accurate count calculation.

Why It Was Needed?

The existing searchRelatedEntities API always applied default "name" sorting even when users didn't specify a sortBy parameter, which could impact performance for large datasets where users don't need sorted results. This forced sorting behaviour couldn't be disabled, making deep pagination slower than necessary. The new parameter gives users the flexibility to opt out of default sorting when they prioritise performance over order, particularly useful for paginated queries on entities with thousands of relationships.

How was this patch tested?

  • Unit tests: Updated existing test cases in DiscoveryRESTTest, EntityDiscoveryServiceTest, and AtlasDiscoveryServiceTest to pass disableDefaultSorting=false, ensuring backward compatibility
  • New test cases: Added testSearchRelatedEntitiesWithDisableDefaultSorting() to verify both sorted (disableDefaultSorting=false) and unsorted (disableDefaultSorting=true) paths return the same entities
  • Pagination tests: Added testSearchRelatedEntitiesWithOffset() to verify correct pagination behaviour across multiple pages
  • Edge case tests: Added tests for deleted entity filtering (testSearchRelatedEntitiesWithDeletedEntities) and approximate count accuracy (testApproximateCountExcludesDeleted) with the new parameter
  • Manual testing: Tested with curl commands on large datasets (50K-100K relationships) to verify performance improvements and correctness
  • No regression for sorted queries

* @throws AtlasBaseException
*/
@GET
@Path("relationship/v2")
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please review if there is need to introduce new API

*/
@GraphTransaction
@Override
public AtlasSearchResult searchRelatedEntitiesV2(String guid, String relation, boolean getApproximateCount, SearchParameters searchParameters, boolean disableDefaultSorting) throws AtlasBaseException {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please incorporate changes in the existing method, to avoid code duplication

@sheetalshah1007 sheetalshah1007 force-pushed the ATLAS-5194 branch 5 times, most recently from fbb2299 to 8c111f4 Compare March 12, 2026 05:10
Comment thread repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java Outdated
@sheetalshah1007 sheetalshah1007 force-pushed the ATLAS-5194 branch 2 times, most recently from 45eca01 to 3092442 Compare March 12, 2026 07:00
@pinal-shah pinal-shah merged commit a807e6d into apache:master Mar 13, 2026
1 check passed
pinal-shah pushed a commit that referenced this pull request Mar 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants