(OFBIZ-13366) : Improved performance and implementation of database-level pagination in FindGeneric.groovy#968
Open
pushprajsinghjadoun wants to merge 3 commits intoapache:trunkfrom
Open
Conversation
…to fetch only limited records instead of fetching all the records from the DB
pushprajsinghjadoun
changed the title
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Improved: Improved performance and implementation of database-level pagination in FindGeneric.groovy (OFBIZ-13366)
Efficiency of the generic entity search and listing functionality within WebTools (FindGeneric.groovy) when handling large datasets.
Implemented:
Database-level pagination using limit() and offset() via direct EntityQuery execution in the Groovy script, eliminating in-memory filtering for large result sets.
Completed:
Refactoring of the data retrieval logic to ensure EntityListIterator is used with a fixed result size, and updating the dynamic form generation to support pre-paginated data.
Fixed:
Performance-related system timeouts and memory exhaustion issues encountered when performing searches on large entities (e.g., Product) without search conditions.
Explanation
The previous implementation of FindGeneric.groovy relied on the performFind service within the dynamically generated XML form’s block. This approach was inefficient for large datasets because pagination was not consistently applied at the database level, often leading to excessive memory usage, timeouts, or server crashes when browsing large entities without filters.
This change moves the data retrieval logic directly into the Groovy script. By using EntityQuery with explicit limit and offset values derived from the VIEW_INDEX and VIEW_SIZE request parameters, only the required subset of records is fetched from the database. Additionally, getResultsSizeAfterPartialList() is used to provide accurate total counts for the UI without loading all records into memory, and the dynamic form renderer is informed that the data is already paginated via the override-list-size attribute. Support for suffixed pagination parameters (e.g., VIEW_INDEX_1) has also been added.
Thanks: The OFBiz community and reviewers for their feedback and guidance.