[SYSTEMDS-3948] Row-wise Sparsity Estimator#2466
Closed
ywcb00 wants to merge 16 commits into
Closed
Conversation
Baunsgaard
reviewed
May 7, 2026
Contributor
Baunsgaard
left a comment
There was a problem hiding this comment.
Some comments. hope it helps.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2466 +/- ##
============================================
+ Coverage 71.36% 71.38% +0.01%
- Complexity 48684 48751 +67
============================================
Files 1570 1571 +1
Lines 188757 188912 +155
Branches 37039 37067 +28
============================================
+ Hits 134712 134849 +137
- Misses 43591 43601 +10
- Partials 10454 10462 +8 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
1f2c869 to
30cc518
Compare
Contributor
Author
|
Thank you for the comments @Baunsgaard . I have updated the PR accordingly. :) |
Baunsgaard
reviewed
May 15, 2026
Contributor
Author
|
Thank you for the valuable comments. :) |
Baunsgaard
approved these changes
May 19, 2026
Contributor
Baunsgaard
left a comment
There was a problem hiding this comment.
PR LGTM, there are only micro optimizations to do now.
…ise sparsity estimator works for the matrix multiplication and bind test cases for now
…to the unity tests for sparsity estimation
…ontainer for row wise sparsity vectors to simplify access and allow storing it with chain nodes
…mator with element-wise and single operations
…wise and single operations NOTE: using average case estimation per row
…ion for diagonal operation
…ro and diag (mv and vm) operations with the row-wise sparsity estimator
… to consolidate all calls to getters before the switch
… for row-wise sparsity vector and apply the corresponding operations directly in the code instead
…sity estimation tests
add some test parameter configurations
…ses or increase assign specific tolerance
…expressions to for loops
…ts to double type for constant values remove comment stating that this would be the best estimate for the respective case
04fcd6f to
73dd10b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hi,
this PR adds the row-wise sparsity estimator from:
Lin, Chunxu, Wensheng Luo, Yixiang Fang, Chenhao Ma, Xilin Liu and Yuchi Ma;
On Efficient Large Sparse Matrix Chain Multiplication;
Proceedings of the ACM on Management of Data 2 (2024): 1 - 27.
Note that the row sparsity propagation, as described in the publication, applies to MM chains only. Other operations use fallback methods for sparsity estimation w/ row sparsity vectors, which create a cut in the sparsity estimation DAG.