SOLR-18328 Add support for standard deviation in rollup for streaming expressions by KhushJain · Pull Request #4691 · apache/solr

KhushJain · 2026-08-01T21:53:14Z

https://issues.apache.org/jira/browse/SOLR-18328

Description

Added support for the std(col) (standard deviation) metric in rollup() streaming expressions.
The std(col) metric was non-functional; the StdMetric class was a stub copy-pasted from MeanMetric — update() was a no-op, getValue() returned null, and newInstance() even returned a MeanMetric instead of a StdMetric.

Solution

Implemented StdMetric to compute the sample standard deviation:

StdMetric.java:

update() now accumulates count, sum, and sumSq from the column value (handling Double/Float/Integer/Long, skipping nulls/other types).
getValue() computes the sample standard deviation sqrt((n*sumSq - sum^2) / (n*(n-1))), returning 0.0 when count <= 1, and honoring outputLong (rounds to a long when set).
Fixed newInstance() to return a StdMetric (was incorrectly returning MeanMetric), and replaced the unused doubleSum/longSum fields with sum/sumSq.

Tests

StreamExpressionTest.testRollupStdMetric (new): validates std(a_i) and std(a_f) alongside count(*) in a rollup(... over="a_s" ...) expression across multiple groups.
StreamingTest.testRollupStream: added StdMetric("a_i") / StdMetric("a_f") to the metrics array and asserted expected values per group.

Checklist

Please review the following and check all that apply:

I have reviewed the guidelines for How to Contribute and my code conforms to the standards described there to the best of my ability.
I have created a Jira issue and added the issue ID to my pull request title.
I have given Solr maintainers access to contribute to my PR branch. (optional but recommended, not available for branches on forks living under an organisation)
I have developed this patch against the main branch.
I have run ./gradlew check.
I have added tests for my changes.
I have added documentation for the Reference Guide
I have added a changelog entry for my change

KhushJain · 2026-08-02T18:24:40Z

Hey @epugh, thanks for reviewing!!

One of the existing flaky test failed testSingleShardInFlightRequestsDuringShutDown

VishnuPriyaChandraSekar · 2026-08-03T21:06:37Z

+  public void update(Tuple tuple) {
+    Object o = tuple.get(columnName);
+    double val;
+    if (o instanceof Double d) {


Could the 'if-else' structure be simplified ?

there definitly are some unusla coding patterns in the streaming code... but we tend to follow them once they exist as there are so many of them!

@epugh Thanks for the clarification :) Is there a documentation about the code standard adopted by the codebase ?
@KhushJain Feel free to fix the if-else or ignore my suggestion :)

we have "tidy" and errrorprone tools that we run int he builds. As far as in the streaming code, nothign formal written, just, look at other java classes ;-). Having said that, I do appreciate your reviewing this PR, and there are lots of PR's out there that need a reviewer to go through them!

Agreed on the quirky instanceof pattern, but keeping it consistent with all its siblings.

@epugh I would be happy to review the PR's :)

VishnuPriyaChandraSekar · 2026-08-03T21:07:16Z

  @Override
-  public void update(Tuple tuple) {}
+  public void update(Tuple tuple) {
+    Object o = tuple.get(columnName);


Could the object 'o' be made immutable by declaring it as final ?

it could be added but the current state is consistent with the existing style.

VishnuPriyaChandraSekar · 2026-08-03T21:08:21Z

@@ -24,15 +24,10 @@
 import org.apache.solr.client.solrj.io.stream.expr.StreamFactory;

 public class StdMetric extends Metric {


Could you include java docs for this class ?

Added docstrings.

VishnuPriyaChandraSekar · 2026-08-03T21:10:32Z

@@ -89,7 +101,13 @@ public String[] getColumns() {

  @Override
  public Number getValue() {


Could you include javadocs to understand the purpose of this method ?

Added docstrings.

epugh

a question about docs, but otherwise sems to follow the pattern of some other ones.. It would be good to answer the other review comments, and tjhen I think we can move forward...

epugh · 2026-08-03T23:47:35Z

 * `over`: (Mandatory) A list of fields to group by.
 * `metrics`: (Mandatory) The list of metrics to compute.
-Currently supported metrics are `sum(col)`, `avg(col)`, `min(col)`, `max(col)`, `count(*)`, `missing(col)`, `countDist(col)`, `per(col, percentile)`.
+Currently supported metrics are `sum(col)`, `avg(col)`, `min(col)`, `max(col)`, `count(*)`, `missing(col)`, `countDist(col)`, `per(col, percentile)`, `std(col)`.


is there any more docs on these indivdual ones or is this mention what the pattern is? Just wondering if folks will know how to use it...

no dedicated per-metric docs exists, all streaming metrics are documented this same inline way.

Support for missing(col), countDist(col) and per(col, percentile) are also added by me in the past.

epugh · 2026-08-03T23:48:50Z

+  public void update(Tuple tuple) {
+    Object o = tuple.get(columnName);
+    double val;
+    if (o instanceof Double d) {


there definitly are some unusla coding patterns in the streaming code... but we tend to follow them once they exist as there are so many of them!

KhushJain · 2026-08-05T01:55:28Z

Thanks @epugh @VishnuPriyaChandraSekar for reviewing it!!

Can we please back port this to 9x as well?

VishnuPriyaChandraSekar

Thanks for addressing my feedback :)

add support for standard deviation in rollup for streaming expressions

7bdc97d

github-actions Bot added documentation Improvements or additions to documentation client:solrj tests labels Aug 1, 2026

epugh self-assigned this Aug 2, 2026

VishnuPriyaChandraSekar suggested changes Aug 3, 2026

View reviewed changes

epugh reviewed Aug 3, 2026

View reviewed changes

Add docstrings

2f93308

VishnuPriyaChandraSekar approved these changes Aug 6, 2026

View reviewed changes

		@@ -24,15 +24,10 @@
		import org.apache.solr.client.solrj.io.stream.expr.StreamFactory;

		public class StdMetric extends Metric {

		@@ -89,7 +101,13 @@ public String[] getColumns() {

		@Override
		public Number getValue() {

Uh oh!

Conversation

KhushJain commented Aug 1, 2026

Description

Solution

Tests

Checklist

Uh oh!

KhushJain commented Aug 2, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

epugh left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

KhushJain commented Aug 5, 2026

Uh oh!

VishnuPriyaChandraSekar left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants