Skip to content

Add more tests for Byzantine roubust aggregator#1042

Open
mina5rovic wants to merge 1 commit intodevelopfrom
byzantine-tests-mina5rovic
Open

Add more tests for Byzantine roubust aggregator#1042
mina5rovic wants to merge 1 commit intodevelopfrom
byzantine-tests-mina5rovic

Conversation

@mina5rovic
Copy link
Collaborator

  • Adds test coverage for Byzantine robust aggregator
  • Covers edge cases and expected behavior
  • No aggregator code changes

Copy link
Collaborator

@JulienVig JulienVig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the additional tests! fast-check is a cool library I hadn't heard of property-based testing before.
Running the fast check test seems to be a bit flaky, alternating between passing and failing.
Here's what it looks like when it fails (I bumped the number of runs to 2000 to have it fail more frequently)

$ npm -w discojs test -- -t 'bounds the marginal influence of a single Byzantine client'

 FAIL   discojs  discojs/src/aggregator/byzantine.spec.ts > ByzantineRobustAggregator > bounds the marginal influence of a single Byzantine client
Error: Property failed after 1306 tests
{ seed: -459713346, path: "1305:65:63:63:63", endOnFailure: true }
Counterexample: [[Number.NaN,0,0,0]]
Shrunk 4 time(s)
Got AssertionError: expected NaN to be less than or equal to 0.4
    at byzantine-tests-mina5rovic/discojs/src/aggregator/byzantine.spec.ts:209:29
    at processTicksAndRejections (node:internal/process/task_queues:105:5)
    at AsyncProperty.run (file:byzantine-tests-mina5rovic/discojs/node_modules/fast-check/lib/esm/check/property/AsyncProperty.generic.js:46:28)
    at asyncRunIt (file:byzantine-tests-mina5rovic/discojs/node_modules/fast-check/lib/esm/check/runner/Runner.js:33:21)
    at byzantine-tests-mina5rovic/discojs/src/aggregator/byzantine.spec.ts:180:5
    at file:byzantine-tests-mina5rovic/node_modules/@vitest/runner/dist/index.js:915:20

Hint: Enable verbose mode in order to have the list of all failing values encountered during the run
 ❯ discojs/src/aggregator/byzantine.spec.ts:209:29
    207|           const deviation = Math.abs(byz - clean);
    208|           const maxAllowed = 2 * clipRadius / n; // realistic tolerance for extreme inputs
    209|           expect(deviation).toBeLessThanOrEqual(maxAllowed);
       |                             ^
    210|         }
    211|       ),
 ❯ AsyncProperty.run discojs/node_modules/fast-check/lib/esm/check/property/AsyncProperty.generic.js:46:28
 ❯ asyncRunIt discojs/node_modules/fast-check/lib/esm/check/runner/Runner.js:33:21
 ❯ discojs/src/aggregator/byzantine.spec.ts:180:5
 ❯ buildError discojs/node_modules/fast-check/lib/esm/check/runner/utils/RunDetailsFormatter.js:126:15
 ❯ asyncThrowIfFailed discojs/node_modules/fast-check/lib/esm/check/runner/utils/RunDetailsFormatter.js:143:11

"nodemon": "3",
"ts-node": "10"
"ts-node": "10",
"fast-check": "^3"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"fast-check": "^3"
"fast-check": "3"

We usually keep the version fixed to avoid unintentional updates and break-downs

"eslint": "9",
"eslint-plugin-cypress": "5",
"eslint-plugin-vue": "10",
"fast-check": "^4.5.3",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you only need to use fast-check for discojs then no need to add it to the whole project. Also this version is different from the one installed in discojs, which one do you want to use?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants