Skip to content

feat(backend-shared, server-core, query-orchestrator): replace crypto md5 hashing with xxhash#10314

Open
h0tw1r3 wants to merge 11 commits intocube-js:masterfrom
h0tw1r3:feat/xxhash
Open

feat(backend-shared, server-core, query-orchestrator): replace crypto md5 hashing with xxhash#10314
h0tw1r3 wants to merge 11 commits intocube-js:masterfrom
h0tw1r3:feat/xxhash

Conversation

@h0tw1r3
Copy link

@h0tw1r3 h0tw1r3 commented Jan 16, 2026

Continuation of PR #9397

This PR just adds a few commits to create a single function as requested (#9397 (comment)) and test coverage.

In addition to the performance improvements with xxhash, my motivation was to remove md5 crypto as a whole because it's not supported when enabling FIPS mode.

Original PR description below

Check List

  • Tests have been run in packages where changes have been made if available
  • Linter has been run for changed code
  • Tests for the changes have been added if not covered yet
  • Docs have been added / updated if required

Switch to using faster xxhash for cache keys and versions of data models instead of slow crypto.createHash('md5').
Use the fastest node xxhash implementation, written in ... of course rust :)

XXH3: generates 64 or 128-bit hashes, using vectorized arithmetic.

Here are some more info for those who is interested:

@h0tw1r3 h0tw1r3 requested review from a team as code owners January 16, 2026 16:51
@github-actions github-actions bot added javascript Pull requests that update Javascript code data source driver pr:community Contribution from Cube.js community members. labels Jan 16, 2026
@h0tw1r3
Copy link
Author

h0tw1r3 commented Jan 16, 2026

@ovr tagging as you requested the shared defaultHasher interface

@h0tw1r3 h0tw1r3 force-pushed the feat/xxhash branch 2 times, most recently from a0edab9 to 2b35b72 Compare January 22, 2026 20:14
@h0tw1r3 h0tw1r3 changed the title feat: replace crypto md5 key/version hashing with single function based on xxhash feat(backend-shared): replace crypto md5 hashing with xxhash Jan 30, 2026
@h0tw1r3 h0tw1r3 changed the title feat(backend-shared): replace crypto md5 hashing with xxhash feat(backend-shared, server-core, query-orchestrator): replace crypto md5 hashing with xxhash Jan 30, 2026
@h0tw1r3
Copy link
Author

h0tw1r3 commented Jan 31, 2026

@KSDaemon curious why did you close the original PR?

@KSDaemon
Copy link
Contributor

@h0tw1r3 Hey! Well... I stepped out of the Cube Core team, and I don't think anyone would take this initiative to finish... So all hope rests on you! :)

@h0tw1r3
Copy link
Author

h0tw1r3 commented Feb 1, 2026

@h0tw1r3 Hey! Well... I stepped out of the Cube Core team, and I don't think anyone would take this initiative to finish... So all hope rests on you! :)

It's ready to go! I don't want to be a bother, but I have no idea how to get anyone's attention for a review. Any help would be greatly appreciated.

Very interested in using Cube, but the customers I work with have strict security guidelines (FIPS). If node can be run in FIPS mode, it opens up a new class of users to Cube.

@KSDaemon
Copy link
Contributor

KSDaemon commented Feb 1, 2026

@h0tw1r3 I'll ping guys from the Core team.

@paveltiunov
Copy link
Member

Hey @h0tw1r3 ! Thanks for contributing! I briefly checked PR and overall FIPS is on our plate to support however changing md5 to anything would be drastically breaking change so it's surely should be done under the flag. Second why is it xxhash and not sha? It isn't FIPS compliant as well. Thanks!

@h0tw1r3
Copy link
Author

h0tw1r3 commented Feb 4, 2026

Hey @h0tw1r3 ! Thanks for contributing! I briefly checked PR and overall FIPS is on our plate to support however changing md5 to anything would be drastically breaking change so it's surely should be done under the flag. Second why is it xxhash and not sha? It isn't FIPS compliant as well. Thanks!

@paveltiunov That's good to hear about FIPS support!

I'm just saying it's impossible to enable node FIPS (ie. node --enable-fips cubejs server or export NODE_OPTIONS=--enable-fips) as MD5 is specifically not supported by FIPS compliant cryptographic modules. There doesn't seem to be an operational requirement for the hashes in the PR to be FIPS, but when using Cube on a FIPS compliant system, node must be run in FIPS mode to accommodate other things that do require FIPS (ie. backend driver connections).

xxhash was selected because of the PR this was based on (fast, well supported), I just completed that work with a generic implementation, such that swapping the defaultHasher would be trivial.

@h0tw1r3
Copy link
Author

h0tw1r3 commented Feb 4, 2026

@paveltiunov implemented md5, sha256, sha512 hashing. defaults to md5, controllable through CUBEJS_HASHING_ALGORITHM env var.

@igorlukanin
Copy link
Member

Hi @h0tw1r3 👋 May I kindly ask you to add CUBEJS_HASHING_ALGORITHM to docs at https://cube.dev/docs/product/configuration/reference/environment-variables in this PR? The relevant files are under docs/pages in this repo.

@h0tw1r3
Copy link
Author

h0tw1r3 commented Feb 4, 2026

@igorlukanin added to the configuration reference.
for clarity, renamed the env var to CUBEJS_HASHER_ALGORITHM

@h0tw1r3
Copy link
Author

h0tw1r3 commented Feb 5, 2026

final changes... in leu of feedback

  • reverted unrelated syntax changes
  • consistent naming conventions (hashing -> hasher)
  • validate algorithm configuration

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

data source driver javascript Pull requests that update Javascript code pr:community Contribution from Cube.js community members.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants