Skip to content

feat(bitop): Support DIFF, DIFF1, ANDOR, and ONE for command BITOP#3388

Open
xuzifu666 wants to merge 2 commits intoapache:unstablefrom
xuzifu666:opt_bit
Open

feat(bitop): Support DIFF, DIFF1, ANDOR, and ONE for command BITOP#3388
xuzifu666 wants to merge 2 commits intoapache:unstablefrom
xuzifu666:opt_bit

Conversation

@xuzifu666
Copy link
Member

issue: #3132

@xuzifu666 xuzifu666 changed the title [WIP]feat(bitop): Support DIFF, DIFF1, ANDOR, and ONE for command BITOP feat(bitop): Support DIFF, DIFF1, ANDOR, and ONE for command BITOP Mar 9, 2026
@sonarqubecloud
Copy link

Copy link
Member

@jihuayu jihuayu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When a key is missing, the DIFF, DIFF1, and ANDOR operations incorrectly shift the position of the first operand, leading to corrupted results.
The root cause is that the implementation simply skips non-existent keys, causing subsequent logic to incorrectly treat the first available fragment (fragments[0]) as the primary operand ($X$).

Scenario: $x$ does not exist, $y$ has bits set.
Command: BITOP DIFF dest x y
Expected: $x \text{ AND NOT } y = 0$.
Current Behavior: The implementation treats $y$ as $X$, resulting in $y$.
Note: DIFF1 and ANDOR fail similarly (resulting in either $0$ or $y$ depending on the specific code path).

The following is a test case that should pass.

	t.Run("BITOP DIFF/DIFF1/ANDOR with missing first source key", func(t *testing.T) {
		require.NoError(t, rdb.FlushDB(ctx).Err())

		require.NoError(t, rdb.SetBit(ctx, "y", 0, 1).Err())
		require.NoError(t, rdb.SetBit(ctx, "y1", 5, 1).Err())
		require.NoError(t, rdb.SetBit(ctx, "y2", 5, 1).Err())

		// DIFF: X & ~(Y1 | ...) where X is missing -> 0
		require.NoError(t, rdb.Do(ctx, "BITOP", "DIFF", "diff_missing_first", "x_missing", "y").Err())
		require.NoError(t, rdb.GetBit(ctx, "diff_missing_first", 0).Err())
		require.EqualValues(t, 0, rdb.GetBit(ctx, "diff_missing_first", 0).Val())

		// DIFF1: (Y1 | ...) & ~X where X is missing -> Y1 | ...
		require.NoError(t, rdb.Do(ctx, "BITOP", "DIFF1", "diff1_missing_first", "x_missing", "y").Err())
		require.NoError(t, rdb.GetBit(ctx, "diff1_missing_first", 0).Err())
		require.EqualValues(t, 1, rdb.GetBit(ctx, "diff1_missing_first", 0).Val())

		// ANDOR: X & (Y1 | ...) where X is missing -> 0
		require.NoError(t, rdb.Do(ctx, "BITOP", "ANDOR", "andor_missing_first", "x_missing", "y1", "y2").Err())
		require.NoError(t, rdb.GetBit(ctx, "andor_missing_first", 5).Err())
		require.EqualValues(t, 0, rdb.GetBit(ctx, "andor_missing_first", 5).Val())
	})

	t.Run("BITOP DIFF/DIFF1/ANDOR with sparse first source fragments", func(t *testing.T) {
		require.NoError(t, rdb.FlushDB(ctx).Err())

		// x only has low bits, y keys only have high bits, so x is logically zero in the high fragment.
		const highOffset = int64(20000)
		require.NoError(t, rdb.SetBit(ctx, "x", 0, 1).Err())
		require.NoError(t, rdb.SetBit(ctx, "y_high", highOffset, 1).Err())
		require.NoError(t, rdb.SetBit(ctx, "y1_high", highOffset, 1).Err())
		require.NoError(t, rdb.SetBit(ctx, "y2_high", highOffset, 1).Err())

		// DIFF: x & ~y_high -> 0 at highOffset because x has no bit there.
		require.NoError(t, rdb.Do(ctx, "BITOP", "DIFF", "diff_sparse_first", "x", "y_high").Err())
		require.NoError(t, rdb.GetBit(ctx, "diff_sparse_first", highOffset).Err())
		require.EqualValues(t, 0, rdb.GetBit(ctx, "diff_sparse_first", highOffset).Val())

		// DIFF1: y_high & ~x -> 1 at highOffset because x has no bit there.
		require.NoError(t, rdb.Do(ctx, "BITOP", "DIFF1", "diff1_sparse_first", "x", "y_high").Err())
		require.NoError(t, rdb.GetBit(ctx, "diff1_sparse_first", highOffset).Err())
		require.EqualValues(t, 1, rdb.GetBit(ctx, "diff1_sparse_first", highOffset).Val())

		// ANDOR: x & (y1_high | y2_high) -> 0 at highOffset because x has no bit there.
		require.NoError(t, rdb.Do(ctx, "BITOP", "ANDOR", "andor_sparse_first", "x", "y1_high", "y2_high").Err())
		require.NoError(t, rdb.GetBit(ctx, "andor_sparse_first", highOffset).Err())
		require.EqualValues(t, 0, rdb.GetBit(ctx, "andor_sparse_first", highOffset).Val())
	})

return rocksdb::Status::OK();
}

rocksdb::Status Bitmap::BitOp(engine::Context &ctx, BitOpFlags op_flag, const std::string &op_name,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function is a bit too long (stretching from lines 459 to 800). Could you extract some of the logic into a separate function?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants