Skip to content

Refactor/use new numbuffer in rust 1.98 - #103

Open
noamteyssier wants to merge 2 commits into
dev-0.9.7from
refactor/use-new-numbuffer-in-rust-1.98
Open

Refactor/use new numbuffer in rust 1.98#103
noamteyssier wants to merge 2 commits into
dev-0.9.7from
refactor/use-new-numbuffer-in-rust-1.98

Conversation

@noamteyssier

Copy link
Copy Markdown
Collaborator

No description provided.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request attempts to replace the external itoa dependency with core::fmt::NumBuffer from the core library to format integers across various reader and block modules. However, core::fmt::NumBuffer is not a valid type in the Rust standard library, and primitive integers do not support the format_into method. To prevent compilation failures, the itoa dependency must be retained, and all changes utilizing NumBuffer and format_into should be reverted back to itoa::Buffer and its format method.

Comment thread Cargo.toml
bytemuck = { version = "1.25.1", features = ["derive", "extern_crate_alloc"] }
byteorder = "1.5.0"
itoa = "1.0.18"
memchr = "2.8.3"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

The type core::fmt::NumBuffer does not exist in the Rust standard library. We must keep the itoa dependency to format integers without allocation.

Suggested change
memchr = "2.8.3"
itoa = "1.0.18"
memchr = "2.8.3"

Comment thread src/bq/reader.rs

// create a reusable buffer for translating record IDs
let mut translater = itoa::Buffer::new();
let mut translater = NumBuffer::new();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

Revert to itoa::Buffer::new() as NumBuffer is not a valid standard library type.

Suggested change
let mut translater = NumBuffer::new();
let mut translater = itoa::Buffer::new();

Comment thread src/bq/reader.rs
for (inner_idx, idx) in (range_start..range_end).enumerate() {
// translate the index
let id_str = translater.format(idx);
let id_str = idx.format_into(&mut translater);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

Revert to translater.format(idx) since format_into is not a method on primitive integers.

Suggested change
let id_str = idx.format_into(&mut translater);
let id_str = translater.format(idx);

Comment thread src/cbq/core/block.rs
is_paired: self.header.is_paired(),
has_headers: self.header.has_headers(),
header_buffer: itoa::Buffer::new(),
header_buffer: NumBuffer::new(),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

Revert to itoa::Buffer::new() since NumBuffer is not a valid standard library type.

Suggested change
header_buffer: NumBuffer::new(),
header_buffer: itoa::Buffer::new(),

Comment thread src/cbq/core/block.rs
/// Preallocated itoa buffer for converting global record index to string
header_buffer: itoa::Buffer,
/// Preallocated NumBuffer for converting global record index to string
header_buffer: NumBuffer<usize>,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

Revert to itoa::Buffer since NumBuffer is not a valid standard library type.

Suggested change
header_buffer: NumBuffer<usize>,
header_buffer: itoa::Buffer,

Comment thread src/cbq/core/block.rs
Comment on lines +834 to +836
fn new(index: usize, buf: &mut NumBuffer<usize>) -> Self {
let mut index_buf = [0u8; 20];
let header_str = itoa_buf.format(index);
let header_str = index.format_into(buf);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

Revert the signature and implementation to use itoa::Buffer.

Suggested change
fn new(index: usize, buf: &mut NumBuffer<usize>) -> Self {
let mut index_buf = [0u8; 20];
let header_str = itoa_buf.format(index);
let header_str = index.format_into(buf);
fn new(index: usize, itoa_buf: &mut itoa::Buffer) -> Self {
let mut index_buf = [0u8; 20];
let header_str = itoa_buf.format(index);

Comment thread src/vbq/reader.rs
block: &'a RecordBlock,
pos: usize,
header_buffer: itoa::Buffer,
header_buffer: NumBuffer<u64>,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

Revert to itoa::Buffer since NumBuffer is not a valid standard library type.

Suggested change
header_buffer: NumBuffer<u64>,
header_buffer: itoa::Buffer,

Comment thread src/vbq/reader.rs
block,
pos: 0,
header_buffer: itoa::Buffer::new(),
header_buffer: NumBuffer::new(),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

Revert to itoa::Buffer::new() since NumBuffer is not a valid standard library type.

Suggested change
header_buffer: NumBuffer::new(),
header_buffer: itoa::Buffer::new(),

Comment thread src/vbq/reader.rs
let mut header_len = 0;
if meta.s_header_span.len == 0 && meta.x_header_span.len == 0 {
let header_str = self.header_buffer.format(index);
let header_str = index.format_into(&mut self.header_buffer);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

Revert to self.header_buffer.format(index) since format_into is not a method on primitive integers.

Suggested change
let header_str = index.format_into(&mut self.header_buffer);
let header_str = self.header_buffer.format(index);

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant