Skip to content

[wip] chore: use quote-based code generation#135

Draft
nyurik wants to merge 11 commits intooxibus:mainfrom
nyurik:syn
Draft

[wip] chore: use quote-based code generation#135
nyurik wants to merge 11 commits intooxibus:mainfrom
nyurik:syn

Conversation

@nyurik
Copy link
Member

@nyurik nyurik commented Jan 28, 2026

This pull request refactors the codebase to modernize code generation and streamline identifier handling, while removing legacy code that is no longer needed. The most significant changes are the migration to token-based code generation using proc-macro2 and quote, the introduction of new utilities for identifier management, and the cleanup of obsolete formatting and error-handling code.

Migration to token-based code generation:

  • Replaced string/Write-based code generation in FeatureConfig with methods that generate TokenStreams using proc-macro2 and quote, enabling more robust and idiomatic macro code generation. (src/feature_config.rs) [1] [2]
  • Updated dependencies in Cargo.toml to use explicit versions of proc-macro2 and a newer version of quote.

Identifier and naming utilities:

  • Added new traits and helper functions in src/utils.rs for generating and sanitizing Rust identifiers (Ident) from DBC names, including support for enum names, multiplexed variant names, and conversion traits (ToIdent, Tokens).
  • Updated ValType in src/signal_type.rs to implement IdentFragment, supporting its use in macro-generated identifiers. [1] [2]

Removal of legacy and unused code:

  • Removed the PadAdapter struct and associated formatting logic, as well as all Write-based formatting utilities, in favor of token-based code generation. (src/pad.rs)
  • Deleted the custom CanError enum and its Display implementation from the includes module, as error handling is now generated directly in the output. (src/includes/errors.rs, src/includes/mod.rs) [1] [2]

Testing and snapshot updates:

  • Refactored the manual test in tests/snapshots.rs to use in-memory code generation and snapshot testing with insta, reflecting the new code generation approach and ensuring test coverage for generated error types.

These changes modernize the code generation pipeline, improve maintainability, and set the stage for further enhancements to the macro and codegen infrastructure.

@codecov
Copy link

codecov bot commented Jan 28, 2026

Codecov Report

❌ Patch coverage is 92.97125% with 44 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/lib.rs 93.22% 5 Missing and 33 partials ⚠️
src/signal_type.rs 0.00% 3 Missing ⚠️
src/utils.rs 96.29% 1 Missing and 1 partial ⚠️
src/feature_config.rs 87.50% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

src/utils.rs Outdated
fn field_name(&self) -> Ident {
sanitize_name(self.get_name(), ToSnakeCase::to_snake_case).ident()
}
fn field_name2(&self, prefix: &str, suffix: &str) -> Ident {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe more descriptive name? field_name_with_affixes or wrap_field_name?

fn field_name2(&self, prefix: &str, suffix: &str) -> Ident {
format!(
"{prefix}{}{suffix}",
sanitize_name(self.get_name(), ToSnakeCase::to_snake_case)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it possible to use field_name to reduce duplication?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

possibly - i would need to check

Comment on lines +75 to +82
let tmp: String;
sanitize_name(
if suffix.is_empty() {
self.get_name()
} else {
tmp = format!("{}{suffix}", self.get_name());
&tmp
},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about this to get rid of tmp String?

Suggested change
let tmp: String;
sanitize_name(
if suffix.is_empty() {
self.get_name()
} else {
tmp = format!("{}{suffix}", self.get_name());
&tmp
},
sanitize_name(
&if suffix.is_empty() {
self.get_name().to_string()
} else {
format!("{}{suffix}", self.get_name())
},
ToShoutySnakeCase::to_shouty_snake_case,
)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tbh, i am not a big fan of optimizing away a single line of code at a cost of memory allocation. It does make sense for complex logic savings, but here... meh

Copy link
Member Author

@nyurik nyurik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @trnila for looking at it. I was mostly experimenting at this stage -- this whole PR is a draft just to keep everyone posted of the direction. I keep looking at the https://github.com/yasuo-ozu/template_quote to see if something like that would offer significantly cleaner codegen... It won't be perfect, but may offer some clarity...

Comment on lines +75 to +82
let tmp: String;
sanitize_name(
if suffix.is_empty() {
self.get_name()
} else {
tmp = format!("{}{suffix}", self.get_name());
&tmp
},
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tbh, i am not a big fan of optimizing away a single line of code at a cost of memory allocation. It does make sense for complex logic savings, but here... meh

fn field_name2(&self, prefix: &str, suffix: &str) -> Ident {
format!(
"{prefix}{}{suffix}",
sanitize_name(self.get_name(), ToSnakeCase::to_snake_case)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

possibly - i would need to check

@trnila
Copy link
Member

trnila commented Feb 23, 2026

Any updates on whether we will use quote or template_quote? quote seems to be more widely used and maintained, and looks like a nice upgrade compared to string templating...

@nyurik
Copy link
Member Author

nyurik commented Feb 23, 2026

@trnila quote crate is the foundation of all such crates. Template_quote is a tool on top of it with some extras. It is really good, but may get a bit hard to read at times... i'm still experimenting with it

@nyurik
Copy link
Member Author

nyurik commented Feb 23, 2026

P.s. the problem with quote is that it is too verbose - you have to construct each snippet of code ahead of things, and later add it to the result - whereas in template_quote you can do it inline - making the code shorter and more readable ... or less, depending on how crazy it gets

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants