Skip to content

fix: avoid NoClassDefFoundError in supportCodegenForJavaSerialization#3424

Merged
chaokunyang merged 2 commits intoapache:mainfrom
siy:fix/codegen-enclosing-class-check
Feb 26, 2026
Merged

fix: avoid NoClassDefFoundError in supportCodegenForJavaSerialization#3424
chaokunyang merged 2 commits intoapache:mainfrom
siy:fix/codegen-enclosing-class-check

Conversation

@siy
Copy link
Contributor

@siy siy commented Feb 25, 2026

Summary

CodegenSerializer.supportCodegenForJavaSerialization() calls cls.getEnclosingClass() before checking Modifier.isStatic(). In classloader-isolated environments (OSGi, custom module systems with per-module classloaders), the enclosing class may not be visible from the class's classloader, causing NoClassDefFoundError.

Two issues fixed:

  1. Evaluation order: Check Modifier.isStatic() first — static nested classes support codegen regardless of enclosing class visibility, so there is no need to load it.

  2. Error handling: When getEnclosingClass() throws, return false instead of re-throwing as RuntimeException. If the enclosing class cannot be loaded, the class cannot function as a non-static inner class in this context, so codegen is simply not applicable.

Reproduction scenario

  • Class Outer.InnerRecord is a static nested record
  • InnerRecord is loaded by ClassLoader A
  • Outer is only visible to ClassLoader B
  • Current code: getEnclosingClass()NoClassDefFoundErrorRuntimeException (crash)
  • Fixed code: isStatic()true (correct result, no class loading)

Check Modifier.isStatic() before calling getEnclosingClass() to avoid
loading the enclosing class unnecessarily. In classloader-isolated
environments (OSGi, custom module systems), the enclosing class may
not be visible from the current classloader, causing NoClassDefFoundError.

Also return false instead of re-throwing when the enclosing class
cannot be loaded — if the enclosing class is not resolvable, the class
cannot be a valid non-static inner class in this context.
@siy siy requested a review from chaokunyang as a code owner February 25, 2026 22:12
…on()

Swap evaluation order so that when codegen is disabled,
supportCodegenForJavaSerialization() is never called. This avoids
triggering getEnclosingClass() in classloader-isolated environments
where withCodegen(false) should completely bypass codegen logic.

Both call sites in getSerializerClass() and getObjectSerializerClass()
had the same issue.
@siy siy mentioned this pull request Feb 25, 2026
3 tasks
Copy link
Collaborator

@chaokunyang chaokunyang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@chaokunyang chaokunyang merged commit de152fa into apache:main Feb 26, 2026
58 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants