Make Grails applications processable by Spring AOT (Leyden AOT Cache + GraalVM Native Image Support) - #16094
Make Grails applications processable by Spring AOT (Leyden AOT Cache + GraalVM Native Image Support)#16094codeconsole wants to merge 115 commits into
Conversation
filteringCodecsByContentTypeSettings took the live GrailsApplication as a
constructor argument, and groovyPagesServlet took a new GroovyPagesServlet().
Spring AOT's ValueCodeGenerator cannot emit code for an arbitrary object, so
processAot aborted the entire run:
UnsupportedTypeValueCodeGenerationException:
Code generation does not support grails.core.DefaultGrailsApplication
Both now use forms the generator understands: a reference by bean name, as
errorsViewStackTracePrinter directly above already does, and an inner bean
definition. This was the first processAot failure for a stock web application
on 7.2.1, 8.0.0-M4 and 8.0.0-SNAPSHOT alike. Further blockers remain behind it,
so this does not by itself make an application AOT-processable.
Constructing the servlet through the registry rather than with new means it now
passes through bean post-processing. Its pluginManager is unaffected -
initFrameworkServlet already autowires the servlet's properties by type, which
covers that setter - but a post-processor whose pointcut matched the servlet
would hand ServletRegistrationBean a proxy in its place. No pointcut in the
framework does.
CoreGrailsPlugin registers a second ConfigurationClassPostProcessor so that @configuration beans contributed by plugins through doWithSpring - which arrive after Spring's own processor has finished - still get parsed. An AOT-optimized context has no ConfigurationClassPostProcessor at all: the configuration classes were parsed at build time and their bean definitions are in the generated initializer. Registering one there parses them a second time, and the re-parse collides with what AOT already emitted: BeanDefinitionStoreException: Invalid bean definition with name 'propertySourcesPlaceholderConfigurer' defined in org.grails.plugins.CoreAutoConfiguration: Bean name derived from @bean method 'propertySourcesPlaceholderConfigurer' clashes with bean name for containing configuration class For a static @bean method AOT emits the bean definition with the configuration class as its bean class, so the re-parse rediscovers CoreAutoConfiguration under the bean name propertySourcesPlaceholderConfigurer and then collides with itself. The processor is skipped when AotDetector.useGeneratedArtifacts() reports generated artifacts are in use; behaviour without AOT is unchanged. This was reachable only after the GSP live-instance beans were fixed, since processAot did not previously get far enough to produce an initializer.
An abstract bean definition is a template: it carries property values for
children to inherit and is never instantiated. Spring's bean-definition code
generator has no representation for one - BeanDefinitionPropertiesCodeGenerator
emits lazyInit, primary, scope, role and synthetic, but not the abstract flag,
and nothing filters abstract definitions out beforehand. The definition is
regenerated as a concrete bean of type Object still carrying the template's
properties, and the context fails applying them:
BeanCreationException: Error creating bean with name
'abstractGrailsResourceLocator': Invalid property 'searchLocations' of bean
class [java.lang.Object]
CoreGrailsPlugin contributes exactly such a template through
AbstractResourceLocatorPostProcessor, and it is public surface - third-party
plugins inherit from it with bean.parent, asset-pipeline's assetResourceLocator
among them - so it cannot simply be removed.
A BeanRegistrationExcludeFilter registered in META-INF/spring/aot.factories
keeps every abstract definition out of generation. Children are unaffected,
because AOT generates them from their merged definition with inherited values
already folded in; a definition contributed dynamically still finds its parent,
since the post-processor that registers the template runs during refresh in an
AOT context as it does in any other.
With this an AOT-processed application starts. Its bean definitions differ from
a normal boot only by the annotation-processing infrastructure AOT replaces:
the configuration, autowired and common-annotation processors, Boot's shared
metadata reader factory, and grailsConfigurationClassPostProcessor.
SiteMeshViewResolver now implements ServletContextAware, so the bean-definition
wrap no longer passes the servlet context as a constructor argument. That
reference was to a bean the container only registers once the web server has
started, which has no bean definition for AOT to resolve, and it failed
processAot for every application with GSP on the classpath:
AotBeanProcessingException: Error processing bean with name 'jspViewResolver'
Caused by: NoSuchBeanDefinitionException: No bean named 'servletContext' available
GrailsSiteMeshViewResolver gains the matching three-argument constructor and
reads the context through the inherited accessor rather than keeping its own
copy. The four-argument constructor stays for callers that build the resolver
directly, which is how the instance-level post-processor still creates it.
This was the last blocker: a stock Grails web application now completes
processAot and starts with spring.aot.enabled=true. Its bean definitions differ
from a normal boot only by the annotation-processing infrastructure AOT
replaces - the configuration, autowired and common-annotation processors,
Boot's shared metadata reader factory, and grailsConfigurationClassPostProcessor.
The sitemesh version is moved to 3.3.0-SNAPSHOT because the change it depends on
is not in 3.3.0-M3. It must be pinned to a released version before this merges.
AOT support is opt-in and needs configuration that is not guessable - the Spring Boot AOT plugin, and generation in production mode, without which the url mappings holder takes its reload-mode proxy shape and generation fails naming a bean unrelated to anything in the application. The deployment guide now covers enabling it, what changes at runtime, and the limitations: registrar-contributed beans are not generated, definitions holding live objects cannot be generated, abstract definitions are excluded, and AOT alone does not make an application native-image ready. Two layers of coverage, because the failures differ in kind. CoreGrailsPluginAotSpec runs the real generator over the core plugin's bean definitions, so a definition holding a live instance fails the build. It also covers the configuration class post-processor being registered normally and withheld when generated artifacts are in use, which no existing test reached. A fourth case registers a definition holding a live instance and asserts generation rejects it, so the check above cannot pass vacuously. Generation succeeding does not mean the application starts, and the post-processor condition is a runtime-only behaviour. The grails-test-examples aot application therefore runs processAot and then starts the packaged jar with spring.aot.enabled=true, asserting the beans an AOT context must still contain. Its check task fails the build if either half regresses.
Spring AOT generates the bean definitions for a context at build time, and it can only do so for beans the container knows how to build. A bean registered as an already-constructed object has nothing to generate from, so the MongoDB datastore could not be processed ahead of time. The MongoDB initializer built the event publisher itself and passed the instance to the datastore constructor. It now registers the publisher as a definition and refers to it by name, so the container builds both. ConfigurableApplicationContextEventPublisher could only be built by passing a context to its constructor, which is the thing a definition cannot do. It now also takes the context from the container through ApplicationContextAware, leaving the existing constructor in place for callers outside a container. Which publisher gets registered still depends on where GORM is being bootstrapped: outside an application context the no-op publisher is used, as before, because the context-aware one would never be given a context there and would fail on the first event it published.
The welcome page shows the Spring Security version only when the dependency is
present, and reached it with Class.getMethod('getVersion').invoke(null). Calling
Method.invoke from a GSP expression goes through Groovy's dynamic dispatch, which
resolves to the private caller-sensitive overload the JDK added for core reflection.
A native image cannot supply the caller argument that overload requires and aborts
the process rather than raising an exception.
Spring's ReflectionUtils performs the invocation from Java, so Groovy never
dispatches on Method.invoke itself. The presence check is unchanged, which keeps the
page compiling for generated applications that did not select Spring Security.
Both copies of the page are updated: the one the profiles CLI writes into a new
application and the one grails-forge serves.
The file was added without one, which fails the release audit.
A scaffolded controller had no views of its own: the resolver expanded a template into GSP source and compiled the result the first time each view was asked for. That costs the first request, and a native image cannot do it at all, because defining a class at run time is what an ahead-of-time image gives up. The templates are now expanded during the build and compiled with the rest of the views, so at run time they are found rather than produced. Only naming is substituted, and the templates defer everything about the domain class to the field tag libraries at render time, so this needs no GORM, no application context and no loading of application classes: the controllers are read with ASM and the domain class name is enough. The generated pages are staged with the application's own before a single compilation. Compiling them separately would produce a second gsp/views.properties, and the archive tasks discard duplicates, so one of the two manifests would be lost and the views it listed would never be found. A view the application declares is not generated over, which keeps a hand-written page ahead of a scaffolded one as it is at run time.
Handling a request reaches the controller, URL mapping, data binding and content negotiation APIs through Groovy's dynamic dispatch, which reads a type's declared methods to choose an overload. An ahead-of-time image keeps only the members something asks for, so these were stripped and dispatch failed at the point of use. Leaving this to the tracing agent does not work, because the agent records only the paths that were exercised. Each of these failed on a path an ordinary check misses: content negotiation runs only for a request that states what it accepts, the method check only for POST, PUT and DELETE, binding only for a request carrying data, and the parameter accessor only once a mapping carries parameters. All four passed a page walk and failed for a real visitor. Each registrar sits in the module owning the API and names its types as strings, registering only those present, so an application that does not use a given plugin is unaffected. None of this varies between applications, which is why it belongs with the framework rather than in every application's metadata.
Calling a closure goes through doCall, and Groovy reads its parameter types reflectively to choose an overload. In an ahead-of-time image those are stripped unless something asks for them, and the call then fails where the closure is used rather than at start-up. The framework ships thousands across its plugins, so naming them would be neither complete nor stable, and leaving it to each application means every one of them rediscovers the same list. They are found while the hints are written instead. That happens during the build, on an ordinary JVM with the whole classpath, so scanning is available; it is only the image that cannot do it. Two kinds of closure are deliberately left out, both of which otherwise fail the build rather than degrading at run time. One is a closure whose declaring class does not resolve: the GSP compiler's task extends an Ant type absent at run time, and its closures reach it through invokedynamic, so nothing in their own bytecode reveals the dependency. The other is a closure naming an absent class in a method body: the JSP closures link happily against a runtime with no JSP API, because loading a class resolves its signatures and not the bodies the image analysis goes on to parse.
Grails reaches an application's artefacts reflectively, and a precompiled page is looked up by the name recorded for its view, so a native image that keeps only the members something asked for leaves both present but unusable. Until now an application had to run under the tracing agent to discover them, which records only the paths that were exercised: a page nobody visited during the trace is a page missing from the image. The build already knows the answer. The compiled classes are on disk and the pages are named in the manifest the GSP compiler writes, so both are read directly. Pages a plugin contributes are read from the manifest inside its artifact as well, because an application renders those as readily as its own and they are not in its own build output. The closure registration is widened at the same time. A plugin declares its descriptor in whatever package it chooses, and the bean definitions in that descriptor are closures the container calls while the context is built, so scanning only the framework's own packages missed them.
A rendered page and a persisted entity are both reached through Groovy, so an image that keeps only the members something asked for serves part of an application and fails on the rest. What fails depends on what was exercised: a page renders until it reaches a tag that was stripped, and a datastore reads until something writes. Three registrars cover the runtimes an application does not own: Tag libraries and the page runtime, found by name wherever a plugin declares them. A flash message only exists after a redirect that set one, and a field is only rendered by a form, so a walk of an application's pages exercises neither and both fail for the first person who edits a record. The scaffolded resource controller, including the protected methods it defines for the write operations, which serving its pages never reaches. The persistence runtime, registered by package so that a datastore's own persisters and query support are covered without the shared module naming any of them. Fields are registered as well as methods: a persister hands work to anonymous inner classes that read the state they captured as properties, which is how removing an entity reaches its session. Each skips a type that does not resolve. The JSP integration and the datastores are compiled against APIs an application need not have, and registering a type the image analysis then cannot parse fails the build rather than degrading at run time. The two build tasks state their dependency on the compilation rather than inferring it from the classes directory, which more than one task writes to.
Matching only the library by name left the classes it declares inside it stripped, so a page rendered until a tag reached one. The fields plugin keeps the bean stack that a nested tag reads its subject from in such a class, which is why a list or a form failed while the pages around them were fine. Found by driving the pages in a browser. An automated check that asks for a URL and looks at the status never nests a tag deeply enough to reach it.
Whether the pages compiled at build time are used is decided by whether a development environment is available, which comes down to whether the application looks like a project on disk. That is the right question for a running JVM, where skipping them lets an edit take effect without a restart. It is the wrong question for an ahead-of-time image. Such an image is a single executable that may be run from anywhere, including the directory it was built in, where the sources are still present and the answer is yes. It then read those sources and tried to compile them, which is the one thing it cannot do, so every page failed. Running the executable from somewhere else worked, which made this look like a property of the working directory rather than of the check. Both the plugin that loads the compiled page registry and the locator that reads it now also accept an image, which cannot compile a page whatever its surroundings suggest. Development is unchanged. The registry is loaded from a plugin descriptor, and a descriptor is Groovy, so the call that asks whether this is an image is itself dispatched dynamically and the class it is made on has to survive into the image.
Each registrar decided for itself whether a type it found could be kept, and the two checks that decision needs were rediscovered a piece at a time, wrongly, three times over. Registering a type that cannot be loaded does not degrade at run time: it fails the image build, because the analysis parses what it is asked to keep. Either check alone lets one case through, which is what made this easy to get wrong. Whether the type and the class declaring it load covers a closure whose enclosing class extends something absent, which the closure never names itself because it reaches it through invokedynamic. Whether the types its bytecode names load covers the opposite, a class that loads cleanly while a method body names an absent one, because loading resolves signatures and not bodies. Both now live with the reason each exists, where a registrar for a new part of the framework will find them.
The task had no tests. Two of these cover what went wrong while it was being written: the pages a plugin contributes, whose absence stopped an application starting because it renders those as readily as its own, and the rule that the manifest rather than the directory listing decides which pages exist, so a class left behind by an earlier build is not recorded.
The registrars were written without the style checks having been run, and they indented array initialisers one level too far and put their imports in the wrong group. Behaviour is unchanged.
The expansion staged the views into a directory and pointed the compilation there, for every project. A project with no scaffolded controller then copied its views on every build and compiled them from somewhere other than where they are, for nothing. Whether anything is scaffolded is read from the controller sources, because the decision is needed before anything has been compiled. A project that does not scaffold is left exactly as it was.
Generating a context ahead of time normally makes the injection annotations unnecessary: the generator reads them and writes the field and method access into the code it emits. Two kinds of injection did not survive that, and neither failed at start-up -- the first request that reached the bean did. A bean's autowire mode was not written out at all. Grails registers much of what it contributes as autowired by name, so a tag library arrived holding null where it expected a message source. The mode is now carried into the generated definition, for whatever set it. An annotated member is only generated for when the generator can see the class declaring it. A bean contributed as an interface built by a supplier hides that class -- the link generator is declared as LinkGenerator and built by a closure -- so its annotated field was injected by nobody. The two processors that read those annotations are registered when running on generated artifacts. Only those two: registering the whole set brings back the processor that reads configuration classes, and reading them again in a context whose configuration is already generated makes a second definition for beans the generated code has contributed, which fails the context outright. (cherry picked from commit dbe0b2f9ff1f205266e0f1b2c9202baf708ad616)
The GSP plugin registered a bean for every tag library from doWithSpring(), which meant registering them again on every start, over whatever was already there. On a running JVM that only made noise. On a context generated ahead of time it replaced definitions that carried the injection the generator had worked out, leaving a tag library holding null where it expected a collaborator, and by-name autowiring did not stand in for it because those collaborators are fields. The definitions are contributed by a post-processor instead, which leaves an existing definition alone. The artefacts are read from the application rather than named, so a tag library still belongs to whoever declared it: the application, another plugin, or one supplied through providedArtefacts. Where a definition is already there but has lost its autowire mode, the mode is raised back to by-name and never lowered, so a definition asking for something else keeps it. (cherry picked from commit 03c353953382fbf50a7924dcb5151f9618bb2591)
A bean declared through the plugin DSL passes its arguments positionally, and a
constructor ending in a variable-argument parameter is called the way the language
allows: one value where the parameter is an array, or a collection where it is an
array of that element type. Building the bean, Spring adapts the argument to the
parameter. Reading the definition to generate code for it, Spring does not -- it
looks the argument up by the parameter's type, and a lone String does not answer
to String[].
The argument is then missed and resolved as a dependency instead, and an array of
a type nobody publishes as a bean resolves to an empty array rather than failing.
So the bean is built, and built wrong: a datastore that maps no classes, or a
servlet registration with no URL mapping, which then falls back to mapping
everything and swallows every request. Nothing is logged, and the bean that goes
wrong is rarely the one that reports it.
Gathering the argument ahead of time means the generator writes out
new String[] {"*.gsp"}, which the lookup does find. An argument already usable as
the array is left alone, and one whose elements would need converting is left to
the resolution that exists today rather than guessed at here.
The tests read the code that is actually generated, because the failure this
guards against is one where every bean is still registered and still built.
(cherry picked from commit b91d09ee1c3b8dcd1ef97e9b025c642705f13366)
A datastore was given the configuration object itself to hold, and a definition holding a property resolver is a definition holding everything that resolver can reach. Generating code for it wrote those values out: the environment of whatever machine ran the build, 223 entries of it in the case at hand, credentials among them, committed into generated source. Worse, the application then read its settings from there rather than from where it runs, so a value that is meant to differ between build and run -- a host, a port, a password -- did not. While code is being generated, and only then, the context's environment is named instead, so the lookup is made where the application runs. Every other time the resolver is held as it always was, which matters for a datastore brought up on its own: its configuration is whatever the caller passed, and there is no environment holding it. The classes a datastore maps are also collected as the array its constructor takes, rather than as a collection that only becomes one because Spring adapts it. Verified on an ahead-of-time application: nothing of the build machine appears in the generated sources, and the datastore resolves its address at startup. (cherry picked from commit f437fdec01ccc40da7b6aa0ef787c336039bbd63)
Running on generated artifacts, the plugins whose doWithSpring() produced these definitions have already run: they ran while the artifacts were being generated, and what they registered was written out as code and registered again from that code, ahead of this phase. Registering over it discarded the instance supplier the generator wrote -- which resolves the constructor, the fields and the injection methods up front -- and replaced it with a definition that has to find all of that by reflection, which is what a generated image does not carry. So the generated definition stands and the one contributed here is dropped. Anything the generator did not produce a definition for is registered as usual, so a bean a plugin contributes conditionally is unaffected, and on a normal start nothing is skipped: a plugin overrides what came before it exactly as it did. This is one guard where the definitions land rather than one at each call site, so it holds for the early registration phase and the post-processor alike. An ahead-of-time application went from 27 overridden definitions to none. The one remaining message is Spring replacing its own import-aware post-processor with an equivalent definition, which comes from two configuration class post-processors each contributing the same registration. grails-spring had no test task wired, having had no tests; it now applies the shared test configuration. (cherry picked from commit f21062e981349f7eb4962fd95b4214f0add5bc47)
The core plugin registers a ConfigurationClassPostProcessor so that @configuration beans contributed by plugins get parsed. A context that annotation configuration has already been set up on has one of its own, and the plugin definitions are registered ahead of it, so it sees them: the second processor parses the same registry over again. While code is being generated the two of them each contribute a registration of the import-aware post-processor, so the generated initializer registers it twice under one name and the second replaces the first on every start -- the last overriding definition message an ahead-of-time application reported. It is now registered only where there is none: a context assembled without that step, such as a test slice registering this plugin's beans on a bare registry, which is what it was for. An ahead-of-time application now starts with no overridden bean definitions at all. (cherry picked from commit fa92bd33942f3ebdf5d76ae200e044cd3ab95374)
Both decorate the same generated block of definition properties, one after the other, and no bean in the application under test happens to draw on both, so nothing was proving they compose. A bean that is autowired by name and takes a variable-argument constructor argument now shows both contributions surviving. (cherry picked from commit 65ce6fbe9e52b7bc197c896118891aec08085296)
A plugin declares them in a closure, and Groovy resolves each call in a closure where the call is written rather than when the closure is compiled, so every call on the registry it is handed is made reflectively. An image keeps a method for that only when something has asked it to, and nothing did: the registry belongs to Spring, an application never names it, and the closures that call it are registered as closures rather than for what they call. So the image refused the first call and the context did not start, reporting a method of an interface that appears nowhere in the application. It stood only as long as an application's own traced metadata happened to cover the framework's API, which is not the application's to know. Verified by a native image, which now starts. (cherry picked from commit c4cfdaf7e8997fe71b56720c61497ce2b0374228)
Three mechanisms answered the same question. grails.env=production is set on
processAot by the Gradle plugin; the same property was set again by hand in the
example application; and UrlMappingsGrailsPlugin asked separately whether code was
being generated.
Only the first is load-bearing. With it set, the plugin's own check changes nothing
-- the environment is already not a reloading one -- so the check, the two seams it
was reached through and the tests that covered them are gone, and that file is back
to what it was. The example application no longer repeats what the plugin does.
The check in the GSP plugin is not the same and stays. It asks whether the code is
being generated, which is not a question about the environment:
isDevelopmentEnvironmentAvailable() asks whether grails-app is on disk, and while
generating it is. Removing it does not fail the build -- it writes the directory
the build ran in into groovyPageResourceLoader:
addPropertyValue("baseResource", "file:/home/build/app/")
which starts cleanly on the machine that generated it and looks for its pages in a
directory that does not exist anywhere else. A check on the generated sources now
fails the example application's build if anything writes a build path into them,
since nothing else would say so.
The guide no longer asks for a property the plugin already sets.
Three places read SpringProperties.getFlag(AbstractAotProcessor.AOT_PROCESSING), each carrying its own explanation of what the flag means. Spring sets it around its own processing and offers no predicate for it, so the reading of it now lives in grails-common beside the other shared ahead-of-time code, which is where GORM and the framework both already reach. AbstractDatastoreInitializer.configurationReference also moves out from between two field declarations to sit with the other methods.
A version that could not be determined was left out either way. Leaving out a default is what lets one be shown at all -- an application without Spring Security says nothing about it rather than saying it does not know. Leaving out one the application named itself is different: it reads as the option having been ignored, which is the very thing an unrecognised option is now warned about. So a version asked for by name reads unknown, and only the defaults are left out. The version lookups move out of createBannerVersions into versionsFor, since deciding this per option needs the option that produced a label. Also gone: colouring an image's output when TERM is set. It cannot tell output being watched from output being redirected, so it wrote escapes into a redirected log; an image that should be coloured can be told so with spring.output.ansi.enabled, which is Spring Boot's own property and needs nothing here. The two seams that existed for a spec to override go with it. And VarargsBeanRegistrationAotProcessor no longer catches Throwable. It meant to tolerate a bean class that will not resolve, which is an Exception; an Error is a JVM out of memory or a class that will not link, and neither is a bean without a variable-argument constructor. A debug line is left so a genuinely broken bean can be traced to here.
extractAotCacheApplication was an Exec that deleted through the project and built its command line in doFirst, which is Project access at execution time and a hard error under the configuration cache. It is a task type now, with the archive as an input, the extracted application as an output, and the deleting and running done through injected FileSystemOperations and ExecOperations. archiveOf resolved the bootJar task to read its archive, so every build that merely configured this project paid for resolving it. It is a Provider<RegularFile> now, which the trace and training tasks take as well. The assets are attached to bootJar through a file collection that matches the task by name rather than through afterEvaluate and findByName. The asset pipeline's plugin id has changed once already and its task name has not, so matching by name is what stays true -- while nothing is resolved to find out whether it is there. The precompiled page manifest is attached only where those are the pages that will be rendered: a deployed application, and one whose code is being written out. A manifest left on the classpath by an earlier build, or shipped inside a plugin, no longer stops a page being reloaded as it is edited in a project on disk. Covered both ways by a spec.
classes() reads a singleton off the context, and whether the context is there when it is asked was left for the reader to work out. It is, on both paths that ask: the early registration phase sets it on the instance it creates, and the @bean method is invoked after Spring has applied ApplicationContextAware. It has to be -- the scan beside it resolves resources through the context and did so before any of this was written -- so nothing new is being relied upon. Said in the javadoc now, and covered by a spec on classes() rather than on the method behind it. It reads the bean factory rather than getAutowireCapableBeanFactory(), which refuses a context that has not been refreshed. This is asked while definitions are still being contributed, which is the reason the classes are left as a singleton at all, so asking must not depend on how far the context has got. And a spec for the other end of the same question: a resource locator that inherited nowhere to look still finds a packaged resource, which is what makes emptying the inherited search locations safe rather than merely quiet.
|
The failures are not snapshot resolution. This is due to a circular dependency on #16139 Removed in e93d383, together with the |
There was a problem hiding this comment.
Thanks — I went through all eleven threads against b22be60 rather than against the replies, and every one is genuinely fixed. Resolved them all.
Spot checks, for the record: GrailsEnvironmentPostProcessor and UrlMappingsGrailsPlugin are byte-identical to the merge base; every new runtime AOT class is under org.apache.grails.*; isGeneratingCode() has three call sites and no remaining copies of the flag read; versionsFor shows unknown only for an option named by the application, with GrailsBannerNativeMarkSpec covering both directions; GroovyPagesGrailsPluginPrecompiledSpec asserts the manifest is absent in development and present when deployed; and ExtractApplicationTask reaches nothing through Project at execution time.
What follows is new, from a pass over the production code as it now stands rather than a re-reading of the earlier points. One blocker and a set of should-fixes; nothing that undoes the design.
The blocker is the generateNativeMetadata ordering: it is wired ahead of the task that produces the pages it reads, so the page half of the metadata is always empty. Nothing executes that task in a test, which is why it is green.
The rest are Gradle-side robustness (two configuration-cache captures that survived the last round, fixed ports with a liveness check that cannot tell one application from another, a cache key that pins the build machine, an unclosed HttpClient), and three on the GORM side, of which the Neo4j one matters most: that path still writes the build machine's environment into generated source, and I would rather it refuse to generate than do that quietly.
| // changes that input and the task can never be up to date -- every build would run | ||
| // the application again to record what the last one already recorded. | ||
| Provider<Directory> beside = project.layout.buildDirectory.dir('aot-cache') | ||
| task.cacheFile.set(beside.map { Directory dir -> dir.file("${project.name}.aot") }) |
There was a problem hiding this comment.
project is captured in a provider that is evaluated after configuration.
beside.map { Directory dir -> dir.file("${project.name}.aot") } reads project.name when the provider is queried — at property finalization or execution — so the closure carries the Project into the task's serialized state. That is the same class of problem as the doFirst block from the last round, just deferred through a map instead.
org.gradle.configuration-cache=false in this repo's gradle.properties, so CI will not catch it; a consumer building with the cache on will. Hoisting String archiveName = project.name outside the closure fixes it.
There was a problem hiding this comment.
0021e820a5. String cacheName = project.name hoisted out of the beside.map { } closure.
|
FYI @codeconsole 5.2.0-M3 is released on asset pipeline |
|
On your open question from the package-move thread: leave the Gradle plugin classes in |
A cache records the classpath it was trained against. The run was given the
archive by file name, so what it wrote down was a bare name, and a bare name is
resolved against the working directory of whatever starts the application later.
The cache was therefore usable only from the extracted directory. Started any
other way -- `java -jar /opt/app/app.jar` from a service unit, or an image whose
working directory is not the one the archive sits in -- the JVM reports
Required classpath entry does not exist: app.jar
ignores the cache and starts as it would have anyway, so the deployment quietly
loses the startup time the cache exists to save. Measured on a demo application,
1201 ms from the extracted directory against 2509 ms from anywhere else.
An absolute entry is matched by prefix substitution instead, which survives both
a different working directory and the directory being moved -- the latter being
what deploying it does. The run is still made in the application directory, so
the archive still finds what sits beside it.
Two configuration-time slips in the same method. The cache file name was built inside a provider closure that read project.name. A provider is queried when the property is finalized or the task runs, so the closure carried the Project into the task's state, which the configuration cache refuses. This repo sets org.gradle.configuration-cache=false, so CI would not have caught it; a consumer building with it on would. And whether compileGroovy and copyAstClasses exist was asked with findByName, which creates the task in order to answer -- so a build paid for tasks it may not have depended on. The names answer the same question without realizing anything.
Both the training run and the trace waited on contains('Started '), which a
container satisfies long before the application is ready: Jetty logs
"Started ServerConnector@..." and "Started Server@..." while the context is
still being built. A run taken for started there is trained or traced half
built, and what it records is quietly thinner than it should be -- a slower
start, or a class missing from an image, rather than anything that failed.
Spring Boot's line has a shape no container's does -- a name, "in", and a
number of seconds -- and that is what is now matched.
The same read was taking the whole log every 250ms in the platform default
charset. StartupLog reads what has been added since it last looked, as UTF-8,
and matches whole lines so one still being written is left for the next look
rather than cut in half and missed.
HttpClient owns a selector thread and an executor, and the daemon outlives the build -- so a client left open is a set of threads left running, and another set for every later build that traces again. Its javadoc claimed nothing was left behind in the daemon, which was the thing that happened.
generateNativeMetadata reads build/gsp-classes/main, which compileGroovyPages
writes. It reached the build through processResources -- and compileGroovyPages
depends on classes, which runs processResources, so consuming the task there put
it ahead of all three:
generateNativeMetadata -> processResources -> classes -> compileGroovyPages
The directory it read was therefore empty on a clean build and stale on any
other. Recording the pages is half of what the task exists for, and it recorded
none of them; an image built from that metadata is missing every page, which
shows up as a page not found at runtime rather than as anything at build time.
It was also an undeclared dependency on another task's output, declared
@InputFiles on a @CacheableTask.
No ordering satisfies both directions, so the output now goes to what runs after
the pages are compiled: bootJar, under BOOT-INF/classes as the pages themselves
are, and the classpath the image is built from. Putting it in the source set
output instead would restore the cycle, because the classes task builds those.
Nothing executed this task in a test, which is why it was green. Two now do: one
asserts compileGroovyPages runs before generateNativeMetadata, the other that
processResources no longer consumes it and bootJar does. Both fail against the
previous wiring.
project.files(tasks.matching { }) keeps a live TaskCollection, and handing it to
bootJar made the task container -- and the project behind it -- reachable from
the archive's input files. Resolving those inputs also ran the predicate against
every task registered, realizing all of them to find one.
Asking the names costs nothing and realizes nothing; only the task that is
actually there is then looked up, and the file collection it returns carries the
dependency on it. Still by name rather than by the plugin that registers it: the
pipeline's plugin id has changed once already, the task name has not, and an
application is free to register the task itself -- which is what the spec for
this does.
Whether the application has started is answered in part by connecting to its port, and a connection cannot tell one application from another. So anything already listening on 18080 or 18081 -- a second build on the same CI agent, a developer's own server, a run left behind by a cancelled build -- answered on the application's behalf: the wait ended at once and what was trained or traced was a recording of somebody else's application, reported as a success. The port is now bound before the run is started, and a port that is busy fails the task with the port and the setting that names it. It is not proof against a race, but the case worth catching is a port that is already busy.
An absolute JDK path declared as an @input on a @CacheableTask puts the machine that ran the build into the cache key, so the task could never be hit on CI or on another developer's machine -- which is most of what the annotation is for. The launcher is nested instead, as GroovyPageForkCompileTask already does, so what is fingerprinted is what the launcher says about itself. It is still set as the provider it is, so reading the project does not provision a JDK.
Three things, now that this publisher is built by the container rather than handed a context. The cast in setApplicationContext was unchecked, so a context that is not configurable threw a ClassCastException out of a container callback -- out of the container's own setup rather than out of anything this was asked to do. It is narrowed instead, and left unset. Nothing guarded against the field staying null. findEventPublisherClass picks this class when either the registry or resourcePatternResolver.resourceLoader is a ConfigurableApplicationContext, but the bean is built by whoever owns the registry -- so where the registry is a plain DefaultListableBeanFactory and only the resource loader is a context, ApplicationContextAware is never applied and the first publishEvent was a NullPointerException from inside GORM. It now says which situation it is in. And the field is volatile: it stopped being final when the no-argument constructor arrived, and it is written by whichever thread refreshes the container and read by whichever one publishes.
…ration The datastore is given the resolved configuration rather than a reference to the environment, because the method that names the environment was added to AbstractDatastoreInitializer and this build resolves that class from a released artifact. Generating code for that definition writes out what it holds -- the build machine's PropertyResolver, its environment variables among them -- into the application's generated source, and ships it. Nothing said it had happened. Refusing is the lesser outcome: an unsupported combination that says so can be worked around, one that quietly ships the builder's environment cannot be noticed. Also both halves of the event publisher condition. Four initializers register grailsDatastoreEventPublisher under one name and whichever drains last defines it for all of them; the other three accept a resource loader that is a context, so testing only the registry here could leave a Hibernate-and-Neo4j application publishing Hibernate's events through a no-op -- GORM events stopping and auto-timestamping stopping, with nothing logged.
org.grails.gradle.plugin.aot -> org.apache.grails.gradle.plugin.aot, for the eight classes this PR adds and their specs. They stay together in an .aot package beside the plugin they belong to; only the root changes. grails-gradle/common already publishes org.apache.grails.gradle.common, so this is the direction that module was already going.
org.grails.plugins.web.TagLibBeanDefinitionsPostProcessor -> org.apache.grails.gsp.taglib.TagLibBeanDefinitionsPostProcessor, with its spec. Newly added code, so it goes under org.apache.grails like the rest of what this PR adds. Not under an .aot package: the reason it exists is that doWithSpring re-registered over generated definitions, but GroovyPagesGrailsPlugin registers it unconditionally and it runs in every environment.
There was a problem hiding this comment.
Assume you'll be updating this version in this PR? I've released your changes on the asset pipeline ...
There was a problem hiding this comment.
Yes — 6d468648a8. asset-pipeline-gradle.version and asset-pipeline-bom.version both 5.2.0-M1 -> 5.2.0-M3.
The release carrying the changes this PR depends on. Both pins move together: asset-pipeline-gradle for the build plugin, asset-pipeline-bom for what the application resolves.
✅ All tests passed ✅🏷️ Commit: 6d46864 Learn more about TestLens at testlens.app. |
Depends on
wondrify/asset-pipeline#465
Makes a Grails application processable by Spring AOT. A stock web application now completes
processAotand starts withspring.aot.enabled=true:Apply the Spring Boot AOT plugin to opt in:
AOT is also the gate on GraalVM native images: Boot refuses to start a native image without a build-time generated initializer, so nothing native was reachable before this.
What was blocking it
Four independent mechanisms, each only visible once the previous was cleared.
Live object instances in bean definitions.
GroovyPagesGrailsPluginpassed the liveGrailsApplicationas a constructor argument and anew GroovyPagesServlet()instance. A bean definition is a recipe;ValueCodeGeneratorcannot emit source that reconstructs an arbitrary object:Both now use forms the generator understands — a reference by bean name and an inner bean definition. This was the first
processAotfailure on 7.2.1, 8.0.0-M4 and 8.0.0-SNAPSHOT alike.A second
ConfigurationClassPostProcessor.CoreGrailsPluginregisters one so that@Configurationbeans contributed by plugins throughdoWithSpring— which arrive after Spring's own processor has finished — still get parsed. An AOT context has none of its own by design, and adding one back parses the configuration classes a second time, colliding with what AOT already emitted:For a static
@Beanmethod AOT emits the definition with the configuration class as its bean class, so the re-parse rediscoversCoreAutoConfigurationunder the bean's own name and collides with itself.Abstract bean definitions.
BeanDefinitionPropertiesCodeGeneratoremitslazyInit,primary,scope,roleandsyntheticbut not the abstract flag, and nothing filters abstract definitions out beforehand — a template is silently regenerated as a concreteObjectbean still carrying the template's properties.CoreGrailsPlugincontributes exactly such a template (abstractGrailsResourceLocator), and it is public surface: third-party plugins inherit from it withbean.parent, asset-pipeline'sassetResourceLocatoramong them. ABeanRegistrationExcludeFilterkeeps them out of generation; children are unaffected because AOT generates them from their merged definition.A servlet context wired as a bean reference. SiteMesh's bean-definition wrap referenced the
servletContextbean, which the container registers only once the web server starts and which has no bean definition at all.Depends on a SiteMesh change
The last item is fixed upstream in
spring-webmvc-sitemesh—SiteMeshViewResolvernow implementsServletContextAwareinstead of taking the context as a constructor argument, sodependencies.gradletracks3.3.0-SNAPSHOT.GrailsSiteMeshViewResolverkeeps its four-argument constructor, so a build on the older release still compiles.Also here
An AOT cache. On JDK 25 or later,
grails.aotCachetrains a JDK AOT cache by running the packaged application once and recording the classes it loaded and the methods it ran:grails { aotCache { enabled = true paths = ['/', '/login', '/book/index'] } }Measured on an application with GORM for Hibernate, security and asset pipeline: 2.594s ordinary, 2.284s with Spring AOT, 0.933s with a cache trained over those paths.
Reachability metadata for a native image.
generateNativeMetadatarecords the application's own artefacts and compiled pages by reading the build output, with no run required.traceNativeMetadataruns the application under GraalVM's agent for the paths and forms it is given, which is the half that reading the build output cannot supply:grails { nativeMetadata { forms = ['/login?username=admin&password=secret', '/book/create'] paths = ['/', '/book'] } }Forms are submitted before paths are asked for, and the session one establishes carries, so a page behind a login is traced by listing its form. The task fails rather than reporting success when what it recorded is not what was asked for.
The banner says how the application was started —
NATIVE,AOT CACHE,AOT— and reports the servlet container it is running on. Spring Security and the container are shown by default now; a version shown by default that cannot be determined is left out rather than shown asunknown, while one asked for by name still readsunknown. See the upgrade guide.Limitations
beanRegistrar()-contributed beans are still excluded from generation by Spring's ownaotProcessingIgnoreRegistrationflag and re-registered at runtime whenGrailsApplicationPostProcessorre-runs. Correct, but it repeats the plugin scan on every boot — the work AOT exists to precompute.GroovyPagesServletis now created by the container rather than withnew, so it passes through bean post-processing. ItspluginManageris unaffected (initFrameworkServletalready autowires by type), but a post-processor whose pointcut matched it would handServletRegistrationBeana proxy. No pointcut in the framework does.Verification
grails-test-examples/aotis built with the Spring Boot AOT plugin and itscheckruns two things against the packaged jar withspring.aot.enabled=true.aotStartupCheckstarts the application and asks it for a page. Rendering that page requires the tag library to have been given the collaborator it takes by name, the page to have been found in the manifest of pages compiled at build time, the action to have reached the view it returned, and the URL mappings to have routed the request — none of which fails at start-up when it is broken. The link on the page is followed rather than compared against a path, so the link generator has to have produced one that leads back.aotGeneratedSourcesCarryNoBuildPathreads the generated sources and fails if any of them holds the directory the build ran in. A definition that carries one builds cleanly, starts cleanly on the machine that generated it, and looks for its resources somewhere that exists nowhere else.