Plugin Support for Java SDK

[donald-pinckney]

Plugin System for Java Temporal SDK

What was changed

Added a plugin system for the Java Temporal SDK, modeled after the Python SDK's plugin architecture but adapted to Java idioms.

New Plugin Interfaces

• WorkflowServiceStubsPlugin (temporal-serviceclient) - Configuration and connection lifecycle for service stubs
• WorkflowClientPlugin (temporal-sdk) - Configuration for workflow clients
• ScheduleClientPlugin (temporal-sdk) - Configuration for schedule clients
• WorkerPlugin (temporal-sdk) - Configuration and lifecycle for worker factories and workers

WorkerPlugin Lifecycle Methods

WorkerPlugin includes lifecycle methods that use a chain pattern where next accepts the same parameters as the method:

• startWorkerFactory(factory, next) - Wraps factory startup, next is Consumer<WorkerFactory>
• shutdownWorkerFactory(factory, next) - Wraps factory shutdown, next is Consumer<WorkerFactory>
• startWorker(taskQueue, worker, next) - Wraps individual worker startup, next is BiConsumer<String, Worker>
• shutdownWorker(taskQueue, worker, next) - Wraps individual worker shutdown, next is BiConsumer<String, Worker>
• replayWorkflowExecution(worker, history, next) - Wraps workflow replay, next is ReplayCallback (custom interface that throws Exception)

Configuration methods are called in forward (registration) order. Lifecycle methods are called in reverse order for proper nesting (first plugin wraps all others)

SimplePlugin

• SimplePlugin (temporal-sdk) - Abstract convenience class implementing all plugin interfaces
  • Can be used via builder: SimplePlugin.newBuilder("name").build()
  • Can be subclassed for custom behavior
  • Provides convenience methods for common operations (interceptors, data converters, workflow/activity registration)

Modified Files

• WorkflowServiceStubsOptions - Added plugins field
• WorkflowServiceStubs - Applies plugin configuration and connection lifecycle
• WorkflowClientOptions - Added plugins field
• WorkflowClientInternalImpl - Applies plugin configuration, propagates from service stubs
• ScheduleClientOptions - Added plugins field
• ScheduleClientImpl - Applies plugin configuration, propagates from service stubs
• WorkerFactoryOptions - Added plugins field
• WorkerFactory - Full plugin lifecycle (configuration, worker init, start/shutdown)
• Worker - Added plugins field and replay plugin chain support

Test Files

• PluginTest.java - Tests subclassing pattern and lifecycle ordering
• SimplePluginBuilderTest.java - Tests builder API comprehensively
• PluginPropagationTest.java - Integration tests for plugin propagation chain
• WorkflowClientOptionsPluginTest.java - Tests options class plugin field handling

Why?

The plugin system provides a higher-level abstraction over the existing interceptor infrastructure, enabling users to:

• Modify configuration during client/worker creation
• Wrap execution lifecycles with setup/teardown logic
• Auto-propagate plugins through the creation chain
• Bundle multiple customizations (interceptors, context propagators, data converters, workflow/activity registrations) into reusable units

Checklist

1. Closes Plugin support #2626, tracked in Plugins to support controlling multiple configuration points at once features#652

2. How was this tested:

   • Unit tests for plugin interfaces and SimplePlugin builder
   • Integration tests for plugin propagation through full chain
   • All tests pass locally

3. Any docs updates needed?

   • Documentation for the new plugin API should be added

---

Design

Plugin Interfaces

Each level in the creation chain has its own plugin interface:

WorkflowServiceStubsPlugin     →  WorkflowClientPlugin  →  WorkerPlugin
                               →  ScheduleClientPlugin

Plugins set at higher levels propagate down automatically. For example, a plugin set on WorkflowServiceStubsOptions that also implements WorkflowClientPlugin will have its configureWorkflowClient() method called when creating a WorkflowClient.

SimplePlugin

SimplePlugin is an abstract class that implements all plugin interfaces with sensible defaults. It can be used two ways:

Builder pattern for declarative configuration:

SimplePlugin plugin = SimplePlugin.newBuilder("my-org.tracing")
    .setDataConverter(myConverter)
    .addWorkerInterceptors(new TracingInterceptor())
    .addClientInterceptors(new LoggingInterceptor())
    .registerWorkflowImplementationTypes(MyWorkflow.class)
    .registerActivitiesImplementations(new MyActivityImpl())
    .onWorkerStart((taskQueue, worker) -> logger.info("Worker started: {}", taskQueue))
    .build();

Subclassing for custom behavior:

public class TracingPlugin extends SimplePlugin {
    private final Tracer tracer;

    public TracingPlugin(Tracer tracer) {
        super("my-org.tracing");
        this.tracer = tracer;
    }

    @Override
    public void configureWorkflowClient(WorkflowClientOptions.Builder builder) {
        builder.setInterceptors(new TracingClientInterceptor(tracer));
    }

    @Override
    public void configureWorkerFactory(WorkerFactoryOptions.Builder builder) {
        builder.setWorkerInterceptors(new TracingWorkerInterceptor(tracer));
    }
}

Usage Example

// Create plugin
SimplePlugin metricsPlugin = SimplePlugin.newBuilder("my-org.metrics")
    .customizeServiceStubs(b -> b.setMetricsScope(myScope))
    .addWorkerInterceptors(new MetricsInterceptor())
    .build();

// Set on service stubs - will propagate to client and workers
WorkflowServiceStubsOptions stubsOptions = WorkflowServiceStubsOptions.newBuilder()
    .setTarget("localhost:7233")
    .setPlugins(metricsPlugin)
    .build();

WorkflowServiceStubs stubs = WorkflowServiceStubs.newServiceStubs(stubsOptions);
WorkflowClient client = WorkflowClient.newInstance(stubs);  // plugin propagates here
WorkerFactory factory = WorkerFactory.newInstance(client);   // and here

[CLAassistant]

All committers have signed the CLA.

[donald-pinckney]

@cretz I think i've addressed everything. Could you do a re-review of the code?

[donald-pinckney]

FYI, I also took a first attempt at replayer support.

[cretz]

Sorry, a few more things I noticed as I got into the impl (thanks for allowing this back and forth)

[donald-pinckney]

@cretz I've addressed all items in this round.

[cretz]

Looks great, love the reduction in complication (also helps plugin authors when what the plugin does is more obvious). Only one more request on that same aspect, then LGTM.

[cretz]

Looks great, thanks! Approved from my POV, but not marking approved because you'll want @maciejdudko or @tconley1428 approval to merge (owners of this SDK).

[tconley1428]

I'm not sure I would bother with the customizers versus just letting them subclass simpleplugin if they want to, but I don't object strenuously.

[cretz]

IIUC, in all of our SDKs we decided to have such a concept where they don't have to subclass to provide a lambda to customize existing values. While I disagreed with this (and disagreed with the general aversion to requiring subclassing to do advanced customization), it was overruled.

Granted we could change this to have "customizers" for the specific fields we offer instead of the options as a whole, that'd make sense to me.

[tconley1428]

The usability of this seems limited compared to at least the other languages I wrote given the lack of ability to see and modify existing configs. For instance, if I want my plugin to add a codec to a dataconverter or change the converter, I can't do that. In other languages I allow you to provide a function as well as a raw value. Not sure how easy that is in Java though.

[tconley1428]

I guess some of the builder stuff makes that tricky. DataConverter in particular is common to need to modify, so maybe just something special for that.

[cretz]

For instance, if I want my plugin to add a codec to a dataconverter or change the converter, I can't do that

Why not? You have an options customizer you can do whatever. But if you're saying you want a data converter customizer instead of an options customizer, that can make sense and more aligns with what other SDKs do.

In other languages I allow you to provide a function as well as a raw value. Not sure how easy that is in Java though.

.NET you couldn't either, but we allowed callables as alternatives for fixed values for every specific simple-plugin option we offered, I think we can do the same here and get rid of the options-level customizers.

[donald-pinckney]

The current support for customizers (customizeClient) sort of let's you do this, but it's a bit janky:

SimplePlugin.newBuilder("my-plugin")                                                                                                                                                                           
    .customizeClient(builder -> {                                                                                                                                                                              
        DataConverter existing = builder.build().getDataConverter();                                                                                                                                           
        builder.setDataConverter(wrapWithCodec(existing, myCodec));                                                                                                                                            
    })                                                                                                                                                                                                         
    .build();

Which isn't a whole lot better than subclassing SimplePlugin with this override:

@Override                                                                                                                                                                                                  
public void configureWorkflowClient(WorkflowClientOptions.Builder builder) {                                                                                                                               
    // Call super first to apply any base configuration                                                                                                                                                    
    super.configureWorkflowClient(builder);                                                                                                                                                                
                                                                                                                                                                                                            
    // Now see what's configured and wrap it                                                                                                                                                               
    DataConverter existing = builder.build().getDataConverter();                                                                                                                                           
    builder.setDataConverter(                                                                                                                                                                              
        new CodecDataConverter(existing, List.of(myCodec)));                                                                                                                                               
}

Here's couple of options:

Option 0: Remove the customizers

Just remove these customizer things like customizeClient. If they want to customize, just have them override configureWorkflowClient. Builder-style supports full replacement only.

Option 1: Add customizeX methods alongside setX methods

// Existing - replaces                                                                                                                                                                                         
public Builder setDataConverter(DataConverter dataConverter)                                                                                                                                                   
                                                                                                                                                                                                                
// New - transforms existing                                                                                                                                                                                   
public Builder customizeDataConverter(UnaryOperator<DataConverter> customizer)                                                                                                                                 
                                                                                                                                                                                                                
// Usage:                                                                                                                                                                                                         
.setDataConverter(myConverter)  // Replace                                                                                                                                                                     
.customizeDataConverter(existing -> wrapWithCodec(existing, myCodec))  // Transform 

Option 2: Only accept functions

public Builder setDataConverter(UnaryOperator<DataConverter> customizer)                                                                                                                                       
                                                                                                                                                                                                                
// Usage:                                                                                                                                                                                                         
.setDataConverter(existing -> myConverter)  // Replace (ignores existing)                                                                                                                                      
.setDataConverter(existing -> wrapWithCodec(existing, myCodec))  // Transform 

I don't really like this, I think it makes call sites hard/weird to read for full replacements.

Option 3: Custom PluginValue<T> wrapper (closer to Python)

@FunctionalInterface                                                                                                                                                                                           
public interface PluginValue<T> {                                                                                                                                                                              
    T resolve(@Nullable T existing);                                                                                                                                                                           
                                                                                                                                                                                                                
    static <T> PluginValue<T> set(T value) { return existing -> value; }                                                                                                                                       
    static <T> PluginValue<T> customize(UnaryOperator<T> fn) { return fn::apply; }                                                                                                                             
}                                                                                                                                                                                                              
                                                                                                                                                                                                                
// Usage                                                                                                                                                                                                       
.dataConverter(PluginValue.set(myConverter))                                                                                                                                                                   
.dataConverter(PluginValue.customize(existing -> wrap(existing)))                                                                                                                                              
.dataConverter(existing -> wrap(existing))  // Works since it's a functional interface 

Seems like a more complicated version of Option 1.

[donald-pinckney]

(FYI i just renamed customizeClient -> customizeWorkflowClient, so its consistent withconfigureWorkflowClient).

[donald-pinckney]

@cretz @tconley1428 In 2f20274 I tried removing the options customizers and adding in a DataConverter customizer. LMK how that looks.