-
Notifications
You must be signed in to change notification settings - Fork 2.1k
Ability to call transform functions within a runtime #2015
Copy link
Copy link
Closed
Labels
domain: parsingAnything related to parsing within VectorAnything related to parsing within Vectordomain: processingAnything related to processing Vector's events (parsing, merging, reducing, etc.)Anything related to processing Vector's events (parsing, merging, reducing, etc.)domain: transformsAnything related to Vector's transform componentsAnything related to Vector's transform componentsdomain: vrlAnything related to the Vector Remap LanguageAnything related to the Vector Remap Languagehave: niceThis feature is nice to have. It is low priority.This feature is nice to have. It is low priority.needs: approvalNeeds review & approval before work can begin.Needs review & approval before work can begin.needs: more demandNeeds more demand before work can begin, +1 or comment to support.Needs more demand before work can begin, +1 or comment to support.needs: requirementsNeeds a a list of requirements before work can be beginNeeds a a list of requirements before work can be begintype: enhancementA value-adding code change that enhances its existing functionality.A value-adding code change that enhances its existing functionality.
Metadata
Metadata
Assignees
Labels
domain: parsingAnything related to parsing within VectorAnything related to parsing within Vectordomain: processingAnything related to processing Vector's events (parsing, merging, reducing, etc.)Anything related to processing Vector's events (parsing, merging, reducing, etc.)domain: transformsAnything related to Vector's transform componentsAnything related to Vector's transform componentsdomain: vrlAnything related to the Vector Remap LanguageAnything related to the Vector Remap Languagehave: niceThis feature is nice to have. It is low priority.This feature is nice to have. It is low priority.needs: approvalNeeds review & approval before work can begin.Needs review & approval before work can begin.needs: more demandNeeds more demand before work can begin, +1 or comment to support.Needs more demand before work can begin, +1 or comment to support.needs: requirementsNeeds a a list of requirements before work can be beginNeeds a a list of requirements before work can be begintype: enhancementA value-adding code change that enhances its existing functionality.A value-adding code change that enhances its existing functionality.
This has been discussed, and clarifies the intent behind #2012. It is becoming clear that runtimes will likely replace the entire transform pipeline, at least for operations that cross the fundamental boundary. We can further encourage this by allowing users to call other "function" transforms from within runtimes.
Note, this should be limited to transforms that do not aggregate, only simple input/output transforms. For example, we should not allow users to call the
merge,dedupe, ortransactiontransforms within a runtime since this would complicate the runtime APIs. I'm open to solutions for these, but I'm not sure they are worth it.Finally, I do not think we should address this in #2000. We should defer this and follow up with this addition, assuming it is possible.