In convert, We are currently bailing out when an ExpressionTool is encountered:
if hasattr(cwl_tool, "expression"):
raise RuntimeError("ExpressionTool not supported yet")Can we support the conversion of ExpressionTool?
If the above clause is removed and we let the processing continue, it crashes because the plan for the activity corresponding to the execution of the ExpressionTool is not found. More specifically, resolve_plan returns None and the program crashes when it tries to do:
plan_tag = plan.id.localpart
Error message:
AttributeError: 'NoneType' object has no attribute 'id'
Adding some prints in resolve_plan:
def _resolve_plan(self, activity):
print("resolving plan for", activity.id)
job_qname = activity.plan()
print(" job qname:", job_qname)
plan = activity.provenance.entity(job_qname)
print(" plan:", plan)
if not plan:
m = SCATTER_JOB_PATTERN.match(str(job_qname))
if m:
plan = activity.provenance.entity(m.groups()[0])
return planWe get:
resolving plan for id:a9f719bd-9bf2-42a4-aa4a-163eb95351dd
job qname: wf:main/
Entity wf:main/ not found in Provenance<urn:uuid:4b66a4db-eb94-43fe-8475-14d38ac3a3bc from /home/simleo/work/wf_run_crate/expression_tool/cwl/ngtax-run-1/metadata/provenance/primary.cwlprov.xml>
plan: None
So the activity.plan() (job_qname) is just wf:main/, with no tool-specific tag after the slash. Compare this to the output for a tool in the conversion of tests/data/revsort-run-1:
resolving plan for id:f81dd60b-46db-4e58-b9f9-5606de1f10de
job qname: wf:main/rev
plan: entity(wf:main/rev, [prov:type='prov:Plan', prov:type='wfdesc:Process'])
Looking at primary.cwlprov.json:
"wasAssociatedWith": {
...
"_:id11": {
"prov:activity": "id:a9f719bd-9bf2-42a4-aa4a-163eb95351dd",
"prov:agent": "id:ed5680f3-84f4-423c-be6f-d9ed9991a436",
"prov:plan": "wf:main/"
},
...
}prov:plan is also wf:main/ for other ExpressionTools used in the workflow. So this is something that's not supported by CWLProv.
The above results have been obtained by trying to convert the RO of an execution of https://gitlab.com/m-unlock/cwl/-/raw/main/workflows/workflow_ngtax.cwl with https://gitlab.com/m-unlock/cwl/-/raw/main/tests/ngtax/ngtax.yaml. The version of cwltool used was 3.1.20240112164112.
In
convert, We are currently bailing out when anExpressionToolis encountered:Can we support the conversion of
ExpressionTool?If the above clause is removed and we let the processing continue, it crashes because the plan for the activity corresponding to the execution of the ExpressionTool is not found. More specifically,
resolve_planreturnsNoneand the program crashes when it tries to do:Error message:
Adding some prints in
resolve_plan:We get:
So the
activity.plan()(job_qname) is justwf:main/, with no tool-specific tag after the slash. Compare this to the output for a tool in the conversion oftests/data/revsort-run-1:Looking at
primary.cwlprov.json:prov:planis alsowf:main/for otherExpressionTools used in the workflow. So this is something that's not supported by CWLProv.The above results have been obtained by trying to convert the RO of an execution of https://gitlab.com/m-unlock/cwl/-/raw/main/workflows/workflow_ngtax.cwl with https://gitlab.com/m-unlock/cwl/-/raw/main/tests/ngtax/ngtax.yaml. The version of cwltool used was
3.1.20240112164112.