Description
transform() edits the template it is given. Translator.translate() documents the opposite:
:returns: a copy of the template with SAM resources replaced with the corresponding CloudFormation
What actually happens, measured on main (ab5503a2) by deep-copying the input, transforming, and comparing:
| template |
caller's dict after transform() |
| bare function |
unchanged |
function + Globals |
Globals section deleted; merged Timeout written into the resource's Properties |
function + Api event |
function properties rewritten in place |
function + explicit AWS::Serverless::Api |
generated DefinitionBody written into the Api resource |
function + explicit AWS::Serverless::HttpApi |
generated DefinitionBody written into the Api resource |
explicit Api alone, state machine |
unchanged |
The consequence is worse than the mutation itself. Because the first transform leaves its own generated DefinitionBody behind in the caller's template, transforming the same object a second time fails:
InvalidDocumentException: Event with id [Api] is invalid.
API method "get" defined multiple times for path "/x".
This hits any caller that holds on to the template — transforming once per region or partition, tooling that shows the user their original template after transforming, or anything that simply calls transform() twice. The error message points at the user's event definition, which is not where the problem is.
I checked for cross-invocation state leakage as well (transform template A, then B, and compare B against a clean run): that is clean. The damage is confined to the object passed in.
Steps to reproduce
import copy
from unittest.mock import MagicMock, patch
from samtranslator.translator.transform import transform
template = {
"Transform": "AWS::Serverless-2016-10-31",
"Resources": {
"Fn": {
"Type": "AWS::Serverless::Function",
"Properties": {
"CodeUri": "s3://bucket/key", "Handler": "index.handler", "Runtime": "python3.11",
"Events": {"Api": {"Type": "Api", "Properties": {
"Path": "/x", "Method": "get", "RestApiId": {"Ref": "Api"}}}},
},
},
"Api": {"Type": "AWS::Serverless::Api", "Properties": {"StageName": "prod"}},
},
}
before = copy.deepcopy(template)
loader = MagicMock()
loader.load.return_value = {}
with patch("boto3.session.Session.region_name", "us-east-1"):
transform(template, {}, loader)
print("input mutated:", template != before) # True
transform(template, {}, loader) # raises InvalidDocumentException
Observed result
input mutated: True, then InvalidDocumentException: API method "get" defined multiple times for path "/x" on the second call.
Expected result
The caller's template is left as it was, and transforming the same template twice produces the same output both times.
Possible fix
Take a copy at the transform() boundary.
One constraint worth recording, because it is not obvious and it rules out the tempting placement: the copy has to be taken before to_py27_compatible_template() runs. That function installs Py27Dict/Py27UniStr wrappers whose hash-ordering state logical ID generation depends on, and copy.deepcopy does not preserve it. I first put the copy at the top of Translator.translate() — the method whose docstring promises a copy — and tests/translator/test_translator.py caught it immediately: test_transform_feature_toggle_0_feature_toggle_api_open_api_version_override fails there because generated logical IDs change. Copying earlier, while the template is still plain dicts and strings, avoids that entirely.
Cost is negligible: on a 246 KiB template with 200 functions, the copy is 3.2 ms against a 1924 ms transform, or 0.2%.
Note this fixes the public transform() entry point. Translator.translate() called directly still mutates its argument, so its docstring is still inaccurate for that path; fixing it there needs __deepcopy__ support on the Py27 types, which is a larger change and I have left it alone.
Additional environment details
- SAM translator
main @ ab5503a2, installed with pip install -e .
- Python 3.12, macOS 15
Description
transform()edits the template it is given.Translator.translate()documents the opposite:What actually happens, measured on
main(ab5503a2) by deep-copying the input, transforming, and comparing:transform()GlobalsGlobalssection deleted; mergedTimeoutwritten into the resource'sPropertiesApieventAWS::Serverless::ApiDefinitionBodywritten into theApiresourceAWS::Serverless::HttpApiDefinitionBodywritten into theApiresourceApialone, state machineThe consequence is worse than the mutation itself. Because the first transform leaves its own generated
DefinitionBodybehind in the caller's template, transforming the same object a second time fails:This hits any caller that holds on to the template — transforming once per region or partition, tooling that shows the user their original template after transforming, or anything that simply calls
transform()twice. The error message points at the user's event definition, which is not where the problem is.I checked for cross-invocation state leakage as well (transform template A, then B, and compare B against a clean run): that is clean. The damage is confined to the object passed in.
Steps to reproduce
Observed result
input mutated: True, thenInvalidDocumentException: API method "get" defined multiple times for path "/x"on the second call.Expected result
The caller's template is left as it was, and transforming the same template twice produces the same output both times.
Possible fix
Take a copy at the
transform()boundary.One constraint worth recording, because it is not obvious and it rules out the tempting placement: the copy has to be taken before
to_py27_compatible_template()runs. That function installsPy27Dict/Py27UniStrwrappers whose hash-ordering state logical ID generation depends on, andcopy.deepcopydoes not preserve it. I first put the copy at the top ofTranslator.translate()— the method whose docstring promises a copy — andtests/translator/test_translator.pycaught it immediately:test_transform_feature_toggle_0_feature_toggle_api_open_api_version_overridefails there because generated logical IDs change. Copying earlier, while the template is still plain dicts and strings, avoids that entirely.Cost is negligible: on a 246 KiB template with 200 functions, the copy is 3.2 ms against a 1924 ms transform, or 0.2%.
Note this fixes the public
transform()entry point.Translator.translate()called directly still mutates its argument, so its docstring is still inaccurate for that path; fixing it there needs__deepcopy__support on the Py27 types, which is a larger change and I have left it alone.Additional environment details
main@ab5503a2, installed withpip install -e .