[Python] Add Firehose Basics scenario + Hello - #7963
Conversation
5953ef3 to
b3151fb
Compare
This comment was marked as outdated.
This comment was marked as outdated.
- Restore Java, Rust, SAP ABAP entries in firehose_metadata.yaml (PutRecord, PutRecordBatch, Scenario_PutRecords were overwritten) - Add stubber-based unit tests (test_firehose_unit.py, 11 tests) - Increase wait_for_stream_active max_wait from 60s to 300s - Remove unused imports (json, datetime, timezone) from wrapper - Remove unused demo_tools.question import from scenario - Add region_name parameter to FirehoseWrapper.from_client() - Fix missing trailing newlines (wrapper, requirements.txt) - Apply Black formatting to all files
…omment - Replace boto3.client type annotation with untyped parameter - Increase wait_for_encryption_enabled max_wait from 60s to 300s - Add inline comment noting update_destination assumes S3 destination
Restoring these languages in firehose_metadata.yaml requires their READMEs to also be regenerated with the current WRITEME format.
There was a problem hiding this comment.
🤖 AI Code Example Review
{
"summary": "This PR substantially addresses all blocking issues from the previous review: tests are now present (both unit and integration), the wrapper is complete and no longer truncated, a polling waiter is included, the scenario runner exists, requirements.txt is added, and unused imports have been removed. The implementation is largely ready to merge with a few minor remaining concerns around entry-point confusion in the README, missing unit tests for error paths, unused imports in the scenario file, and a conftest.py for pytest marks.",
"detailed_review": "1. [RESOLVED] Tests added: Both test_firehose_unit.py and test_firehose_integ.py are present and comprehensive. Unit tests cover all wrapper methods using botocore Stubber, including partial-failure batch handling. This was the most critical gap from the previous review — well done.\n\n2. [RESOLVED] Truncated delete_delivery_stream: The method is now complete and properly handles ResourceInUseException.\n\n3. [RESOLVED] Stream-active polling: wait_for_stream_active() and wait_for_encryption_enabled() methods are included in the wrapper, and both are called correctly in the scenario.\n\n4. [RESOLVED] Unused imports: json, datetime, and timezone were previously flagged as unused in firehose_wrapper.py. They no longer appear in that file. The scenario file (firehose_basics_scenario.py) imports all of them and uses them, so that's fine.\n\n5. [REMAINING] README entry point is wrong: python/example_code/firehose/README.md line 86 says python firehose_wrapper.py as the entry point for the 'Learn the basics' scenario. The actual entry point is python scenarios/firehose_basics_scenario.py. The wrapper class has no if __name__ == '__main__' block and cannot be run directly. This will confuse users.\n\n6. [REMAINING] Missing conftest.py for pytest mark registration: test_firehose_integ.py and test_firehose_unit.py use @pytest.mark.integ, but no conftest.py is present to register this mark. Without it, pytest will emit PytestUnknownMarkWarning and running pytest -m integ may not work as expected. The comparable ControlTower and HealthImaging examples all include a conftest.py with markers = ["integ: ...] in a pytest.ini_options-compatible file.\n\n7. [REMAINING] Scenario file has unused import sys and unused # noqa comment: firehose_basics_scenario.py imports sys (only used for sys.path.append, which is acceptable) but also time which is used — that's fine. However, firehose_hello.py is imported via from firehose_hello import hello_firehose inside a test function body (test_firehose_integ.py, line 83) rather than at the top, which is an unusual pattern. This is minor but inconsistent with project conventions.\n\n8. [NEW] No unit tests for error code paths: The unit tests all test the happy path. None test ClientError branches (e.g., ResourceInUseException on create, ResourceNotFoundException on describe). Adding one or two error-path tests would give better coverage and is consistent with comparable examples.\n\n9. [NEW] wait_for_encryption_enabled max_wait default is 120s in scenario call but 300s in wrapper default: In firehose_basics_scenario.py line ~118, wait_for_encryption_enabled is called with max_wait=120. StartDeliveryStreamEncryption can take several minutes. The previous review noted this for wait_for_stream_active (which was corrected to 300s), but the encryption wait is still 120s. This may cause timeout failures in integration tests.\n\n10. [NEW] firehose_hello.py is placed in the repo root of firehose/, not in a hello/ subdirectory: The metadata YAML references github: python/example_code/firehose and the README references firehose_hello.py, but the PR description says it's in hello/hello_firehose.py. The actual diff shows the file at python/example_code/firehose/firehose_hello.py (root level), not hello/. The test imports from firehose_hello which matches root placement, but the PR description is misleading. This is cosmetic but should be clarified.",
"inline_comments": [
{
"path": "python/example_code/firehose/README.md",
"line": 86,
"body": "The entry point shown here (python firehose_wrapper.py) is incorrect — the wrapper class has no __main__ block. The correct entry point is python scenarios/firehose_basics_scenario.py. This will cause confusion for users following the README."
},
{
"path": "python/example_code/firehose/scenarios/firehose_basics_scenario.py",
"line": 116,
"body": "max_wait=120 may be too short for StartDeliveryStreamEncryption to complete. Encryption enablement can take several minutes in practice. Consider increasing to at least 300 seconds, consistent with the wait_for_stream_active call above."
},
{
"path": "python/example_code/firehose/test/test_firehose_integ.py",
"line": 83,
"body": "Importing hello_firehose inside the test function body is non-standard. Move the import to the top of the file with the other imports for consistency with project conventions."
},
{
"path": "python/example_code/firehose/test/test_firehose_unit.py",
"line": 20,
"body": "Consider adding a conftest.py in the test/ directory that registers the integ mark to avoid PytestUnknownMarkWarning. Comparable examples (e.g., ControlTower) include this file."
},
{
"path": "python/example_code/firehose/firehose_wrapper.py",
"line": 430,
"body": "wait_for_encryption_enabled default max_wait=300 is correct here, but the scenario overrides it to 120s (line ~118 of firehose_basics_scenario.py). Ensure the scenario caller uses a value that reflects realistic SSE enablement time (≥300s recommended)."
}
]
}
This review was generated automatically using Amazon Bedrock. It compares your changes against existing examples and coding guidelines. Please use your judgment — this is advisory, not authoritative.
Summary
This PR adds Python code examples for Firehose generated by the CodeLoom
automated pipeline.
Files Added
python/example_code/firehose/hello/hello_firehose.py— Hello World examplepython/example_code/firehose/scenarios/firehose_basics_scenario.py— Scenariopython/example_code/firehose/firehose_wrapper.py— Service wrapper classpython/example_code/firehose/test/test_firehose_integ.py— Integration testspython/example_code/firehose/requirements.txt— Dependencies.doc_gen/metadata/firehose_metadata.yaml— Metadata for doc generationOperations Covered
(See specification for full list)
Generation Details
amazon-firehose-spec-20260707-181316amazon-firehose-codegen-20260707-181557Review Checklist