Skip to content

[Python] Add Firehose Basics scenario + Hello - #7963

Open
scmacdon wants to merge 11 commits into
mainfrom
codeloom/firehose-basics-20260707
Open

[Python] Add Firehose Basics scenario + Hello#7963
scmacdon wants to merge 11 commits into
mainfrom
codeloom/firehose-basics-20260707

Conversation

@scmacdon

@scmacdon scmacdon commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR adds Python code examples for Firehose generated by the CodeLoom
automated pipeline.

Files Added

  • python/example_code/firehose/hello/hello_firehose.py — Hello World example
  • python/example_code/firehose/scenarios/firehose_basics_scenario.py — Scenario
  • python/example_code/firehose/firehose_wrapper.py — Service wrapper class
  • python/example_code/firehose/test/test_firehose_integ.py — Integration tests
  • python/example_code/firehose/requirements.txt — Dependencies
  • .doc_gen/metadata/firehose_metadata.yaml — Metadata for doc generation

Operations Covered

(See specification for full list)

Generation Details

  • Spec Operation ID: amazon-firehose-spec-20260707-181316
  • Codegen Operation ID: amazon-firehose-codegen-20260707-181557
  • Generated: 2026-07-07 18:20 UTC
  • Pipeline Version: 1.0 (PoC)

Review Checklist

  • Code compiles/runs without errors
  • Snippet tags are correct and complete
  • Metadata YAML matches code structure
  • Scenario workflow makes logical sense for the service
  • Error handling covers realistic failure cases
  • Hello example demonstrates basic connectivity
  • Integration tests pass
  • No hardcoded credentials or account-specific values

@scmacdon scmacdon added auto-generated Python This issue relates to the AWS SDK for Python (boto3) needs-review labels Jul 7, 2026
@scmacdon scmacdon self-assigned this Jul 7, 2026
@scmacdon
scmacdon force-pushed the codeloom/firehose-basics-20260707 branch from 5953ef3 to b3151fb Compare July 8, 2026 19:53
@scmacdon scmacdon added the On Call Review needed This work needs an on-call review label Jul 8, 2026
@brmur
brmur requested review from rlhagerm and removed request for rlhagerm July 9, 2026 12:33
@github-actions

This comment was marked as outdated.

@scmacdon

scmacdon commented Jul 9, 2026

Copy link
Copy Markdown
Contributor Author

@awsdocs awsdocs deleted a comment from github-actions Bot Jul 10, 2026
github-actions[bot]

This comment was marked as resolved.

@rlhagerm rlhagerm removed the On Call Review needed This work needs an on-call review label Jul 13, 2026
github-actions[bot]

This comment was marked as resolved.

@scmacdon scmacdon closed this Jul 22, 2026
- Restore Java, Rust, SAP ABAP entries in firehose_metadata.yaml
  (PutRecord, PutRecordBatch, Scenario_PutRecords were overwritten)
- Add stubber-based unit tests (test_firehose_unit.py, 11 tests)
- Increase wait_for_stream_active max_wait from 60s to 300s
- Remove unused imports (json, datetime, timezone) from wrapper
- Remove unused demo_tools.question import from scenario
- Add region_name parameter to FirehoseWrapper.from_client()
- Fix missing trailing newlines (wrapper, requirements.txt)
- Apply Black formatting to all files
@brmur brmur reopened this Jul 23, 2026
github-actions[bot]

This comment was marked as resolved.

…omment

- Replace boto3.client type annotation with untyped parameter
- Increase wait_for_encryption_enabled max_wait from 60s to 300s
- Add inline comment noting update_destination assumes S3 destination
github-actions[bot]

This comment was marked as resolved.

github-actions[bot]

This comment was marked as resolved.

Restoring these languages in firehose_metadata.yaml requires their
READMEs to also be regenerated with the current WRITEME format.
@github-actions github-actions Bot added Rust This issue relates to the AWS SDK for Rust (Preview) Java-v2 This issue relates to the AWS SDK for Java V2 SAP ABAP This issue relates to the AWS SDK for SAP ABAP labels Jul 23, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Code Example Review

{
"summary": "This PR substantially addresses all blocking issues from the previous review: tests are now present (both unit and integration), the wrapper is complete and no longer truncated, a polling waiter is included, the scenario runner exists, requirements.txt is added, and unused imports have been removed. The implementation is largely ready to merge with a few minor remaining concerns around entry-point confusion in the README, missing unit tests for error paths, unused imports in the scenario file, and a conftest.py for pytest marks.",
"detailed_review": "1. [RESOLVED] Tests added: Both test_firehose_unit.py and test_firehose_integ.py are present and comprehensive. Unit tests cover all wrapper methods using botocore Stubber, including partial-failure batch handling. This was the most critical gap from the previous review — well done.\n\n2. [RESOLVED] Truncated delete_delivery_stream: The method is now complete and properly handles ResourceInUseException.\n\n3. [RESOLVED] Stream-active polling: wait_for_stream_active() and wait_for_encryption_enabled() methods are included in the wrapper, and both are called correctly in the scenario.\n\n4. [RESOLVED] Unused imports: json, datetime, and timezone were previously flagged as unused in firehose_wrapper.py. They no longer appear in that file. The scenario file (firehose_basics_scenario.py) imports all of them and uses them, so that's fine.\n\n5. [REMAINING] README entry point is wrong: python/example_code/firehose/README.md line 86 says python firehose_wrapper.py as the entry point for the 'Learn the basics' scenario. The actual entry point is python scenarios/firehose_basics_scenario.py. The wrapper class has no if __name__ == '__main__' block and cannot be run directly. This will confuse users.\n\n6. [REMAINING] Missing conftest.py for pytest mark registration: test_firehose_integ.py and test_firehose_unit.py use @pytest.mark.integ, but no conftest.py is present to register this mark. Without it, pytest will emit PytestUnknownMarkWarning and running pytest -m integ may not work as expected. The comparable ControlTower and HealthImaging examples all include a conftest.py with markers = ["integ: ...] in a pytest.ini_options-compatible file.\n\n7. [REMAINING] Scenario file has unused import sys and unused # noqa comment: firehose_basics_scenario.py imports sys (only used for sys.path.append, which is acceptable) but also time which is used — that's fine. However, firehose_hello.py is imported via from firehose_hello import hello_firehose inside a test function body (test_firehose_integ.py, line 83) rather than at the top, which is an unusual pattern. This is minor but inconsistent with project conventions.\n\n8. [NEW] No unit tests for error code paths: The unit tests all test the happy path. None test ClientError branches (e.g., ResourceInUseException on create, ResourceNotFoundException on describe). Adding one or two error-path tests would give better coverage and is consistent with comparable examples.\n\n9. [NEW] wait_for_encryption_enabled max_wait default is 120s in scenario call but 300s in wrapper default: In firehose_basics_scenario.py line ~118, wait_for_encryption_enabled is called with max_wait=120. StartDeliveryStreamEncryption can take several minutes. The previous review noted this for wait_for_stream_active (which was corrected to 300s), but the encryption wait is still 120s. This may cause timeout failures in integration tests.\n\n10. [NEW] firehose_hello.py is placed in the repo root of firehose/, not in a hello/ subdirectory: The metadata YAML references github: python/example_code/firehose and the README references firehose_hello.py, but the PR description says it's in hello/hello_firehose.py. The actual diff shows the file at python/example_code/firehose/firehose_hello.py (root level), not hello/. The test imports from firehose_hello which matches root placement, but the PR description is misleading. This is cosmetic but should be clarified.",
"inline_comments": [
{
"path": "python/example_code/firehose/README.md",
"line": 86,
"body": "The entry point shown here (python firehose_wrapper.py) is incorrect — the wrapper class has no __main__ block. The correct entry point is python scenarios/firehose_basics_scenario.py. This will cause confusion for users following the README."
},
{
"path": "python/example_code/firehose/scenarios/firehose_basics_scenario.py",
"line": 116,
"body": "max_wait=120 may be too short for StartDeliveryStreamEncryption to complete. Encryption enablement can take several minutes in practice. Consider increasing to at least 300 seconds, consistent with the wait_for_stream_active call above."
},
{
"path": "python/example_code/firehose/test/test_firehose_integ.py",
"line": 83,
"body": "Importing hello_firehose inside the test function body is non-standard. Move the import to the top of the file with the other imports for consistency with project conventions."
},
{
"path": "python/example_code/firehose/test/test_firehose_unit.py",
"line": 20,
"body": "Consider adding a conftest.py in the test/ directory that registers the integ mark to avoid PytestUnknownMarkWarning. Comparable examples (e.g., ControlTower) include this file."
},
{
"path": "python/example_code/firehose/firehose_wrapper.py",
"line": 430,
"body": "wait_for_encryption_enabled default max_wait=300 is correct here, but the scenario overrides it to 120s (line ~118 of firehose_basics_scenario.py). Ensure the scenario caller uses a value that reflects realistic SSE enablement time (≥300s recommended)."
}
]
}


This review was generated automatically using Amazon Bedrock. It compares your changes against existing examples and coding guidelines. Please use your judgment — this is advisory, not authoritative.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-generated Java-v2 This issue relates to the AWS SDK for Java V2 needs-review Python This issue relates to the AWS SDK for Python (boto3) Rust This issue relates to the AWS SDK for Rust (Preview) SAP ABAP This issue relates to the AWS SDK for SAP ABAP

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants