Skip to content

[Python] Add firehose Basics scenario + Hello - #8009

Open
scmacdon wants to merge 21 commits into
awsdocs:mainfrom
scmacdon:codeloom/firehose-basics-20260723
Open

[Python] Add firehose Basics scenario + Hello#8009
scmacdon wants to merge 21 commits into
awsdocs:mainfrom
scmacdon:codeloom/firehose-basics-20260723

Conversation

@scmacdon

@scmacdon scmacdon commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR adds Python code examples for firehose generated by the CodeLoom
automated pipeline.

Files Added

  • python/example_code/firehose/hello/hello_firehose.py — Hello World example
  • python/example_code/firehose/scenarios/firehose_basics_scenario.py — Scenario
  • python/example_code/firehose/firehose_wrapper.py — Service wrapper class
  • python/example_code/firehose/test/test_firehose_integ.py — Integration tests
  • python/example_code/firehose/requirements.txt — Dependencies
  • .doc_gen/metadata/firehose_metadata.yaml — Metadata for doc generation

Operations Covered

(See specification for full list)

Generation Details

  • Spec Operation ID: amazon-firehose-spec-20260723-145332
  • Codegen Operation ID: amazon-firehose-codegen-20260723-150835
  • Generated: 2026-07-23 15:44 UTC
  • Pipeline Version: 1.0 (PoC)

Review Checklist

  • Code compiles/runs without errors
  • Snippet tags are correct and complete
  • Metadata YAML matches code structure
  • Scenario workflow makes logical sense for the service
  • Error handling covers realistic failure cases
  • Hello example demonstrates basic connectivity
  • Integration tests pass
  • No hardcoded credentials or account-specific values

@scmacdon scmacdon added auto-generated Python This issue relates to the AWS SDK for Python (boto3) needs-review labels Jul 23, 2026
github-actions[bot]

This comment was marked as resolved.

@scmacdon scmacdon self-assigned this Jul 23, 2026
github-actions[bot]

This comment was marked as resolved.

github-actions[bot]

This comment was marked as resolved.

github-actions[bot]

This comment was marked as resolved.

github-actions[bot]

This comment was marked as resolved.

github-actions[bot]

This comment was marked as resolved.

github-actions[bot]

This comment was marked as resolved.

github-actions[bot]

This comment was marked as resolved.

github-actions[bot]

This comment was marked as resolved.

github-actions[bot]

This comment was marked as resolved.

@scmacdon scmacdon added the Basics A basic code example showing the core actions for a particular service. label Jul 23, 2026
github-actions[bot]

This comment was marked as resolved.

github-actions[bot]

This comment was marked as resolved.

github-actions[bot]

This comment was marked as resolved.

github-actions[bot]

This comment was marked as resolved.

@scmacdon
scmacdon force-pushed the codeloom/firehose-basics-20260723 branch from 7263cf3 to 6b36e88 Compare July 23, 2026 22:02
github-actions[bot]

This comment was marked as resolved.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Code Example Review

This PR has addressed most of the blocking issues from the previous review (import error fixed, conftest.py added, unused imports removed, batch chunking implemented, static bucket name replaced with dynamic CFN bucket, cleanup now empties bucket before stack deletion). A few minor issues remain, but nothing blocking. The code is well-structured, tested, and largely follows AWS SDK example best practices.

Detailed Review

  1. [ADDRESSED] Import error in test files: The previous blocking from scenario_firehose_basics import FirehoseScenario import error has been fixed — the test file now imports from firehose_wrapper correctly and the conftest.py adds the parent directory to sys.path. This was the most critical blocker and is resolved.

  2. [ADDRESSED] conftest.py added: python/example_code/firehose/test/conftest.py now exists and properly patches sys.path, resolving the import resolution issue for both firehose_wrapper and firehose_hello.

  3. [ADDRESSED] Unused imports removed: firehose_wrapper.py no longer imports json, Optional, datetime, or timezone. Only logging, time, typing, boto3, and botocore are imported — all are used.

  4. [ADDRESSED] Batch chunking in put_record_batch: The put_record_batch method now properly chunks records into batches of up to 500 (configurable), aggregates FailedPutCount and RequestResponses across all chunks, and returns a unified response. This resolves the previous spec-compliance concern.

  5. [ADDRESSED] Static S3 bucket name replaced: The CFN template now uses Type: AWS::S3::Bucket without a fixed BucketName, letting CloudFormation generate a unique name. This resolves the re-run collision issue.

  6. [ADDRESSED] Cleanup ordering and bucket emptying: The _cleanup method now waits for stream deletion before emptying the S3 bucket, and _empty_bucket uses bucket.object_versions.all().delete(). This resolves the previous CFN stack deletion failure risk.

  7. [REMAINING MINOR] test_firehose_scenario.py is truncated in the diff: The diff shows the file ends mid-line (stubb) at the end of TestFirehoseHello.test_hello_firehose. The full file content shows stubber.deactivate() is called, but the truncated diff makes it hard to verify the file is complete. This appears to be a diff artifact — the full file content looks complete. Not blocking.

  8. [REMAINING MINOR] No retry/backoff in put_record/put_record_batch: The spec explicitly requires 'retries with exponential back-off and jitter' for ServiceUnavailableException. The code logs a message suggesting retries but does not implement them. The existing scenarios/firehose-put-actions/firehose.py (which this PR also modifies) uses backoff.on_exception as a good pattern. Consider adding tenacity or documenting this limitation more explicitly. Low severity for an educational example, but the spec calls for it.

  9. [MINOR] README.md entry for scenario points to wrong file: The README says python firehose_wrapper.py to run the scenario, but the scenario entry point is python scenarios/firehose_basics_scenario.py. The wrapper file does not have a runnable __main__ block that exercises the full scenario. Users following the README will run the wrong file.

  10. [MINOR] test_firehose_integ.py not visible in the diff: The previous review flagged blocking issues with test_firehose_integ.py (wrong import name scenario_firehose_basics). This file is referenced in the previous review but does not appear in the current diff — it's unclear if it was removed or simply not shown. If it still exists with the wrong import, it remains a blocker. The test file shown in this PR (test_firehose_scenario.py) is correct, but the integ test file status is unknown.


This review was generated automatically using Amazon Bedrock. It compares your changes against existing examples and coding guidelines. Please use your judgment — this is advisory, not authoritative.

Comment thread python/example_code/firehose/README.md
Comment thread python/example_code/firehose/scenarios/firehose_basics_scenario.py
Comment thread python/example_code/firehose/firehose_wrapper.py
Comment thread python/example_code/firehose/test/test_firehose_scenario.py
@brmur
brmur force-pushed the codeloom/firehose-basics-20260723 branch from 7afdd1b to a309596 Compare July 27, 2026 15:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-generated Basics A basic code example showing the core actions for a particular service. needs-review Python This issue relates to the AWS SDK for Python (boto3)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant