Skip to content

docs: add bash_executor example#1391

Open
akihikokuroda wants to merge 3 commits into
generative-computing:mainfrom
akihikokuroda:shellexample
Open

docs: add bash_executor example#1391
akihikokuroda wants to merge 3 commits into
generative-computing:mainfrom
akihikokuroda:shellexample

Conversation

@akihikokuroda

Copy link
Copy Markdown
Member

Pull Request

Issue

Fixes #1390

Description

Adds a comprehensive example demonstrating safe Python code generation with bash_executor orchestration for CLI data processing tasks. Shows how to avoid shell operators by using Python control flow for orchestrating multiple bash commands.

Key Features:

  • LLM generates Python code with sequential bash_executor() calls
  • Validates generated code with PythonSyntaxValid requirement
  • Executes safely via python_tool with configurable execution tiers
  • 5 working examples covering file finding, counting, displaying, and aggregation
  • Comprehensive documentation with usage patterns and best practices

Changes

New Files

  • docs/examples/requirements/code_generation_and_execution_bash.py - Full example with 5 data processing tasks

Modified Files

  • mellea/stdlib/tools/_bash_patterns.py - Relaxed semicolon guardrail to allow escaped semicolons (\;) for find -exec syntax
  • docs/examples/requirements/README.md - Added comprehensive documentation section for the new example
  • docs/examples/README.md - Updated requirements section highlighting the new example

Examples Provided

Example 1: Find and count ERROR entries

User request: Find all ERROR entries in the log files and count them
Output: Total ERROR entries found in all logs: 6

Example 2: Count total lines in files

User request: List all log files and count the total number of lines across all of them
Output: Total number of lines across all log files: 24

Example 3: Display file contents

User request: Show the contents of all data files in the data directory
Output: Displays all CSV data files

Example 4: Find and display WARN entries

User request: Find all WARN entries in the log files and display the files containing them
Output: logs/app_1.log: contains 2 WARN entries
logs/app_2.log: contains 2 WARN entries
logs/app_3.log: contains 2 WARN entries

Example 5: Aggregate ERROR entries

User request: Count how many log files contain ERROR entries
Output: Total ERROR entries across all log files: 6

Technical Details

Key Design Principles

  1. No Shell Operators - Avoids pipes (|), redirects (>, >>), and unescaped semicolons
  2. Python Control Flow - Uses loops, conditionals, and variables for orchestration
  3. ExecutionResult API - Checks result.success, result.stdout, result.skip_message
  4. Safe Execution - Uses python_tool tier for subprocess isolation
  5. Output Parsing - Extracts and transforms command output in Python

Bash Guardrail Enhancement

Relaxed the semicolon guardrail in _bash_patterns.py to allow escaped semicolons (\;):

  • Before: All semicolons were rejected
  • After: Escaped semicolons (\;) are allowed for find -exec ... \; patterns
  • Impact: Enables legitimate find command patterns while blocking shell chaining

LLM Prompt Enhancements

Enhanced the prompt with:

  • Directory structure documentation (logs/, data/, config/)
  • Common file operations (cat, find, grep, wc -l)
  • grep pattern tips for timestamped logs (avoid ^ anchors)
  • ExecutionResult API documentation
  • Multiple code examples showing correct patterns

Testing

All 5 examples execute successfully with correct output
All 49 bash guardrail tests pass
Code passes ruff formatting (ruff format)
Code passes ruff linting (ruff check)
Code passes mypy type checking (mypy)

Verification

# Run the example
uv run python docs/examples/requirements/code_generation_and_execution_bash.py

# Run tests
uv run pytest test/stdlib/tools/test_bash_guardrails.py -v

# Check code quality
uv run ruff format docs/examples/requirements/code_generation_and_execution_bash.py
uv run ruff check docs/examples/requirements/code_generation_and_execution_bash.py
uv run mypy docs/examples/requirements/code_generation_and_execution_bash.py

Documentation

Comprehensive documentation added to docs/examples/requirements/README.md:
- Feature overview and prerequisites
- Usage examples (default, custom workspace, interactive)
- Pipeline steps and design principles
- Generated code patterns
- Output examples and key design principles

Related Issues

Addresses use cases for:
- Safe CLI data processing with LLM-generated code
- Orchestrating multiple bash commands via Python
- Avoiding shell operators in generated code
- Processing and aggregating bash command outputs

Backward Compatibility

Backward compatible - Only adds new example and relaxes guardrail (allows more patterns, doesn't restrict existing valid patterns)

Impact

- Users: New pattern for safe bash command orchestration
- Developers: Clear example of python_tool usage with bash_executor
- Codebase: Enhanced guardrails support legitimate use cases
- Documentation: Comprehensive guide for CLI data processing workflows


### Testing
- [ ] Tests added to the respective file if code was changed
- [ ] New code has 100% coverage if code was added
- [ ] Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

### Attribution
- [x] AI coding assistants used

---

### Adding a new component, requirement, sampling strategy, or tool?

If your PR adds or modifies one of the types below, check the matching box. A checklist of type-specific review items will be posted as a comment.

<!-- DO NOT DELETE THIS SECTION — managed by .github/workflows/pr-update.yml. Checking a box is fine; removing the boxes will break checklist updates. -->
<!-- If your PR implements more than one, please split it into separate PRs. -->

- [ ] Component
- [ ] Requirement
- [ ] Sampling Strategy
- [ ] Tool

NOTE: Please ensure you have an issue that has been acknowledged by a core contributor and routed you to open a pull request against this repository. Otherwise, please open an issue before continuing with this pull request.

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
@akihikokuroda akihikokuroda changed the title Shellexample docs: add bash_executor example Jul 14, 2026
@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Jul 14, 2026
@akihikokuroda akihikokuroda marked this pull request as ready for review July 14, 2026 19:29
@akihikokuroda akihikokuroda requested a review from a team as a code owner July 14, 2026 19:29
Comment thread mellea/stdlib/tools/_bash_patterns.py Outdated
# Semicolon: substring check (dangerous even in some quote contexts)
if ";" in arg:
# Allow escaped semicolons (\;) used in find -exec syntax
if ";" in arg and "\\;" not in arg:

@AngeloDanducci AngeloDanducci Jul 14, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I am understanding the code trace correctly it looks like this:

BashEnvironment._validate_command (in shell.py:834)
shlex.split (shell.py:849)
_is_dangerous_command(shell.py:868)
check_all_patterns(shell.py:144)
check_all_patterns(_bash_patterns.py:248)
ShellOperatorPattern.check(_bash_patterns.py:91)

This check final check was updated to allow escaped semicolons specifically (;).

However if since shlex.split is already called, is the backslash not already removed?

ie
command as typed : find logs -name '*.log' -exec cat {} \;
after shlex.split
shlex.split(command) : ['find', 'logs', '-name', '*.log', '-exec', 'cat', '{}', ';']

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AngeloDanducci your are right. There are issues processing "". Some code are changed to fix this issue.

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docs: Add Python code generation example with bash_executor orchestration

2 participants