Skip to content
This repository was archived by the owner on Apr 23, 2025. It is now read-only.

Fix testing regression #10

Merged
ipv1337 merged 7 commits into from
Apr 14, 2025
Merged

Fix testing regression #10

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
e84589e
Fix pytest.ini timeout setting format
ipv1337 Apr 14, 2025
8dfbaa8
Add script to find hanging tests
ipv1337 Apr 14, 2025
19e9e13
Update scripts/find_hanging_tests.sh
ipv1337 Apr 14, 2025
03dd001
Fix PR issues: Make timeouts configurable, improve error handling, an…
ipv1337 Apr 14, 2025
a1af2eb
Fix CI failures: Make test directory configurable, improve error hand…
ipv1337 Apr 14, 2025
03e6569
CI fix: Improve path handling in GitHub Actions environment
ipv1337 Apr 14, 2025
f036972
Update GitHub workflow to better handle missing tests and make CI mor…
ipv1337 Apr 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/python-ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ jobs:
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install build pytest pytest-cov ruff
pip install build pytest pytest-cov pytest-timeout ruff
pip install -e .
# Show installed packages
pip list
Expand Down
2 changes: 1 addition & 1 deletion pytest.ini
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ testpaths = test_dir
python_files = test_*.py
python_classes = Test*
python_functions = test_*
timeout = 30 # Set default timeout for all tests to 30 seconds
timeout = 30
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot Apr 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Removing the comment here makes the purpose of the timeout less clear. Consider adding a brief comment explaining why the timeout is set, or referring to documentation where the timeout strategy is described.

timeout = 30  # Set default timeout for all tests to 30 seconds

filterwarnings =
ignore::DeprecationWarning
ignore::pytest.PytestCollectionWarning
74 changes: 74 additions & 0 deletions scripts/find_hanging_tests.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
#!/bin/bash

# Script to find hanging tests
echo "Running tests individually to find hanging tests..."

# Clean up cache files first
find . -name "__pycache__" -type d -exec rm -rf {} + 2>/dev/null || true
find . -name "*.pyc" -exec rm -f {} + 2>/dev/null || true

# Set timeout (in seconds)
TIMEOUT=15
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot Apr 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Consider making the timeout configurable via an environment variable. This would allow users to adjust the timeout based on their specific needs and hardware. Also, consider adding a comment about the units of the timeout value (seconds).

Suggested change
TIMEOUT=15
TIMEOUT=${TIMEOUT:-15} # Timeout in seconds, default to 15 if not set


# Function to run a single test with timeout
run_test_with_timeout() {
TEST_FILE=$1
echo "Testing: $TEST_FILE"
if timeout $TIMEOUT python -m pytest "$TEST_FILE" -v; then
echo "✅ $TEST_FILE completed successfully"
else
EXIT_CODE=$?
if [ $EXIT_CODE -eq 124 ]; then
echo "❌ $TEST_FILE TIMEOUT - Test is hanging!"
else
echo "❌ $TEST_FILE failed with exit code $EXIT_CODE"
fi
fi
echo "----------------------------------------"
}

# Test files to check - base list from most critical files
TEST_FILES=(
"test_dir/test_file_tools.py"
"test_dir/test_system_tools.py"
"test_dir/test_directory_tools.py"
"test_dir/improved/test_quality_tools.py"
"test_dir/improved/test_summarizer_tool.py"
"test_dir/improved/test_tree_tool.py"
"test_dir/test_models_base.py"
"test_dir/test_model_basic.py"
"test_dir/test_model_integration.py"
"test_dir/test_gemini_model.py"
"test_dir/test_gemini_model_advanced.py"
"test_dir/test_gemini_model_coverage.py"
"test_dir/test_gemini_model_error_handling.py"
"test_dir/test_ollama_model.py"
"test_dir/test_ollama_model_advanced.py"
"test_dir/test_ollama_model_coverage.py"
"test_dir/test_ollama_model_context.py"
"test_dir/test_ollama_model_error_handling.py"
"test_dir/test_config.py"
"test_dir/test_config_comprehensive.py"
"test_dir/test_config_edge_cases.py"
"test_dir/test_config_missing_methods.py"
"test_dir/test_main.py"
"test_dir/test_main_comprehensive.py"
"test_dir/test_main_edge_cases.py"
"test_dir/test_main_improved.py"
"test_dir/test_task_complete_tool.py"
"test_dir/test_tools_base.py"
"test_dir/test_tools_init_coverage.py"
"test_dir/test_utils.py"
"test_dir/test_utils_comprehensive.py"
"test_dir/test_test_runner_tool.py"
"test_dir/test_basic_functions.py"
"test_dir/test_tools_basic.py"
"test_dir/test_tree_tool_edge_cases.py"
)
Comment thread
ipv1337 marked this conversation as resolved.
Outdated

# Run each test file individually
for TEST_FILE in "${TEST_FILES[@]}"; do
run_test_with_timeout "$TEST_FILE"
done

echo "Test scan complete. Check output for any hanging tests."
117 changes: 113 additions & 4 deletions scripts/run_coverage_ci.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,22 +1,131 @@
#!/bin/bash
# Script to generate coverage for CI pipeline
# Script to generate coverage for CI pipeline with timeouts to prevent hanging

set -e # Exit on error
set -x # Print commands for debugging

echo "Starting coverage generation for CI..."

# Set up coverage directory
mkdir -p coverage_html

# Run pytest with coverage enabled and generate reports
# Clean up any pycache files to avoid import conflicts
find . -name "__pycache__" -type d -exec rm -rf {} + 2>/dev/null || true
find . -name "*.pyc" -delete

# Run tests in smaller batches with timeouts
echo "Running test suite with coverage enabled..."

# First, run the basic tools tests which are known to work
echo "Running tools tests (known to work well)..."
python -m pytest \
--cov=src.cli_code \
--cov-report=xml:coverage.xml \
--cov-report=html:coverage_html \
--cov-report=term \
test_dir/test_file_tools.py test_dir/test_directory_tools.py test_dir/test_system_tools.py \
test_dir/improved/test_quality_tools.py test_dir/improved/test_summarizer_tool.py test_dir/improved/test_tree_tool.py
--timeout=60 \
test_dir/test_file_tools.py \
test_dir/test_system_tools.py \
test_dir/test_directory_tools.py \
test_dir/improved/test_quality_tools.py \
test_dir/improved/test_summarizer_tool.py \
test_dir/improved/test_tree_tool.py

# Now run the model tests separately
echo "Running model tests..."
python -m pytest \
--cov=src.cli_code \
--cov-append \
--cov-report=xml:coverage.xml \
--cov-report=html:coverage_html \
--cov-report=term \
--timeout=60 \
test_dir/test_models_base.py \
test_dir/test_model_basic.py \
test_dir/test_model_integration.py

# Run gemini model tests individually
for test_file in \
test_dir/test_gemini_model.py \
test_dir/test_gemini_model_advanced.py \
test_dir/test_gemini_model_coverage.py \
test_dir/test_gemini_model_error_handling.py; do
echo "Running $test_file with timeout..."
python -m pytest \
--cov=src.cli_code \
--cov-append \
--timeout=60 \
"$test_file" || echo "Warning: $test_file timed out or failed"
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot Apr 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using || echo "Warning: $test_file timed out or failed" will prevent the script from exiting if a test fails, but it might mask important error information. Consider logging the specific error message and exit code for better debugging. You can capture the exit code using $? immediately after the pytest command and include it in the warning message.

Suggested change
"$test_file" || echo "Warning: $test_file timed out or failed"
"$test_file" || { echo "Warning: $test_file timed out or failed with exit code $?"; }

done

# Run ollama model tests individually
for test_file in \
test_dir/test_ollama_model.py \
test_dir/test_ollama_model_advanced.py \
test_dir/test_ollama_model_coverage.py \
test_dir/test_ollama_model_context.py \
test_dir/test_ollama_model_error_handling.py; do
echo "Running $test_file with timeout..."
python -m pytest \
--cov=src.cli_code \
--cov-append \
--timeout=60 \
"$test_file" || echo "Warning: $test_file timed out or failed"
done

# Run config tests individually
for test_file in \
test_dir/test_config.py \
test_dir/test_config_comprehensive.py \
test_dir/test_config_edge_cases.py \
test_dir/test_config_missing_methods.py; do
echo "Running $test_file with timeout..."
python -m pytest \
--cov=src.cli_code \
--cov-append \
--timeout=60 \
"$test_file" || echo "Warning: $test_file timed out or failed"
done

# Run main tests individually
for test_file in \
test_dir/test_main.py \
test_dir/test_main_comprehensive.py \
test_dir/test_main_edge_cases.py \
test_dir/test_main_improved.py; do
echo "Running $test_file with timeout..."
python -m pytest \
--cov=src.cli_code \
--cov-append \
--timeout=60 \
"$test_file" || echo "Warning: $test_file timed out or failed"
done

# Run remaining tests individually
for test_file in \
test_dir/test_task_complete_tool.py \
test_dir/test_tools_base.py \
test_dir/test_tools_init_coverage.py \
test_dir/test_utils.py \
test_dir/test_utils_comprehensive.py \
test_dir/test_test_runner_tool.py \
test_dir/test_basic_functions.py \
test_dir/test_tools_basic.py \
test_dir/test_tree_tool_edge_cases.py; do
echo "Running $test_file with timeout..."
python -m pytest \
--cov=src.cli_code \
--cov-append \
--timeout=60 \
"$test_file" || echo "Warning: $test_file timed out or failed"
done

# Generate a final coverage report
python -m pytest \
--cov=src.cli_code \
--cov-report=xml:coverage.xml \
--cov-report=html:coverage_html \
--cov-report=term

echo "Coverage report generated in coverage.xml and coverage_html/"

Expand Down
116 changes: 113 additions & 3 deletions scripts/test_coverage_local.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,21 +1,131 @@
#!/bin/bash
# Script to test coverage generation locally
# Script to test coverage generation locally with timeouts to prevent hanging

set -e # Exit on error
set -x # Print commands for debugging

echo "Starting local test coverage generation..."

# Set up coverage directory
mkdir -p coverage_html

# Run pytest with coverage enabled and generate reports
# Clean up any pycache files to avoid import conflicts
find . -name "__pycache__" -type d -exec rm -rf {} + 2>/dev/null || true
find . -name "*.pyc" -delete

# Run tests in smaller batches with timeouts
echo "Running test suite with coverage enabled..."

# First, run the basic tools tests which are known to work
echo "Running tools tests (known to work well)..."
python -m pytest \
--cov=src.cli_code \
--cov-report=xml:coverage.xml \
--cov-report=html:coverage_html \
--cov-report=term \
test_dir/test_file_tools.py test_dir/test_directory_tools.py test_dir/test_system_tools.py
--timeout=30 \
test_dir/test_file_tools.py \
test_dir/test_system_tools.py \
test_dir/test_directory_tools.py \
test_dir/improved/test_quality_tools.py \
test_dir/improved/test_summarizer_tool.py \
test_dir/improved/test_tree_tool.py

# Now run the model tests separately
echo "Running model tests..."
python -m pytest \
--cov=src.cli_code \
--cov-append \
--cov-report=xml:coverage.xml \
--cov-report=html:coverage_html \
--cov-report=term \
--timeout=30 \
test_dir/test_models_base.py \
test_dir/test_model_basic.py \
test_dir/test_model_integration.py

# Run gemini model tests individually
for test_file in \
test_dir/test_gemini_model.py \
test_dir/test_gemini_model_advanced.py \
test_dir/test_gemini_model_coverage.py \
test_dir/test_gemini_model_error_handling.py; do
echo "Running $test_file with timeout..."
python -m pytest \
--cov=src.cli_code \
--cov-append \
--timeout=30 \
"$test_file" || echo "Warning: $test_file timed out or failed"
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot Apr 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using || echo "Warning: $test_file timed out or failed" will prevent the script from exiting if a test fails, but it might mask important error information. Consider logging the specific error message and exit code for better debugging. You can capture the exit code using $? immediately after the pytest command and include it in the warning message.

Suggested change
"$test_file" || echo "Warning: $test_file timed out or failed"
"$test_file" || { echo "Warning: $test_file timed out or failed with exit code $?"; }

done

# Run ollama model tests individually
for test_file in \
test_dir/test_ollama_model.py \
test_dir/test_ollama_model_advanced.py \
test_dir/test_ollama_model_coverage.py \
test_dir/test_ollama_model_context.py \
test_dir/test_ollama_model_error_handling.py; do
echo "Running $test_file with timeout..."
python -m pytest \
--cov=src.cli_code \
--cov-append \
--timeout=30 \
"$test_file" || echo "Warning: $test_file timed out or failed"
done

# Run config tests individually
for test_file in \
test_dir/test_config.py \
test_dir/test_config_comprehensive.py \
test_dir/test_config_edge_cases.py \
test_dir/test_config_missing_methods.py; do
echo "Running $test_file with timeout..."
python -m pytest \
--cov=src.cli_code \
--cov-append \
--timeout=30 \
"$test_file" || echo "Warning: $test_file timed out or failed"
done

# Run main tests individually
for test_file in \
test_dir/test_main.py \
test_dir/test_main_comprehensive.py \
test_dir/test_main_edge_cases.py \
test_dir/test_main_improved.py; do
echo "Running $test_file with timeout..."
python -m pytest \
--cov=src.cli_code \
--cov-append \
--timeout=30 \
"$test_file" || echo "Warning: $test_file timed out or failed"
done

# Run remaining tests individually
for test_file in \
test_dir/test_task_complete_tool.py \
test_dir/test_tools_base.py \
test_dir/test_tools_init_coverage.py \
test_dir/test_utils.py \
test_dir/test_utils_comprehensive.py \
test_dir/test_test_runner_tool.py \
test_dir/test_basic_functions.py \
test_dir/test_tools_basic.py \
test_dir/test_tree_tool_edge_cases.py; do
echo "Running $test_file with timeout..."
python -m pytest \
--cov=src.cli_code \
--cov-append \
--timeout=30 \
"$test_file" || echo "Warning: $test_file timed out or failed"
done

# Generate a final coverage report
python -m pytest \
--cov=src.cli_code \
--cov-report=xml:coverage.xml \
--cov-report=html:coverage_html \
--cov-report=term

echo "Coverage report generated in coverage.xml and coverage_html/"
echo "This is the format SonarCloud expects."
Expand Down
Loading