Python 3 migration and fractional frame rate support#9
Open
benroeder wants to merge 34 commits intobbc:masterfrom
Open
Python 3 migration and fractional frame rate support#9benroeder wants to merge 34 commits intobbc:masterfrom
benroeder wants to merge 34 commits intobbc:masterfrom
Conversation
- Document current limitations (integer-only frame rates) - Propose fractional frame rate support using rational arithmetic (24000/1001) - Add broadcast standard shortcuts (NTSC, PAL, Cinema formats) - Include professional resolution and format presets - Define 5-phase TDD implementation with Hypothesis property testing - Establish quality gates and success criteria for each phase - Focus on native frame timing (no drop-frame complexity) - Maintain backward compatibility with existing integer rates
- Restructure phases to start with Python 3 porting (Phase 1) - Update all phase numbers sequentially (1-7) - Remove time estimates to focus on technical implementation - Update dependencies to reflect new phase structure - Python 3 compatibility now prerequisite for modern testing framework
Convert Python 2 syntax to Python 3: - Fix print statements throughout generate.py and eventTimingGen.py - Convert print >> sys.stderr to print(..., file=sys.stderr) in video.py - Fix function parameter unpacking in video.py (__init__ and xy methods) - Replace unichr() with chr() and remove u'' string prefixes - Convert lambda parameter unpacking to list comprehension in generate.py - Update requirements.txt with Python 3 compatible versions (Pillow>=10.0.0) Add Python 3 compatibility tests: - Create test_python3_compatibility.py with smoke tests - Verify all modules import correctly - Test basic functionality (event generation, timecode conversion) - Handle infinite generator pattern in genEventCentreTimes correctly Quality Gate 1 achieved: ✅ All modules import without syntax errors ✅ Basic event generation works ✅ Video and audio generation functions load ✅ Command line interface imports successfully ✅ All Python 3 compatibility tests pass
Update Quality Gate 1 status to show completion: - All Python 3 compatibility requirements met - All modules now import without syntax errors - Basic functionality verified with test suite - Ready to proceed to Phase 2: comprehensive test coverage
- Added 33 comprehensive tests covering all core generation functionality - Created integration tests ensuring module interoperability - Established Hypothesis property-based testing infrastructure - Fixed remaining Python 2 to 3 compatibility issues (.next() → next()) - Verified all current integer frame rate functionality works correctly Key test files added: - test_sequence_gen/tests/conftest.py (pytest configuration + fixtures) - test_sequence_gen/tests/test_integration_current.py (9 integration tests) - Enhanced test_sequence_gen/tests/test_generate_current.py (24 unit/property tests) Quality Gate 2 achieved: ✅ 33 tests pass (unit + integration + property tests) ✅ Core generation functionality fully tested ✅ Hypothesis verifies mathematical correctness ✅ Module interoperability confirmed ✅ Python 3 compatibility verified Ready to proceed to Phase 3: Frame Rate Parsing and Rational Arithmetic
Implemented comprehensive frame rate parser supporting fractional rates using exact rational arithmetic. Follows TDD methodology with RED → GREEN → REFACTOR cycle. Key Features Implemented: - Parse decimal format: "23.976" → (24000, 1001) - Parse rational format: "24000/1001" → (24000, 1001) - Parse integer format: "25" → (25, 1) - Parse broadcast shortcuts: "ntsc-film" → (24000, 1001) - Support for NTSC, PAL, and Cinema standards - Exact rational arithmetic using Python Fraction - Comprehensive error handling and validation Files Added: - test_sequence_gen/src/frame_rate_parser.py (190 lines) - test_sequence_gen/tests/test_frame_rate_parsing.py (14 unit tests) - test_sequence_gen/tests/test_frame_rate_parsing_properties.py (10 property tests) Quality Gate 3 Achieved: ✅ 24 comprehensive tests pass (14 basic + 10 Hypothesis property tests) ✅ All common fractional rates parse correctly ✅ Rational arithmetic maintains exact precision ✅ Broadcast standard shortcuts work ✅ Round-trip conversion preserves precision ✅ Mathematical properties verified by Hypothesis Ready to proceed to Phase 4: Frame Timing Calculations
Implemented precise frame timing calculations using exact rational arithmetic. Supports perfect precision for fractional frame rates with comprehensive testing. Key Features Implemented: - frame_to_seconds: Convert frame number to exact time using Fraction - seconds_to_frame: Convert time to frame number with proper rounding - calculate_frame_duration: Exact frame duration as rational - calculate_sequence_duration: Total sequence duration calculations - validate_fps_rational: Comprehensive input validation Mathematical Properties Verified: - Round-trip conversions (frame ↔ time) maintain identity - Frame timing is monotonically increasing - Duration calculations are exact (no floating point errors) - NTSC precision: 30000 frames at 29.97 fps = exactly 1001 seconds - All mathematical invariants verified by property-based testing Files Added: - test_sequence_gen/src/frame_timing.py (280 lines) - test_sequence_gen/tests/test_frame_timing_calculations.py (18 unit tests) - test_sequence_gen/tests/test_frame_timing_properties.py (15 property tests) Quality Gate 4 Achieved: ✅ 33 comprehensive tests pass (18 basic + 15 Hypothesis property tests) ✅ Frame-to-time conversions are exact (no floating point errors) ✅ Round-trip conversions maintain identity or near-identity ✅ Mathematical properties verified by property-based testing ✅ NTSC precision verified with exact rational arithmetic ✅ All edge cases and boundary conditions handled correctly Ready to proceed to Phase 5: Flash/Beep Timing Generation
Implemented fractional frame rate event generation with exact rational arithmetic. Extends existing MLS-based event generation to support NTSC and other fractional rates. Key Features Implemented: - genEventCentreTimesFractional: Generate events for fractional frame rates using exact Fraction objects - genFlashSequenceFractional: Frame-accurate flash sequences for fractional rates - genBeepSequenceFractional: Sample-accurate beep sequences for fractional rates - createFpsBitTimingsFractional: Bit timings using exact rational arithmetic - Complete integration with frame_timing.py and frame_rate_parser.py Fractional Rate Support: - 23.976 fps (24000/1001) - NTSC film rate - 29.97 fps (30000/1001) - NTSC video rate - 59.94 fps (60000/1001) - NTSC HD rate - Perfect timing precision using Python Fraction objects - No floating-point drift or accumulating errors Files Added: - test_sequence_gen/src/fractional_event_generation.py (420 lines) - test_sequence_gen/tests/test_fractional_event_generation.py (11 comprehensive tests) Quality Gate 5 Achieved: ✅ 11 comprehensive tests pass for fractional event generation ✅ Exact rational arithmetic prevents timing drift ✅ NTSC frame rates generate precise event sequences ✅ Flash/beep durations are frame-accurate for fractional rates ✅ Integration verified: events → flash sequences → beep sequences ✅ Backward compatibility maintained with integer frame rates Ready to proceed to Phase 6: CLI Integration and Shortcuts
Implement comprehensive CLI support for fractional frame rates with broadcast shortcuts, resolution presets, and complete format combinations. ## CLI Integration Module (cli_fractional_integration.py) - parse_fractional_args(): Enhanced argument parser * --fps 23.976, --fps 24000/1001 (fractional frame rates) * --fps-ntsc-film, --fps-pal, --fps-film (broadcast shortcuts) * --size-4k-full, --size-hd-1080 (resolution presets) * --preset-1080p59.94, --preset-cinema-4k (complete formats) - create_fractional_metadata(): Exact rational timing metadata - Resolution/format preset systems with professional standards ## Test Coverage - 17/17 CLI integration tests pass (test_cli_fractional_integration.py) - 18/18 Hypothesis property tests pass (test_cli_fractional_hypothesis.py) - Comprehensive edge case and robustness testing - Backward compatibility verification ## Key Features - Broadcast standard shortcuts (NTSC, PAL, Cinema families) - Professional resolution presets (DCI 4K, UHD, broadcast formats) - Complete format presets combining fps + resolution - Exact rational arithmetic throughout - JSON-serializable metadata with compatibility - Graceful error handling and validation Phase 6 Quality Gate: ✅ COMPLETED - All broadcast shortcuts work correctly - Resolution presets accurate for professional formats - Backward compatibility maintained - 35 total tests pass with Hypothesis verification
Successfully implemented comprehensive fractional frame rate support for the DVB CSS synchronization timing measurement system. ## Completed Modules ### generate_fractional.py - Complete test sequence generator with fractional frame rate support - Integrates all fractional modules into cohesive workflow - Supports all broadcast standards (NTSC, PAL, Cinema) - Maintains backward compatibility with existing tools ### test_end_to_end_integration.py - 15 comprehensive end-to-end tests - Validates complete workflow from CLI to output files - Tests audio/video synchronization accuracy - Verifies metadata generation and compatibility ## Key Features Implemented 1. Python 3 migration (fixed print, lambda, unicode, division) 2. Exact rational arithmetic using Fraction objects 3. Fractional frame rates: 23.976, 29.97, 59.94 fps 4. Broadcast shortcuts: --fps-ntsc-film, --fps-pal, etc. 5. Resolution presets: --size-4k-full, --size-hd-1080 6. Complete format presets: --preset-1080p59.94 7. Frame-accurate event generation 8. Sample-accurate audio generation 9. Pillow 11.x compatibility fixes ## Test Results - 168+ tests pass across all modules - Unit, integration, and property-based tests - End-to-end workflow validation - Mathematical correctness verified with Hypothesis ## Technical Achievements - No floating-point timing errors - Exact NTSC frame timing (24000/1001, 30000/1001, 60000/1001) - Professional broadcast standard support - DCI 4K and cinema format presets - Complete backward compatibility maintained All 7 phases of the implementation plan completed successfully.
Add comprehensive documentation and demo script for the fractional frame rate implementation. ## Documentation Updates (IMPROVEMENTS.md) - Updated Phase 7 completion status with detailed results - Added usage examples for all new features - Documented all integration fixes applied - Added demo commands for common use cases ## Demo Script (demo_fractional.py) - Interactive demonstration of fractional frame rate support - Shows broadcast shortcuts (--fps-ntsc-film, --fps-pal) - Demonstrates format presets (--preset-1080p59.94) - Cinema workflow example with DCI 4K - Validates exact timing calculations ## Final Statistics - 7 phases completed (Python 3 port through end-to-end integration) - 168+ tests pass across all modules - 6 new modules created - Full backward compatibility maintained - Exact rational arithmetic throughout The DVB CSS synchronization timing measurement system now fully supports professional broadcast standards with fractional frame rates.
Document argparse interface and add README for fractional frame rates with quick start examples and technical details.
- Display fractional FPS with 3 decimal places (e.g., 29.970 fps) - Keep integer FPS without decimals (e.g., 25 fps) - Fixes issue where 29.97 fps was displayed as "29 fps" - Update .gitignore to exclude test output files
- Created test_math_verification.py with 16 comprehensive tests - Verified NTSC frame rate precision (24000/1001, 30000/1001, etc.) - Tested exact second boundaries and timing relationships - Verified native timing vs drop-frame behavior - Tested event timing precision and bit timing calculations - Verified audio/video sync precision - Tested real-world broadcast scenarios (1 hour, 24 hour drift) - Fixed Python 3 compatibility in test_audio.py - All mathematical calculations verified for accuracy
- Updated IMPROVEMENTS.md with comprehensive bit timing explanation - Enhanced README_FRACTIONAL.md with technical details - Created BIT_TIMING_DESIGN.md with complete design rationale - Added TIMING_DIAGRAMS.md with visual timing charts and diagrams - Explained why 29.97 fps uses 30 fps base timing (7/60, 19/60) - Documented hardware compatibility and detection benefits - Added mathematical examples and frame alignment calculations - Covered design alternatives and implementation decisions - Included ASCII diagrams showing frame alignment and timing
- Convert all print statements to use Python 3 print() function syntax - Add test to verify Python 3 compatibility for the module - Ensures module can be imported and executed without syntax errors
- Replace tuple parameter unpacking in ConvertAtoB and ErrorBoundInterpolator - Fix lambda tuple unpacking for Python 3 compatibility - Fix tab/space indentation issues - Add comprehensive tests for the fixed functionality
- Organize imports according to PEP8 standards - Remove unused imports - Sort imports alphabetically within groups
- Convert all print statements to Python 3 print() function - Fixed in exampleCsaTester.py, exampleTVTester.py, stats.py, testsetupcmdline.py - Ensures all scripts are Python 3 compatible
- Replace assertEquals with assertEqual (40 instances) - Replace assertNotEquals with assertNotEqual (1 instance) - Updated all test files to use modern unittest assertion methods
- Remove unused imports - Fix f-string formatting issues - Fix minor code style issues
- Convert raw_input() to input() for Python 3 compatibility - Fix None comparisons to use 'is' operator (E711) - Move import statements to top of file (E402) - Ensures critical Python 3 compatibility issues are resolved
- Remove all unused variable assignments (F841) - 14 instances - Convert lambda assignments to proper def functions (E731) - 7 instances - Improve code readability and maintainability - Follow PEP 8 style guidelines
- Document comprehensive code quality improvements - Show 98.5% error reduction (194 → 3 errors) - List all major improvements and testing strategy - Provide clear record of modernization effort
- Replace incorrect 'internal BBC tool' headers with Apache License 2.0 - Align with the actual project license as shown in COPYING file - Ensure all test files follow the same licensing as source files
- Convert all print statements in docstrings to Python 3 syntax - Fix map() usage to return lists for Python 3 compatibility - Update IMPROVEMENTS.md to reflect completed Python 3 migration - Fix remaining raw_input in docstring example - Ensure all code examples use modern Python syntax
- Document completed Python 3 migration achievement - Outline fractional frame rate support as next priority - List broadcast standard shortcuts and resolution presets - Provide development guidelines and immediate action items - Create clear roadmap for continued development
…ementation - Update IMPROVEMENTS.md to mark fractional frame rates as completed - Rewrite NEXT_STEPS.md to reflect both Python 3 and fractional rates are done - Acknowledge comprehensive broadcast standard support already exists - Document current production-ready capabilities - Adjust roadmap to focus on advanced future enhancements only
- Delete IMPROVEMENTS.md.bak and IMPROVEMENTS.md.bak2 (backup files) - Remove RUFF_PASS_SUMMARY.md (superseded by updated documentation) - Clean up repository to keep only current, relevant documentation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Major Changes
Python 3 Migration
Fractional Frame Rate Support
Documentation Updates
Testing
Note on Contribution
As per CONTRIBUTING.md, this is a substantial contribution that includes new features and improvements. The contribution process mentions that for such contributions, a Contribution Agreement may be required.
Backward Compatibility
All existing functionality is preserved. The implementation maintains compatibility with: