Data pipeline refactor

[networmix]

No description provided.

[Copilot]

Pull Request Overview

This PR represents a comprehensive refactor of the NetGraph data pipeline, replacing the previous workflow step pattern with a new structured results schema and updating numerous test files to align with the modernized API.

Key Changes:

• Migrated from TrafficMatrixPlacementAnalysis to TrafficMatrixPlacement with updated execution patterns
• Replaced capacity envelope analysis with MaxFlow step producing unified flow results
• Introduced new results schema with standardized flow_results structure replacing ad-hoc envelope data

Reviewed Changes

Copilot reviewed 80 out of 81 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
tests/workflow/test_traffic_matrix_placement_analysis.py	Updated test patterns to use new TrafficMatrixPlacement API and flow_results schema
tests/workflow/test_tm_analysis_perf_safety.py	Migrated to new TrafficMatrixPlacement class with Results store interface
tests/workflow/test_notebook_analysis.py	Updated PackageManager tests to use check_packages instead of installation methods
tests/workflow/test_network_stats.py	Converted to new Results schema with step-scoped data storage
tests/workflow/test_namespace_alignment.py	Updated to use new results export format and step data structure
tests/workflow/test_msd_perf_safety.py	Migrated to MaximumSupportedDemand class with new Results API
tests/workflow/test_maximum_supported_demand.py	Updated to use MaximumSupportedDemand and new results schema
tests/workflow/test_cost_power_efficiency.py	Converted to new Results store with step-scoped data access
tests/workflow/test_capacity_envelope_analysis.py	Renamed to MaxFlow step with unified flow results structure
tests/workflow/test_build_graph.py	Updated to support both in-memory and serialized graph representations
tests/workflow/test_base.py	Simplified to use new step metadata storage pattern
tests/workflow/test_analysis_integration.py	Updated workflow integration tests for new MaxFlow API
tests/workflow/analysis/test_placement_matrix.py	Converted to use flow_results data structure instead of envelope patterns
tests/workflow/analysis/test_capacity_matrix.py	Significantly simplified to work with new flow results schema
tests/scenario/test_scenario.py	Updated scenario execution tests for new results export format
tests/results/test_serialisation.py	Updated serialization tests for new step-scoped results structure
tests/results/test_result.py	Converted to use new Results API with enter/exit step pattern
tests/report/test_report.py	Updated sample data to reflect new results schema
tests/profiling/test_reporter_smoke.py	Removed placeholder test file
tests/monte_carlo/test_results.py	Retained only SensitivityResults tests after API simplification
tests/monte_carlo/test_functions.py	Updated to work with new FlowIterationResult return types
tests/integration/test_template_examples.py	Updated to use node_link_to_graph for graph reconstruction
tests/integration/test_scenario_4.py	Converted to use new results schema and graph deserialization
tests/integration/test_scenario_3.py	Updated capacity analysis tests for MaxFlow API
tests/integration/test_scenario_2.py	Updated to use graph deserialization from new results format
tests/integration/test_scenario_1.py	Updated basic scenario tests for new results structure
tests/integration/test_error_cases.py	Updated error handling tests to use graph deserialization
tests/integration/test_data_templates.py	Updated workflow templates to use MaxFlow instead of CapacityEnvelopeAnalysis
tests/integration/scenario_4.yaml	Updated YAML configuration to use MaxFlow step type
tests/integration/scenario_3.yaml	Updated YAML configuration to use MaxFlow step type
tests/integration/helpers.py	Enhanced helper functions to work with new results schema
tests/flows/test_policy.py	Updated test comments for flow placement tolerance behavior
tests/failure/test_manager_integration.py	Updated to work with FlowIterationResult return types
tests/failure/test_manager.py	Simplified convenience method tests for new API contracts
tests/dsl/test_examples.py	Updated workflow execution tests for new results access pattern

Comments suppressed due to low confidence (1)

tests/workflow/test_capacity_envelope_analysis.py:178

• [nitpick] This line contains nested conditional logic that could be simplified. Consider extracting the flow results validation into a helper method or using guard clauses for better readability.

        )