Skip to content

Commit b4dde06

Browse files
Add Multi-Period-modeling and stochastic modeling to flixopt (#348)
* V3.0.0/main (#284) * Bugfix plot_node_balance_pie() * Scenarios/fixes (#252) * BUGFIX missing conversion to TimeSeries * BUGFIX missing conversion to TimeSeries * Bugfix node_balance with flow_hours: Negate correctly * Scenarios/filter (#253) * Add containts and startswith to filter_solution * Scenarios/drop suffix (#251) Drop suffixes in plots and add the option to drop suffixes to sanitize_dataset() * Scenarios/bar plot (#254) * Add stacked bar style to plotting methods * Rename mode to style (line, bar, area, ...) * Bugfix plotting * Fix example_calculation_types.py * Scenarios/fixes (#255) * Fix indexing issue with only one scenario * Bugfix Cooling Tower * Add option for balanced Storage Flows (equalize size of charging and discharging) * Add option for balanced Storage Flows * Change error to warning (non-fixed size with piecewise conversion AND fixed_flow_rate with OnOff) * Bugfix in DataConverter * BUGFIX: Typo (total_max/total_min in Effect) * Bugfix in node_balance() (negating did not work when using flow_hours mode * Scenarios/effects (#256) * Add methods to track effect shares of components and Flows * Add option to include flows when retrieving effects * Add properties and methods to store effect results in a dataset * Reorder methods * Rename and improve docs * Bugfix test class name * Fix the Network algorithm to calculate the sum of parallel paths, and be independent on nr of nodes and complexity of the network * Add tests for the newtork chaining and the results of effect shares * Add methods to check for circular references * Add test to check for circular references * Update cycle checker to return the found cycles * Add checks in results to confirm effects are computed correctly * BUGFIX: Remove +1 from prior testing * Add option for grouped bars to plotting.with_plotly() and make lines of stacked bar plots invisible * Reconstruct FlowSystem in CalculationResults on demand. DEPRECATION in CalculationResults * ruff check * Bugfix: save flow_system data, not the flow_system * Update tests * Scenarios/datasets results (#257) * Use dataarray instead of dataset * Change effects dataset to dataarray and use nan when no share was found * Add method for flow_rates dataset * Add methods to get flow_rates and flow_hours as datasets * Rename the dataarrays to the flow * Preserve index order * Improve filter_edges_dataset() * Simplify _create_flow_rates_dataarray() * Add dataset for sizes of Flows * Extend results structure to contain flows AND start/end infos * Add FlowResults Object * BUGFIX:Typo in _ElementResults.constraints * Add flows to results of Nodes * Simplify dataarray creation and improve FlowResults * Add nice docstrings * Improve filtering of flow results * Improve filtering of flow results. Add attribute of component * Add big dataarray with all variables but indexed * Revert "Add big dataarray with all variables but indexed" This reverts commit 08cd8a1. * Improve filtering method for coords filter and add error handling for restoring the flow system * Remove unnecessary methods in results .from_json() * Ensure consistent coord ordering in Effects dataarray * Rename get_effects_per_component() * Make effects_per_component() a dataset instead of a dataarray * Improve backwards compatability * ruff check * ruff check * Scenarios/deprecation (#258) * Deprecate .active_timesteps * Improve logger warning * Starting release notes * Bugfix in plausibility_check: Index 0 * Set bargap to 0 in stacked bars * Ensure the size is always properly indexed in results. * ruff check * BUGFIX in extract data, that causes coords in linopy to be incorrect (scalar xarray.DataArrays) * Improve yaml formatting for model documentation (#259) * Make the size/capacity a TimeSeries (#260) * Scenarios/plot network (#262) * Catch bug in plot_network with 2D arrays * Add plot_network() to test_io.py * Update deploy-docs.yaml: Run on Release publishing instead of creation and only run for stable releases (vx.y.z) * Bugfix DataConverter and add tests (#263) * Fix doc deployment to not publish on non stable releases * Remove unused code * Remove legend placing for better auto placing in plotly * Fix plotly dependency * Improve validation when adding new effects * Moved release notes to CHANGELOG.md * Try to add to_dataset to Elements * Remove TimeSeries * Remove TimeSeries * Rename conversion method to pattern: to_... * Move methods to FlowSystem * Drop nan values across time dimension if present * Allow lists of values to create DataArray * Update resolving of FlowSystem * Simplify TimeSeriesData * Move TImeSeriesData to Structure and simplyfy to inherrit from xarray.DataArray * Adjust IO * Move TimeSeriesData back to core.py and fix Conversion * Adjust IO to account for attrs of DataArrays in a Dataset * Rename transforming and connection methods in FlowSystem * Compacted IO methods * Remove infos() * remove from_dict() and to_dict() * Update __str__ of Interface * Improve str and repr * Improve str and repr * Add docstring * Unify IO stuff in Interface class * Improve test tu utilize __eq__ method * Make Interface class more robust and improve exceptions * Add option to copy Interfaces (And the FlowSystem) * Make a copy of a FLowSytsem that gets reused in a second Calculation * Remove test_timeseries.py * Reorganizing Datatypes * Remove TImeSeries and TimeSeriesCollection entirely * Remove old method * Add option to get structure with stats of dataarrays * Change __str__ method * Remove old methods * remove old imports * Add isel, sel and resample methods to FlowSystem * Remove need for timeseries with extra timestep * Simplify IO of FLowSystem * Remove parameter timesteps from IO * Improve Exceptions and Docstrings * Improve isel sel and resample methods * Change test * Bugfix * Improve * Improve * Add test for Storage Bounds * Add test for Storage Bounds * CHANGELOG.md * ruff check * Improve types * CHANGELOG.md * Bugfix in Storage * Revert changes in example_calculation_types.py * Revert changes in simple_example.py * Add convenient access to Elements in FlowSystem * Get Aggregated Calculation Working * Segmented running with wrong results * Use new persistent FLowSystem to create Calculations upfront * Improve SegmentedCalcualtion * Improve SegmentedCalcualtion * Fix SegmentedResults IO * ruff check * Update example * Updated logger essages to use .label_full instead of .label * Re-add parameters. Use deprecation warning instead * Update changelog * Improve warning message * Merge * Merge * Fit scenario weights to model coords when transforming * Merge * Removing logic between minimum, maximum and fixed size from InvestParameters * Remove selected_timesteps * Improve TypeHints * New property on InvestParameters for min/max/fixed size * Move logic for InvestParameters in Transmission to from Model to Interface * Make transformation of data more hierarchical (Flows after Components) * Add scenario validation * Change Transmission to have a "balanced" attribute. Change Tests accordingly * Improve index validations * rename method in tests * Update DataConverter * Add DataFrame Support back * Add copy() to DataConverter * Update fit_to_model_coords to take a list of coords * Make the DataConverter more universal by accepting a list of coords/dims * Update DataConverter for n-d arrays * Update DataConverter for n-d arrays * Add extra tests for 3-dims * Add FLowSystemDimension Type * Revert some logic about the fit_to_model coords * Adjust FLowSystem IO for scenarios * BUGFIX: Raise Exception instead of logging * Change usage of TimeSeriesData * Adjust logic to handle non scalars * Adjust logic to _resolve_dataarray_reference into separate method * Update IO of FlowSystem * Improve get_coords() * Adjust FlowSystem init for correct IO * Add scenario to sel and isel methods, and dont normalize scenario weights * Improve scenario_weights_handling * Add warning for not scaled weights * Update test_scenarios.py * Improve util method * Add objective to solution dataset. * Update handling of scenario_weights update tests * Ruff check. Fix type hints * Fix type hints and improve None handling * Fix coords in AggregatedCalculation * Improve Error Messages of DataConversion * Allow multi dim data conversion and broadcasting by length * Improve DataConverter to handle multi-dim arrays * Rename methods and remove unused code * Improve DataConverter by better splitting handling per datatype. Series only matches index (for one dim). Numpy matches shape * Add test for error handling * Update scenario example * Fix Handling of TimeSeriesData * Improve DataConverter * Fix resampling of the FlowSystem * Improve Warning Message * Add example that leverages resampling * Add example that leverages resampling adn fixing of Investments * Add flag to Calculation if its modeled * Make flag for connected_and_transformed FLowSystem public * Make Calcualtion Methods return themselfes to make them chainable * Improve example * Improve Unreleased CHANGELOG.md * Add year coord to FlowSystem * Improve dimension handling * Change plotting to use an indexer instead * Change plotting to use an indexer instead * Use tuples to set dimensions in Models * Bugfix in validation logic and test * Improve Errors * Improve weights handling and rescaling if None * Fix typehint * Update Broadcasting in Storage Bounds and improve type hints * Make .get_model_coords() return an actual xr.Coordinates Object * Improve get_coords() * Rename SystemModel to FlowSystemModel * First steps * Improve Feature Patterns * Improve acess to variables via short names * Improve * Add naming options to big_m_binary_bounds() * Fix and improve FLowModeling with Investment * Improve * Tyring to improve the Methods for bounding variables in different scenarios * Improve BoundingPatterns * Improve BoundingPatterns * Improve BoundingPatterns * Fix duration Modeling * Fix On + Size * Fix InvestmentModel * Fix Models * Update constraint names in test * Fix OnOffModel for multiple Flows * Update constraint names in tests * Simplify * Improve handling of vars/cons and models * Revising the basic structure of a class Model * Revising the basic structure of a class Model * Simplify and focus more on own Model class * Update tests * Improve state computation in ModelingUtilities * Improve handling of previous flowrates * Imropove repr and submodel acess * Update access pattern in tests * Fix PiecewiseEffects and StorageModel * Fix StorageModel and Remove PreventSimultaniousUseModel * Fix Aggregation and SegmentedCalculation * Update tests * Loosen precision in tests * Update test_on_hours_computation.py and some types * Rename class Model to Submodel * rename sub_model to submodel everywhere * rename self.model to self.submodel everywhere * Rename .model with .submodel if its only a submodel * Rename .sub_models with .submodels * Improve repr * Improve repr * Include def do_modeling() into __init__() of models * Make properties private * Improve Inheritance of Models * V3.0.0/plotting (#285) * Use indexer to reliably plot solutions with and wihtout scenarios/years * ruff check * Improve typehints * Update CHANGELOG.md * Bugfix from renaming to .submodel * Bugfix from renaming to .submodel * Improve indexer in results plotting * rename register_submodel() to .add_submodels() adn add SUbmodels collection class * Add nice repr to FlowSystemModel and Submodel * Bugfix .variables and .constraints * Add type checks to modeling.py * Improve assertion in tests * Improve docstrings and register ElementModels directly in FlowSystemModel * Improve __repr__() * ruff check * Use new method to compare sets in tests * ruff check * Update Contribute.md, some dependencies and add pre-commit * Pre commit hook * Run Pre-Commit Hook for the first time * Fix link in README.md * Update Effect name in tests to be 'costs' instead of 'Costs' Everywhere Simplify testing by creating a Element Library * Improve some of the modeling and coord handling * Add tests with years and scenarios * Update tests to run with multiple coords * Fix Effects dataset computation in case of empty effects * Update Test for multiple dims Fix Dim order in scaled_bounds_with_state Bugfix logic in .use_switch_on * Fix test with multiple dims * Fix test with multiple dims * New test * New test for previous flow_rates * V3.0.0/main fit to model coords improve (#295) * Change fit_to_model_coords to work with a Collection of dims * Improve fit_to_model_coords * Improve CHANGELOG.md * Update pyproject.toml * new ruff check * Merge branch 'main' into dev # Conflicts: # CHANGELOG.md # flixopt/network_app.py * Update CHANGELOG.md * Fix Error message * Revert changes * Feature/v3/update (#352) * Remove need for timeseries with extra timestep * Simplify IO of FLowSystem * Remove parameter timesteps from IO * Improve Exceptions and Docstrings * Improve isel sel and resample methods * Change test * Bugfix * Improve * Improve * Add test for Storage Bounds * Add test for Storage Bounds * CHANGELOG.md * ruff check * Improve types * CHANGELOG.md * Bugfix in Storage * Revert changes in example_calculation_types.py * Revert changes in simple_example.py * Add convenient access to Elements in FlowSystem * Get Aggregated Calculation Working * Segmented running with wrong results * Use new persistent FLowSystem to create Calculations upfront * Improve SegmentedCalcualtion * Improve SegmentedCalcualtion * Fix SegmentedResults IO * ruff check * Update example * Updated logger essages to use .label_full instead of .label * Re-add parameters. Use deprecation warning instead * Update changelog * Improve warning message * Merge * Merge * Fit scenario weights to model coords when transforming * Merge * Removing logic between minimum, maximum and fixed size from InvestParameters * Remove selected_timesteps * Improve TypeHints * New property on InvestParameters for min/max/fixed size * Move logic for InvestParameters in Transmission to from Model to Interface * Make transformation of data more hierarchical (Flows after Components) * Add scenario validation * Change Transmission to have a "balanced" attribute. Change Tests accordingly * Improve index validations * rename method in tests * Update DataConverter * Add DataFrame Support back * Add copy() to DataConverter * Update fit_to_model_coords to take a list of coords * Make the DataConverter more universal by accepting a list of coords/dims * Update DataConverter for n-d arrays * Update DataConverter for n-d arrays * Add extra tests for 3-dims * Add FLowSystemDimension Type * Revert some logic about the fit_to_model coords * Adjust FLowSystem IO for scenarios * BUGFIX: Raise Exception instead of logging * Change usage of TimeSeriesData * Adjust logic to handle non scalars * Adjust logic to _resolve_dataarray_reference into separate method * Update IO of FlowSystem * Improve get_coords() * Adjust FlowSystem init for correct IO * Add scenario to sel and isel methods, and dont normalize scenario weights * Improve scenario_weights_handling * Add warning for not scaled weights * Update test_scenarios.py * Improve util method * Add objective to solution dataset. * Update handling of scenario_weights update tests * Ruff check. Fix type hints * Fix type hints and improve None handling * Fix coords in AggregatedCalculation * Improve Error Messages of DataConversion * Allow multi dim data conversion and broadcasting by length * Improve DataConverter to handle multi-dim arrays * Rename methods and remove unused code * Improve DataConverter by better splitting handling per datatype. Series only matches index (for one dim). Numpy matches shape * Add test for error handling * Update scenario example * Fix Handling of TimeSeriesData * Improve DataConverter * Fix resampling of the FlowSystem * Improve Warning Message * Add example that leverages resampling * Add example that leverages resampling adn fixing of Investments * Add flag to Calculation if its modeled * Make flag for connected_and_transformed FLowSystem public * Make Calcualtion Methods return themselfes to make them chainable * Improve example * Improve Unreleased CHANGELOG.md * Add year coord to FlowSystem * Improve dimension handling * Change plotting to use an indexer instead * Change plotting to use an indexer instead * Use tuples to set dimensions in Models * Bugfix in validation logic and test * Improve Errors * Improve weights handling and rescaling if None * Fix typehint * Update Broadcasting in Storage Bounds and improve type hints * Make .get_model_coords() return an actual xr.Coordinates Object * Improve get_coords() * Rename SystemModel to FlowSystemModel * First steps * Improve Feature Patterns * Improve acess to variables via short names * Improve * Add naming options to big_m_binary_bounds() * Fix and improve FLowModeling with Investment * Improve * Tyring to improve the Methods for bounding variables in different scenarios * Improve BoundingPatterns * Improve BoundingPatterns * Improve BoundingPatterns * Fix duration Modeling * Fix On + Size * Fix InvestmentModel * Fix Models * Update constraint names in test * Fix OnOffModel for multiple Flows * Update constraint names in tests * Simplify * Improve handling of vars/cons and models * Revising the basic structure of a class Model * Revising the basic structure of a class Model * Simplify and focus more on own Model class * Update tests * Improve state computation in ModelingUtilities * Improve handling of previous flowrates * Imropove repr and submodel acess * Update access pattern in tests * Fix PiecewiseEffects and StorageModel * Fix StorageModel and Remove PreventSimultaniousUseModel * Fix Aggregation and SegmentedCalculation * Update tests * Loosen precision in tests * Update test_on_hours_computation.py and some types * Rename class Model to Submodel * rename sub_model to submodel everywhere * rename self.model to self.submodel everywhere * Rename .model with .submodel if its only a submodel * Rename .sub_models with .submodels * Improve repr * Improve repr * Include def do_modeling() into __init__() of models * Make properties private * Improve Inheritance of Models * V3.0.0/plotting (#285) * Use indexer to reliably plot solutions with and wihtout scenarios/years * ruff check * Improve typehints * Update CHANGELOG.md * Bugfix from renaming to .submodel * Bugfix from renaming to .submodel * Improve indexer in results plotting * rename register_submodel() to .add_submodels() adn add SUbmodels collection class * Add nice repr to FlowSystemModel and Submodel * Bugfix .variables and .constraints * Add type checks to modeling.py * Improve assertion in tests * Improve docstrings and register ElementModels directly in FlowSystemModel * Improve __repr__() * ruff check * Use new method to compare sets in tests * ruff check * Update Contribute.md, some dependencies and add pre-commit * Pre commit hook * Run Pre-Commit Hook for the first time * Fix link in README.md * Update Effect name in tests to be 'costs' instead of 'Costs' Everywhere Simplify testing by creating a Element Library * Improve some of the modeling and coord handling * Add tests with years and scenarios * Update tests to run with multiple coords * Fix Effects dataset computation in case of empty effects * Update Test for multiple dims Fix Dim order in scaled_bounds_with_state Bugfix logic in .use_switch_on * Fix test with multiple dims * Fix test with multiple dims * New test * New test for previous flow_rates * Add Model for YearAwareInvestments * Add FlowSystem.years_per_year attribute and "years_of_last_year" parameter to FlowSystem() * Add YearAwareInvestmentModel * Add new Interface * Improve YearAwareInvestmentModel * Rename and improve * Move piecewise_effects * COmbine TImingInvestment into a single interface * Add model tests for investment * Add size_changes variables * Add size_changes variables * Improve InvestmentModel * Improve InvestmentModel * Rename parameters * remove old code * Add a duration_in_years to the InvestTimingParameters * Improve handling of fixed_duration * Improve validation and make Investment/divestment optional by default * Rename some vars and improve previous handling * Add validation for previous size * Change fit_to_model_coords to work with a Collection of dims * Improve fit_to_model_coords * Improve test * Update transform_data() * Add new "year of investment" coord to FlowSystem * Add 'year_of_investment' dimension to FlowSystem * Improve InvestmentTiming * Improve InvestmentTiming * Add specific_effect back * add effects_by_investment_year back * Add year_of_investment to FLowSystem.sel() * Improve Interface * Handle selection of years properly again * Temp * Make ModelingPrimitives.consecutive_duration_tracking() dim-agnostic * Use new lifetime variable and constraining methods * Improve Plausibility check * Improve InvestmentTImingParameters * Improve weights * Adjust test * Remove old classes * V3.0.0/main fit to model coords improve (#295) * Change fit_to_model_coords to work with a Collection of dims * Improve fit_to_model_coords * ruff format * Revert changes * Update type hints * Revert changes introduced by new Multiperiod Invest parameters * Improve CHnagelog and docstring of Storage * Improve Changelog * Improve InvestmentModel * Improve InvestmentModel to have 2 cases. One without years and one with * Improve InvestmentModel to have 2 cases. One without years and one with years. Further, remove investment_scenarios parameter * Revert some changes regarding Investments * Typo * Remove Investment test file (only local testing) * More reverted changes * More reverted changes * Add years_of_last_year to docstring * Revert change from Investment * Revert change from Investment * Remove old todos.txt file * Fix typos in CHANGELOG.md * Improve usage of name_prefix to intelligently join with the label * Ensure IO of years_of_last_year * Typo * Typo * activat tests on pulls to feature/v3 * activat tests on pulls to feature/v3/main * Feature/v3/low-impact-improvements (#355) * Fix typo * Prefer robust scalar extraction for timestep sizes in aggregation * Improve docs and error messages * Update examples * Use validated timesteps * Remove unnessesary import * Use FlowSystem.model instead of FlowSystem.submodel * Fix Error message * Improve CHANGELOG.md * Use self.standard_effect instead of provate self._standard_effect and update docstring * in calculate_all_conversion_paths, use `collections.deque` for efficiency on large graphs * Make aggregation_parameters.hours_per_period more robust by using rounding * Improve import and typos * Improve docstring * Use validated timesteps * Improve error * Improve warning * Improve type hint * Improve CHANGELOG.md: typos, wording and duplicate entries * Improve CI (#357) Separate example testing from other tests by marking them. By default, purest doesn't run the example tests * Feature/v3/data converter (#356) * Update DataConverter * Update tests of error messages * Update tests of error messages * Update Dataconverter to allow bool values * fix tests * Improve code order of prefix in transform_data() * Move pytest-xdist to dev deps * Fix transform_data to not pass a prefix to flow * Move to unreleased Add emojis to CHANGELOG.md * Feature/v3/feature/308 rename effect domains (#365) * Rename effect domains * Rename effect domains * Ensure backwards compatability * Improve * Improve * Bugfix IO with deprectaed params * Add guards for extra kwargs * Add guards for extra kwargs * centralize logic for deprectaed params * Move handlign from centralized back to classes in a dedicated method * Improce property handling * Move handling to Interface class * Getting lost * Revert "Getting lost" This reverts commit 3c0db76. * Revert "Move handling to Interface class" This reverts commit 09bdeec. * Revert "Improce property handling" This reverts commit 5fe2c64. * Revert "Move handlign from centralized back to classes in a dedicated method" This reverts commit 9f4c1f6. * Revert "centralize logic for deprectaed params" This reverts commit 4a82574. * Add "" to warnings * Revert change in examples * Improve BackwardsCompatibleDataset * Add unit tests for backwards compatability * Remove backwards compatible dataset * Renamed maximum_temporal_per_hour to maximum_per_hour and minimum_temporal_per_hour to minimum_per_hour * Add entires to CHANGELOG.md * Remove backwards compatible dataset * Remove unused imports * Move to unreleased * Catch up on missed renamings from merge * Catch up on missed renamings from merge * Typo * Typo * Several small improvements or potential future bug preventions * Feature/v3/feature/305 rename specific share to other effects to specific share from effect (#366) * Step 1 * Bugfix * Make fit_effects_to_model_coords() more flexible * Fix dims * Update conftest.py * Typos * Improve Effect examples * Add extra validation for Effect Shares * Feature/v3/feature/367 rename year dimension to period (#370) * The framework now uses "period" instead of "year" as the dimension name and "periodic" instead of "nontemporal" for the effect domain * Update CHANGELOG.md * Remove periods_of_last_period parameter and adjust weights calculation * Bugfix * Bugfix * Switch from "as_time_series": bool to "dims": [time, period, scenario] arguments * Improve normalization of weights * Update tests * Typos in docs * Improve docstrings * Improve docstrings * Update CHANGELOG.md * Improved tests: added extra time+scenarios combination * Add rename and improve CHANGELOG.md * Made CHANGELOG.md more concise * Simplify array summation and improve `np.isclose` usage in `modeling` and `aggregation` modules. * Make storage and load profile methods flexible by introducing `timesteps_length` parameter; update test configurations accordingly. * Refine error messages in `ModelingPrimitives` to correctly reference updated method names. * Enhance test fixtures by adding `ids` for parameterized tests, improve input flexibility with dynamic timestep length, and refine error message sorting logic. * Refactor variable selection and constraint logic in `aggregation.py` for handling more than only a time dimension * Adjust constraint in `aggregation.py` to enforce stricter summation limit (1 instead of 1.1) * Reverse transition constraint inequality for consistency in `modeling.py`. * Update dependency to use h5netcdf instead of netcdf4 * Feature/v3/several improvements (#372) * Update deprecated properties to use new aggregation attributes in `core.py`. * Refactor `drop_constant_arrays` in `core.py` to improve clarity, add type hints, and enhance logging for dropped variables. * Bugfix example_calculation_types.py and two_stage_optimization.py * Use time selection more explicitly * Refactor plausibility checks in `components.py` to handle string-based `initial_charge_state` more robustly and simplify capacity bounds retrieval using `InvestParameters`. * Refactor `create_transmission_equation` in `components.py` to handle `relative_losses` gracefully when unset and simplify the constraint definition. * Update pytest `addopts` formatting in `pyproject.toml` to work with both unix and windows * Refine null value handling when resolving dataarrays` to check for 'time' dimension before dropping all-null values. * Refactor flow system restoration to improve exception handling and ensure logger state resets. * Refactor imports in `elements.py` to remove unused `ModelingPrimitives` from `features` and include it from `modeling` instead. * Refactor `count_consecutive_states` in `modeling.py` to enhance documentation, improve edge case handling, and simplify array processing. * Refactor `drop_constant_arrays` to handle NaN cases with `skipna` and sort dropped variables for better logging; streamline logger state restoration in `results.py`. * Temp * Improve NAN handling in count_consecutive_states() * Refactor plausibility checks in `components.py` to prevent initial capacity from constraining investment decisions and improve error messaging. * Feature/v3/feature/no warnings in tests (#373) * Refactor examples to not use deprectaed patterns * Refactor tests to replace deprecated `sink`/`source` properties with `inputs`/`outputs` in component definitions. * Use 'h' instead of deprectaed 'H' in coordinate freq in tests; adjust `xr.concat` in `results.py` to use `join='outer'` for safer merging. * Refactor plot tests to use non-interactive backends, save plots as files, and close figures to prevent memory leaks. * Refactor plot tests to use non-interactive Plotly renderer (`json`), add cleanup with `tearDown`, and ensure compatibility with non-interactive Matplotlib backends. * Configure pytest filters to treat most warnings as errors, ignore specific third-party warnings, and display all warnings from internal code. * Revert "Configure pytest filters to treat most warnings as errors, ignore specific third-party warnings, and display all warnings from internal code." This reverts commit 0928b26. * Refactor plotting logic to prevent memory leaks, improve backend handling, and add test fixtures for cleanup and non-interactive configurations. * Update pytest filterwarnings to treat most warnings as errors, ignore specific third-party warnings, and display all internal warnings. * Suppress specific third-party warnings in `__init__.py` to reduce noise for end users. * Update pytest warning filters: treat internal warnings as errors, revert treating most third-party warnings as errors. * Suppress additional third-party warnings in `__init__.py` to minimize runtime noise. * Update pytest warning filters: suppress specific third-party warnings and add detailed context for `__init__.py` filters. * Sync and consolidate third-party warning filters in `__init__.py` and `pyproject.toml` to suppress runtime noise effectively. * Expand and clarify third-party warning filters in `__init__.py` and `pyproject.toml` for improved runtime consistency and reduced noise. * Update deprecated code in tests * Refactor backend checks in `plotting.py` and streamline test fixtures for consistency in handling non-interactive backends. * Refactor plotting logic to handle test environments explicitly, remove unused Plotly configuration, and improve figure cleanup in tests. * Add entry to CHANGELOG.md * Typos in example * Reogranize Docs (#377) * Improve effects parameter naming in InvestParameters (#389) * FIrst Try * Improve deprecation * Update usage of deprectated parameters * Improve None handling * Add extra kwargs handling * Improve deprecation * Use custom method for kwargs * Add deprecation method * Apply deprecation method to other classes * Apply to effects.py as well * Update usage of deprectaed parameters * Update CHANGELOG.md * Update Docs * Feature/v3/feature/test examples dependent (#390) * Update example test to run dependent examples in order * Update example test to run dependent examples in order * Update CHANGELOG.md * Improve test directory handling * Improve test directory handling * Typo * Feature/v3/feature/rename investparameter optional to mandatory (#392) * Change .optional to .mandatory * Change .optional to .mandatory * Remove not needed properties * Improve deprectation warnings * Improve deprectation of "optional" * Remove all usages of old "optional" parameter in code * Typo * Imrpove readability * Adjust some logging levels * Add scenarios and periods to repr and str of FlowSystem * Feature/v3/feature/386 use better default logging colors and dont log to file by default (#394) * Fix `charge_state` Constraint in `Storage` leading to incorrect losses in discharge and therefore incorrect charge states and discharge values (#347) * Fix equation in Storage * Fix test for equation in Storage * Update CHANGELOG.md * Improve Changelog Message * Fix CHANGELOG.md * Simplify changes from next release * Update CHANGELOG.md * Fix CHANGELOG.md * chore(deps): update dependency mkdocs-material to v9.6.20 (#369) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Improve renovate.json to automerge ruff despite 0.x version * chore(deps): update dependency tsam to v2.3.9 (#379) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * chore(deps): update dependency ruff to v0.13.2 (#378) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Feature/Improve Configuration options and handling (#385) * Refactor configuration management: remove dataclass-based schema and simplify CONFIG structure. * Refactor configuration loading: switch from `os` to `pathlib`, streamline YAML loading logic. * Refactor logging setup: split handler creation into dedicated functions, simplify configuration logic. * Improve logging configurability and safety - Add support for `RotatingFileHandler` to prevent large log files. - Introduce `console` flag for optional console logging. - Default to `NullHandler` when no handlers are configured for better library behavior. * Temp * Temp * Temp * Temp * Temp * Temp * Refactor configuration and logging: remove unused `merge_configs` function, streamline logging setup, and encapsulate `_setup_logging` as an internal function. * Remove unused `change_logging_level` import and export. * Add tests for config.py * Expand `config.py` test coverage: add tests for custom config loading, logging setup, dict roundtrip, and attribute modification. * Expand `test_config.py` coverage: add modeling config persistence test, refine logging reset, and improve partial config load assertions. * Expand `test_config.py` coverage: add teardown for state cleanup and reset modeling config in setup. * Add `CONFIG.reset()` method and expand test coverage to verify default restoration * Refactor `CONFIG` to centralize defaults in `_DEFAULTS` and ensure `reset()` aligns with them; add test to verify consistency. * Refactor `_DEFAULTS` to use `MappingProxyType` for immutability, restructure config hierarchy, and simplify `reset()` implementation for maintainability; update tests accordingly. * Mark `TestConfigModule` tests to run in a single worker with `@pytest.mark.xdist_group` to prevent global config interference. * Add default log file * Update CHANGELOG.md * Readd change_logging_level() for backwards compatability * Add more options to config.py * Add a docstring to config.y * Add a docstring to config.y * rename parameter message_format * Improve color config * Improve color config * Update CHANGELOG.md * Improve color handling * Improve color handling * Remove console Logging explicityl from examples * Make log to console the default * Make log to console the default * Add individual level parameters for console and file * Add extra Handler section * Use dedicated levels for both handlers * Switch back to not use Handlers * Revert "Switch back to not use Handlers" This reverts commit 05bbccb. * Revert "Use dedicated levels for both handlers" This reverts commit ed0542b. * Revert "Add extra Handler section" This reverts commit a133cc8. * Revert "Add individual level parameters for console and file" This reverts commit 19f81c9. * Fix CHANGELOG.md * Update CHANGELOG.md * Fix CHANGELOG.md * Allow blank issues * Change default logging behaviour to other colors and no file logging * Use white for INFO * Use terminal default for INFO * Explicitly use stdout for StreamHandler * Use terminal default for Logging color * Add option for loggger name * Update CHANGELOG.md * Ensure custom formats are being applied * Catch empty config files * Update test to match new defaults --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Fix warnings filter * Remove config file (#391) * Remove config file * Remove yaml in MANIFEST.in * Improve config console logger: Allow stderr and improve multiline format (#395) * Add some validation to config.py * Improve file Permission handling in config.py * Remove unwanted return * Improve Docstrings in config.py * Improve Docstrings in config.py * Typo * Use code block in docstring * Allow stderr for console logging * Make docstrings more compact * Make docstrings more compact * Updated to actually use stderr * Simplify format() * Improve format * Add extra validation * Update CHANGELOG.md * Feature/v3/feature/381 feature equalize sizes and or flow rates between scenarios (#396) * First try * Centralize in FlowSystem * Add centralized handling * Logical Bug * Add to IO * Add test * Add some error handling and logging * Rename variable * Change parameter naming * Remove not needed method * Refactor to reduce duplication * Change defaults * Change defaults * Change defaults * Update docs * Update docs * Update docs * Update docs * Feature/v3/feature/Linked investments over multiple periods * Reorganize InvestmentParameters to always create the binary investment variable * Add new variable that indicates wether investment was taken, independent of period and allow linked periods * Improve Handling of linked periods * Improve Handling of linked periods * Add examples * Typos * Fix: reference invested only after it exists * Improve readbility of equation * Update from Merge * Improve InvestmentModel * Improve readability * Improve readability and reorder methods * Improve logging * Improve InvestmentModel * Rename to "invested" * Update CHANGELOG.md * Bugfix * Improve docstring * Improve InvestmentModel to be more inline with the previous Version * Improve Exceptions and add a meaningfull comment in InvestParameters * Typo * Feature/v3/feature/common resources in examples (#401) * Typo * Typos in scenario_example.py * Improve data files in examples * Improve data files in examples * Handle local install more gracefully with __version__ * Remove bad example * Increase timeout in examples * Improve test_examples.py * Improve example * Fixx Error message in test * Fix: Dependecy issue with python 3.10 * run ci on more branches if there are prs * Minor improvements and Update to the CHANGELOG.md * Feature/v3/feature/last minute improvements (#403) * Typos oin CHANGELOG.md * Add error handling in exmaple * Surface warnings during tests (avoid hiding deprecations) * Add missing docs file * Imrpve Releasnotes of v2.2.0 * Improve docs * Remove some filterwarnings from tsam --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
1 parent f3e765c commit b4dde06

83 files changed

Lines changed: 14804 additions & 6360 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/python-app.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ on:
55
branches: [main] # Only main branch
66
tags: ['v*.*.*']
77
pull_request:
8-
branches: [main, dev]
8+
branches: [main, 'dev*', 'dev/**', 'feature/**']
99
types: [opened, synchronize, reopened]
1010
paths-ignore:
1111
- 'docs/**'
@@ -88,7 +88,7 @@ jobs:
8888
uv pip install --system .[dev]
8989
9090
- name: Run tests
91-
run: pytest -v -p no:warnings --numprocesses=auto
91+
run: pytest -v --numprocesses=auto
9292

9393
test-examples:
9494
runs-on: ubuntu-24.04

CHANGELOG.md

Lines changed: 146 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,7 @@ Please keep the format of the changelog consistent with the other releases, so t
4141
4242
---
4343
44+
4445
## [Unreleased] - ????-??-??
4546
4647
### ✨ Added
@@ -62,12 +63,149 @@ Please keep the format of the changelog consistent with the other releases, so t
6263
### 📝 Docs
6364
6465
### 👷 Development
65-
- Enable blank issues
6666
6767
### 🚧 Known Issues
6868
69+
---
6970
Until here -->
7071

72+
## [3.0.0] - 2025-10-13
73+
**Summary**: This release introduces new model dimensions (periods and scenarios) for multi-period investments and stochastic modeling, along with a redesigned effect sharing system and enhanced I/O capabilities.
74+
75+
### ✨ Added
76+
77+
**New model dimensions:**
78+
79+
- **Period dimension**: Enables multi-period investment modeling with distinct decisions in each period for transformation pathway optimization
80+
- **Scenario dimension**: Supports stochastic modeling with weighted scenarios for robust decision-making under uncertainty (demand, prices, weather)
81+
- Control variable independence across scenarios via `scenario_independent_sizes` and `scenario_independent_flow_rates` parameters
82+
- By default, investment sizes are shared across scenarios while flow rates vary per scenario
83+
84+
**Redesigned effect sharing system:**
85+
86+
Effects now use intuitive `share_from_*` syntax that clearly shows contribution sources:
87+
88+
```python
89+
costs = fx.Effect('costs', '', 'Total costs',
90+
share_from_temporal={'CO2': 0.2}, # From temporal effects
91+
share_from_periodic={'land': 100}) # From periodic effects
92+
```
93+
94+
This replaces `specific_share_to_other_effects_*` parameters and inverts the direction for clearer relationships.
95+
96+
**Enhanced I/O and data handling:**
97+
98+
- NetCDF/JSON serialization for all Interface objects and FlowSystem with round-trip support
99+
- FlowSystem manipulation: `sel()`, `isel()`, `resample()`, `copy()`, `__eq__()` methods
100+
- Direct access to FlowSystem from results without manual restoring (lazily loaded)
101+
- New `FlowResults` class and precomputed DataArrays for sizes/flow_rates/flow_hours
102+
- `effects_per_component` dataset for component impact evaluation, including all indirect effects through effect shares
103+
104+
**Other additions:**
105+
106+
- Balanced storage - charging and discharging sizes can be forced equal via `balanced` parameter
107+
- New Storage parameters: `relative_minimum_final_charge_state` and `relative_maximum_final_charge_state` for final state control
108+
- Improved filter methods in results
109+
- Example for 2-stage investment decisions leveraging FlowSystem resampling
110+
111+
### 💥 Breaking Changes
112+
113+
- `relative_minimum_charge_state` and `relative_maximum_charge_state` don't have an extra timestep anymore.
114+
- Renamed class `SystemModel` to `FlowSystemModel`
115+
- Renamed class `Model` to `Submodel`
116+
- Renamed `mode` parameter in plotting methods to `style`
117+
- Renamed investment binary variable `is_invested` to `invested` in `InvestmentModel`
118+
- `Calculation.do_modeling()` now returns the `Calculation` object instead of its `linopy.Model`. Callers that previously accessed the linopy model directly should now use `calculation.do_modeling().model` instead of `calculation.do_modeling()`.
119+
120+
### ♻️ Changed
121+
122+
- FlowSystems cannot be shared across multiple Calculations anymore. A copy of the FlowSystem is created instead, making every Calculation independent
123+
- Each Subcalculation in `SegmentedCalculation` now has its own distinct `FlowSystem` object
124+
- Type system overhaul - added clear separation between temporal and non-temporal data throughout codebase for better clarity
125+
- Enhanced FlowSystem interface with improved `__repr__()` and `__str__()` methods
126+
- Improved Model Structure - Views and organisation is now divided into:
127+
- Model: The main Model (linopy.Model) that is used to create and store the variables and constraints for the FlowSystem.
128+
- Submodel: The base class for all submodels. Each is a subset of the Model, for simpler access and clearer code.
129+
- Made docstrings in `config.py` more compact and easier to read
130+
- Improved format handling in configuration module
131+
- Enhanced console output to support both `stdout` and `stderr` stream selection
132+
- Added `show_logger_name` parameter to `CONFIG.Logging` for displaying logger names in messages
133+
134+
### 🗑️ Deprecated
135+
136+
- The `agg_group` and `agg_weight` parameters of `TimeSeriesData` are deprecated and will be removed in a future version. Use `aggregation_group` and `aggregation_weight` instead.
137+
- The `active_timesteps` parameter of `Calculation` is deprecated and will be removed in a future version. Use the new `sel(time=...)` method on the FlowSystem instead.
138+
- The assignment of Bus Objects to Flow.bus is deprecated and will be removed in a future version. Use the label of the Bus instead.
139+
- The usage of Effects objects in Dicts to assign shares to Effects is deprecated and will be removed in a future version. Use the label of the Effect instead.
140+
- **InvestParameters** parameters renamed for improved clarity around investment and retirement effects:
141+
- `fix_effects``effects_of_investment`
142+
- `specific_effects``effects_of_investment_per_size`
143+
- `divest_effects``effects_of_retirement`
144+
- `piecewise_effects``piecewise_effects_of_investment`
145+
- **Effect** parameters renamed:
146+
- `minimum_investment``minimum_periodic`
147+
- `maximum_investment``maximum_periodic`
148+
- `minimum_operation``minimum_temporal`
149+
- `maximum_operation``maximum_temporal`
150+
- `minimum_operation_per_hour``minimum_per_hour`
151+
- `maximum_operation_per_hour``maximum_per_hour`
152+
- **Component** parameters renamed:
153+
- `Source.source``Source.outputs`
154+
- `Sink.sink``Sink.inputs`
155+
- `SourceAndSink.source``SourceAndSink.outputs`
156+
- `SourceAndSink.sink``SourceAndSink.inputs`
157+
- `SourceAndSink.prevent_simultaneous_sink_and_source``SourceAndSink.prevent_simultaneous_flow_rates`
158+
159+
### 🔥 Removed
160+
161+
- **Effect share parameters**: The old `specific_share_to_other_effects_*` parameters were replaced WITHOUT DEPRECATION
162+
- `specific_share_to_other_effects_operation``share_from_temporal` (with inverted direction)
163+
- `specific_share_to_other_effects_invest``share_from_periodic` (with inverted direction)
164+
165+
### 🐛 Fixed
166+
167+
- Enhanced NetCDF I/O with proper attribute preservation for DataArrays
168+
- Improved error handling and validation in serialization processes
169+
- Better type consistency across all framework components
170+
- Added extra validation in `config.py` to improve error handling
171+
172+
### 📝 Docs
173+
174+
- Reorganized mathematical notation docs: moved to lowercase `mathematical-notation/` with subdirectories (`elements/`, `features/`, `modeling-patterns/`)
175+
- Added comprehensive documentation pages: `dimensions.md` (time/period/scenario), `effects-penalty-objective.md`, modeling patterns
176+
- Enhanced all element pages with implementation details, cross-references, and "See Also" sections
177+
- Rewrote README and landing page with clearer vision, roadmap, and universal applicability emphasis
178+
- Removed deprecated `docs/SUMMARY.md`, updated `mkdocs.yml` for new structure
179+
- Tightened docstrings in core modules with better cross-referencing
180+
- Added recipes section to docs
181+
182+
### 🚧 Known Issues
183+
184+
- IO for single Interfaces/Elements to Datasets might not work properly if the Interface/Element is not part of a fully transformed and connected FlowSystem. This arises from Numeric Data not being stored as xr.DataArray by the user. To avoid this, always use the `to_dataset()` on Elements inside a FlowSystem that's connected and transformed.
185+
186+
### 👷 Development
187+
188+
- **Centralized deprecation pattern**: Added `_handle_deprecated_kwarg()` helper method to `Interface` base class that provides reusable deprecation handling with consistent warnings, conflict detection, and optional value transformation. Applied across 5 classes (InvestParameters, Source, Sink, SourceAndSink, Effect) reducing deprecation boilerplate by 72%.
189+
- FlowSystem data management simplified - removed `time_series_collection` pattern in favor of direct timestep properties
190+
- Change modeling hierarchy to allow for more flexibility in future development. This leads to minimal changes in the access and creation of Submodels and their variables.
191+
- Added new module `.modeling` that contains modeling primitives and utilities
192+
- Clearer separation between the main Model and "Submodels"
193+
- Improved access to the Submodels and their variables, constraints and submodels
194+
- Added `__repr__()` for Submodels to easily inspect its content
195+
- Enhanced data handling methods
196+
- `fit_to_model_coords()` method for data alignment
197+
- `fit_effects_to_model_coords()` method for effect data processing
198+
- `connect_and_transform()` method replacing several operations
199+
- **Testing improvements**: Eliminated warnings during test execution
200+
- Updated deprecated code patterns in tests and examples (e.g., `sink`/`source``inputs`/`outputs`, `'H'``'h'` frequency)
201+
- Refactored plotting logic to handle test environments explicitly with non-interactive backends
202+
- Added comprehensive warning filters in `__init__.py` and `pyproject.toml` to suppress third-party library warnings
203+
- Improved test fixtures with proper figure cleanup to prevent memory leaks
204+
- Enhanced backend detection and handling in `plotting.py` for both Matplotlib and Plotly
205+
- Always run dependent tests in order
206+
207+
---
208+
71209
## [2.2.0] - 2025-10-11
72210
**Summary:** This release is a Configuration and Logging management release.
73211

@@ -77,9 +215,16 @@ Until here -->
77215
- Added configurable log format settings: `CONFIG.Logging.date_format` and `CONFIG.Logging.format`
78216
- Added configurable console settings: `CONFIG.Logging.console_width` and `CONFIG.Logging.show_path`
79217
- Added `CONFIG.Logging.Colors` nested class for customizable log level colors using ANSI escape codes (works with both standard and Rich handlers)
218+
- All examples now enable console logging to demonstrate proper logging usage
219+
- Console logging now outputs to `sys.stdout` instead of `sys.stderr` for better compatibility with output redirection
220+
221+
### 💥 Breaking Changes
222+
- Console logging is now disabled by default (`CONFIG.Logging.console = False`). Enable it explicitly in your scripts with `CONFIG.Logging.console = True` and `CONFIG.apply()`
223+
- File logging is now disabled by default (`CONFIG.Logging.file = None`). Set a file path to enable file logging
80224

81225
### ♻️ Changed
82226
- Logging and Configuration management changed
227+
- Improved default logging colors: DEBUG is now gray (`\033[90m`) for de-emphasized messages, INFO uses terminal default color (`\033[0m`) for clean output
83228

84229
### 🗑️ Deprecated
85230
- `change_logging_level()` function is now deprecated in favor of `CONFIG.Logging.level` and `CONFIG.apply()`. Will be removed in version 3.0.0.

MANIFEST.in

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ include CHANGELOG.md
55
include pyproject.toml
66

77
# Include package source and data
8-
recursive-include flixopt *.py *.yaml
8+
recursive-include flixopt *.py
99

1010
# Exclude everything else
1111
global-exclude *.pyc *.pyo __pycache__

0 commit comments

Comments
 (0)