Add Multi-Period-modeling and stochastic modeling to flixopt

.github/workflows/python-app.yaml

@@ -5,7 +5,7 @@ on:
     branches: [main]        # Only main branch
     tags: ['v*.*.*']
   pull_request:
-    branches: [main, dev]
+    branches: [main, dev, feature/v3/main]
     types: [opened, synchronize, reopened]
     paths-ignore:
       - 'docs/**'
@@ -90,6 +90,35 @@ jobs:
       - name: Run tests
         run: pytest -v -p no:warnings --numprocesses=auto
 
+  test-examples:
+    runs-on: ubuntu-24.04
+    timeout-minutes: 45
+    needs: lint
+    # Only run examples on releases (tags)
+    if: startsWith(github.ref, 'refs/tags/v')
+
+    steps:
+      - name: Check out code
+        uses: actions/checkout@v5
+
+      - name: Set up uv
+        uses: astral-sh/setup-uv@v6
+        with:
+          version: "0.8.19"
+          enable-cache: true
+
+      - name: Set up Python ${{ env.PYTHON_VERSION }}
+        uses: actions/setup-python@v6
+        with:
+          python-version: ${{ env.PYTHON_VERSION }}
+
+      - name: Install dependencies
+        run: |
+          uv pip install --system .[dev] "pytest-xdist==3.*"
+
+      - name: Run example tests
+        run: pytest -v -p no:warnings -m examples
+
   security:
     name: Security Scan
     runs-on: ubuntu-24.04
@@ -129,7 +158,7 @@ jobs:
     runs-on: ubuntu-24.04
     permissions:
       contents: write
-    needs: [lint, test, security]
+    needs: [lint, test, test-examples, security]
     if: startsWith(github.ref, 'refs/tags/v')
 
     steps:
@@ -168,7 +197,7 @@ jobs:
   publish-testpypi:
     name: Publish to TestPyPI
     runs-on: ubuntu-24.04
-    needs: [test, create-release]  # Run after tests and release creation
+    needs: [test, test-examples, create-release]  # Run after tests and release creation
     if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')  # Only on tag push
     environment:
       name: testpypi

CHANGELOG.md

@@ -29,6 +29,98 @@ Please remove all irrelevant sections before releasing.
 
 Until here -->
 
+## [Unreleased] - ????-??-??
+This release brings multi-year investments and stochastic modeling to flixopt.
+Furthermore, I/O methods were improved, and resampling and selection of parts of the FlowSystem are now possible.
+Several internal improvements were made to the codebase.
+
+
+### Multi-year investments
+A flixopt model might be modeled with a "year" dimension.
+This enables modeling transformation pathways over multiple years with several investment decisions
+
+### Stochastic modeling
+A flixopt model can be modeled with a scenario dimension.
+Scenarios can be weighted and variables can be equated across scenarios. This enables modeling uncertainties in the flow system, such as:
+* Different demand profiles
+* Different price forecasts
+* Different weather conditions
+
+Common use cases are:
+* Find the best overall investment decision for possible scenarios (robust decision-making)
+* Find the best dispatch for the most important assets under uncertain price and weather conditions
+
+The weighted sum of the total objective effect of each scenario is used as the objective of the optimization.
+
+#### Improved Data handling: I/O, resampling and more through xarray
+* IO for all Interfaces and the FlowSystem with round-trip serialization support
+    * NetCDF export/import capabilities for all Interface objects and FlowSystem
+    * JSON export for documentation purposes
+    * Recursive handling of nested Interface objects
+* FlowSystem data manipulation methods
+   * `sel()` and `isel()` methods for temporal data selection
+   * `resample()` method for temporal resampling
+   * `copy()` method to create a copy of a FlowSystem, including all underlying Elements and their data
+   * `__eq__()` method for FlowSystem comparison
+
+* Core data handling improvements
+   * `get_dataarray_stats()` function for statistical summaries
+   * Enhanced `DataConverter` class with better TimeSeriesData support
+
+
+### Added
+* FlowSystem restoring: The used FlowSystem is now accessible directly from the results without manual restoring (lazily). All parameters can be safely accessed anytime after the solve.
+* FlowResults added as a new class to store the results of Flows. They can now be accessed directly.
+* Added precomputed DataArrays for `size`s, `flow_rate`s and `flow_hour`s.
+* Added `effects_per_component()`-Dataset to Results that stores the direct (and indirect) effects of each component. This greatly improves the evaluation of the impact of individual Components, even with many and complex effects.
+* Improved filter methods in `results.py`
+* Balanced storage - Storage charging and discharging sizes can now be forced to be equal when optimizing their size by the `balanced` parameter.
+* Added Example for 2-stage Investment decisions leveraging the resampling of a FlowSystem
+* New Storage Parameter: `relative_minimum_final_charge_state` and `relative_maximum_final_charge_state` parameter for final state control. Default to last value of `relative_minimum_charge_state` and `relative_maximum_charge_state`, which will prevent change of behaviour for most users.
+
+### Changed
+* **BREAKING**: `relative_minimum_charge_state` and `relative_maximum_charge_state` don't have an extra timestep anymore.
+* **BREAKING**: Renamed class `SystemModel` to `FlowSystemModel`
+* **BREAKING**: Renamed class `Model` to `Submodel`
+* **BREAKING**: Renamed `mode` parameter in plotting methods to `style`
+* FlowSystems cannot be shared across multiple Calculations anymore. A copy of the FlowSystem is created instead, making every Calculation independent
+* Each Subcalculation in `SegmentedCalculation` now has its own distinct `FlowSystem` object
+* Type system overhaul - added clear separation between temporal and non-temporal data throughout codebase for better clarity
+* Enhanced FlowSystem interface with improved `__repr__()` and `__str__()` methods
+* Improved Model Structure - Views and organisation is now divided into:
+  * Model: The main Model (linopy.Model) that is used to create and store the variables and constraints for the flow_system.
+  * Submodel: The base class for all submodels. Each is a subset of the Model, for simpler access and clearer code.
+
+### Deprecated
+* The `agg_group` and `agg_weight` parameters of `TimeSeriesData` are deprecated and will be removed in a future version. Use `aggregation_group` and `aggregation_weight` instead.
+* The `active_timesteps` parameter of `Calculation` is deprecated and will be removed in a future version. Use the new `sel(time=...)` method on the FlowSystem instead.
+* The assignment of Bus Objects to Flow.bus is deprecated and will be removed in a future version. Use the label of the Bus instead.
+* The usage of Effects objects in Dicts to assign shares to Effects is deprecated and will be removed in a future version. Use the label of the Effect instead.
+
+### Removed
+
+### Fixed
+* Enhanced NetCDF I/O with proper attribute preservation for DataArrays
+* Improved error handling and validation in serialization processes
+* Better type consistency across all framework components
+
+### Known issues
+* IO for single Interfaces/Elements to Datasets might not work properly if the Interface/Element is not part of a fully transformed and connected FlowSystem. This arises from Numeric Data not being stored as xr.DataArray by the user. To avoid this, always use the `to_dataset()` on Elements inside a FlowSystem that's connected and transformed.
+
+### *Development*
+* **BREAKING**: Calculation.do_modeling() now returns the Calculation object instead of its linopy.Model
+* FlowSystem data management simplified - removed `time_series_collection` pattern in favor of direct timestep properties
+* Change modeling hierarchy to allow for more flexibility in future development. This leads to minimal changes in the access and creation of Submodels and their variables.
+* Added new module `.modeling`that contains Modelling primitives and utilities
+* Clearer separation between the main Model and "Submodels"
+* Improved access to the Submodels and their variables, constraints and submodels
+* Added `__repr__()` for Submodels to easily inspect its content
+* Enhanced data handling methods
+   * `fit_to_model_coords()` method for data alignment
+   * `fit_effects_to_model_coords()` method for effect data processing
+   * `connect_and_transform()` method replacing several operations
+
+
 ## [2.1.9] - 2025-09-23
 Small Bugfix which was supposed to be fixed in 2.1.8
 
@@ -85,7 +177,7 @@ There are no changes or new features.
 ## [2.1.6] - 2025-09-02
 
 ### Changed
-- `Sink`, `Source` and `SourceAndSink` now accept multiple `flows` as `inputs` and `outputs` instead of just one. This enables to model more use cases using these classes. [[#291](https://github.com/flixOpt/flixopt/pull/291) by [@FBumann](https://github.com/FBumann)]
+- `Sink`, `Source` and `SourceAndSink` now accept multiple `flows` as `inputs` and `outputs` instead of just one. This enables modeling more use cases using these classes. [[#291](https://github.com/flixOpt/flixopt/pull/291) by [@FBumann](https://github.com/FBumann)]
 - Further, both `Sink` and `Source` now have a `prevent_simultaneous_flow_rates` argument to prevent simultaneous flow rates of more than one of their Flows. [[#291](https://github.com/flixOpt/flixopt/pull/291) by [@FBumann](https://github.com/FBumann)]
 
 ### Added

examples/01_Simple/simple_example.py

@@ -66,7 +66,8 @@
         discharging=fx.Flow('Q_th_unload', bus='Fernwärme', size=1000),
         capacity_in_flow_hours=fx.InvestParameters(fix_effects=20, fixed_size=30, optional=False),
         initial_charge_state=0,  # Initial storage state: empty
-        relative_maximum_charge_state=1 / 100 * np.array([80, 70, 80, 80, 80, 80, 80, 80, 80, 80]),
+        relative_maximum_charge_state=1 / 100 * np.array([80, 70, 80, 80, 80, 80, 80, 80, 80]),
+        relative_maximum_final_charge_state=0.8,
         eta_charge=0.9,
         eta_discharge=1,  # Efficiency factors for charging/discharging
         relative_loss_per_hour=0.08,  # 8% loss per hour. Absolute loss depends on current charge state

examples/03_Calculation_types/example_calculation_types.py

@@ -45,9 +45,9 @@
 
     # TimeSeriesData objects
     TS_heat_demand = fx.TimeSeriesData(heat_demand)
-    TS_electricity_demand = fx.TimeSeriesData(electricity_demand, agg_weight=0.7)
-    TS_electricity_price_sell = fx.TimeSeriesData(-(electricity_demand - 0.5), agg_group='p_el')
-    TS_electricity_price_buy = fx.TimeSeriesData(electricity_price + 0.5, agg_group='p_el')
+    TS_electricity_demand = fx.TimeSeriesData(electricity_demand, aggregation_weight=0.7)
+    TS_electricity_price_sell = fx.TimeSeriesData(-(electricity_demand - 0.5), aggregation_group='p_el')
+    TS_electricity_price_buy = fx.TimeSeriesData(electricity_price + 0.5, aggregation_group='p_el')
 
     flow_system = fx.FlowSystem(timesteps)
     flow_system.add_elements(
@@ -136,7 +136,10 @@
     a_strom_tarif = fx.Source(
         'Stromtarif',
         source=fx.Flow(
-            'P_el', bus='Strom', size=1000, effects_per_flow_hour={costs.label: TS_electricity_price_buy, CO2: 0.3}
+            'P_el',
+            bus='Strom',
+            size=1000,
+            effects_per_flow_hour={costs.label: TS_electricity_price_buy, CO2.label: 0.3},
         ),
     )
 
@@ -161,12 +164,12 @@
     if full:
         calculation = fx.FullCalculation('Full', flow_system)
         calculation.do_modeling()
-        calculation.solve(fx.solvers.HighsSolver(0, 60))
+        calculation.solve(fx.solvers.HighsSolver(0.01 / 100, 60))
         calculations.append(calculation)
 
     if segmented:
         calculation = fx.SegmentedCalculation('Segmented', flow_system, segment_length, overlap_length)
-        calculation.do_modeling_and_solve(fx.solvers.HighsSolver(0, 60))
+        calculation.do_modeling_and_solve(fx.solvers.HighsSolver(0.01 / 100, 60))
         calculations.append(calculation)
 
     if aggregated:
@@ -175,7 +178,7 @@
             aggregation_parameters.time_series_for_low_peaks = [TS_electricity_demand, TS_heat_demand]
         calculation = fx.AggregatedCalculation('Aggregated', flow_system, aggregation_parameters)
         calculation.do_modeling()
-        calculation.solve(fx.solvers.HighsSolver(0, 60))
+        calculation.solve(fx.solvers.HighsSolver(0.01 / 100, 60))
         calculations.append(calculation)
 
     # Get solutions for plotting for different calculations
@@ -191,34 +194,35 @@ def get_solutions(calcs: list, variable: str) -> xr.Dataset:
     # --- Plotting for comparison ---
     fx.plotting.with_plotly(
         get_solutions(calculations, 'Speicher|charge_state').to_dataframe(),
-        mode='line',
+        style='line',
         title='Charge State Comparison',
         ylabel='Charge state',
     ).write_html('results/Charge State.html')
 
     fx.plotting.with_plotly(
         get_solutions(calculations, 'BHKW2(Q_th)|flow_rate').to_dataframe(),
-        mode='line',
+        style='line',
         title='BHKW2(Q_th) Flow Rate Comparison',
         ylabel='Flow rate',
     ).write_html('results/BHKW2 Thermal Power.html')
 
     fx.plotting.with_plotly(
         get_solutions(calculations, 'costs(operation)|total_per_timestep').to_dataframe(),
-        mode='line',
+        style='line',
         title='Operation Cost Comparison',
         ylabel='Costs [€]',
     ).write_html('results/Operation Costs.html')
 
     fx.plotting.with_plotly(
         pd.DataFrame(get_solutions(calculations, 'costs(operation)|total_per_timestep').to_dataframe().sum()).T,
-        mode='bar',
+        style='stacked_bar',
         title='Total Cost Comparison',
         ylabel='Costs [€]',
     ).update_layout(barmode='group').write_html('results/Total Costs.html')
 
     fx.plotting.with_plotly(
-        pd.DataFrame([calc.durations for calc in calculations], index=[calc.name for calc in calculations]), 'bar'
+        pd.DataFrame([calc.durations for calc in calculations], index=[calc.name for calc in calculations]),
+        'stacked_bar',
     ).update_layout(title='Duration Comparison', xaxis_title='Calculation type', yaxis_title='Time (s)').write_html(
         'results/Speed Comparison.html'
     )