Add Multi-Period-modeling and stochastic modeling to flixopt

.github/workflows/python-app.yaml

@@ -5,7 +5,7 @@ on:
     branches: [main]        # Only main branch
     tags: ['v*.*.*']
   pull_request:
-    branches: [main, dev]
+    branches: [main, dev, feature/v3/main]
     types: [opened, synchronize, reopened]
     paths-ignore:
       - 'docs/**'

CHANGELOG.md

@@ -68,6 +68,98 @@ Please keep the format of the changelog consistent with the other releases, so t
 Until here -->
 ---
 
+## [Unreleased] - ????-??-??
+This release brings multi-year investments and stochastic modeling to flixopt.
+Furthermore, I/O methods were improved, and resampling and selection of parts of the FlowSystem are now possible.
+Several internal improvements were made to the codebase.
+
+
+### Multi-year investments
+A flixopt model might be modeled with a "year" dimension.
+This enables modeling transformation pathways over multiple years with several investment decisions
+
+### Stochastic modeling
+A flixopt model can be modeled with a scenario dimension.
+Scenarios can be weighted and variables can be equated across scenarios. This enables modeling uncertainties in the flow system, such as:
+* Different demand profiles
+* Different price forecasts
+* Different weather conditions
+
+Common use cases are:
+* Find the best overall investment decision for possible scenarios (robust decision-making)
+* Find the best dispatch for the most important assets under uncertain price and weather conditions
+
+The weighted sum of the total objective effect of each scenario is used as the objective of the optimization.
+
+#### Improved Data handling: I/O, resampling and more through xarray
+* IO for all Interfaces and the FlowSystem with round-trip serialization support
+    * NetCDF export/import capabilities for all Interface objects and FlowSystem
+    * JSON export for documentation purposes
+    * Recursive handling of nested Interface objects
+* FlowSystem data manipulation methods
+   * `sel()` and `isel()` methods for temporal data selection
+   * `resample()` method for temporal resampling
+   * `copy()` method to create a copy of a FlowSystem, including all underlying Elements and their data
+   * `__eq__()` method for FlowSystem comparison
+
+* Core data handling improvements
+   * `get_dataarray_stats()` function for statistical summaries
+   * Enhanced `DataConverter` class with better TimeSeriesData support
+
+
+### Added
+* FlowSystem restoring: The used FlowSystem is now accessible directly from the results without manual restoring (lazily). All parameters can be safely accessed anytime after the solve.
+* FlowResults added as a new class to store the results of Flows. They can now be accessed directly.
+* Added precomputed DataArrays for `size`s, `flow_rate`s and `flow_hour`s.
+* Added `effects_per_component()`-Dataset to Results that stores the direct (and indirect) effects of each component. This greatly improves the evaluation of the impact of individual Components, even with many and complex effects.
+* Improved filter methods in `results.py`
+* Balanced storage - Storage charging and discharging sizes can now be forced to be equal when optimizing their size by the `balanced` parameter.
+* Added Example for 2-stage Investment decisions leveraging the resampling of a FlowSystem
+* New Storage Parameter: `relative_minimum_final_charge_state` and `relative_maximum_final_charge_state` parameter for final state control. Default to last value of `relative_minimum_charge_state` and `relative_maximum_charge_state`, which will prevent change of behaviour for most users.
+
+### Changed
+* **BREAKING**: `relative_minimum_charge_state` and `relative_maximum_charge_state` don't have an extra timestep anymore.
+* **BREAKING**: Renamed class `SystemModel` to `FlowSystemModel`
+* **BREAKING**: Renamed class `Model` to `Submodel`
+* **BREAKING**: Renamed `mode` parameter in plotting methods to `style`
+* FlowSystems cannot be shared across multiple Calculations anymore. A copy of the FlowSystem is created instead, making every Calculation independent
+* Each Subcalculation in `SegmentedCalculation` now has its own distinct `FlowSystem` object
+* Type system overhaul - added clear separation between temporal and non-temporal data throughout codebase for better clarity
+* Enhanced FlowSystem interface with improved `__repr__()` and `__str__()` methods
+* Improved Model Structure - Views and organisation is now divided into:
+  * Model: The main Model (linopy.Model) that is used to create and store the variables and constraints for the flow_system.
+  * Submodel: The base class for all submodels. Each is a subset of the Model, for simpler access and clearer code.
+
+### Deprecated
+* The `agg_group` and `agg_weight` parameters of `TimeSeriesData` are deprecated and will be removed in a future version. Use `aggregation_group` and `aggregation_weight` instead.
+* The `active_timesteps` parameter of `Calculation` is deprecated and will be removed in a future version. Use the new `sel(time=...)` method on the FlowSystem instead.
+* The assignment of Bus Objects to Flow.bus is deprecated and will be removed in a future version. Use the label of the Bus instead.
+* The usage of Effects objects in Dicts to assign shares to Effects is deprecated and will be removed in a future version. Use the label of the Effect instead.
+
+### Removed
+
+### Fixed
+* Enhanced NetCDF I/O with proper attribute preservation for DataArrays
+* Improved error handling and validation in serialization processes
+* Better type consistency across all framework components
+
+### Known issues
+* IO for single Interfaces/Elements to Datasets might not work properly if the Interface/Element is not part of a fully transformed and connected FlowSystem. This arises from Numeric Data not being stored as xr.DataArray by the user. To avoid this, always use the `to_dataset()` on Elements inside a FlowSystem that's connected and transformed.
+
+### *Development*
+* **BREAKING**: Calculation.do_modeling() now returns the Calculation object instead of its linopy.Model
+* FlowSystem data management simplified - removed `time_series_collection` pattern in favor of direct timestep properties
+* Change modeling hierarchy to allow for more flexibility in future development. This leads to minimal changes in the access and creation of Submodels and their variables.
+* Added new module `.modeling`that contains Modelling primitives and utilities
+* Clearer separation between the main Model and "Submodels"
+* Improved access to the Submodels and their variables, constraints and submodels
+* Added `__repr__()` for Submodels to easily inspect its content
+* Enhanced data handling methods
+   * `fit_to_model_coords()` method for data alignment
+   * `fit_effects_to_model_coords()` method for effect data processing
+   * `connect_and_transform()` method replacing several operations
+
+
 ## [2.1.9] - 2025-09-23
 
 **Summary:** Small bugfix release addressing network visualization error handling.

examples/01_Simple/simple_example.py

@@ -66,7 +66,8 @@
         discharging=fx.Flow('Q_th_unload', bus='Fernwärme', size=1000),
         capacity_in_flow_hours=fx.InvestParameters(fix_effects=20, fixed_size=30, optional=False),
         initial_charge_state=0,  # Initial storage state: empty
-        relative_maximum_charge_state=1 / 100 * np.array([80, 70, 80, 80, 80, 80, 80, 80, 80, 80]),
+        relative_maximum_charge_state=1 / 100 * np.array([80, 70, 80, 80, 80, 80, 80, 80, 80]),
+        relative_maximum_final_charge_state=0.8,
         eta_charge=0.9,
         eta_discharge=1,  # Efficiency factors for charging/discharging
         relative_loss_per_hour=0.08,  # 8% loss per hour. Absolute loss depends on current charge state

examples/03_Calculation_types/example_calculation_types.py

@@ -45,9 +45,9 @@
 
     # TimeSeriesData objects
     TS_heat_demand = fx.TimeSeriesData(heat_demand)
-    TS_electricity_demand = fx.TimeSeriesData(electricity_demand, agg_weight=0.7)
-    TS_electricity_price_sell = fx.TimeSeriesData(-(electricity_demand - 0.5), agg_group='p_el')
-    TS_electricity_price_buy = fx.TimeSeriesData(electricity_price + 0.5, agg_group='p_el')
+    TS_electricity_demand = fx.TimeSeriesData(electricity_demand, aggregation_weight=0.7)
+    TS_electricity_price_sell = fx.TimeSeriesData(-(electricity_demand - 0.5), aggregation_group='p_el')
+    TS_electricity_price_buy = fx.TimeSeriesData(electricity_price + 0.5, aggregation_group='p_el')
 
     flow_system = fx.FlowSystem(timesteps)
     flow_system.add_elements(
@@ -136,7 +136,10 @@
     a_strom_tarif = fx.Source(
         'Stromtarif',
         source=fx.Flow(
-            'P_el', bus='Strom', size=1000, effects_per_flow_hour={costs.label: TS_electricity_price_buy, CO2: 0.3}
+            'P_el',
+            bus='Strom',
+            size=1000,
+            effects_per_flow_hour={costs.label: TS_electricity_price_buy, CO2.label: 0.3},
         ),
     )
 
@@ -161,12 +164,12 @@
     if full:
         calculation = fx.FullCalculation('Full', flow_system)
         calculation.do_modeling()
-        calculation.solve(fx.solvers.HighsSolver(0, 60))
+        calculation.solve(fx.solvers.HighsSolver(0.01 / 100, 60))
         calculations.append(calculation)
 
     if segmented:
         calculation = fx.SegmentedCalculation('Segmented', flow_system, segment_length, overlap_length)
-        calculation.do_modeling_and_solve(fx.solvers.HighsSolver(0, 60))
+        calculation.do_modeling_and_solve(fx.solvers.HighsSolver(0.01 / 100, 60))
         calculations.append(calculation)
 
     if aggregated:
@@ -175,7 +178,7 @@
             aggregation_parameters.time_series_for_low_peaks = [TS_electricity_demand, TS_heat_demand]
         calculation = fx.AggregatedCalculation('Aggregated', flow_system, aggregation_parameters)
         calculation.do_modeling()
-        calculation.solve(fx.solvers.HighsSolver(0, 60))
+        calculation.solve(fx.solvers.HighsSolver(0.01 / 100, 60))
         calculations.append(calculation)
 
     # Get solutions for plotting for different calculations
@@ -191,34 +194,35 @@ def get_solutions(calcs: list, variable: str) -> xr.Dataset:
     # --- Plotting for comparison ---
     fx.plotting.with_plotly(
         get_solutions(calculations, 'Speicher|charge_state').to_dataframe(),
-        mode='line',
+        style='line',
         title='Charge State Comparison',
         ylabel='Charge state',
     ).write_html('results/Charge State.html')
 
     fx.plotting.with_plotly(
         get_solutions(calculations, 'BHKW2(Q_th)|flow_rate').to_dataframe(),
-        mode='line',
+        style='line',
         title='BHKW2(Q_th) Flow Rate Comparison',
         ylabel='Flow rate',
     ).write_html('results/BHKW2 Thermal Power.html')
 
     fx.plotting.with_plotly(
         get_solutions(calculations, 'costs(operation)|total_per_timestep').to_dataframe(),
-        mode='line',
+        style='line',
         title='Operation Cost Comparison',
         ylabel='Costs [€]',
     ).write_html('results/Operation Costs.html')
 
     fx.plotting.with_plotly(
         pd.DataFrame(get_solutions(calculations, 'costs(operation)|total_per_timestep').to_dataframe().sum()).T,
-        mode='bar',
+        style='stacked_bar',
         title='Total Cost Comparison',
         ylabel='Costs [€]',
     ).update_layout(barmode='group').write_html('results/Total Costs.html')
 
     fx.plotting.with_plotly(
-        pd.DataFrame([calc.durations for calc in calculations], index=[calc.name for calc in calculations]), 'bar'
+        pd.DataFrame([calc.durations for calc in calculations], index=[calc.name for calc in calculations]),
+        'stacked_bar',
     ).update_layout(title='Duration Comparison', xaxis_title='Calculation type', yaxis_title='Time (s)').write_html(
         'results/Speed Comparison.html'
     )

examples/04_Scenarios/scenario_example.py

@@ -0,0 +1,137 @@
+"""
+This script shows how to use the flixopt framework to model a simple energy system.
+"""
+
+import numpy as np
+import pandas as pd
+
+import flixopt as fx
+
+if __name__ == '__main__':
+    # Create datetime array starting from '2020-01-01' for the given time period
+    timesteps = pd.date_range('2020-01-01', periods=9, freq='h')
+    scenarios = pd.Index(['Base Case', 'High Demand'])
+    years = pd.Index([2020, 2021, 2022])
+
+    # --- Create Time Series Data ---
+    # Heat demand profile (e.g., kW) over time and corresponding power prices
+    heat_demand_per_h = pd.DataFrame(
+        {'Base Case': [30, 0, 90, 110, 110, 20, 20, 20, 20], 'High Demand': [30, 0, 100, 118, 125, 20, 20, 20, 20]},
+        index=timesteps,
+    )
+    power_prices = np.array([0.08, 0.09, 0.10])
+
+    flow_system = fx.FlowSystem(timesteps=timesteps, years=years, scenarios=scenarios, weights=np.array([0.5, 0.6]))
+
+    # --- Define Energy Buses ---
+    # These represent nodes, where the used medias are balanced (electricity, heat, and gas)
+    flow_system.add_elements(fx.Bus(label='Strom'), fx.Bus(label='Fernwärme'), fx.Bus(label='Gas'))
+
+    # --- Define Effects (Objective and CO2 Emissions) ---
+    # Cost effect: used as the optimization objective --> minimizing costs
+    costs = fx.Effect(
+        label='costs',
+        unit='€',
+        description='Kosten',
+        is_standard=True,  # standard effect: no explicit value needed for costs
+        is_objective=True,  # Minimizing costs as the optimization objective
+    )
+
+    # CO2 emissions effect with an associated cost impact
+    CO2 = fx.Effect(
+        label='CO2',
+        unit='kg',
+        description='CO2_e-Emissionen',
+        specific_share_to_other_effects_operation={costs.label: 0.2},
+        maximum_operation_per_hour=1000,  # Max CO2 emissions per hour
+    )
+
+    # --- Define Flow System Components ---
+    # Boiler: Converts fuel (gas) into thermal energy (heat)
+    boiler = fx.linear_converters.Boiler(
+        label='Boiler',
+        eta=0.5,
+        Q_th=fx.Flow(
+            label='Q_th',
+            bus='Fernwärme',
+            size=50,
+            relative_minimum=0.1,
+            relative_maximum=1,
+            on_off_parameters=fx.OnOffParameters(),
+        ),
+        Q_fu=fx.Flow(label='Q_fu', bus='Gas'),
+    )
+
+    # Combined Heat and Power (CHP): Generates both electricity and heat from fuel
+    chp = fx.linear_converters.CHP(
+        label='CHP',
+        eta_th=0.5,
+        eta_el=0.4,
+        P_el=fx.Flow('P_el', bus='Strom', size=60, relative_minimum=5 / 60, on_off_parameters=fx.OnOffParameters()),
+        Q_th=fx.Flow('Q_th', bus='Fernwärme'),
+        Q_fu=fx.Flow('Q_fu', bus='Gas'),
+    )
+
+    # Storage: Energy storage system with charging and discharging capabilities
+    storage = fx.Storage(
+        label='Storage',
+        charging=fx.Flow('Q_th_load', bus='Fernwärme', size=1000),
+        discharging=fx.Flow('Q_th_unload', bus='Fernwärme', size=1000),
+        capacity_in_flow_hours=fx.InvestParameters(fix_effects=20, fixed_size=30, optional=False),
+        initial_charge_state=0,  # Initial storage state: empty
+        relative_maximum_charge_state=np.array([80, 70, 80, 80, 80, 80, 80, 80, 80]) * 0.01,
+        relative_maximum_final_charge_state=0.8,
+        eta_charge=0.9,
+        eta_discharge=1,  # Efficiency factors for charging/discharging
+        relative_loss_per_hour=0.08,  # 8% loss per hour. Absolute loss depends on current charge state
+        prevent_simultaneous_charge_and_discharge=True,  # Prevent charging and discharging at the same time
+    )
+
+    # Heat Demand Sink: Represents a fixed heat demand profile
+    heat_sink = fx.Sink(
+        label='Heat Demand',
+        sink=fx.Flow(label='Q_th_Last', bus='Fernwärme', size=1, fixed_relative_profile=heat_demand_per_h),
+    )
+
+    # Gas Source: Gas tariff source with associated costs and CO2 emissions
+    gas_source = fx.Source(
+        label='Gastarif',
+        source=fx.Flow(label='Q_Gas', bus='Gas', size=1000, effects_per_flow_hour={costs.label: 0.04, CO2.label: 0.3}),
+    )
+
+    # Power Sink: Represents the export of electricity to the grid
+    power_sink = fx.Sink(
+        label='Einspeisung', sink=fx.Flow(label='P_el', bus='Strom', effects_per_flow_hour=-1 * power_prices)
+    )
+
+    # --- Build the Flow System ---
+    # Add all defined components and effects to the flow system
+    flow_system.add_elements(costs, CO2, boiler, storage, chp, heat_sink, gas_source, power_sink)
+
+    # Visualize the flow system for validation purposes
+    flow_system.plot_network(show=True)
+
+    # --- Define and Run Calculation ---
+    # Create a calculation object to model the Flow System
+    calculation = fx.FullCalculation(name='Sim1', flow_system=flow_system)
+    calculation.do_modeling()  # Translate the model to a solvable form, creating equations and Variables
+
+    # --- Solve the Calculation and Save Results ---
+    calculation.solve(fx.solvers.HighsSolver(mip_gap=0, time_limit_seconds=30))
+
+    calculation.results.plot_heatmap('CHP(Q_th)|flow_rate')
+
+    # --- Analyze Results ---
+    calculation.results['Fernwärme'].plot_node_balance_pie()
+    calculation.results['Fernwärme'].plot_node_balance(style='stacked_bar')
+    calculation.results['Storage'].plot_node_balance()
+    calculation.results.plot_heatmap('CHP(Q_th)|flow_rate')
+
+    # Convert the results for the storage component to a dataframe and display
+    df = calculation.results['Storage'].node_balance_with_charge_state()
+    print(df)
+    calculation.results['Storage'].plot_charge_state(engine='matplotlib')
+
+    # Save results to file for later usage
+    calculation.results.to_file()
+    fig, ax = calculation.results['Storage'].plot_charge_state(engine='matplotlib')