Breaking dam 2D fails in parallel

[MakisH]

When running the breaking dam 2D tutorial in parallel:

# fluid-openfoam
./run.sh -parallel 

# solid-calculix
./run.sh

the CalculiX participant crashes at t=0.6s. Reported by a user in the forum.

• Tutorials state (last commit / release): 9894b79 (but much earlier versions should be relevant as well)
• Versions of solvers and adapters used: OpenFOAM v2512, OpenFOAM adapter f7e2d, CalculiX v2.20, CalculiX adapter 11993
• preCICE version: v3.3.1
• Log files:
  • fluid-openfoam.log
  • solid-calculix.log
  • precice-Solid-convergence.log
  • precice-Solid-iterations.log

I have not yet compared with the serial case, I will add more results once available.

[precice-bot]

This issue has been mentioned on preCICE Forum on Discourse. There might be relevant details there:

https://precice.discourse.group/t/issues-with-calculix-precice-interfoam-simulation-when-running-flap-case-in-parallel/2720/2

[Geethapranay1]

hi @MakisH,

As I am going through the codebase in more depth and learning more about it, this is my current understanding of the issue. Coming from a mechanical engineering background, I have some familiarity with the physical aspects, and based on that, these are my takes on this bug.

what i observed from logs

• preCICE coupling converges cleanly up to time-window 299 (t ≈ 0.598 s).
• at time-window 300 (t = 0.600 s), interface loads received by CalculiX jump sharply:
  • average force increases by an order of magnitude
  • residual forces reach O(10⁶)
• shortly after, ccx_preCICE aborts with:

  corrupted double-linked list / Aborted (core dumped)
  

  This indicates heap/memory corruption in the solid solver or adapter it could be a hard crash, not a clean non-convergence stop.
• OpenFOAM then reports:

  Receiving data ... failed: read: End of file
  

  This appears to be a secondary failure caused by the solid crash.

---

this could be a cause:

A numerical instability / load spike around dam impact in the parallel fluid run triggers a pathological state in ccx_preCICE. The heap corruption suggests a robustness bug in the adapter or solver rather than a graceful divergence this caused the force spike pushes CalculiX into undefined behavior rather than triggering a controlled coupling failure.

Reading FORCES coupling data.
 increment 299 attempt 2 
 increment size= 2.000000e-03
 sum of previous increments=5.960000e-01
 actual step time=5.980000e-01
 actual total time=5.980000e-01

 iteration 1

 Using up to 1 cpu(s) for the stress calculation.

 Using up to 1 cpu(s) for the energy calculation.

 Using up to 1 cpu(s) for the symmetric stiffness/mass contributions.

 Factoring the system of equations using the symmetric spooles solver
 Using up to 1 cpu(s) for spooles.

 Using up to 1 cpu(s) for the stress calculation.

 Using up to 1 cpu(s) for the energy calculation.

 average force= 2117.561525
 time avg. forc= 424.442060
 largest residual force= 6563.055752 in node 730 and dof 1
 largest increment of disp= 1.692905e-02
 largest correction to disp= 1.611154e-02 in node 1092 and dof 1

 no convergence

 iteration 2

 Using up to 1 cpu(s) for the symmetric stiffness/mass contributions.

 Factoring the system of equations using the symmetric spooles solver
 Using up to 1 cpu(s) for spooles.

 Using up to 1 cpu(s) for the stress calculation.

 Using up to 1 cpu(s) for the energy calculation.

 average force= 1956.152143
 time avg. forc= 423.902229
 largest residual force= 333.453314 in node 730 and dof 1
 largest increment of disp= 1.692905e-02
 largest correction to disp= 1.882575e-03 in node 1104 and dof 2

 no convergence

 iteration 3

 Using up to 1 cpu(s) for the symmetric stiffness/mass contributions.

 Factoring the system of equations using the symmetric spooles solver
 Using up to 1 cpu(s) for spooles.

 Using up to 1 cpu(s) for the stress calculation.

 Using up to 1 cpu(s) for the energy calculation.

 average force= 1955.581004
 time avg. forc= 423.900319
 largest residual force= 8.109278 in node 479 and dof 2
 largest increment of disp= 1.692905e-02
 largest correction to disp= 1.044222e-04 in node 1095 and dof 2

 no convergence

 iteration 4

 Using up to 1 cpu(s) for the symmetric stiffness/mass contributions.

 Factoring the system of equations using the symmetric spooles solver
 Using up to 1 cpu(s) for spooles.

 Using up to 1 cpu(s) for the stress calculation.

 Using up to 1 cpu(s) for the energy calculation.

 average force= 1955.591712
 time avg. forc= 423.900354
 largest residual force= 0.001536 in node 479 and dof 2
 largest increment of disp= 1.692905e-02
 largest correction to disp= 1.423890e-06 in node 1458 and dof 2

 convergence

Adapter writing coupling data...
Writing DISPLACEMENTS coupling data.
Adapter calling advance()...
---[precice] �[0m relative convergence measure: relative two-norm diff of data "Displacement" = 5.72e-08, limit = 1.00e-03, normalization = 3.22e+00, conv = true
---[precice] �[0m relative convergence measure: relative two-norm diff of data "Force" = 1.12e-04, limit = 1.00e-03, normalization = 3.10e+03, conv = true
---[precice] �[0m All converged
---[precice] �[0m Time window completed
---[precice] �[0m it 1 (min: 1, max: 30), time-window 300, t 0.598 (max: 1), Dt 0.002, max-dt 0.002
 initial energy (at start of step) = 0.000000e+00

 since start of the step: 
 external work = 8.857410e+02
 work performed by the damping forces = 0.000000e+00
 netto work = 8.857410e+02

 actual energy: 
 internal energy = 8.556602e+02
 kinetic energy = 7.125242e+01
 elastic contact energy = 0.000000e+00
 energy lost due to friction = 0.000000e+00
 total energy  = 9.269126e+02

 energy increase = 9.269126e+02

 energy balance (absolute) = 4.117162e+01 
 energy balance (relative) = 22.596250 % 

 Using up to 1 cpu(s) for the stress calculation.

Adjusting time step for transient step
precice_dt = 0.002000, ccx_dt = 0.005000 (dtheta = 0.000500, tper = 10.000000) -> dt = 0.002000 (dtheta = 0.000200)
Adapter reading coupling data...
Reading FORCES coupling data.
Adapter writing checkpoint...
 increment 300 attempt 1 
 increment size= 2.000000e-03
 sum of previous increments=5.980000e-01
 actual step time=6.000000e-01
 actual total time=6.000000e-01

 iteration 1

 Using up to 1 cpu(s) for the stress calculation.

 Using up to 1 cpu(s) for the energy calculation.

 Using up to 1 cpu(s) for the symmetric stiffness/mass contributions.

 Factoring the system of equations using the symmetric spooles solver
 Using up to 1 cpu(s) for spooles.

 Using up to 1 cpu(s) for the stress calculation.

 Using up to 1 cpu(s) for the energy calculation.

 average force= 52991.256000
 time avg. forc= 599.124873
 largest residual force= 5422384.881289 in node 393 and dof 1
 largest increment of disp= 4.758304e-02
 largest correction to disp= 4.758304e-02 in node 1401 and dof 2

---

after studying a bit this migrations could be helpful

1. In breaking-dam-2d/fluid-openfoam/system/fvSchemes:

// current (potentially unstable for parallel FSI dam-break):
ddtSchemes
{
    default CrankNicolson 0.9;
}

// this fix may be a one:
ddtSchemes
{
    default Euler;
}

why this could be helpful: Crank–Nicolson is less numerically dissipative than Euler. In violent free-surface + moving-mesh FSI, it can amplify oscillations and produce sharper load transitions. Euler damps those oscillations, which often prevents the force spike that triggers the solid failure. This has been reported to improve stability for dam-break cases around t = 0.5–0.6 s, particularly in parallel runs.

2. halving the timestep (e.g. 0.002 → 0.001) limits the magnitude of sudden inter-solver force jumps at each window.

i am trying this steps

1. compare serial vs. parallel OpenFOAM force history at the interface around t = 0.6 s which would confirm whether the load spike is parallel-specific (e.g. due to domain decomposition artifacts at the interface).
2. Run ccx_preCICE under ASAN or Valgrind to locate the exact source of the memory corruption, independent of what triggers it.