SuperLU_DIST solver interface

[jhale]

Joint work with @chrisrichardson.

This adds an optional SuperLU_DIST interface to DOLFINx for solving systems constructed using MatrixCSR. The purpose of this is to provide a reasonable MPI-capable LU solver for DOLFINx builds without PETSc on all platforms.

The implementation is pretty light, and does not include SuperLU headers into our headers.

In future PRs:

• dolfinx.fem.LinearProblem class for beginners.
• To discuss: dolfinx.fem.NonlinearProblem class for beginners.
• Enable user to select solver options

[garth-wells]

MatrixCSR has more than one template argument. Should probably expand to include them all, or template over the matrix.

Related to this, any small tweaks that would allow us to use the SuperLU_DIST GPU support?

[jhale]

What SuperLU_DIST actually accepts is not very general. We could probably template over MatrixCSR and then use concepts to constrain.

On the second point, on my reading you add +cuda or +rocm on the Spack build for superlu-dist and then (partial) GPU offload is enabled by default. I will try this another time.

https://dl.acm.org/doi/epdf/10.1145/3577197

[schnellerhase]

It would be nice if we could separate the MatrixCSR to SuperLu-Matrix conversion logic from the solver wrapper. This allows for custom usability on the C++ side (not forcing the usage of SuperLUDistSolver) and, thinking about why we need to wrap this solver in the first place, if there was a superlu4py this conversion is the only thing we would need on dolfinx's side.

[jhale]

Let's stay on track here; superlu_dist has a perfectly well designed C interface, we don't want to include SuperLU_dist headers into DOLFINx headers, or impose memory management choices onto C++ users (note the subtle lifetime issues on A and SuperMatrix).

We can of course split out to another class SuperLUMatrix which SuperLUDistSolver takes, but does it really add anything?

[jhale]

The function for creating the supermatrix is now factored out into the anonymous namespace.

[jhale]

I've managed to split our SuperLUMatrix and SuperLUSolver in the branch:

https://github.com/FEniCS/dolfinx/tree/jhale/split-superlu-solver-matrix

There is a slight breaking of abstractions at the start of solve which can be fixed.

If this is the direction we want to go in (probably) I will finish up the Python wrap.

All comments addressed - new PR with split on Matrix and Solver will come soon.