Fides optimizer: clarify use of Hessian and FIM

[dilpath]

Currently it looks like a user needs to specify pypesto.optimize.FidesOptimizer(..., hessian_update=None) to ensure Fides uses the provided Hessian. It appears that default behavior does not use the Hessian from the objective when possible.

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.81%. Comparing base (f0d3f39) to head (9eedf62).
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #1706      +/-   ##
===========================================
- Coverage    83.84%   83.81%   -0.04%     
===========================================
  Files          164      164              
  Lines        14345    14346       +1     
===========================================
- Hits         12028    12024       -4     
- Misses        2317     2322       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[FFroehlich]

from what I recall fides.HybridFixed() initially uses the provided Hessian approximation and then switches to BFGS after a fixed number of iterations. Based on the benchmarking I did back then, this is a reasonable default for FIM approximations, which is likely what most users would work with.

[dweindl]

At least in the current (develop) version, the docstring doesn't match the implementation:

pyPESTO/pypesto/optimize/optimizer.py

Lines 1702 to 1705 in 86a0076

	hessian_update:
	Hessian update strategy. If this is ``None``, a hybrid approximation
	that switches from the ``problem.objective`` provided Hessian (
	approximation) to a BFGS approximation will be used.

hessian_update=None would use the provided Hessian, not the hybrid scheme.

[dilpath]

Thanks for the feedback -- I tried to resolve this by changing some docstrings.

[PaulJonasJost]

Thanks for this 🙏🏼

[PaulJonasJost]

I find it puzzling that we print a warning in a default setting. Rather a debug? (unless we expect the user to set a value usually)

[dilpath]

Sure, done now (also for the old warning). I also updated the docstring again to make explicit the assumption that the problem objective is actually the FIM.