Documentation: expand sparse least squares example to other algorithms (Douglas-Rachford)

[patwa67]

I managed to get most algorithms running on the Sparse regression example, but not the DRLS algorithm. I guess it should be applicable to sparse regression, but what needs to be done with:
drls = ProximalAlgorithms.DRLS(tol=1e-4,verbose=true)
solution_dr, iterations_dr = drls(x0=zeros(n_features+1),f=training_loss, g=reg,gamma=1e-7)

[lostella]

In that example, a function can be passed as f argument to the proximal-gradient-based algorithms, since gradients will be taken with respect to f via automatic differentiation.

For other algorithms, including DouglasRachford and DRLS, the proximal mapping of f is needed. Therefore a regular Julia function will not work. Instead, you can construct an equivalent function using eg ProximalOperators. For this specific example, something along the lines of this should work, for appropriate A and b and scaling coefficient: this is used a little below exactly with DRLS.

I think this is once again a documentation issue: feel free to leave this open, I’ll just adjust the title.

[patwa67]

It seems as if ProximalAlgorithms.DRLS(tol = 10 * TOL, directions=acc) doesn't care about the values of the tol parameter.

[lostella]

It seems as if ProximalAlgorithms.DRLS(tol = 10 * TOL, directions=acc) doesn't care about the values of the tol parameter.

Do you have an example?

It definitely cares about the value of tol, but like other Newton-type methods it may suddenly converge fast to the solution, and therefore appear to be insensitive to the tolerance.

[patwa67]

It's a rather large data set, can I send it by email? I have tested your test_small_lasso.jl and DRLS works fine on that small data, but not on my larger data. The interesting point is that all other methods from that file works fine on my data.

[lostella]

How big is it? Otherwise you could upload the data somewhere and share the link here, in case it’s fine to have the data publicly available

[patwa67]

test_lasso.zip

[lostella]

@patwa67 looks like there is a bug in the way DRLS displays progress:

ProximalAlgorithms.jl/src/algorithms/drls.jl

Lines 188 to 190 in 3de04a9

	default_display(it, ::DRLSIteration, state::DRLSState) = @printf(
	"%5d | %.3e | %.3e | %.3e\n", it, state.gamma / state.gamma, norm(state.res, Inf), state.tau,
	)

Fixing it, thanks!