Comments on: SPGL1 and TV minimization ?

By: SPGL1 and TV: Answers from SPGL1 Authors « Le Petit Chercheur Illustré

SPGL1 and TV: Answers from SPGL1 Authors « Le Petit Chercheur Illustré — Tue, 02 Sep 2008 21:48:24 +0000

[...] SPGL1 Authors Following the writing of my previous post, which obtained various interesting comments (many thanks to Gabriel Peyré, Igor Caron and Pierre Vandergheynst), I sent a mail to Michael P. [...]

By: jackdurden

jackdurden — Tue, 26 Aug 2008 12:43:09 +0000

Thank you Gabriel. Very good review. I noticed however that the correct link is :

http://www.ceremade.dauphine.fr/~peyre/cs-tv/OptimReview.pdf

Best,
Laurent

By: Gabriel Peyré

Gabriel Peyré — Mon, 25 Aug 2008 18:04:17 +0000

I have put here
http://www.ceremade.dauphine.fr/~peyre/upload/cs-tv/OptimReview.pdf
more details about iterative thresholding and Chambolle’s algorithm !

By: jackdurden

jackdurden — Sat, 23 Aug 2008 22:05:12 +0000

No. I didn’t notice that new version. Thank you. BTW, I sent a mail to the authors of spgl1 about our recent TV discussion above.

Laurent

By: Igor Carron

Igor Carron — Fri, 22 Aug 2008 16:33:55 +0000

Laurent,

Surely you have noted that there is new version of SPGL1 (v. 1.6). It’ll be featured on Nuit Blanche on Monday.

Cheers,

Igor.

By: Pierre Vandergheynst

Pierre Vandergheynst — Wed, 20 Aug 2008 10:37:04 +0000

I see, that’s interesting since it generalizes the class of measurement matrices for which you can now apply the algo. Some testing is needed

By: Gabriel Peyré

Gabriel Peyré — Wed, 20 Aug 2008 10:23:03 +0000

Ouch sorry, the iterations should be in fact

By: Gabriel Peyré

Gabriel Peyré — Wed, 20 Aug 2008 10:21:56 +0000

Apparently I have some problems with my old school ASCII equations … let’s try in LaTeX …

The initial iteration reads

and the modified one

where should be larger than the operator norm of A for the iterations to converge.

By: Gabriel Peyré

Gabriel Peyré — Wed, 20 Aug 2008 09:48:23 +0000

Dear Pierre and Laurent,

Actually yes (if I am not wrong), the algorithm generalizes to the case where A is not an orthogonal projector. You need to replace the orthogonal projection on the constraints Au=b

u <- u + A^+ ( b – A u )

(where A^+=A* is the pseudo inverse, equal to A* because we are dealing with an orthogonal projector), by a gradient descent step

u than the operator norm of A (same as in the paper of Daubechies et al, and Combettes proved that it can be extended to twice the operator norm).

Then you apply Chambolle’s algorithm with a regularization parameter lambda/tau instead of lambda.

This corresponds to proximal iterations, since Chambolle algorithm is actually computing the proximal operator associated to the TV norm.

Intuitively, as emphasized by Laurent, this corresponds to replacing the soft thresholding by the ROF resolution, which are prox operator of respectively the L1 norm in an ortho-basis and the TV norm.

I am not sure wether the update of lambda to match the L2 constraint is still garantied to work in this case.

By: jackdurden

jackdurden — Wed, 20 Aug 2008 08:03:23 +0000

Or another one is to generalize the update of the regularization parameter of A. Chambolle (that allows the convergence to the constraint problem), to the Iterative Soft Thresholding procedure of TwIST, where the measurement matrix can be a RIP matrix (not necessarily an orthoprojector).

Laurent