From Manifold Identification to Newton Acceleration on Intersections: Sparse Stiefel Optimization

Abstract

We study a Newton acceleration for sparse composite optimization on the Stiefel manifold. The main difficulty is geometric: the active manifold identified by the nonsmooth regularizer may fail to intersect the Stiefel manifold transversely, which obstructs a Riemannian Newton step on the identified manifold. In the transverse case, we prove local identification of the ManPG tangent proximal mapping. For nontransverse cases, we introduce an off-diagonally perturbed Stiefel family that generically restores the identification geometry while yielding an \(O(\|Δ\|_F)\)-KKT guarantee for the original problem. We also derive verifiable support-level conditions for clean intersection, which cover nontransverse sparse patterns and yield the smooth moving local models used by the Newton correction. Based on these results, we propose MIX, a safeguarded ManPG/Newton-CG method on moving identified intersections. In the general clean-intersection setting, we prove global descent and KKT-residual guarantees for MIX. In the transverse or generically perturbed cases, if the sequence has an accumulation point satisfying certain regularity assumptions and the second-order sufficient condition (SOSC), then the full sequence converges to that point, with finite active-manifold identification and a local Q-superlinear rate. Numerical experiments on compressed modes and sparse PCA show that MIX substantially improves efficiency while preserving solution quality. Beyond the Stiefel manifold, we also outline how the safeguarded global-convergence mechanism of MIX extends to general smooth equality-constrained manifolds.

Disclosure

“X, and the use of Newton corrections on identified intersections—were formulated by the authors. ChatGPT assisted with specific searches of clean-intersection counterexamples. In preparing Version 3 of this manuscript, the authors used ChatGPT (GPT-5.6 Sol) to assist in developing and checking the revised local convergence analysis, in particular the proof that removes the sequence convergence assumption from the local convergence analysis of MIX. ChatGPT was additionally used f”

PDF page 49

Classification

Proof ideas or individual proof-step assistance

Multiplier

8

Verified

2026-08-23T18:52:33.135747Z

Structural counts