Improved Pattern Complexity for Personal Nonsmooth Nonconvex Optimization

We research differentially non-public (DP) optimization algorithms for stochastic and empirical goals that are neither clean nor convex, and suggest strategies that return a Goldstein-stationary level with pattern complexity bounds that enhance on current works.
We begin by offering a single-pass (ϵ,δ)(epsilon,delta)(ϵ,δ)-DP algorithm that returns an (α,β)(alpha,beta)(α,β)-stationary level so long as the dataset is of dimension Ω~(1/αβ3+d/ϵαβ2+d3/4/ϵ1/2αβ5/2)widetilde{Omega}left(1/alphabeta^{3}+d/epsilonalphabeta^{2}+d^{3/4}/epsilon^{1/2}alphabeta^{5/2}proper)Ω(1/αβ3+d/ϵαβ2+d3/4/ϵ1/2αβ5/2), which is Ω(d)Omega(sqrt{d})Ω(d​) occasions smaller than the algorithm of Zhang et al. [2024] for this process, the place ddd is the dimension.
We then present a multi-pass polynomial time algorithm which additional improves the pattern complexity to Ω~(d/β2+d3/4/ϵα1/2β3/2)widetilde{Omega}left(d/beta^2+d^{3/4}/epsilonalpha^{1/2}beta^{3/2}proper)Ω(d/β2+d3/4/ϵα1/2β3/2), by designing a pattern environment friendly ERM algorithm, and proving that Goldstein-stationary factors generalize from the empirical loss to the inhabitants loss.

† Work partially carried out throughout Apple internship