Frank–Wolfe algorithm

The Frank–Wolfe algorithm is an iterative first-order optimization algorithm for constrained convex optimization. Known as the conditional gradient method, reduced gradient algorithm and the convex combination algorithm, the method was proposed by Marguerite Frank and Philip Wolfe in 1956. In each iteration, the Frank–Wolfe algorithm considers a linear approximation of the objective function, moves towards a minimizer of this linear function. Suppose D is a compact convex set in a vector space and f: D → R is a convex differentiable real-valued function; the Frank–Wolfe algorithm solves the optimization problem Minimize f subject to x ∈ D. Initialization: Let k ← 0, let x 0 be any point in D. Step 1. Direction-finding subproblem: Find s k solving Minimize s T ∇ f Subject to s ∈ D Step 2. Step size determination: Set γ ← 2 k + 2, or alternatively find γ that minimizes f subject to 0 ≤ γ ≤ 1. Step 3. Update: Let x k + 1 ← x k + γ, let k ← k + 1 and go to Step 1. While competing methods such as gradient descent for constrained optimization require a projection step back to the feasible set in each iteration, the Frank–Wolfe algorithm only needs the solution of a linear problem over the same set in each iteration, automatically stays in the feasible set.

The convergence of the Frank–Wolfe algorithm is sublinear in general: the error in the objective function to the optimum is O after k iterations, so long as the gradient is Lipschitz continuous with respect to some norm. The same convergence rate can be shown if the sub-problems are only solved approximately; the iterates of the algorithm can always be represented as a sparse convex combination of the extreme points of the feasible set, which has helped to the popularity of the algorithm for sparse greedy optimization in machine learning and signal processing problems, as well as for example the optimization of minimum–cost flows in transportation networks. If the feasible set is given by a set of linear constraints the subproblem to be solved in each iteration becomes a linear program. While the worst-case convergence rate with O can not be improved in general, faster convergence can be obtained for special problem classes, such as some convex problems. Since f is convex, for any two points x, y ∈ D we have: f ≥ f + T ∇ f This holds for the optimal solution x ∗.

That is, f ≥ f + T ∇ f. The best lower bound with respect to a given point x is given by f ≥ f + T ∇ f ( x

Zygmunt Zaremba

Zygmunt Witalis Zaremba, pseudonyms Andrzej Czarski, Wit Smrek, was Polish socialist activist and publicist. Zaremba was member of the Youth Association for Progress and Independence, Polish Socialist Party - Opposition member of the Polish Socialist Party and its Central Executive Committee. Since 1918 he stayed in Poland, he was member of Polish Socialist Party authorities – Supreme Council and Central Executive Committee. During the years of 1921–1924 he was a vice-president of its Supreme Council. In the years of 1922–1935 Zaremba was a deputy of a Sejm. During the Invasion of Poland he organised the Robotnicza Brygada Obrony Warszawy. Zaremba was a co-founder of conspiratory Polish Socialist Party - Freedom-Equality-Independence and its administration member. In the years of 1944–1945 representative of the Council of National Unity. In 1946 he moved to Paris, where he became a president of the Central Committee of the Polish Socialist Party. In 1949 he co-founded Political Council in London.

He was a president and co-founder of the International Socialist Office and until 1964, president of the Central-East Socialist Europe Union. Zaremba was a co-author of Program Polski Ludowej, he was an editor of party-press Robotnik, Związkowiec. An editor of Światło and Droga. Czerski, Andrzej. Od Borysowa do Rygi. —.. Racjonalizacja - kryzys - proletariat. —.. PPS w Polsce niepodległej. —.. Obrona Warszawy. —.. Obrona Warszawy. London. —.. Powstanie sierpniowe. —.. Les transformations sociales en Pologne. Paris. —.. Wojna i konspiracja. London. —.. Przemiany w ruchu komunistycznym. Paris. —.. Wspomnienia. Pokolenie przełomu 1905-1919. "Zaremba Zygmunt Witalis". Internetowa encyklopedia PWN. Wydawnictwo Naukowe PWN. Retrieved 2007-11-26. "Zaremba Zygmunt Witalis". WIEM Encyklopedia. Retrieved 2007-11-26. Zygmunt Zaremba at WorldCat