# Convex optimization

Convex optimization is a subfield of mathematical optimization that studies the problem of minimizing convex functions over convex sets. Many classes of convex optimization problems admit polynomial-time algorithms, whereas mathematical optimization is in general NP-hard.

Convex optimization has applications in a wide range of disciplines, such as automatic control systems, estimation and signal processing, communications and networks, electronic circuit design, data analysis and modeling, finance, statistics (optimal experimental design), and structural optimization, where the approximation concept has proven to be efficient.  With recent advancements in computing and optimization algorithms, convex programming is nearly as straightforward as linear programming.

## Definition

A convex optimization problem is an optimization problem in which the objective function is a convex function and the feasible set is a convex set. A function $f$ mapping some subset of $\mathbb {R} ^{n}$ into $\mathbb {R} \cup \{\pm \infty \}$ is convex if its domain is convex and for all $\theta \in [0,1]$ and all $x,y$ in its domain, the following condition holds:$f(\theta x+(1-\theta )y)\leq \theta f(x)+(1-\theta )f(y)$ . A set S is convex if for all members $x,y\in S$ and all $\theta \in [0,1]$ , we have that $\theta x+(1-\theta )y\in S$ .

Concretely, a convex optimization problem is the problem of finding some $\mathbf {x^{\ast }} \in C$ attaining

$\inf\{f(\mathbf {x} ):\mathbf {x} \in C\}$ ,

where the objective function $f$ is convex, as is the feasible set $C$ .   If such a point exists, it is referred to as an optimal point; the set of all optimal points is called the optimal set. If $f$ is unbounded below over $C$ or the infimum is not attained, then the optimization problem is said to be unbounded. Otherwise, if $C$ is the empty set, then the problem is said to be infeasible.

### Standard form

A convex optimization problem is said to be in the standard form if it is written as

{\begin{aligned}&{\underset {\mathbf {x} }{\operatorname {minimize} }}&&f(\mathbf {x} )\\&\operatorname {subject\ to} &&g_{i}(\mathbf {x} )\leq 0,\quad i=1,\dots ,m\\&&&h_{i}(\mathbf {x} )=0,\quad i=1,\dots ,p,\end{aligned}} where $x\in \mathbb {R} ^{n}$ is the optimization variable, the functions $f,g_{1},\ldots ,g_{m}$ are convex, and the functions $h_{1},\ldots ,h_{p}$ are affine.  In this notation, the function $f$ is the objective function of the problem, and the functions $g_{i}$ and $h_{i}$ are referred to as the constraint functions. The feasible set of the optimization problem is the set consisting of all points $x\in \mathbb {R} ^{n}$ satisfying $g_{1}(x)\leq 0,\ldots ,g_{m}(x)\leq 0$ and $h_{1}(x)=0,\ldots ,h_{p}(x)=0$ . This set is convex because the sublevel sets of convex functions are convex, affine sets are convex, and the intersection of convex sets is convex. 

Many optimization problems can be equivalently formulated in this standard form. For example, the problem of maximizing a concave function $f$ can be re-formulated equivalently as the problem of minimizing the convex function $-f$ ; as such, the problem of maximizing a concave function over a convex set is often referred to as a convex optimization problem.

## Properties

The following are useful properties of convex optimization problems:

• every local minimum is a global minimum;
• the optimal set is convex;
• if the objective function is strictly convex, then the problem has at most one optimal point.

These results are used by the theory of convex minimization along with geometric notions from functional analysis (in Hilbert spaces) such as the Hilbert projection theorem, the separating hyperplane theorem, and Farkas' lemma.

## Examples

The following problem classes are all convex optimization problems, or can be reduced to convex optimization problems via simple transformations:  

## Lagrange multipliers

Consider a convex minimization problem given in standard form by a cost function $f(x)$ and inequality constraints $g_{i}(x)\leq 0$ for $1\leq i\leq m$ . Then the domain ${\mathcal {X}}$ is:

${\mathcal {X}}=\left\{x\in X\vert g_{1}(x),\ldots ,g_{m}(x)\leq 0\right\}.$ The Lagrangian function for the problem is

$L(x,\lambda _{0},\lambda _{1},\ldots ,\lambda _{m})=\lambda _{0}f(x)+\lambda _{1}g_{1}(x)+\cdots +\lambda _{m}g_{m}(x).$ For each point $x$ in $X$ that minimizes $f$ over $X$ , there exist real numbers $\lambda _{0},\lambda _{1},\ldots ,\lambda _{m},$ called Lagrange multipliers, that satisfy these conditions simultaneously:

1. $x$ minimizes $L(y,\lambda _{0},\lambda _{1},\ldots ,\lambda _{m})$ over all $y\in X,$ 2. $\lambda _{0},\lambda _{1},\ldots ,\lambda _{m}\geq 0,$ with at least one $\lambda _{k}>0,$ 3. $\lambda _{1}g_{1}(x)=\cdots =\lambda _{m}g_{m}(x)=0$ (complementary slackness).

If there exists a "strictly feasible point", that is, a point $z$ satisfying

$g_{1}(z),\ldots ,g_{m}(z)<0,$ then the statement above can be strengthened to require that $\lambda _{0}=1$ .

Conversely, if some $x$ in $X$ satisfies (1)–(3) for scalars $\lambda _{0},\ldots ,\lambda _{m}$ with $\lambda _{0}=1$ then $x$ is certain to minimize $f$ over $X$ .

## Algorithms

Convex optimization problems can be solved by the following contemporary methods:

Subgradient methods can be implemented simply and so are widely used. Dual subgradient methods are subgradient methods applied to a dual problem. The drift-plus-penalty method is similar to the dual subgradient method, but takes a time average of the primal variables.

## Extensions

Extensions of convex optimization include the optimization of biconvex, pseudo-convex, and quasiconvex functions. Extensions of the theory of convex analysis and iterative methods for approximately solving non-convex minimization problems occur in the field of generalized convexity, also known as abstract convex analysis.

## See also

This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.