may include equality constraints, as presented below:
For solving this problem, a function called lagrangian is defined as follows:
This new function depends on the original decision variables x and a new variable λ, known as Lagrange multiplier or dual variable. By means of the lagrangian function, a constrained optimization problem was transformed into an unconstrained optimization problem that can be solved numerically, namely:
by a small abuse of notation, ∂f / ∂x is used instead of ∇f, which is the formal representation for the n-dimentional case, (the same for L and g). Notice that the optimal conditions of L imply optimal solution in f but also feasibility in terms of the constraint.
The first condition implies that the gradient of the objective function must be parallel to the gradient of the constraint and, the Lagrange multiplier is the proportionality constant. Besides this geometrical interpretation, Lagrange multipliers have another interesting interpretation. Suppose a local optimum x~ is found for a constrained optimization problem, and we want to know the sensibility of this optimum with respect to a. The following derivative can be calculated, relating the change of the objective function with respect to a change in the constrain:
The first two terms in the right-hand side of the equation vanishes, in view of the optimal conditions of x~; thus, the following expresion is obtained:
this means that λ is the variation of the lagrangian (and hence the objective function), for a small variation on the parameter a (see Chapter 3 for more details about dual variables).
Example 2.8
Consider the following optimization problem
If the problem were unconstrained, the solution would be x = 0, y = 0; however, the solution must fulfill the constraint x + y = 1. Therefore, a new function is defined as follows:
with the following optimal conditions
