2.26) leads to the following condition:
This condition implies that 0 is a local optimum of g; moreover,
Notice that g is a function of one variable, then optimal conditiong′ = 0 is met, regardless the direction of Δx. Therefore, the optimum of a multivariate function is given when the gradient is zero ∇f(x~)=0). This condition permits to find local optimal points, as presented in the next section. Two questions are still open: in what conditions are the optimum global? And, when is the solution unique? We will answer these relevant questions in the next chapter. For now, let us see how to find the optimum using the gradient.
2.5 The gradient method
The gradient method is, perhaps, the most simple and well-known algorithm for solving optimization problems. Cauchy invented the basic method in the 19th century, but the computed advent leads to different applications that encompass power systems operation and machine learning. Let us consider the following unconstrained optimization problem:
where the objective function f :
The gradient method consists in applying this iteration until the gradient is small enough, i.e., until ‖∇f(x)‖ ≥ ϵ. It is easier to understand the algorithm by considering concrete problems and their implementation in Python, as given in the next examples.
Example 2.4
Consider the following optimization problem:
The gradient of this function is presented below:
We require to find a value (x, y) such that this gradient is zero. Therefore, we use the gradient method. The algorithm starts from an initial point (for example x = 10, y = 10) and calculate new points as follows:
This step can be implemented in a script in Python, as presented below:
import numpy as np x = 10 y = 10 t = 0.03 for k in range(50): dx = 20*x + np.exp(x+y) dy = 30*y + np.exp(x+y) x += -t*dx y += -t*dy print('grad:',np.abs([dx,dy])) print('argmin:',x,y)
In the first line, we import the module NumPy with the alias np. This module contains mathematical functions such as sin, cos, exp, ln among others. The gradient introduces two components dx and dy, which are evaluated in each iteration and added to the previous point (x,y). We repeat the process 50 times and print the value of the gradient each iteration. Notice that all the indented statements belong to the for-statement, and hence the gradient is printed in each iteration. In contrast, the argmin is printed only at the end of the process.
Example 2.5
Python allows calculating the gradient automatically using the module AutoGrad. It is quite intuitive to use. Consider the following script, which solves the same problem presented in the previous example:
import autograd.numpy as np from autograd import grad # gradient calculation def f(x): z = 10.0*x[0]**2 + 15*x[1]**2 + np.exp(x[0]+x[1]) return z g = grad(f) # create a funtion g that returns the gradient x = np.array([10.0,10.0]) t = 0.03 for k in range(50): dx = g(x) x = x -t*dx print('argmin:',x)
In this case, we defined a function f and its gradient g where (x, y) was replaced by a vector (x0, x1). The module NumPy was loaded using autograd.numpy to obtain a gradient function automatically. The code executes the same 50 iterations, obtaining the same result. The reader should execute and compare the two codes in terms of time calculation and results.
Example 2.6
Consider a small photovoltaic system formed by three solar panels A, B, and C, placed as depicted in Figure 2.6. Each solar system has a power electronic converter that requires to be connected to a common point E before transmitted
