How to Achieve Good Parameter Optimization Using The New Methodology in UTMOST III Optimizer

Introduction

What is the goal of optimization?

The optimization is the task of finding the absolute best set of admissible conditions to achieve your objective, formulated in mathematical terms. The goal of optimization is, given a system, to find the setting of it’s parameters so that to obtain the optimal performance. The performance of the system is given by an evaluation function. Optimization problems are commonly found in a wide range of fields, and it is also of central concern to many problems.

The basic approach in all cases is usually the same: User selects or designs a “merit-function” that measures the agreement between the data and the model with a particular choice of parameters. The merit function is generally designed so that small values represent close agreement. The parameters and the model are then adjusted to achieve a minimum in the merit function, yielding best-parameters set. The adjustment process is thus a problem in minimization in many dimensions. The computational wish is always the same: do it quickly and cheaply. Often the computational effort is dominated by the cost of evaluating the merit function (and perhaps its partial derivatives). In such cases, the wishes are sometimes replaced by a simple goal: Evaluate the merit-function as few times as possible.

Finding a global extremum is, in general, a very difficult problem.

Two standard methods are widely used:

find local extrema starting from widely varying starting values of the independent variables and then pick the most extreme of these

perturb a local extremum by taking a finite amplitude step away from it, and then see if your method returns you a better point, or always to the same one (Simulated Annealing)[1]

Unfortunately, there is no perfect optimization algorithm. and the choice of the optimization method is based on the following consideration: A selection must be made between methods that need only evaluations of the “merit-function” to minimize and those that also require evaluations of the derivatives of that functions. Algorithms using the derivative are somewhat more powerful than those using only the function, but not always enough so as to compensate for the additional calculations of derivatives.