3.1. Introduction

Generic Tool for Optimization (GTOpt) is a software package for multi- and single- objective nonlinear optimization. Optimization problems of this kind arise in almost all engineering and scientific applications. GTOpt implements variety of modern methods, however, most of the technical details are hidden from the end user.

Optimization is a process of finding the values of model parameters (design variables), which lead to the best performance of model under investigation. Model performance is usually quantified in terms of several fitness functions constructed from measured model responses. GTOpt offers a variety of methods to conduct optimization of one or multiple model performance criteria and allows to efficiently solve engineering optimization problems, where the prime (but not the only) difficulty is the expensiveness of objectives and constraints evaluations. However, this does not mean that GTOpt cannot efficiently optimize your cheap to evaluate, semi-analytical model. To the contrary, GTOpt provides a diversity of optimization methods and algorithms covering majority of mathematical programming topics.

Through all the GTOpt development history a few basic principles were always pursuit:

1. Algorithm efficiency.

   No matter what type of optimization problem is considered, implemented in GTOpt methods are ought to be the most effective in terms of required number of model evaluations. Everything else (including, for instance, CPU time) is considered to be of lower priority. And we are doing all the best to constantly ensure this (GTOpt is routinely tested against other solvers and previous GTOpt versions).

2. Simplicity to use by non-experts.

   No doubt, optimization branch of mathematical sciences is rather intricate and highly developed. We cannot expect deep understanding of optimization methods from the end user, however, efficiency of getting the solution utterly depends on proper algorithm selection. To solve this issue GTOpt never asks, which optimization method is to be applied. Instead, it inquires for basic properties of the model (such as evaluation expensiveness, smoothness of model responses, or expected degree of multi-modality), which are hopefully simple enough to be easily accessible. The choice of specific optimization algorithm is then based on gathered information.

3. Highest customization degree.

   Simplicity to use by non-experts implies that none of implemented optimization methods are visible to end user, moreover, user essentially never knows, which algorithm will be used to solve his problem. But this was proven to be unacceptable for expert-level customers (as well as for GTOpt developers), which want to have full control over the whole optimization process. Therefore, the entire optimization stack of GTOpt is ought to be highly customizable and we proud that these capabilities are provided all the way down to the lowest level numerical routines.

4. “Don’t ever reinvent the wheel”.

   For GTOpt development team this well known idiom translates to: “Don’t spend time in implementing particular optimization method unless you have an original idea making your implementation competitive”. Despite of simplicity, this principle has two major consequences:

   • Every implemented in GTOpt algorithm possesses its own original features which make it unique and cutting-edge efficient.
   • In well-known and fairly investigated cases a good freely available implementation is to be reused (for example, in linear programming, where we have little hope to invent something new in reasonable time).

Prior to discussing supported types and flavors of optimization problems, we should emphasize that GTOpt had always been oriented towards real-life engineering applications. Apart from requiring each and every algorithm to be:

• sufficiently insensitive to numerical noise in model responses, and
• stable with respect to occasional undefined model behavior,

this trend dictates serious changes in what is meant by the solution of optimization problem. This question is by no means void: as it often happens in applications proper problem formulation could only be found by trial-and-error analysis, which might become prohibitive due to expected high computational cost of underlying models. Therefore, we cannot afford to have single solution of the problem, all other solution candidates might equally important.

Peculiarities of what is understood by optimal solution in GTOpt are discussed in Definitions, while some details on treatment of noisy problems are summarized in section Noisy Problems.

To briefly describe GTOpt optimization capabilities and available methods, let us introduce some notations:

• N — the number of design variables perhaps subject to box bounds.
• K — the number of criteria (objective functions) to consider.
• M — the number of constraints defining feasible domain, excluding box bounds on design variables.

Then the general form of optimization problems served GTOpt optimizer might be represented as (this formulation does not concerns robust optimization cases, for which separate treatment is required, see below):

\[\begin{split}\begin{array}{cl} \min\limits_x \, f^i(x) & K\,\,\, \mbox{objective functions} \\ c^j_L \,\le\, c^j(x)\,\le\, c^j_U & M \,\,\, \mbox{generic constraints} \\[9pt] x^k_L \,\le\, x^k\,\le\, x^k_U & N\,\,\,\mbox{box bounds} \end{array}\end{split}\]

Note that box bounds on design variables are treated specially and are respected always. It is guaranteed that no one model evaluation could ever happen outside of imposed box bounds.

Main classes of problems, which can be efficiently solved with GTOpt are summarized in the following table.

Problem Type	N	K	M
Linear	\(\lesssim 10^6\)	1, linear	\(\lesssim 10^6\), linear
Quadratic indefinite	\(\lesssim 10^4\)	1, quadratic	\(\lesssim 10^4\), linear
Single-objective unconstrained	\(\lesssim 10^4\)	1	0
Single-objective constrained	\(\lesssim 10^4\)	1	\(\lesssim 10^4\)
Constraint satisfaction	\(\lesssim 10^4\)	0	\(\lesssim 10^4\)
Multi-objective	\(\lesssim 10^4\)	\(\lesssim 10\)	\(\lesssim 10^4\)

Note that cited numbers are only approximate estimates, precise values crucially depend upon the considered problem. Nevertheless, they are surely valid by order of magnitude. Within this table relevant problem data (objectives, constraints) are assumed to be generic non-linear functions unless explicitly stated otherwise. It should be noted that effective solution of the above types of problems requires at very least cutting-edge efficient implementation of the variety of numerical algorithms, perhaps not directly related to optimization per se. Section Numerical Methods provides a brief summary of numerical methods implemented in GTOpt.

It is crucial that the optimization functionality of GTOpt is not exhausted by the above “classical” problem formulations. Moreover, efficient methods to solve these cases provide foundation for algorithms, aimed to deal with real-life applications. Perhaps, the most important here is the family of optimization strategies based on surrogate modeling, which are believed to be most efficient in the majority of engineering applications. It is worth to mention that GTOpt surrogate based optimization supports all types of problems including multi-objective robust formulations, therefore, SBO in GTOpt is indeed an universal tool to investigate and eventually solve real-life applications. See Surrogate Based Optimization.

In many practical cases expensiveness of model evaluation is not the prime obstacle to get the solution. Often problem data (objectives, constraints) intrinsically depend upon external parameters, the values of which are known only approximately. In this case it is natural to look for the solutions, which are minimally sensitive to variations of not-well known parameters. Clearly, this approach requires problem reformulation, which is commonly known as robust counterpart of original optimization task. GTOpt also provides cutting-edge effective robust optimization functionality, which works well even with expensive to evaluate models — see Robust Optimization.

Last but not least functionality of GTOpt to be mentioned here concerns integer design variables. No doubt, mixed-integer black-box optimization is incomparably more complex than its continuous counterpart. Although no theoretically sound results are available here, GTOpt provides respective functionality, based on surrogate modeling approach, which covers all mentioned above types of problems, including robust formulations — see Mixed Integer Variables.

Finally, let us mention that the abovementioned variety of methods is tightly bound up with the GTOpt optimization stack architecture summarized in section Solver Architecture. Its universality allows to easily add new optimization algorithms, replace existing ones with advanced implementations and configure virtually all aspects of optimizer in a few steps.

GTOpt is designed to efficiently solve optimization problems which arise in engineering applications, e.g. aircraft wing or helicopter blade shape optimization. Typically relevant functions in these cases are very slow, so they require a lot of CPU time even for a single objective function evaluation. Thus the primal goal of GTOpt is to reduce the number of user function calls as much as possible. All other characteristics, like CPU time spent, are considered not as much important.