April 9, 2018
pSeven Beats MOPTA08 Automotive Benchmark
MOPTA08 is a multidisciplinary design optimization (MDO) benchmark problem based on a real-life problem from the automotive industry. First, it was presented at the MOPTA 2008 Conference by Don Jones, a technical fellow at General Motors. It states a large-scale multidisciplinary mass optimization of a vehicle in a crash test simulation. Real simulation can optimistically compute about 60 points/day. It was highly desirable to solve the optimization problem in ≤ 1 month (30 days).
The original publication by Miguel F. Anjos is available at http://www.miguelanjos.com/jones-benchmark.
GM crash test simulation example (from wired.com)
MOPTA08 benchmark uses a blackbox based on kriging response surfaces. These response surfaces are trained on data from General Motors crash test simulation.
- 1 objective function to be minimized - mass
- 124 variables normalized to [0,1]
- 68 inequality constraints of form gi (x) ≤ 0
- Constraints well normalized: 0.05 means 5% over requirement, etc.
- Test problem comes with the initial feasible point with objective ~251.07
A good performance would be comparable or better than derivative-free optimization algorithm - Powell’s COBYLA:
- Number of evaluations = ~ 15 x Number of variables
- Fully feasible solution (no constraint violations)
- Objective function value ≤ 228 (at least 80% of potential reduction)
"Anything better is exciting" - states Don Jones, the author of this benchmark.
Objective function dependencies in MOPTA08 blackbox (green – feasible, red – infeasible)
- Max evaluations budget is 1860 points
- Known optimum appears to have objective ~222.74
- The budget is obviously very small for such big number of variables!
The significant number of design variables excludes the use of Surrogate-Based Optimization (SBO) methods so a local gradient-based method Sequential Quadratic Programming (SQP) is used.
The solution is divided into two stages Neval = NIeval + NIIeval
- Stage-I: algorithm works as usual for NIeval evaluations
- Stage-II: if a better feasible solution is not yet found, solve Constraints Satisfaction Problem (CSP) within NIIeval evaluations
An actual budget division is assumed to be ~3:1
- NIeval = 1460 points
- NIIeval = 400 points
MOPTA08 optimization workflow in pSeven
- pSeven allowed to reach feasible objective value ~227.56 in 1860 evaluations
- The “effective” number of required evaluations is ~1650:
- Stage-II solution (feasibility restoration) used the designs evaluated on ~1250th iteration
- In other words, the budget in pSeven could be reduced to 1650 iterations
MOPTA08 optimization history in pSeven (green – feasible, red – infeasible)
- Our solution is close to being “exciting” in the terminology of this benchmark
- pSeven significantly outperforms the most of the results presented in the original publication