Example 6 - Multi-objective optimization for an ellipse function¶

In this example, we will demonstrate how Bayesian Optimization can perform multi-objective optimization (MOO) and create a Pareto front. We will use a hyperthetical function which has a shape of an ellispe:

\[y_1 = x\]

\[y_2 = \sqrt{(1 - x^2/4)}\]

where

\[y_1^2 + y_2^2/4 = 1\]

x is the only input parameter. y1 and y2 are two output reponses which cannot be optimized jointly.

Multi-objective optimization derives a set of solutions that define the tradeoff between competing objectives. The boundary defined by the entire feasible solution set is called the Pareto front.

In nextorch, we implement weighted sum method to construct the Pareto front. It is commonly used for convex problems. A set of objectives are scalarized to a single objective by adding each objective pre-multiplied by a user-supplied weight. The weight of an objective is chosen in proportion to its relative importance. The optimization is simply performed with respected to the scalarized objective. By varying the weight combinations, we can construct the whole Pareto front.

For this example, the scalarized objective can be written as,

\[y = w_1 y_1 + w_2 y_2\]

where the weights

\[w_1, w_2 \in [0, 1]\]

\[w_1 + w_2 = 1\]

The details of this example is summarized in the table below:

Key Item	Description
Goal	Maximization, two objectives
Objective function	Ellipse function
Input (X) dimension	1
Output (Y) dimension	2
Analytical form available?	Yes
Acqucision function	Expected improvement (EI)
Initial Sampling	Latin hypercube

Next, we will go through each step in Bayesian Optimization.

1. Import `nextorch` and other packages¶

[7]:

import time
import numpy as np
from nextorch import plotting, bo, doe, utils, io

2. Define the objective function and the design space¶

We import the PFR model, and wrap it in a Python function called PFR as the objective function objective_func.

The ranges of the input X are specified.

[8]:

#%% Define the objective function
def ellipse(X_real):
    """ellipse function

    Parameters
    ----------
    X_real : numpy matrix
        input parameter

    Returns
    -------
    Y_real: numpy matrix
        y1 and y2
    """
    if len(X_real.shape) < 2:
        X_real = np.expand_dims(X_real, axis=1) #If 1D, make it 2D array

    y1 = X_real.copy()
    y2 = np.sqrt(1 - X_real**2/4)

    Y_real = np.concatenate((y1, y2), axis = 1)

    return Y_real # y1, y2


# Objective function
objective_func = ellipse


#%% Define the design space
X_name = ['x']

# two outputs
Y_names = [r'$\rm y_1$', r'$\rm y_2$']

# combine X and Y names
var_names = X_name + Y_names

# Set the operating range for each parameter
X_ranges =  [[0, 2]]

# Get the information of the design space
n_dim = 1 # the dimension of inputs
n_objective = 2 # the dimension of outputs

3. Define the initial sampling plan¶

Here we use LHC design with 10 points for the initial sampling. The initial reponse in a real scale Y_init_real is computed from the objective function.

[9]:

#%% Initial Sampling
# Latin hypercube design with 10 initial points
n_init_lhc = 10
X_init_lhc = doe.latin_hypercube(n_dim = n_dim, n_points = n_init_lhc, seed= 1)
# Get the initial responses
Y_init_lhc = bo.eval_objective_func(X_init_lhc, X_ranges, objective_func)

4. Initialize an `Experiment` object¶

In this example, we use an WeightedMOOExperiment object, a class designed for multi-objective optimization. It can handle multiple weight combinations, perform the scalarized objective optimization automatically, and construct the entire Pareto front.

An WeightedMOOExperiment is a subclass of Experiment. It requires all key components as Experiment: - Name of the experiment, used for output folder name - Input independent variables X: X_init or X_init_real - List of X ranges: X_ranges - Output dependent variables Y: Y_init or Y_init_real

Optional: - unit_flag: True if the input X matrix is a unit scale, else False - objective_func: Used for test plotting - maximize: True if we look for maximum, else False for minimum

Additionally, weights is required for WeightedMOOExperiment.set_optim_specs function. It defines a list of weights for objective 1. The weights of objective 2 is 1 minus that of objective 1. Under the hood, each weight combination correponds to a single Experiment object, each with a different scalarized objective.

Some progress status will be printed out while initializing all single Experiment objects.

[10]:

#%% Initialize an multi-objective Experiment object
# Set its name, the files will be saved under the folder with the same name
Exp_lhc = bo.WeightedMOOExperiment('ellipse_MOO')
# Import the initial data
Exp_lhc.input_data(X_init_lhc,
                   Y_init_lhc,
                   X_ranges = X_ranges,
                   X_names = X_name,
                   Y_names = Y_names,
                   unit_flag = False)

# Set the optimization specifications
# here we set the objective function, minimization by default
# 10 weights, 10 Experiments
n_exp = 21 # number of single Experiments

# Set a weight vector for objective 1
weights_obj_1 = np.linspace(0, 1, n_exp)
weights_obj_2 = 1 - weights_obj_1

# Set a timer
start_time = time.time()
Exp_lhc.set_optim_specs(objective_func = objective_func,
                        maximize = True,
                        weights = weights_obj_1)
end_time = time.time()
print('Initializing {} Experiments takes {:.2f} minutes.'.format(n_exp, (end_time-start_time)/60))

Initializing 21 experiments
Iter 10/100: 5.723519325256348
Iter 20/100: 5.232956886291504
Iter 30/100: 4.613386154174805
Iter 40/100: 1.9704792499542236
Iter 50/100: 1.512966513633728
Iter 60/100: 1.4102624654769897
Iter 70/100: 1.3249999284744263
Iter 80/100: 1.2746305465698242
Iter 90/100: 1.2467479705810547
Iter 100/100: 1.225650668144226
Initializing experiments 4.76 %
Iter 10/100: 5.723519325256348
Iter 20/100: 5.232956886291504
Iter 30/100: 4.613386154174805
Iter 40/100: 1.9704792499542236
Iter 50/100: 1.512966513633728
Iter 60/100: 1.4102624654769897
Iter 70/100: 1.3249999284744263
Iter 80/100: 1.2746305465698242
Iter 90/100: 1.2467479705810547
Iter 100/100: 1.225650668144226
Initializing experiments 9.52 %
Iter 10/100: 5.723519325256348
Iter 20/100: 5.232956886291504
Iter 30/100: 4.613386154174805
Iter 40/100: 1.9704792499542236
Iter 50/100: 1.512966513633728
Iter 60/100: 1.4102624654769897
Iter 70/100: 1.3249999284744263
Iter 80/100: 1.2746305465698242
Iter 90/100: 1.2467479705810547
Iter 100/100: 1.225650668144226
Initializing experiments 14.29 %
Iter 10/100: 5.723519325256348
Iter 20/100: 5.232956886291504
Iter 30/100: 4.613386154174805
Iter 40/100: 1.9704792499542236
Iter 50/100: 1.512966513633728
Iter 60/100: 1.4102624654769897
Iter 70/100: 1.3249999284744263
Iter 80/100: 1.2746305465698242
Iter 90/100: 1.2467479705810547
Iter 100/100: 1.225650668144226
Initializing experiments 19.05 %
Iter 10/100: 5.723519325256348
Iter 20/100: 5.232956886291504
Iter 30/100: 4.613386154174805
Iter 40/100: 1.9704792499542236
Iter 50/100: 1.512966513633728
Iter 60/100: 1.4102624654769897
Iter 70/100: 1.3249999284744263
Iter 80/100: 1.2746305465698242
Iter 90/100: 1.2467479705810547
Iter 100/100: 1.225650668144226
Initializing experiments 23.81 %
Iter 10/100: 5.723519325256348
Iter 20/100: 5.232956886291504
Iter 30/100: 4.613386154174805
Iter 40/100: 1.9704792499542236
Iter 50/100: 1.512966513633728
Iter 60/100: 1.4102624654769897
Iter 70/100: 1.3249999284744263
Iter 80/100: 1.2746305465698242
Iter 90/100: 1.2467479705810547
Iter 100/100: 1.225650668144226
Initializing experiments 28.57 %
Iter 10/100: 5.723519325256348
Iter 20/100: 5.232956886291504
Iter 30/100: 4.613386154174805
Iter 40/100: 1.9704792499542236
Iter 50/100: 1.512966513633728
Iter 60/100: 1.4102624654769897
Iter 70/100: 1.3249999284744263
Iter 80/100: 1.2746305465698242
Iter 90/100: 1.2467479705810547
Iter 100/100: 1.225650668144226
Initializing experiments 33.33 %
Iter 10/100: 5.723519325256348
Iter 20/100: 5.232956886291504
Iter 30/100: 4.613386154174805
Iter 40/100: 1.9704792499542236
Iter 50/100: 1.512966513633728
Iter 60/100: 1.4102624654769897
Iter 70/100: 1.3249999284744263
Iter 80/100: 1.2746305465698242
Iter 90/100: 1.2467479705810547
Iter 100/100: 1.225650668144226
Initializing experiments 38.10 %
Iter 10/100: 5.723519325256348
Iter 20/100: 5.232956886291504
Iter 30/100: 4.613386154174805
Iter 40/100: 1.9704792499542236
Iter 50/100: 1.512966513633728
Iter 60/100: 1.4102624654769897
Iter 70/100: 1.3249999284744263
Iter 80/100: 1.2746305465698242
Iter 90/100: 1.2467479705810547
Iter 100/100: 1.225650668144226
Initializing experiments 42.86 %
Iter 10/100: 5.723519325256348
Iter 20/100: 5.232956886291504
Iter 30/100: 4.613386154174805
Iter 40/100: 1.9704792499542236
Iter 50/100: 1.512966513633728
Iter 60/100: 1.4102624654769897
Iter 70/100: 1.3249999284744263
Iter 80/100: 1.2746305465698242
Iter 90/100: 1.2467479705810547
Iter 100/100: 1.225650668144226
Initializing experiments 47.62 %
Iter 10/100: 5.723519325256348
Iter 20/100: 5.232956886291504
Iter 30/100: 4.613386154174805
Iter 40/100: 1.9704792499542236
Iter 50/100: 1.512966513633728
Iter 60/100: 1.4102624654769897
Iter 70/100: 1.3249999284744263
Iter 80/100: 1.2746305465698242
Iter 90/100: 1.2467479705810547
Iter 100/100: 1.225650668144226
Initializing experiments 52.38 %
Iter 10/100: 5.723519325256348
Iter 20/100: 5.232956886291504
Iter 30/100: 4.613386154174805
Iter 40/100: 1.9704792499542236
Iter 50/100: 1.512966513633728
Iter 60/100: 1.4102624654769897
Iter 70/100: 1.3249999284744263
Iter 80/100: 1.2746305465698242
Iter 90/100: 1.2467479705810547
Iter 100/100: 1.225650668144226
Initializing experiments 57.14 %
Iter 10/100: 5.723519325256348
Iter 20/100: 5.232956886291504
Iter 30/100: 4.613386154174805
Iter 40/100: 1.9704792499542236
Iter 50/100: 1.512966513633728
Iter 60/100: 1.4102624654769897
Iter 70/100: 1.3249999284744263
Iter 80/100: 1.2746305465698242
Iter 90/100: 1.2467479705810547
Iter 100/100: 1.225650668144226
Initializing experiments 61.90 %
Iter 10/100: 5.723519325256348
Iter 20/100: 5.232956886291504
Iter 30/100: 4.613386154174805
Iter 40/100: 1.9704792499542236
Iter 50/100: 1.512966513633728
Iter 60/100: 1.4102624654769897
Iter 70/100: 1.3249999284744263
Iter 80/100: 1.2746305465698242
Iter 90/100: 1.2467479705810547
Iter 100/100: 1.225650668144226
Initializing experiments 66.67 %
Iter 10/100: 5.723519325256348
Iter 20/100: 5.232956886291504
Iter 30/100: 4.613386154174805
Iter 40/100: 1.9704792499542236
Iter 50/100: 1.512966513633728
Iter 60/100: 1.4102624654769897
Iter 70/100: 1.3249999284744263
Iter 80/100: 1.2746305465698242
Iter 90/100: 1.2467479705810547
Iter 100/100: 1.225650668144226
Initializing experiments 71.43 %
Iter 10/100: 5.723519325256348
Iter 20/100: 5.232956886291504
Iter 30/100: 4.613386154174805
Iter 40/100: 1.9704792499542236
Iter 50/100: 1.512966513633728
Iter 60/100: 1.4102624654769897
Iter 70/100: 1.3249999284744263
Iter 80/100: 1.2746305465698242
Iter 90/100: 1.2467479705810547
Iter 100/100: 1.225650668144226
Initializing experiments 76.19 %
Iter 10/100: 5.723519325256348
Iter 20/100: 5.232956886291504
Iter 30/100: 4.613386154174805
Iter 40/100: 1.9704792499542236
Iter 50/100: 1.512966513633728
Iter 60/100: 1.4102624654769897
Iter 70/100: 1.3249999284744263
Iter 80/100: 1.2746305465698242
Iter 90/100: 1.2467479705810547
Iter 100/100: 1.225650668144226
Initializing experiments 80.95 %
Iter 10/100: 5.723519325256348
Iter 20/100: 5.232956886291504
Iter 30/100: 4.613386154174805
Iter 40/100: 1.9704792499542236
Iter 50/100: 1.512966513633728
Iter 60/100: 1.4102624654769897
Iter 70/100: 1.3249999284744263
Iter 80/100: 1.2746305465698242
Iter 90/100: 1.2467479705810547
Iter 100/100: 1.225650668144226
Initializing experiments 85.71 %
Iter 10/100: 5.723519325256348
Iter 20/100: 5.232956886291504
Iter 30/100: 4.613386154174805
Iter 40/100: 1.9704792499542236
Iter 50/100: 1.512966513633728
Iter 60/100: 1.4102624654769897
Iter 70/100: 1.3249999284744263
Iter 80/100: 1.2746305465698242
Iter 90/100: 1.2467479705810547
Iter 100/100: 1.225650668144226
Initializing experiments 90.48 %
Iter 10/100: 5.723519325256348
Iter 20/100: 5.232956886291504
Iter 30/100: 4.613386154174805
Iter 40/100: 1.9704792499542236
Iter 50/100: 1.512966513633728
Iter 60/100: 1.4102624654769897
Iter 70/100: 1.3249999284744263
Iter 80/100: 1.2746305465698242
Iter 90/100: 1.2467479705810547
Iter 100/100: 1.225650668144226
Initializing experiments 95.24 %
Iter 10/100: 5.723519325256348
Iter 20/100: 5.232956886291504
Iter 30/100: 4.613386154174805
Iter 40/100: 1.9704792499542236
Iter 50/100: 1.512966513633728
Iter 60/100: 1.4102624654769897
Iter 70/100: 1.3249999284744263
Iter 80/100: 1.2746305465698242
Iter 90/100: 1.2467479705810547
Iter 100/100: 1.225650668144226
Initializing experiments 100.00 %
Initializing 21 Experiments takes 0.18 minutes.

5. Run trials¶

At each weight combinations, we perform an optimization task for the scalarized objective (a single Experiment). WeightedMOOExperiment.run_exp_auto run these tasks automatically by using the default choice of acqucision function, Expected improvement (EI). It takes in the number of trials required for each Experiment. The number of trials needs to be large enough which allows Bayesian Optimization algorithm to converge to the optimum. Nevertheless, the optimization of y1 and y2 are rather trivial due to their simple analytical expression. We will do 10 trials for each experiment. The total number of calls for the objective function is n_trails * n_exp (=210).

Some progress status will be printed out during the training.

[11]:

# Set the number of iterations for each experiments
n_trials_lhc = 10
# Set a timer
start_time = time.time()
Exp_lhc.run_exp_auto(n_trials_lhc)

end_time = time.time()
print('Optimizing {} Experiments takes {:.2f} minutes.'.format(n_exp, (end_time-start_time)/60))

Running 21 experiments
Iter 10/100: 0.6268913745880127
Iter 20/100: 0.6080061197280884
Iter 30/100: 0.5958243608474731
Iter 40/100: 0.5888171195983887
Iter 50/100: 0.5849026441574097
Iter 60/100: 0.5811474323272705
Iter 70/100: 0.5798935294151306
Iter 80/100: 0.5778980255126953
Iter 90/100: 0.5769942998886108
Iter 100/100: 0.5764856338500977
Iter 10/100: -0.033872902393341064
Iter 20/100: -0.035207390785217285
Iter 10/100: -0.5689945220947266
Iter 10/100: -1.0354827642440796
Iter 10/100: -0.7298715114593506
Iter 20/100: -0.7343025207519531
Iter 30/100: -0.7352707386016846
Iter 40/100: -0.736293613910675
Iter 10/100: 1.1797966957092285
Iter 10/100: 0.6048701405525208
Iter 10/100: 0.4924396872520447
Iter 10/100: 0.27277737855911255
Iter 10/100: 0.07920187711715698
Running experiments 4.76 %
Iter 10/100: 1.3681230545043945
Iter 20/100: 1.3586927652359009
Iter 30/100: 1.3537731170654297
Iter 40/100: 1.352138638496399
Iter 50/100: 1.3514869213104248
Iter 10/100: 0.8164217472076416
Iter 10/100: 1.7088263034820557
Iter 10/100: 2.4450523853302
Iter 10/100: 2.7939138412475586
Iter 10/100: 2.84529447555542
Iter 10/100: 2.5444865226745605
Iter 10/100: 2.4124679565429688
Iter 10/100: 2.9309582710266113
Iter 10/100: 2.6305296421051025
Running experiments 9.52 %
Iter 10/100: 1.3681230545043945
Iter 20/100: 1.3586927652359009
Iter 30/100: 1.3537731170654297
Iter 40/100: 1.352138638496399
Iter 50/100: 1.3514869213104248
Iter 10/100: 2.7727854251861572
Iter 10/100: 1.7118747234344482
Iter 20/100: 1.7064145803451538
Iter 30/100: 1.7056821584701538
Iter 10/100: 2.402613639831543
Iter 20/100: 2.205653190612793
Iter 10/100: 2.525617837905884
Iter 10/100: 1.5938324928283691
Iter 20/100: 1.5867812633514404
Iter 30/100: 1.5856637954711914
Iter 40/100: 1.585084080696106
Iter 10/100: 2.1382551193237305
Iter 10/100: 2.728966236114502
Iter 10/100: 2.477208137512207
Iter 20/100: 2.468743085861206
Iter 30/100: 2.461470603942871
Iter 40/100: 2.455725908279419
Iter 50/100: 2.4519991874694824
Iter 60/100: 2.4501705169677734
Iter 70/100: 2.4495506286621094
Iter 10/100: 2.821336030960083
Running experiments 14.29 %
Iter 10/100: 1.3681230545043945
Iter 20/100: 1.3586927652359009
Iter 30/100: 1.3537731170654297
Iter 40/100: 1.352138638496399
Iter 50/100: 1.3514869213104248
Iter 10/100: 2.7832698822021484
Iter 10/100: 1.7214404344558716
Iter 20/100: 1.7145824432373047
Iter 30/100: 1.714752197265625
Iter 40/100: 1.7146503925323486
Iter 10/100: 2.1657233238220215
Iter 20/100: 1.9761927127838135
Iter 30/100: 1.973067283630371
Iter 10/100: 3.582399368286133
Iter 10/100: 2.1284360885620117
Iter 20/100: 2.023477077484131
Iter 10/100: 1.8532170057296753
Iter 20/100: 1.8445892333984375
Iter 30/100: 1.8408777713775635
Iter 40/100: 1.8391926288604736
Iter 50/100: 1.837859034538269
Iter 60/100: 1.8367865085601807
Iter 70/100: 1.8363769054412842
Iter 10/100: 3.0369486808776855
Iter 10/100: 2.749030113220215
Iter 10/100: 2.444004535675049
Running experiments 19.05 %
Iter 10/100: 1.3681230545043945
Iter 20/100: 1.3586927652359009
Iter 30/100: 1.3537731170654297
Iter 40/100: 1.352138638496399
Iter 50/100: 1.3514869213104248
Iter 10/100: 2.792799472808838
Iter 10/100: 1.7301276922225952
Iter 20/100: 1.7241005897521973
Iter 10/100: 2.4612932205200195
Iter 20/100: 2.2553839683532715
Iter 10/100: 3.5523085594177246
Iter 10/100: 3.3533549308776855
Iter 10/100: 2.72404146194458
Iter 10/100: 3.0637030601501465
Iter 20/100: 2.996230125427246
Iter 30/100: 2.98677921295166
Iter 40/100: 2.970736026763916
Iter 50/100: 2.964110851287842
Iter 60/100: 2.9593610763549805
Iter 70/100: 2.956413745880127
Iter 80/100: 2.95393443107605
Iter 90/100: 2.952077865600586
Iter 100/100: 2.950751781463623
Iter 10/100: 3.02113676071167
Iter 10/100: 2.8103878498077393
Running experiments 23.81 %
Iter 10/100: 1.3681230545043945
Iter 20/100: 1.3586927652359009
Iter 30/100: 1.3537731170654297
Iter 40/100: 1.352138638496399
Iter 50/100: 1.3514869213104248
Iter 10/100: 2.7726142406463623
Iter 10/100: 1.7114776372909546
Iter 20/100: 1.7054736614227295
Iter 30/100: 1.704662799835205
Iter 10/100: 2.3739192485809326
Iter 20/100: 2.1449532508850098
Iter 10/100: 2.467848300933838
Iter 10/100: 1.5681406259536743
Iter 10/100: 2.89255428314209
Iter 10/100: 2.5576491355895996
Iter 10/100: 2.888643264770508
Iter 10/100: 2.7233901023864746
Running experiments 28.57 %
Iter 10/100: 1.3681230545043945
Iter 20/100: 1.3586927652359009
Iter 30/100: 1.3537731170654297
Iter 40/100: 1.352138638496399
Iter 50/100: 1.3514869213104248
Iter 10/100: 2.768392324447632
Iter 10/100: 3.0067899227142334
Iter 10/100: 2.4555914402008057
Iter 10/100: 2.5896177291870117
Iter 10/100: 3.644348621368408
Iter 10/100: 3.2617673873901367
Iter 20/100: 2.9901938438415527
Iter 30/100: 2.9693527221679688
Iter 40/100: 2.9529311656951904
Iter 50/100: 2.9234466552734375
Iter 60/100: 2.7408971786499023
Iter 70/100: 2.7194924354553223
Iter 80/100: 2.7093825340270996
Iter 90/100: 2.7049508094787598
Iter 10/100: 2.6234514713287354
Iter 10/100: 2.6215763092041016
Iter 20/100: 2.621387004852295
Iter 10/100: 2.4118151664733887
Running experiments 33.33 %
Iter 10/100: 1.3681230545043945
Iter 20/100: 1.3586927652359009
Iter 30/100: 1.3537731170654297
Iter 40/100: 1.352138638496399
Iter 50/100: 1.3514869213104248
Iter 10/100: 2.7873880863189697
Iter 10/100: 3.0451865196228027
Iter 10/100: 2.526772975921631
Iter 10/100: 2.615600347518921
Iter 10/100: 2.4838993549346924
Iter 10/100: 3.529007911682129
Iter 10/100: 3.1605377197265625
Iter 10/100: 3.135444164276123
Iter 10/100: 2.971815347671509
Running experiments 38.10 %
Iter 10/100: 1.3681230545043945
Iter 20/100: 1.3586927652359009
Iter 30/100: 1.3537731170654297
Iter 40/100: 1.352138638496399
Iter 50/100: 1.3514869213104248
Iter 10/100: 2.8159055709838867
Iter 10/100: 3.099329948425293
Iter 10/100: 2.9936540126800537
Iter 10/100: 2.6377127170562744
Iter 10/100: 2.5236144065856934
Iter 10/100: 3.3664932250976562
Iter 10/100: 3.1159825325012207
Iter 10/100: 3.318685531616211
Iter 20/100: 3.3025097846984863
Iter 30/100: 3.3010096549987793
Iter 10/100: 3.1649389266967773
Running experiments 42.86 %
Iter 10/100: 1.3681230545043945
Iter 20/100: 1.3586927652359009
Iter 30/100: 1.3537731170654297
Iter 40/100: 1.352138638496399
Iter 50/100: 1.3514869213104248
Iter 10/100: 2.806105136871338
Iter 10/100: 3.795691728591919
Iter 10/100: 3.424565315246582
Iter 10/100: 3.394320011138916
Iter 10/100: 3.3966333866119385
Iter 10/100: 3.2527337074279785
Iter 10/100: 2.967480182647705
Iter 10/100: 3.010803461074829
Iter 10/100: 3.059370994567871
Running experiments 47.62 %
Iter 10/100: 1.3681230545043945
Iter 20/100: 1.3586927652359009
Iter 30/100: 1.3537731170654297
Iter 40/100: 1.352138638496399
Iter 50/100: 1.3514869213104248
Iter 10/100: 2.790883779525757
Iter 10/100: 3.8096256256103516
Iter 10/100: 3.474247932434082
Iter 10/100: 3.3924412727355957
Iter 10/100: 3.2531042098999023
Iter 10/100: 3.260343074798584
Iter 10/100: 3.186786413192749
Iter 10/100: 2.96463680267334
Iter 10/100: 3.132071018218994
Running experiments 52.38 %
Iter 10/100: 1.3681230545043945
Iter 20/100: 1.3586927652359009
Iter 30/100: 1.3537731170654297
Iter 40/100: 1.352138638496399
Iter 50/100: 1.3514869213104248
Iter 10/100: 2.8003363609313965
Iter 10/100: 3.850367307662964
Iter 10/100: 3.432464599609375
Iter 10/100: 3.1861889362335205
Iter 20/100: 2.9225378036499023
Iter 10/100: 3.770076274871826
Iter 10/100: 3.3576788902282715
Iter 20/100: 3.3390133380889893
Iter 10/100: 3.208883285522461
Iter 10/100: 2.976409912109375
Iter 20/100: 2.9732766151428223
Iter 30/100: 2.9721429347991943
Iter 10/100: 3.070483446121216
Running experiments 57.14 %
Iter 10/100: 1.3681230545043945
Iter 20/100: 1.3586927652359009
Iter 30/100: 1.3537731170654297
Iter 40/100: 1.352138638496399
Iter 50/100: 1.3514869213104248
Iter 10/100: 4.107368469238281
Iter 10/100: 2.678363084793091
Iter 10/100: 3.8013393878936768
Iter 10/100: 3.5047974586486816
Iter 10/100: 3.3746180534362793
Iter 10/100: 3.256424903869629
Iter 10/100: 3.160844326019287
Iter 10/100: 2.9091625213623047
Iter 10/100: 2.9976139068603516
Running experiments 61.90 %
Iter 10/100: 1.3681230545043945
Iter 20/100: 1.3586927652359009
Iter 30/100: 1.3537731170654297
Iter 40/100: 1.352138638496399
Iter 50/100: 1.3514869213104248
Iter 10/100: 2.84079909324646
Iter 10/100: 3.9083783626556396
Iter 10/100: 3.6155099868774414
Iter 10/100: 3.472865343093872
Iter 10/100: 3.3664140701293945
Iter 10/100: 3.005737781524658
Iter 10/100: 3.1068363189697266
Iter 10/100: 3.2274274826049805
Iter 20/100: 3.19210147857666
Iter 30/100: 3.1876795291900635
Iter 40/100: 3.181887149810791
Iter 10/100: 3.100501537322998
Running experiments 66.67 %
Iter 10/100: 1.3681230545043945
Iter 20/100: 1.3586927652359009
Iter 30/100: 1.3537731170654297
Iter 40/100: 1.352138638496399
Iter 50/100: 1.3514869213104248
Iter 10/100: 2.7783145904541016
Iter 10/100: 3.8912203311920166
Iter 10/100: 3.555548667907715
Iter 10/100: 3.453873634338379
Iter 10/100: 3.447695016860962
Iter 10/100: 3.2042813301086426
Iter 10/100: 2.876983165740967
Iter 20/100: 2.7637088298797607
Iter 30/100: 2.526716709136963
Iter 40/100: 2.516010284423828
Iter 50/100: 2.5086259841918945
Iter 60/100: 2.5029947757720947
Iter 70/100: 2.499117136001587
Iter 80/100: 2.4977645874023438
Iter 90/100: 2.4963526725769043
Iter 10/100: 2.3474879264831543
Iter 10/100: 3.5107550621032715
Running experiments 71.43 %
Iter 10/100: 1.3681230545043945
Iter 20/100: 1.3586927652359009
Iter 30/100: 1.3537731170654297
Iter 40/100: 1.352138638496399
Iter 50/100: 1.3514869213104248
Iter 10/100: 2.8144121170043945
Iter 10/100: 3.9295337200164795
Iter 10/100: 3.6426448822021484
Iter 10/100: 3.504629611968994
Iter 10/100: 3.306974172592163
Iter 10/100: 3.1403579711914062
Iter 10/100: 3.278944253921509
Iter 10/100: 3.1242928504943848
Iter 10/100: 2.9754934310913086
Running experiments 76.19 %
Iter 10/100: 1.3681230545043945
Iter 20/100: 1.3586927652359009
Iter 30/100: 1.3537731170654297
Iter 40/100: 1.352138638496399
Iter 50/100: 1.3514869213104248
Iter 10/100: 2.8298919200897217
Iter 10/100: 3.9584975242614746
Iter 10/100: 3.683565616607666
Iter 10/100: 2.5122361183166504
Iter 20/100: 2.517245292663574
Iter 10/100: 2.420309066772461
Iter 20/100: 2.4105477333068848
Iter 30/100: 2.405535936355591
Iter 40/100: 2.402311325073242
Iter 50/100: 2.4005789756774902
Iter 60/100: 2.399474620819092
Iter 70/100: 2.399174690246582
Iter 10/100: 2.2599289417266846
Iter 20/100: 2.258845806121826
Iter 10/100: 3.464040756225586
Iter 10/100: 3.1137373447418213
Iter 10/100: 2.8950583934783936
Running experiments 80.95 %
Iter 10/100: 1.3681230545043945
Iter 20/100: 1.3586927652359009
Iter 30/100: 1.3537731170654297
Iter 40/100: 1.352138638496399
Iter 50/100: 1.3514869213104248
Iter 10/100: 3.846512794494629
Iter 10/100: 3.9135282039642334
Iter 10/100: 2.5095925331115723
Iter 10/100: 2.4661977291107178
Iter 20/100: 2.4610347747802734
Iter 30/100: 2.45853590965271
Iter 40/100: 2.4564826488494873
Iter 50/100: 2.4551258087158203
Iter 10/100: 2.2126548290252686
Iter 20/100: 2.2110421657562256
Iter 10/100: 3.5815441608428955
Iter 10/100: 3.211308002471924
Iter 10/100: 2.9164204597473145
Iter 10/100: 2.580092191696167
Running experiments 85.71 %
Iter 10/100: 1.3681230545043945
Iter 20/100: 1.3586927652359009
Iter 30/100: 1.3537731170654297
Iter 40/100: 1.352138638496399
Iter 50/100: 1.3514869213104248
Iter 10/100: 4.068588733673096
Iter 10/100: 1.6629219055175781
Iter 20/100: 1.6554893255233765
Iter 30/100: 1.6533725261688232
Iter 10/100: 3.4025254249572754
Iter 10/100: 3.1672234535217285
Iter 10/100: 2.805464267730713
Iter 10/100: 1.6840810775756836
Iter 20/100: 1.5795583724975586
Iter 10/100: 1.4441430568695068
Iter 20/100: 1.4393901824951172
Iter 30/100: 1.4373559951782227
Iter 40/100: 1.4359850883483887
Iter 50/100: 1.4341460466384888
Iter 10/100: 1.3799595832824707
Iter 10/100: 2.972493886947632
Running experiments 90.48 %
Iter 10/100: 1.422130823135376
Iter 20/100: 1.410322666168213
Iter 30/100: 1.4083017110824585
Iter 40/100: 1.4067916870117188
Iter 10/100: 4.036256313323975
Iter 10/100: 2.767047166824341
Iter 10/100: 4.087111473083496
Iter 10/100: 3.7735586166381836
Iter 10/100: 3.578563928604126
Iter 10/100: 3.443025588989258
Iter 10/100: 3.1358792781829834
Iter 20/100: 2.556431770324707
Iter 30/100: 2.051668882369995
Iter 10/100: 1.9847800731658936
Iter 20/100: 1.9776380062103271
Iter 30/100: 1.9747257232666016
Iter 40/100: 1.972986102104187
Iter 50/100: 1.9720332622528076
Iter 60/100: 1.971361756324768
Iter 70/100: 1.9712002277374268
Iter 10/100: 1.8015400171279907
Running experiments 95.24 %
Iter 10/100: 1.5131380558013916
Iter 20/100: 1.4888651371002197
Iter 30/100: 1.4848350286483765
Iter 40/100: 1.484344720840454
Iter 10/100: 4.059488773345947
Iter 10/100: 3.0640549659729004
Iter 10/100: 2.785538911819458
Iter 20/100: 2.783586025238037
Iter 30/100: 2.782855987548828
Iter 40/100: 2.781691074371338
Iter 10/100: 2.198711395263672
Iter 10/100: 1.6213393211364746
Iter 10/100: 1.1084954738616943
Iter 20/100: 1.1074461936950684
Iter 30/100: 1.106899380683899
Iter 10/100: 0.650718092918396
Iter 10/100: 0.23726671934127808
Iter 20/100: 0.23644304275512695
Iter 10/100: -0.13481086492538452
Running experiments 100.00 %
Optimizing 21 Experiments takes 0.76 minutes.

6. Visualize the Pareto front¶

We can get the Pareto set directly from the WeightedMOOExperiment object by using WeightedMOOExperiment.get_optim.

To visualize the Pareto front, y1 values are plotted against y2 values. The scatter points resemble an ellispe shape, incidating the method is able to map out the entire front.

[12]:

# Extract the set of optimal solutions
Y_real_opts, X_real_opts = Exp_lhc.get_optim()
weight_names = [r'$\rm w_1$', r'$\rm w_2$']

# Parse the optimum into a table
data_opt = io.np_to_dataframe([weights_obj_1, weights_obj_2, X_real_opts, Y_real_opts], weight_names + var_names, n = n_exp)
display(data_opt.round(decimals=2))

# Make the pareto plots
plotting.pareto_front_exp(Exp_lhc, fill = True, diagonal = False)

	$\rm w_1$	$\rm w_2$	x	$\rm y_1$	$\rm y_2$
0	0.00	1.00	0.00	0.00	1.00
1	0.05	0.95	0.27	0.27	0.99
2	0.10	0.90	0.20	0.40	0.98
3	0.15	0.85	0.33	0.66	0.94
4	0.20	0.80	0.41	0.83	0.91
5	0.25	0.75	1.13	1.13	0.83
6	0.30	0.70	1.33	1.33	0.75
7	0.35	0.65	0.73	1.47	0.68
8	0.40	0.60	1.56	1.56	0.62
9	0.45	0.55	0.84	1.68	0.54
10	0.50	0.50	1.76	1.76	0.47
11	0.55	0.45	0.95	1.91	0.30
12	0.60	0.40	0.95	1.91	0.30
13	0.65	0.35	0.95	1.91	0.30
14	0.70	0.30	0.95	1.91	0.30
15	0.75	0.25	1.99	1.99	0.11
16	0.80	0.20	2.00	2.00	0.00
17	0.85	0.15	2.00	2.00	0.00
18	0.90	0.10	2.00	2.00	0.00
19	0.95	0.05	2.00	2.00	0.00
20	1.00	0.00	2.00	2.00	0.00

../_images/examples_06_ellipse_MOO_12_1.png

Thumbnail of this notebook