Example 7 - Multi-objective optimization for plug flow reactor

In this example, we will demonstrate how Bayesian Optimization can perform multi-objective optimization (MOO) and create a Pareto front. We will use the PFR model from example 5. Two output variables will be generated from the model: yield (y1) and selectivity (y2). Unfortunately, these two variables cannot be maximized simultaneously. An increase in yield would lead to a decrease in selectivity, and vice versa.

For this example, the scalarized objective can be written as,

\[y = w_1 y_1 + w_2 y_2\]

where the weights

\[w_1, w_2 \in [0, 1]\]
\[w_1 + w_2 = 1\]

The details of this example is summarized in the table below:

Key Item

Description

Goal

Maximization, two objectives

Objective function

PFR model

Input (X) dimension

3

Output (Y) dimension

2

Analytical form available?

Yes

Acqucision function

Expected improvement (EI)

Initial Sampling

Latin hypercube

Next, we will go through each step in Bayesian Optimization.

1. Import nextorch and other packages

2. Define the objective function and the design space

We import the PFR model, and wrap it in a Python function called PFR as the objective function objective_func.

The ranges of the input X are specified.

[1]:

import os
import sys
import time
from IPython.display import display

project_path = os.path.abspath(os.path.join(os.getcwd(), '..\..'))
sys.path.insert(0, project_path)

# Set the path for objective function
objective_path = os.path.join(project_path, 'examples', 'PFR')
sys.path.insert(0, objective_path)

import numpy as np
from nextorch import plotting, bo, doe, utils, io

[2]:

#%% Define the objective function
from fructose_pfr_model_function import Reactor

def PFR(X_real):
    """PFR model

    Parameters
    ----------
    X_real : numpy matrix
        reactor parameters:
        T, pH and tf in real scales

    Returns
    -------
    Y_real: numpy matrix
        reactor yield and selectivity
    """
    if len(X_real.shape) < 2:
        X_real = np.expand_dims(X_real, axis=1) #If 1D, make it 2D array

    Y_real = []
    for i, xi in enumerate(X_real):
        Conditions = {'T_degC (C)': xi[0], 'pH': xi[1], 'tf (min)' : 10**xi[2]}
        yi = Reactor(**Conditions)
        Y_real.append(yi)

    Y_real = np.array(Y_real)

    return Y_real # yield, selectivity


# Objective function
objective_func = PFR


#%% Define the design space
# Three input temperature C, pH, log10(residence time)
X_name_list = ['T', 'pH', r'$\rm log_{10}(tf_{min})$']
X_units = [r'$\rm ^{o}C $', '', '']

# Add the units
X_name_with_unit = []
for i, var in enumerate(X_name_list):
    if not X_units[i]  == '':
        var = var + ' ('+ X_units[i] + ')'
    X_name_with_unit.append(var)

# two outputs
Y_name_with_unit = ['Yield %', 'Selectivity %']

# combine X and Y names
var_names = X_name_with_unit + Y_name_with_unit

# Set the operating range for each parameter
X_ranges =  [[140, 200], # Temperature ranges from 140-200 degree C
             [0, 1], # pH values ranges from 0-1
             [-2, 2]] # log10(residence time) ranges from -2-2


# Get the information of the design space
n_dim = len(X_name_list) # the dimension of inputs
n_objective = len(Y_name_with_unit) # the dimension of outputs

3. Define the initial sampling plan

Here we use LHC design with 10 points for the initial sampling. The initial reponse in a real scale Y_init_real is computed from the objective function.

[3]:

#%% Initial Sampling
# Latin hypercube design with 10 initial points
n_init_lhc = 10
X_init_lhc = doe.latin_hypercube(n_dim = n_dim, n_points = n_init_lhc, seed= 1)
# Get the initial responses
Y_init_lhc = bo.eval_objective_func(X_init_lhc, X_ranges, objective_func)

# Compare the two sampling plans
plotting.sampling_3d(X_init_lhc,
                     X_names = X_name_with_unit,
                     X_ranges = X_ranges,
                     design_names = 'LHC')

C:\Users\yifan\Anaconda3\envs\torch\lib\site-packages\scipy\integrate\_ode.py:1182: UserWarning: dopri5: larger nsteps is needed
  self.messages.get(istate, unexpected_istate_msg)))
../_images/examples_07_PFR_MOO_6_1.png

4. Initialize an Experiment object

In this example, we use an WeightedMOOExperiment object, a class designed for multi-objective optimization. It can handle multiple weight combinations, perform the scalarized objective optimization automatically, and construct the entire Pareto front.

An WeightedMOOExperiment is a subclass of Experiment. It requires all key components as Experiment. Additionally, weights is required for WeightedMOOExperiment.set_optim_specs function. It defines a list of weights for objective 1. The weights of objective 2 is 1 minus that of objective 1. Under the hood, each weight combination correponds to a single Experiment object, each with a different scalarized objective.

Some progress status will be printed out while initializing all single Experiment objects.

[4]:

#%% Initialize an multi-objective Experiment object
# Set its name, the files will be saved under the folder with the same name
Exp_lhc = bo.WeightedMOOExperiment('PFR_yield_MOO')
# Import the initial data
Exp_lhc.input_data(X_init_lhc,
                   Y_init_lhc,
                   X_ranges = X_ranges,
                   X_names = X_name_with_unit,
                   Y_names = Y_name_with_unit,
                   unit_flag = True)

# Set the optimization specifications
# here we set the objective function, minimization by default
# 10 weights, 10 Experiments
n_exp = 21 # number of single Experiments

# Set a weight vector for objective 1
weights_obj_1 = np.linspace(0, 1, n_exp)
weights_obj_2 = 1 - weights_obj_1

# Set a timer
start_time = time.time()
Exp_lhc.set_optim_specs(objective_func = objective_func,
                        maximize = True,
                        weights = weights_obj_1)
end_time = time.time()
print('Initializing {} Experiments takes {:.2f} minutes.'.format(n_exp, (end_time-start_time)/60))

Initializing 21 experiments
Iter 10/100: 5.538654327392578
Iter 20/100: 5.134474277496338
Iter 30/100: 4.929288864135742
Initializing experiments 4.76 %
Iter 10/100: 5.538654327392578
Iter 20/100: 5.134474277496338
Iter 30/100: 4.929288864135742
Initializing experiments 9.52 %
Iter 10/100: 5.538654327392578
Iter 20/100: 5.134474277496338
Iter 30/100: 4.929288864135742
Initializing experiments 14.29 %
Iter 10/100: 5.538654327392578
Iter 20/100: 5.134474277496338
Iter 30/100: 4.929288864135742
Initializing experiments 19.05 %
Iter 10/100: 5.538654327392578
Iter 20/100: 5.134474277496338
Iter 30/100: 4.929288864135742
Initializing experiments 23.81 %
Iter 10/100: 5.538654327392578
Iter 20/100: 5.134474277496338
Iter 30/100: 4.929288864135742
Initializing experiments 28.57 %
Iter 10/100: 5.538654327392578
Iter 20/100: 5.134474277496338
Iter 30/100: 4.929288864135742
Initializing experiments 33.33 %
Iter 10/100: 5.538654327392578
Iter 20/100: 5.134474277496338
Iter 30/100: 4.929288864135742
Initializing experiments 38.10 %
Iter 10/100: 5.538654327392578
Iter 20/100: 5.134474277496338
Iter 30/100: 4.929288864135742
Initializing experiments 42.86 %
Iter 10/100: 5.538654327392578
Iter 20/100: 5.134474277496338
Iter 30/100: 4.929288864135742
Initializing experiments 47.62 %
Iter 10/100: 5.538654327392578
Iter 20/100: 5.134474277496338
Iter 30/100: 4.929288864135742
Initializing experiments 52.38 %
Iter 10/100: 5.538654327392578
Iter 20/100: 5.134474277496338
Iter 30/100: 4.929288864135742
Initializing experiments 57.14 %
Iter 10/100: 5.538654327392578
Iter 20/100: 5.134474277496338
Iter 30/100: 4.929288864135742
Initializing experiments 61.90 %
Iter 10/100: 5.538654327392578
Iter 20/100: 5.134474277496338
Iter 30/100: 4.929288864135742
Initializing experiments 66.67 %
Iter 10/100: 5.538654327392578
Iter 20/100: 5.134474277496338
Iter 30/100: 4.929288864135742
Initializing experiments 71.43 %
Iter 10/100: 5.538654327392578
Iter 20/100: 5.134474277496338
Iter 30/100: 4.929288864135742
Initializing experiments 76.19 %
Iter 10/100: 5.538654327392578
Iter 20/100: 5.134474277496338
Iter 30/100: 4.929288864135742
Initializing experiments 80.95 %
Iter 10/100: 5.538654327392578
Iter 20/100: 5.134474277496338
Iter 30/100: 4.929288864135742
Initializing experiments 85.71 %
Iter 10/100: 5.538654327392578
Iter 20/100: 5.134474277496338
Iter 30/100: 4.929288864135742
Initializing experiments 90.48 %
Iter 10/100: 5.538654327392578
Iter 20/100: 5.134474277496338
Iter 30/100: 4.929288864135742
Initializing experiments 95.24 %
Iter 10/100: 5.538654327392578
Iter 20/100: 5.134474277496338
Iter 30/100: 4.929288864135742
Initializing experiments 100.00 %
Initializing 21 Experiments takes 0.10 minutes.

5. Run trials

At each weight combinations, we perform an optimization task for the scalarized objective (a single Experiment). MOOExperiment.run_exp_auto run these tasks automatically by using the default choice of acqucision function, Expected improvement (EI). It takes in the number of trials required for each Experiment. The number of trials needs to be large enough which allows Bayesian Optimization algorithm to converge to the optimum. In this example, the evalution would be slow since the total number of calls for the objective function is n_trails * n_exp (=420).

Some progress status will be printed out during the training.

[5]:

# Set the number of iterations for each experiments
n_trials_lhc = 20
# Set a timer
start_time = time.time()
Exp_lhc.run_exp_auto(n_trials_lhc)

end_time = time.time()
print('Optimizing {} Experiments takes {:.2f} minutes.'.format(n_exp, (end_time-start_time)/60))

Running 21 experiments
Iter 10/100: 4.183126449584961
Iter 20/100: 3.943657875061035
Iter 30/100: 3.821441411972046
Iter 40/100: 3.7874295711517334
Iter 50/100: 3.7611007690429688
Iter 60/100: 3.740879535675049
Iter 70/100: 3.726250171661377
Iter 80/100: 3.7125136852264404
Iter 90/100: 3.6984825134277344
Iter 100/100: 3.6746041774749756
Iter 10/100: 3.4573545455932617
Iter 20/100: 3.429495096206665
Iter 30/100: 3.4093880653381348
Iter 40/100: 3.394003391265869
Iter 50/100: 3.381753444671631
Iter 60/100: 3.371739387512207
Iter 70/100: 3.363407611846924
Iter 80/100: 3.3563222885131836
Iter 90/100: 3.3501975536346436
Iter 100/100: 3.344837188720703
Iter 10/100: 2.8463380336761475
Iter 20/100: 2.837150812149048
Iter 30/100: 2.82894229888916
Iter 40/100: 2.822157144546509
Iter 50/100: 2.816131114959717
Iter 60/100: 2.8108115196228027
Iter 70/100: 2.806081771850586
Iter 80/100: 2.801821231842041
Iter 90/100: 2.7979626655578613
Iter 100/100: 2.7944416999816895
Iter 10/100: 2.574374198913574
Iter 20/100: 2.5690503120422363
Iter 30/100: 2.565063238143921
Iter 40/100: 2.561746120452881
Iter 50/100: 2.558790445327759
Iter 60/100: 2.556117534637451
Iter 70/100: 2.5536534786224365
Iter 80/100: 2.5513782501220703
Iter 90/100: 2.549259901046753
Iter 100/100: 2.5472750663757324
Iter 10/100: 1.5234571695327759
Iter 10/100: 1.1090248823165894
Iter 20/100: 1.0986133813858032
Iter 30/100: 1.0857274532318115
Iter 40/100: 1.0708922147750854
Iter 50/100: 1.0544151067733765
Iter 60/100: 1.035851240158081
Iter 70/100: 1.0148699283599854
Iter 80/100: 0.9941424131393433
Iter 90/100: 0.9771177768707275
Iter 100/100: 0.964246392250061
Running experiments 4.76 %
Iter 10/100: 4.192499160766602
Iter 20/100: 3.9529428482055664
Iter 30/100: 3.831742763519287
Iter 40/100: 3.796140193939209
Iter 50/100: 3.767244338989258
Iter 60/100: 3.7446751594543457
Iter 70/100: 3.7275662422180176
Iter 80/100: 3.7110443115234375
Iter 90/100: 3.6865947246551514
Iter 100/100: 3.6715922355651855
Iter 10/100: 3.4896395206451416
Iter 20/100: 3.4615116119384766
Iter 30/100: 3.4427924156188965
Iter 40/100: 3.428633451461792
Iter 50/100: 3.4172816276550293
Iter 60/100: 3.4081029891967773
Iter 70/100: 3.400411605834961
Iter 80/100: 3.3938984870910645
Iter 90/100: 3.3882663249969482
Iter 100/100: 3.383334159851074
Iter 10/100: 3.1104068756103516
Iter 20/100: 3.093045711517334
Iter 10/100: 2.8651111125946045
Iter 20/100: 2.862767457962036
Iter 10/100: 2.1090023517608643
Iter 20/100: 2.0865976810455322
Iter 30/100: 2.0738320350646973
Iter 40/100: 2.053180456161499
Iter 50/100: 2.011249542236328
Iter 60/100: 1.9108006954193115
Iter 10/100: 1.7170190811157227
Iter 20/100: 1.6912057399749756
Iter 30/100: 1.6790626049041748
Iter 40/100: 1.6675033569335938
Iter 50/100: 1.6573301553726196
Iter 60/100: 1.6517471075057983
Iter 70/100: 1.646225094795227
Iter 80/100: 1.6422417163848877
Iter 90/100: 1.6398708820343018
Iter 100/100: 1.6389484405517578
Iter 10/100: 1.3792803287506104
Iter 20/100: 1.3776271343231201
Running experiments 9.52 %
Iter 10/100: 4.202007293701172
Iter 20/100: 3.9629874229431152
Iter 30/100: 3.8437693119049072
Iter 40/100: 3.8060431480407715
Iter 50/100: 3.774838924407959
Iter 60/100: 3.7496395111083984
Iter 70/100: 3.729433059692383
Iter 80/100: 3.7033073902130127
Iter 90/100: 3.6848976612091064
Iter 100/100: 3.6696574687957764
Iter 10/100: 3.2637853622436523
Iter 20/100: 3.2380785942077637
Iter 30/100: 3.2200615406036377
Iter 40/100: 3.2049813270568848
Iter 50/100: 3.1929802894592285
Iter 60/100: 3.1829843521118164
Iter 70/100: 3.1744251251220703
Iter 80/100: 3.1670732498168945
Iter 90/100: 3.160616397857666
Iter 100/100: 3.1548988819122314
Iter 10/100: 2.9491140842437744
Iter 20/100: 2.9409291744232178
Iter 30/100: 2.9342904090881348
Iter 40/100: 2.92840313911438
Iter 50/100: 2.9232897758483887
Iter 60/100: 2.9187629222869873
Iter 70/100: 2.914721965789795
Iter 80/100: 2.9110889434814453
Iter 90/100: 2.9077930450439453
Iter 100/100: 2.904794216156006
Iter 10/100: 2.8163998126983643
Iter 20/100: 2.7970774173736572
Iter 30/100: 2.7905726432800293
Iter 40/100: 2.7876992225646973
Iter 50/100: 2.7855775356292725
Iter 60/100: 2.784355878829956
Iter 70/100: 2.782971143722534
Iter 80/100: 2.781294822692871
Iter 90/100: 2.779057502746582
Iter 100/100: 2.7753849029541016
Running experiments 14.29 %
Iter 10/100: 4.213242053985596
Iter 20/100: 3.9768388271331787
Iter 30/100: 3.8606607913970947
Iter 40/100: 3.8214311599731445
Iter 50/100: 3.788301944732666
Iter 60/100: 3.7604873180389404
Iter 70/100: 3.7360353469848633
Iter 80/100: 3.7082712650299072
Iter 90/100: 3.6892638206481934
Iter 100/100: 3.6734728813171387
Iter 10/100: 3.259338617324829
Iter 20/100: 3.2336459159851074
Iter 30/100: 3.2167367935180664
Iter 40/100: 3.2020585536956787
Iter 50/100: 3.1900086402893066
Iter 60/100: 3.1802124977111816
Iter 70/100: 3.1718547344207764
Iter 80/100: 3.164635181427002
Iter 90/100: 3.1583313941955566
Iter 100/100: 3.1527552604675293
Iter 10/100: 2.625965118408203
Iter 10/100: 2.598964214324951
Iter 20/100: 2.594447135925293
Iter 30/100: 2.5899317264556885
Iter 40/100: 2.5839786529541016
Iter 50/100: 2.5752041339874268
Iter 60/100: 2.5626444816589355
Iter 70/100: 2.55106258392334
Iter 80/100: 2.546975612640381
Iter 10/100: 2.6011908054351807
Iter 20/100: 2.5817620754241943
Iter 30/100: 2.577695846557617
Iter 40/100: 2.5759317874908447
Iter 50/100: 2.574352264404297
Iter 60/100: 2.5726046562194824
Iter 70/100: 2.570800304412842
Iter 80/100: 2.5691092014312744
Iter 90/100: 2.567507743835449
Iter 100/100: 2.5660133361816406
Running experiments 19.05 %
Iter 10/100: 4.2357587814331055
Iter 20/100: 4.007113456726074
Iter 30/100: 3.896786689758301
Iter 40/100: 3.857685089111328
Iter 50/100: 3.8231849670410156
Iter 60/100: 3.7919468879699707
Iter 70/100: 3.754218816757202
Iter 80/100: 3.731685161590576
Iter 90/100: 3.7117695808410645
Iter 100/100: 3.695369243621826
Iter 10/100: 3.531158208847046
Iter 20/100: 3.493222713470459
Iter 30/100: 3.4688239097595215
Iter 40/100: 3.4429397583007812
Iter 50/100: 3.4296822547912598
Iter 60/100: 3.4182872772216797
Iter 70/100: 3.409618377685547
Iter 80/100: 3.402182102203369
Iter 90/100: 3.3956637382507324
Iter 100/100: 3.389950752258301
Iter 10/100: 3.044795036315918
Iter 20/100: 3.0378670692443848
Iter 30/100: 3.0306663513183594
Iter 40/100: 3.0248615741729736
Iter 50/100: 3.0199697017669678
Iter 60/100: 3.0156431198120117
Iter 70/100: 3.0118346214294434
Iter 80/100: 3.0084547996520996
Iter 90/100: 3.005422353744507
Iter 100/100: 3.0026888847351074
Iter 10/100: 2.82901930809021
Iter 10/100: 2.4936790466308594
Iter 20/100: 2.482510566711426
Iter 30/100: 2.479318141937256
Iter 40/100: 2.477273464202881
Iter 50/100: 2.475417375564575
Iter 60/100: 2.4737589359283447
Iter 70/100: 2.4723100662231445
Iter 80/100: 2.4710168838500977
Iter 90/100: 2.469841718673706
Iter 100/100: 2.468778133392334
Iter 10/100: 2.2979273796081543
Running experiments 23.81 %
Iter 10/100: 4.3033366203308105
Iter 20/100: 4.040682315826416
Iter 30/100: 3.940248966217041
Iter 40/100: 3.883049488067627
Iter 50/100: 3.847900152206421
Iter 60/100: 3.818920373916626
Iter 70/100: 3.7958993911743164
Iter 80/100: 3.776582717895508
Iter 90/100: 3.7600111961364746
Iter 100/100: 3.746065616607666
Iter 10/100: 3.0213255882263184
Iter 20/100: 3.004502058029175
Iter 30/100: 2.991016387939453
Iter 40/100: 2.980783700942993
Iter 50/100: 2.972836971282959
Iter 60/100: 2.966517448425293
Iter 70/100: 2.961379289627075
Iter 80/100: 2.957129716873169
Iter 90/100: 2.9535670280456543
Iter 100/100: 2.950540542602539
Iter 10/100: 2.3653154373168945
Iter 20/100: 2.3589930534362793
Iter 30/100: 2.3493404388427734
Iter 10/100: 2.3882510662078857
Iter 20/100: 2.374083995819092
Iter 10/100: 2.266921043395996
Iter 20/100: 2.2592601776123047
Iter 30/100: 2.256514549255371
Iter 40/100: 2.254836320877075
Iter 50/100: 2.2527852058410645
Iter 60/100: 2.249049663543701
Iter 70/100: 2.2405784130096436
Iter 80/100: 2.2271900177001953
Iter 90/100: 2.222461700439453
Running experiments 28.57 %
Iter 10/100: 4.327762603759766
Iter 20/100: 4.066593170166016
Iter 30/100: 3.9814743995666504
Iter 40/100: 3.934433937072754
Iter 50/100: 3.9047579765319824
Iter 60/100: 3.879936695098877
Iter 70/100: 3.858074188232422
Iter 80/100: 3.8390607833862305
Iter 90/100: 3.822331428527832
Iter 100/100: 3.808030128479004
Iter 10/100: 2.715298891067505
Iter 20/100: 2.698517322540283
Iter 30/100: 2.686617851257324
Iter 40/100: 2.677753448486328
Iter 50/100: 2.6706783771514893
Iter 60/100: 2.665015935897827
Iter 70/100: 2.6604135036468506
Iter 80/100: 2.65661883354187
Iter 90/100: 2.6534438133239746
Iter 100/100: 2.650754451751709
Iter 10/100: 2.409078359603882
Iter 20/100: 2.3934006690979004
Iter 30/100: 2.381629228591919
Iter 40/100: 2.3569188117980957
Iter 50/100: 2.3251750469207764
Iter 10/100: 3.1016969680786133
Iter 20/100: 3.0532844066619873
Iter 30/100: 2.9963619709014893
Iter 40/100: 2.950112819671631
Iter 50/100: 2.914475917816162
Iter 60/100: 2.889267921447754
Iter 70/100: 2.871184825897217
Iter 80/100: 2.8566994667053223
Iter 90/100: 2.8448870182037354
Iter 100/100: 2.8338623046875
Running experiments 33.33 %
Iter 10/100: 4.324654579162598
Iter 20/100: 4.087574481964111
Iter 30/100: 3.9937455654144287
Iter 40/100: 3.9489364624023438
Iter 50/100: 3.915102005004883
Iter 60/100: 3.8902993202209473
Iter 70/100: 3.867396354675293
Iter 80/100: 3.847477912902832
Iter 90/100: 3.829805374145508
Iter 100/100: 3.814542770385742
Iter 10/100: 3.4012608528137207
Iter 20/100: 3.380945920944214
Iter 30/100: 3.3653554916381836
Iter 40/100: 3.352790355682373
Iter 50/100: 3.3427515029907227
Iter 60/100: 3.33457612991333
Iter 70/100: 3.3278236389160156
Iter 80/100: 3.322157859802246
Iter 90/100: 3.3173532485961914
Iter 100/100: 3.313225269317627
Iter 10/100: 2.981856346130371
Iter 20/100: 2.9768524169921875
Iter 30/100: 2.972660541534424
Iter 40/100: 2.9695072174072266
Iter 50/100: 2.9670674800872803
Iter 60/100: 2.96513032913208
Iter 70/100: 2.9635848999023438
Iter 80/100: 2.9623265266418457
Iter 90/100: 2.961319923400879
Iter 100/100: 2.960494041442871
Iter 10/100: 2.7133312225341797
Iter 10/100: 2.5203986167907715
Iter 20/100: 2.5087876319885254
Iter 30/100: 2.5054311752319336
Iter 10/100: 3.6808419227600098
Iter 20/100: 3.6589903831481934
Iter 30/100: 3.635673999786377
Iter 40/100: 3.593378782272339
Iter 50/100: 3.531568765640259
Iter 60/100: 3.468132734298706
Iter 70/100: 3.417191743850708
Iter 80/100: 3.3785934448242188
Iter 90/100: 3.3447208404541016
Iter 10/100: 3.133139133453369
Running experiments 38.10 %
Iter 10/100: 4.308869361877441
Iter 20/100: 4.080162048339844
Iter 30/100: 3.9897780418395996
Iter 40/100: 3.942074775695801
Iter 50/100: 3.905938148498535
Iter 60/100: 3.8791182041168213
Iter 70/100: 3.85599684715271
Iter 80/100: 3.835876941680908
Iter 90/100: 3.8180642127990723
Iter 100/100: 3.8026459217071533
Iter 10/100: 3.385486125946045
Iter 20/100: 3.3649988174438477
Iter 30/100: 3.3488574028015137
Iter 40/100: 3.336035966873169
Iter 50/100: 3.325617551803589
Iter 60/100: 3.3170762062072754
Iter 70/100: 3.309950351715088
Iter 80/100: 3.3039138317108154
Iter 90/100: 3.2987399101257324
Iter 100/100: 3.294255495071411
Iter 10/100: 2.994837999343872
Iter 20/100: 2.989408016204834
Iter 30/100: 2.9849205017089844
Iter 40/100: 2.9811692237854004
Iter 50/100: 2.978114604949951
Iter 60/100: 2.975583076477051
Iter 70/100: 2.973444700241089
Iter 80/100: 2.9716556072235107
Iter 90/100: 2.970095634460449
Iter 100/100: 2.9687724113464355
Running experiments 42.86 %
Iter 10/100: 4.2896199226379395
Iter 20/100: 4.060002326965332
Iter 30/100: 3.9805502891540527
Iter 40/100: 3.935711622238159
Iter 50/100: 3.89778470993042
Iter 60/100: 3.867400646209717
Iter 70/100: 3.84146785736084
Iter 80/100: 3.818765878677368
Iter 90/100: 3.800374984741211
Iter 100/100: 3.785367727279663
Iter 10/100: 3.355480670928955
Iter 20/100: 3.335116386413574
Iter 30/100: 3.3194992542266846
Iter 40/100: 3.3068442344665527
Iter 50/100: 3.2966861724853516
Iter 60/100: 3.2883410453796387
Iter 70/100: 3.281372547149658
Iter 80/100: 3.2754673957824707
Iter 90/100: 3.270388126373291
Iter 100/100: 3.2659764289855957
Iter 10/100: 3.354487419128418
Iter 20/100: 3.257333755493164
Iter 30/100: 3.181497573852539
Iter 40/100: 3.128741979598999
Iter 50/100: 3.092146396636963
Iter 60/100: 3.0633621215820312
Iter 70/100: 3.016573190689087
Running experiments 47.62 %
Iter 10/100: 4.269384384155273
Iter 20/100: 4.036097049713135
Iter 30/100: 3.956967830657959
Iter 40/100: 3.917466878890991
Iter 50/100: 3.884977340698242
Iter 60/100: 3.856931686401367
Iter 70/100: 3.83450984954834
Iter 80/100: 3.815646171569824
Iter 90/100: 3.7983057498931885
Iter 100/100: 3.7824535369873047
Iter 10/100: 3.5855088233947754
Iter 20/100: 3.538978099822998
Iter 30/100: 3.5194807052612305
Iter 40/100: 3.50742769241333
Iter 50/100: 3.4966883659362793
Iter 60/100: 3.4883522987365723
Iter 70/100: 3.481724262237549
Iter 80/100: 3.4760384559631348
Iter 90/100: 3.4711759090423584
Iter 100/100: 3.466938018798828
Iter 10/100: 3.014127731323242
Iter 10/100: 4.009910583496094
Iter 20/100: 3.8147077560424805
Iter 30/100: 3.6857662200927734
Iter 40/100: 3.5440354347229004
Iter 50/100: 3.430243492126465
Iter 60/100: 3.343637466430664
Iter 10/100: 1.291305422782898
Iter 20/100: 1.1535416841506958
Iter 30/100: 1.0392926931381226
Iter 40/100: 0.9585567712783813
Iter 50/100: 0.9319885969161987
Iter 60/100: 0.9203016757965088
Iter 70/100: 0.9146329164505005
Iter 80/100: 0.9073351621627808
Iter 90/100: 0.899975061416626
Iter 100/100: 0.8925250172615051
Iter 10/100: 0.7404468059539795
Iter 20/100: 0.7309198379516602
Iter 30/100: 0.722187876701355
Iter 40/100: 0.7159903049468994
Iter 50/100: 0.7140557765960693
Running experiments 52.38 %
Iter 10/100: 4.24954891204834
Iter 20/100: 4.013315200805664
Iter 30/100: 3.929490089416504
Iter 40/100: 3.8879947662353516
Iter 50/100: 3.85597825050354
Iter 60/100: 3.8311948776245117
Iter 70/100: 3.81209397315979
Iter 80/100: 3.7961320877075195
Iter 90/100: 3.78178334236145
Iter 100/100: 3.7686400413513184
Iter 10/100: 3.728586435317993
Iter 20/100: 3.6857690811157227
Iter 30/100: 3.6371445655822754
Iter 40/100: 3.6015782356262207
Iter 50/100: 3.5836873054504395
Iter 60/100: 3.5725035667419434
Iter 70/100: 3.563994884490967
Iter 80/100: 3.5575110912323
Iter 90/100: 3.552342176437378
Iter 100/100: 3.547945022583008
Iter 10/100: 3.0973877906799316
Iter 20/100: 3.091139316558838
Iter 30/100: 3.085817337036133
Iter 40/100: 3.0816128253936768
Iter 50/100: 3.078092098236084
Iter 60/100: 3.0750582218170166
Iter 70/100: 3.0724422931671143
Iter 80/100: 3.070138931274414
Iter 90/100: 3.068106174468994
Iter 100/100: 3.0662856101989746
Iter 10/100: 3.4786646366119385
Iter 10/100: 2.8341007232666016
Iter 20/100: 2.64926815032959
Iter 30/100: 2.5075297355651855
Iter 40/100: 2.386505365371704
Iter 50/100: 2.250684976577759
Iter 60/100: 2.0919649600982666
Iter 70/100: 1.9604047536849976
Iter 80/100: 1.8736344575881958
Iter 90/100: 1.8226978778839111
Iter 100/100: 1.7817519903182983
Running experiments 57.14 %
Iter 10/100: 4.231907844543457
Iter 20/100: 3.9923453330993652
Iter 30/100: 3.8998003005981445
Iter 40/100: 3.854917526245117
Iter 50/100: 3.825338125228882
Iter 60/100: 3.8025853633880615
Iter 70/100: 3.783451557159424
Iter 80/100: 3.7666378021240234
Iter 90/100: 3.7513856887817383
Iter 100/100: 3.7375965118408203
Iter 10/100: 3.3618111610412598
Iter 20/100: 3.3384246826171875
Iter 30/100: 3.3230743408203125
Iter 40/100: 3.3103740215301514
Iter 50/100: 3.301095962524414
Iter 60/100: 3.2935872077941895
Iter 70/100: 3.287412643432617
Iter 80/100: 3.2822389602661133
Iter 90/100: 3.277810573577881
Iter 100/100: 3.2739574909210205
Iter 10/100: 2.8273673057556152
Iter 20/100: 2.821549892425537
Iter 30/100: 2.816445827484131
Iter 40/100: 2.812380313873291
Iter 50/100: 2.8089542388916016
Iter 60/100: 2.806025981903076
Iter 70/100: 2.8034679889678955
Iter 80/100: 2.8012218475341797
Iter 90/100: 2.7992420196533203
Iter 100/100: 2.797447919845581
Iter 10/100: 2.4721884727478027
Iter 10/100: 3.5104899406433105
Iter 10/100: 3.5036730766296387
Iter 20/100: 3.3827452659606934
Iter 30/100: 3.242931365966797
Running experiments 61.90 %
Iter 10/100: 4.21761417388916
Iter 20/100: 3.968418598175049
Iter 30/100: 3.866288185119629
Iter 40/100: 3.8248445987701416
Iter 50/100: 3.7981743812561035
Iter 60/100: 3.775869369506836
Iter 70/100: 3.75789475440979
Iter 80/100: 3.742050886154175
Iter 90/100: 3.725677728652954
Iter 100/100: 3.6955957412719727
Iter 10/100: 3.210015296936035
Iter 20/100: 3.1909613609313965
Iter 30/100: 3.176583766937256
Iter 40/100: 3.165592670440674
Iter 50/100: 3.157040596008301
Iter 60/100: 3.1502163410186768
Iter 70/100: 3.1446499824523926
Iter 80/100: 3.140005588531494
Iter 90/100: 3.1360692977905273
Iter 100/100: 3.1326818466186523
Iter 10/100: 4.305145263671875
Iter 20/100: 4.11566686630249
Iter 30/100: 3.9762392044067383
Iter 40/100: 3.858733892440796
Iter 50/100: 3.777026414871216
Iter 60/100: 3.724503755569458
Iter 70/100: 3.678652763366699
Iter 10/100: 1.7842563390731812
Iter 10/100: 1.5445109605789185
Iter 20/100: 1.3738981485366821
Iter 30/100: 1.1933400630950928
Iter 40/100: 1.0904693603515625
Iter 50/100: 0.9962801337242126
Iter 60/100: 0.9030078053474426
Iter 70/100: 0.8893696069717407
Iter 80/100: 0.8822472095489502
Iter 90/100: 0.8727449774742126
Iter 100/100: 0.8671132326126099
Iter 10/100: 0.7131938934326172
Running experiments 66.67 %
Iter 10/100: 4.2019853591918945
Iter 20/100: 3.9419808387756348
Iter 30/100: 3.837428092956543
Iter 40/100: 3.8007242679595947
Iter 50/100: 3.7751007080078125
Iter 60/100: 3.753335475921631
Iter 70/100: 3.7360897064208984
Iter 80/100: 3.7180685997009277
Iter 90/100: 3.6790480613708496
Iter 10/100: 3.19596266746521
Iter 20/100: 3.175471305847168
Iter 30/100: 3.1599788665771484
Iter 40/100: 3.1481316089630127
Iter 50/100: 3.1392228603363037
Iter 60/100: 3.132174015045166
Iter 70/100: 3.126483917236328
Iter 80/100: 3.121776580810547
Iter 90/100: 3.1178131103515625
Iter 100/100: 3.114429473876953
Iter 10/100: 4.095574378967285
Iter 20/100: 4.0110321044921875
Iter 30/100: 3.9170353412628174
Iter 40/100: 3.808236598968506
Iter 50/100: 3.7195887565612793
Iter 60/100: 3.6612706184387207
Iter 70/100: 3.6163887977600098
Iter 80/100: 3.571155548095703
Iter 10/100: 3.325558662414551
Iter 20/100: 3.2416443824768066
Iter 30/100: 3.1724395751953125
Iter 40/100: 3.1069352626800537
Iter 50/100: 3.043196201324463
Iter 60/100: 3.0109784603118896
Iter 70/100: 2.9985575675964355
Iter 80/100: 2.987231731414795
Iter 90/100: 2.98197865486145
Iter 100/100: 2.980949640274048
Running experiments 71.43 %
Iter 10/100: 4.1849164962768555
Iter 20/100: 3.919314384460449
Iter 30/100: 3.8144149780273438
Iter 40/100: 3.781383514404297
Iter 50/100: 3.756462574005127
Iter 60/100: 3.734680652618408
Iter 70/100: 3.7174665927886963
Iter 80/100: 3.689645767211914
Iter 10/100: 3.1689453125
Iter 20/100: 3.146244525909424
Iter 30/100: 3.1300625801086426
Iter 40/100: 3.1181833744049072
Iter 50/100: 3.109285831451416
Iter 60/100: 3.1024084091186523
Iter 70/100: 3.0969088077545166
Iter 80/100: 3.0924267768859863
Iter 90/100: 3.0886731147766113
Iter 100/100: 3.085484504699707
Iter 10/100: 2.662224531173706
Iter 10/100: 4.318136215209961
Iter 20/100: 4.1365251541137695
Iter 30/100: 4.008059978485107
Iter 40/100: 3.8980300426483154
Iter 50/100: 3.825746536254883
Iter 60/100: 3.776838779449463
Iter 10/100: 1.3312270641326904
Iter 20/100: 0.7367361187934875
Iter 30/100: 0.5863953232765198
Iter 40/100: 0.42217427492141724
Iter 50/100: 0.3366854190826416
Iter 60/100: 0.2727733254432678
Iter 70/100: 0.1908671259880066
Iter 80/100: 0.10183042287826538
Iter 90/100: 0.07299381494522095
Iter 100/100: 0.0689307451248169
Iter 10/100: 0.0467032790184021
Iter 20/100: 0.03195005655288696
Iter 30/100: 0.01862102746963501
Iter 40/100: 0.0072234272956848145
Iter 50/100: 0.0002059340476989746
Iter 60/100: -0.0019521713256835938
Running experiments 76.19 %
Iter 10/100: 4.167107105255127
Iter 20/100: 3.8985886573791504
Iter 30/100: 3.7950246334075928
Iter 40/100: 3.7629122734069824
Iter 50/100: 3.7377209663391113
Iter 60/100: 3.7162299156188965
Iter 70/100: 3.693418502807617
Iter 10/100: 3.1484408378601074
Iter 20/100: 3.122892379760742
Iter 30/100: 3.1051223278045654
Iter 40/100: 3.092883586883545
Iter 50/100: 3.083995819091797
Iter 60/100: 3.077157497406006
Iter 70/100: 3.0717930793762207
Iter 80/100: 3.0674514770507812
Iter 90/100: 3.0638630390167236
Iter 100/100: 3.0608367919921875
Iter 10/100: 3.440340280532837
Iter 10/100: 1.893911600112915
Iter 20/100: 1.3174219131469727
Iter 30/100: 1.1033706665039062
Iter 40/100: 0.9360932111740112
Iter 50/100: 0.8396105170249939
Iter 60/100: 0.7836467623710632
Iter 70/100: 0.7624371647834778
Iter 80/100: 0.7543555498123169
Iter 90/100: 0.7471371293067932
Iter 100/100: 0.739630937576294
Iter 10/100: 0.729762077331543
Iter 20/100: 0.7080086469650269
Iter 30/100: 0.6935639977455139
Iter 40/100: 0.6839908957481384
Iter 50/100: 0.678776741027832
Running experiments 80.95 %
Iter 10/100: 4.153910160064697
Iter 20/100: 3.8822178840637207
Iter 30/100: 3.7793972492218018
Iter 40/100: 3.7497386932373047
Iter 50/100: 3.724039077758789
Iter 60/100: 3.7032485008239746
Iter 70/100: 3.6861729621887207
Iter 80/100: 3.64206862449646
Iter 10/100: 3.1149826049804688
Iter 20/100: 3.0923609733581543
Iter 30/100: 3.0748605728149414
Iter 40/100: 3.0640947818756104
Iter 50/100: 3.0561563968658447
Iter 60/100: 3.0501749515533447
Iter 70/100: 3.045583486557007
Iter 80/100: 3.0418944358825684
Iter 90/100: 3.038858413696289
Iter 100/100: 3.0363235473632812
Iter 10/100: 3.4575304985046387
Iter 10/100: 1.8057708740234375
Iter 20/100: 1.6318808794021606
Iter 30/100: 1.155457615852356
Iter 40/100: 1.0085206031799316
Iter 10/100: 0.8157492876052856
Iter 20/100: 0.7766554951667786
Iter 30/100: 0.7625764012336731
Iter 40/100: 0.7498197555541992
Iter 50/100: 0.7476480603218079
Running experiments 85.71 %
Iter 10/100: 4.137479782104492
Iter 20/100: 3.863901138305664
Iter 30/100: 3.766801357269287
Iter 40/100: 3.7383928298950195
Iter 50/100: 3.713679313659668
Iter 60/100: 3.695427417755127
Iter 70/100: 3.6824865341186523
Iter 80/100: 3.670405387878418
Iter 90/100: 3.658336639404297
Iter 100/100: 3.6301817893981934
Iter 10/100: 3.115670680999756
Iter 20/100: 3.0953657627105713
Iter 30/100: 3.082263708114624
Iter 40/100: 3.0732223987579346
Iter 50/100: 3.0665721893310547
Iter 60/100: 3.061490058898926
Iter 70/100: 3.057508945465088
Iter 80/100: 3.0542638301849365
Iter 90/100: 3.051555633544922
Iter 100/100: 3.049233913421631
Iter 10/100: 1.599379539489746
Iter 20/100: 1.2741429805755615
Iter 30/100: 1.0750927925109863
Iter 40/100: 0.9733838438987732
Iter 50/100: 0.9162384867668152
Iter 60/100: 0.8806850910186768
Iter 70/100: 0.8564313650131226
Iter 80/100: 0.8376065492630005
Iter 90/100: 0.8212110996246338
Iter 100/100: 0.80511873960495
Iter 10/100: 0.8490427732467651
Iter 20/100: 0.82326340675354
Iter 30/100: 0.8001021146774292
Iter 40/100: 0.792844831943512
Running experiments 90.48 %
Iter 10/100: 4.124499797821045
Iter 20/100: 3.8506946563720703
Iter 30/100: 3.754164695739746
Iter 40/100: 3.725597858428955
Iter 50/100: 3.701223611831665
Iter 60/100: 3.684680461883545
Iter 70/100: 3.6740479469299316
Iter 80/100: 3.6650710105895996
Iter 90/100: 3.6569013595581055
Iter 100/100: 3.6486730575561523
Iter 10/100: 3.5730185508728027
Iter 20/100: 3.468404769897461
Iter 30/100: 3.4317686557769775
Iter 40/100: 3.4108030796051025
Iter 50/100: 3.397139072418213
Iter 60/100: 3.387254476547241
Iter 70/100: 3.3797590732574463
Iter 80/100: 3.3738489151000977
Iter 90/100: 3.3690011501312256
Iter 100/100: 3.364896297454834
Iter 10/100: 3.0050697326660156
Iter 20/100: 2.9981346130371094
Iter 30/100: 2.9936652183532715
Iter 40/100: 2.9896156787872314
Iter 50/100: 2.98671293258667
Iter 60/100: 2.9842090606689453
Iter 70/100: 2.982088565826416
Iter 80/100: 2.980268955230713
Iter 90/100: 2.978654384613037
Iter 100/100: 2.97723650932312
Iter 10/100: 4.817776679992676
Iter 20/100: 4.668087005615234
Iter 30/100: 4.617497444152832
Iter 40/100: 4.59060001373291
Iter 50/100: 4.572688102722168
Iter 60/100: 4.559967041015625
Iter 70/100: 4.5503082275390625
Iter 80/100: 4.542576789855957
Iter 90/100: 4.536053657531738
Iter 100/100: 4.5303544998168945
Iter 10/100: 2.927225112915039
Iter 20/100: 2.754561424255371
Iter 30/100: 2.71384596824646
Iter 10/100: 2.3468918800354004
Iter 20/100: 2.17037034034729
Iter 30/100: 1.964663028717041
Iter 40/100: 1.6867530345916748
Iter 50/100: 1.4216029644012451
Iter 60/100: 1.292809247970581
Iter 70/100: 1.226317048072815
Iter 80/100: 1.1827774047851562
Iter 90/100: 1.1525368690490723
Iter 100/100: 1.1266021728515625
Iter 10/100: 0.9597659111022949
Iter 20/100: 0.9402386546134949
Iter 30/100: 0.9222845435142517
Iter 40/100: 0.9046022891998291
Iter 50/100: 0.8899242877960205
Iter 60/100: 0.8771964907646179
Iter 70/100: 0.8655204772949219
Iter 80/100: 0.8565382361412048
Iter 90/100: 0.8531126976013184
Iter 10/100: 0.7129508256912231
Iter 10/100: 0.6523348093032837
Iter 10/100: 0.422880083322525
Running experiments 95.24 %
Iter 10/100: 4.115649700164795
Iter 20/100: 3.839968204498291
Iter 30/100: 3.7422776222229004
Iter 40/100: 3.713578224182129
Iter 50/100: 3.6893129348754883
Iter 60/100: 3.673434257507324
Iter 70/100: 3.663466453552246
Iter 80/100: 3.655313014984131
Iter 90/100: 3.6482152938842773
Iter 100/100: 3.6412577629089355
Iter 10/100: 3.6149697303771973
Iter 20/100: 3.58750581741333
Iter 30/100: 3.562439203262329
Iter 40/100: 3.52493953704834
Iter 50/100: 3.4991097450256348
Iter 60/100: 3.4780125617980957
Iter 70/100: 3.4638020992279053
Iter 80/100: 3.453963279724121
Iter 90/100: 3.446803331375122
Iter 100/100: 3.441380023956299
Iter 10/100: 4.4300618171691895
Iter 20/100: 4.380061149597168
Iter 30/100: 4.345699310302734
Iter 40/100: 4.32703161239624
Iter 50/100: 4.309081554412842
Iter 60/100: 4.290075302124023
Iter 70/100: 4.2679572105407715
Iter 80/100: 4.24143123626709
Iter 90/100: 4.209858417510986
Iter 100/100: 4.174708843231201
Iter 10/100: 2.2221338748931885
Iter 20/100: 2.006805181503296
Iter 30/100: 1.9161155223846436
Iter 40/100: 1.8616777658462524
Iter 50/100: 1.8212940692901611
Iter 60/100: 1.7982035875320435
Iter 70/100: 1.7845244407653809
Iter 80/100: 1.7772663831710815
Iter 90/100: 1.7728285789489746
Iter 100/100: 1.769514799118042
Running experiments 100.00 %
Optimizing 21 Experiments takes 12.19 minutes.

6. Visualize the Pareto front

We can get the Pareto set directly from the MOOExperiment object by using MOOExperiment.get_optim.

To visualize the Pareto front, y1 values are plotted against y2 values. The region below $ y=x $ is infeasible for the PFR model and we have no Pareto points fall below the line, incidating the method is validate.

[6]:

# Extract the set of optimal solutions
Y_real_opts, X_real_opts = Exp_lhc.get_optim()
weight_names = [r'$\rm w_1$', r'$\rm w_2$']

# Parse the optimum into a table
data_opt = io.np_to_dataframe([weights_obj_1, weights_obj_2, X_real_opts, Y_real_opts], weight_names + var_names, n = n_exp)
display(data_opt.round(decimals=2))

# Make the pareto plots
plotting.pareto_front_exp(Exp_lhc, fill = False)
$\rm w_1$ $\rm w_2$ T ($\rm ^{o}C $) pH $\rm log_{10}(tf_{min})$ Yield % Selectivity %
0 0.00 1.00 140.00 1.00 -2.00 0.10 65.35
1 0.05 0.95 140.00 1.00 -2.00 0.10 65.35
2 0.10 0.90 140.00 1.00 -2.00 0.10 65.35
3 0.15 0.85 182.38 0.93 -1.09 24.92 61.44
4 0.20 0.80 189.04 0.21 -1.67 40.67 58.90
5 0.25 0.75 200.00 0.00 -2.00 48.18 56.54
6 0.30 0.70 191.41 1.00 -0.91 42.37 58.51
7 0.35 0.65 200.00 1.00 -1.09 45.70 57.72
8 0.40 0.60 200.00 1.00 -1.02 47.73 56.82
9 0.45 0.55 200.00 1.00 -0.98 48.66 56.20
10 0.50 0.50 200.00 1.00 -0.94 49.29 55.61
11 0.55 0.45 184.80 1.00 -0.52 47.37 55.52
12 0.60 0.40 182.79 1.00 -0.40 48.03 54.00
13 0.65 0.35 189.75 0.72 -0.93 48.24 55.37
14 0.70 0.30 195.96 0.86 -0.87 49.73 53.37
15 0.75 0.25 198.14 1.00 -0.81 49.90 53.95
16 0.80 0.20 161.34 1.00 0.33 44.83 50.45
17 0.85 0.15 200.00 1.00 -0.83 50.10 53.34
18 0.90 0.10 200.00 1.00 -0.86 50.07 53.99
19 0.95 0.05 200.00 1.00 -0.79 49.85 52.14
20 1.00 0.00 200.00 1.00 -0.82 50.05 52.94
../_images/examples_07_PFR_MOO_12_1.png

References:

  1. Desir, P.; Saha, B.; Vlachos, D. G. Energy Environ. Sci. 2019.

  2. Swift, T. D.; Bagia, C.; Choudhary, V.; Peklaris, G.; Nikolakis, V.; Vlachos, D. G. ACS Catal. 2014, 4 (1), 259–267

  3. The PFR model can be found on GitHub: https://github.com/VlachosGroup/Fructose-HMF-Model

Thumbnail of this notebook