Surrogates

Package related to surrogate models.

Classes summary

Surrogates.Surrogate.Surrogate(...)	Abstract class for surrogate models.
Surrogates.BNN_MCD.BNN_MCD(f_sim_archive, ...)	Class for Bayesian Neural Network approximated via Monte Carlo Dropout.
Surrogates.BLR_ANN.BLR_ANN(f_sim_archive, ...)	Class for Bayesian Linear Regressor with Artificial Neural Network basis functions.
Surrogates.iKRG.iKRG(f_sim_archive, pb, ...)	Class for interpolating Kriging surrogate model.
Surrogates.rKRG.rKRG(f_sim_archive, pb, ...)	Class for regression Kriging surrogate model.
Surrogates.GP.GP(f_sim_archive, pb, ...)	Class for Gaussian Process.
Surrogates.GP_MO.GP_MO(f_sim_archive, pb, ...)	Class for Gaussian Process multitask model (multi dimensional targets only).

Surrogate (abstract)

class Surrogates.Surrogate.Surrogate(f_sim_archive, pb, n_train_samples, f_train_log, f_trained_model)

Bases: ABC

Abstract class for surrogate models.

Parameters:

• f_sim_archive (str) – filename where are stored the simulated candidates

• pb (Problem) – problem the surrogate is associated with

• n_train_samples (positive int or inf, not zero) – number of training samples to extract from the end of f_sim_archive, if float(‘inf’) all the samples from f_sim_archive are considered

• f_train_log (str) – filename where will be recorded training log

• f_trained_model (str) – filename where will be recorded the trained surrogate model

abstract perform_training()

Trains the surrogate model.

abstract perform_prediction(candidates)

Returns the predicted objective value of the candidates and the uncertainty around it. Both quantities are normalized.

Parameters:

candidates (np.ndarray) – candidates

Returns:

predicted objective values and uncertainties

Return type:

(np.ndarray, np.ndarray)

perform_distance(candidates)

Returns the distance from each normalized candidate to the set of normalized known candidates.

Parameters:

candidates (np.ndarray) – candidates

Returns:

distances from the normalized candidates to the set of normalized known candidates

Return type:

np.ndarray

load_sim_archive()

Returns the past simulated candidates.

Returns:

past simulated decision vectors along with their objective value

Return type:

tuple(np.ndarray,np.ndarray)

abstract denormalize_predictions(preds)

Denormalize predicted objective value.

abstract normalize_obj_vals(obj_vals)

Normalized objective values.

abstract load_trained_model()

Loads a model trained in the past.

Approximated Bayesian Neural Network

Monte Carlo Dropout

class Surrogates.BNN_MCD.BNN_MCD(f_sim_archive, pb, n_train_samples, f_train_log, f_trained_model, n_pred_subnets)

Bases: Surrogate

Class for Bayesian Neural Network approximated via Monte Carlo Dropout.

Parameters:

• f_sim_archive (str) – filename where are stored the simulated candidates

• pb (Problem) – problem the surrogate is associated with

• n_train_samples (positive int or inf, not zero) – number of training samples to extract from the end of f_sim_archive, if float(‘inf’) all the samples from f_sim_archive are considered

• f_train_log (str) – filename where will be recorded training log

• f_trained_model (str) – filename where will be recorded the trained surrogate model

• y_bounds (np.ndarray) – lower and upper bounds of the objective values found in the training set

• outputs_scaler (sklearn.preprocessing) – objective values normalizer

• n_pred_subnets (positive int, not zero) – number of sub-networks

• model (tf.keras.Model) – Keras model for the network

perform_training()

Trains the surrogate model.

perform_prediction(candidates)

Returns the predicted objective value of the candidates and the uncertainty around it. Both quantities are normalized.

Parameters:

candidates (np.ndarray) – candidates

Returns:

predicted objective values and uncertainties

Return type:

(np.ndarray, np.ndarray)

denormalize_predictions(preds)

Denormalize predicted objective value.

normalize_obj_vals(obj_vals)

Normalized objective values.

load_trained_model()

Loads a model trained in the past.

load_sim_archive()

Returns the past simulated candidates.

Returns:

past simulated decision vectors along with their objective value

Return type:

tuple(np.ndarray,np.ndarray)

perform_distance(candidates)

Returns the distance from each normalized candidate to the set of normalized known candidates.

Parameters:

candidates (np.ndarray) – candidates

Returns:

distances from the normalized candidates to the set of normalized known candidates

Return type:

np.ndarray

Bayesian Linear Regressor

Artificial Neural Network basis functions

class Surrogates.BLR_ANN.BLR_ANN(f_sim_archive, pb, n_train_samples, f_train_log, f_trained_model)

Bases: Surrogate

Class for Bayesian Linear Regressor with Artificial Neural Network basis functions.

Parameters:

• f_sim_archive (str) – filename where are stored the simulated candidates

• pb (Problem) – problem the surrogate is associated with

• n_train_samples (positive int or inf, not zero) – number of training samples to extract from the end of f_sim_archive, if float(‘inf’) all the samples from f_sim_archive are considered

• f_train_log (str) – filename where will be recorded training log

• f_trained_model (str) – filename where will be recorded the trained surrogate model

• model (pybnn.DNGO) – DNGO model for the network

perform_training()

Trains the surrogate model.

perform_prediction(candidates)

Returns the predicted objective value of the candidates and the uncertainty around it. Both quantities are normalized.

Parameters:

candidates (np.ndarray) – candidates

Returns:

predicted objective values and uncertainties

Return type:

(np.ndarray, np.ndarray)

perform_distance(candidates)

Returns the distance from each normalized candidate to the set of normalized known individuals.

Parameters:

candidates (np.ndarray) – candidates

Returns:

distances from the normalized candidates to the set of normalized known individuals

Return type:

np.ndarray

denormalize_predictions(preds)

Denormalize predicted objective value.

normalize_obj_vals(obj_val)

Normalized objective values.

load_trained_model()

Loads a model trained in the past.

load_sim_archive()

Returns the past simulated candidates.

Returns:

past simulated decision vectors along with their objective value

Return type:

tuple(np.ndarray,np.ndarray)

Gaussian Processes

Interpolation Kriging

class Surrogates.iKRG.iKRG(f_sim_archive, pb, n_train_samples, f_train_log, f_trained_model)

Bases: Surrogate

Class for interpolating Kriging surrogate model.

Parameters:

• f_sim_archive (str) – filename where are stored the simulated candidates

• pb (Problem) – problem the surrogate is associated with

• n_train_samples (positive int or inf, not zero) – number of training samples to extract from the end of f_sim_archive, if float(‘inf’) all the samples from f_sim_archive are considered

• f_train_log (str) – filename where will be recorded training log

• f_trained_model (str) – filename where will be recorded the trained surrogate model

• y_bounds (np.ndarray) – lower and upper bounds of the objective values found in the training set

• outputs_scaler (sklearn.preprocessing) – objective values normalizer

• model (pyKriging.krige.kriging) – kriging model

If you encounter some “np.float deprecated” error while using iKRG.py, please refer to the following page <https://github.com/capaulson/pyKriging/pull/53/files>

perform_training()

Trains the surrogate model.

add_point(candidate, obj_val)

Partial update (used in q-EGO)

Parameters:

• candidate (np.ndarray) – new candidate to add to the training set

• obj_val (np.ndarray) – objective value associated to the candidate

perform_prediction(candidates)

Returns the predicted objective value of the candidates and the uncertainty around it. Both quantities are normalized.

Parameters:

candidates (np.ndarray) – candidates

Returns:

predicted objective values and uncertainties

Return type:

(np.ndarray, np.ndarray)

denormalize_predictions(preds)

Denormalize predicted objective value.

normalize_obj_vals(obj_vals)

Normalized objective values.

load_trained_model()

Loads a model trained in the past.

load_sim_archive()

Returns the past simulated candidates.

Returns:

past simulated decision vectors along with their objective value

Return type:

tuple(np.ndarray,np.ndarray)

perform_distance(candidates)

Returns the distance from each normalized candidate to the set of normalized known candidates.

Parameters:

candidates (np.ndarray) – candidates

Returns:

distances from the normalized candidates to the set of normalized known candidates

Return type:

np.ndarray

Regression Kriging

class Surrogates.rKRG.rKRG(f_sim_archive, pb, n_train_samples, f_train_log, f_trained_model)

Bases: Surrogate

Class for regression Kriging surrogate model.

Parameters:

• f_sim_archive (str) – filename where are stored the simulated candidates

• pb (Problem) – problem the surrogate is associated with

• n_train_samples (positive int or inf, not zero) – number of training samples to extract from the end of f_sim_archive, if float(‘inf’) all the samples from f_sim_archive are considered

• f_train_log (str) – filename where will be recorded training log

• f_trained_model (str) – filename where will be recorded the trained surrogate model

• y_bounds (np.ndarray) – lower and upper bounds of the objective values found in the training set

• outputs_scaler (sklearn.preprocessing) – objective values normalizer

• model (pyKriging.krige.regression_kriging) – regression_kriging model

If you encounter some “np.float deprecated” error while using rKRG.py, please refer to the following page <https://github.com/capaulson/pyKriging/pull/53/files>

perform_training()

Trains the surrogate model.

add_point(candidate, obj_val)

Partial update (used in q-EGO)

Parameters:

• candidate (np.ndarray) – new candidate to add to the training set

• obj_val (np.ndarray) – objective value associated to the candidate

perform_prediction(candidates)

Returns the predicted objective value of the candidates and the uncertainty around it. Both quantities are normalized.

Parameters:

candidates (np.ndarray) – candidates

Returns:

predicted objective values and uncertainties

Return type:

(np.ndarray, np.ndarray)

denormalize_predictions(preds)

Denormalize predicted objective value.

normalize_obj_vals(obj_vals)

Normalized objective values.

load_trained_model()

Loads a model trained in the past.

load_sim_archive()

Returns the past simulated candidates.

Returns:

past simulated decision vectors along with their objective value

Return type:

tuple(np.ndarray,np.ndarray)

perform_distance(candidates)

Returns the distance from each normalized candidate to the set of normalized known candidates.

Parameters:

candidates (np.ndarray) – candidates

Returns:

distances from the normalized candidates to the set of normalized known candidates

Return type:

np.ndarray

GP with different kernels

class Surrogates.GP.GP(f_sim_archive, pb, n_train_samples, f_train_log, f_trained_model, kernel)

Bases: Surrogate

Class for Gaussian Process.

Parameters:

• f_sim_archive (str) – filename where are stored the simulated candidates

• pb (Problem) – problem the surrogate is associated with

• n_train_samples (positive int or inf, not zero) – number of training samples to extract from the end of f_sim_archive, if float(‘inf’) all the samples from f_sim_archive are considered

• f_train_log (str) – filename where will be recorded training log

• f_trained_model (str) – filename where will be recorded the trained surrogate model

• likelihood (gpytorch.likelihoods.GaussianLikelihood) – likelihood function

• outputs_scaler (sklearn.preprocessing) – objective values normalizer

• kernel (str) – covariance function

• model (gpytorch.models.ExactGP) – Gaussian Process model

perform_training()

Trains the surrogate model.

perform_prediction(candidates)

Returns the predicted objective value of the candidates and the uncertainty around it. Both quantities are normalized.

Parameters:

candidates (np.ndarray) – candidates

Returns:

predicted objective values and uncertainties

Return type:

(np.ndarray, np.ndarray)

denormalize_predictions(preds)

Denormalize predicted objective value.

normalize_obj_vals(obj_vals)

Normalized objective values.

load_trained_model()

Loads a model trained in the past.

load_sim_archive()

Returns the past simulated candidates.

Returns:

past simulated decision vectors along with their objective value

Return type:

tuple(np.ndarray,np.ndarray)

perform_distance(candidates)

Returns the distance from each normalized candidate to the set of normalized known candidates.

Parameters:

candidates (np.ndarray) – candidates

Returns:

distances from the normalized candidates to the set of normalized known candidates

Return type:

np.ndarray

GP for multiple ojbectives

class Surrogates.GP_MO.GP_MO(f_sim_archive, pb, n_train_samples, f_train_log, f_trained_model, kernel)

Bases: Surrogate

Class for Gaussian Process multitask model (multi dimensional targets only).

Parameters:

• f_sim_archive (str) – filename where are stored the simulated candidates

• pb (Problem) – problem the surrogate is associated with

• n_train_samples (positive int or inf, not zero) – number of training samples to extract from the end of f_sim_archive, if float(‘inf’) all the samples from f_sim_archive are considered

• f_train_log (str) – filename where will be recorded training log

• f_trained_model (str) – filename where will be recorded the trained surrogate model

• likelihood (gpytorch.likelihoods.GaussianLikelihood) – likelihood function

• outputs_scaler (sklearn.preprocessing) – objective values normalizer

• kernel (str) – covariance function

• model (gpytorch.models.ExactGP) – Gaussian Process model

perform_training()

Trains the surrogate model.

perform_prediction(candidates)

Returns the predicted objective value of the candidates and the uncertainty around it. Both quantities are normalized.

Parameters:

candidates (np.ndarray) – candidates

Returns:

predicted objective values and uncertainties

Return type:

(np.ndarray, np.ndarray)

denormalize_predictions(preds)

Denormalize predicted objective value.

normalize_obj_vals(obj_vals)

Normalized objective values.

load_trained_model()

Loads a model trained in the past.

load_sim_archive()

Returns the past simulated candidates.

Returns:

past simulated decision vectors along with their objective value

Return type:

tuple(np.ndarray,np.ndarray)

perform_distance(candidates)

Returns the distance from each normalized candidate to the set of normalized known candidates.

Parameters:

candidates (np.ndarray) – candidates

Returns:

distances from the normalized candidates to the set of normalized known candidates

Return type:

np.ndarray