The ModEA Benchmark

Task: control the algorithm components of CMA-ES on BBOB functions
Cost: negative objective value
Number of hyperparameters to control: 11 categorical
State Information: generation size, step size, remaining budget, function ID, instance ID
Noise Level: fairly large, depends on target function
Instance space: the BBOB functions with ids, starting point and starting sigma as well as population size

This benchmark uses the ModEA package to enable dynamic control of several algorithm components of CMA-ES. The components of the algorithm that can be selected or changed are: sequential execution, active update, elitism, orthogonal sampling, convergence threshold enabled, step size adaption scheme, mirrored sampling, the base sampler, weight option, local restarts and bound correction. The goal in the optimization is to find the global function minimum before the cutoff, so the cost is defined as the current negativ objective value.

Just like the ModCMA benchmark (which provides a very similar problem with a different backend), this benchmark is challenging due to the large configuration space. It is an advanced benchmark that should likely not be the starting point for the development of DAC methods.

class dacbench.benchmarks.modea_benchmark.ModeaBenchmark(config_path=None, config=None)

Bases: AbstractBenchmark

Benchmark with default configuration & relevant functions for Modea

get_environment()

Return ModeaEnv env with current configuration

Returns

Modea environment

Return type

ModeaEnv

read_instance_set(test=False)

Read path of instances from config into list

class dacbench.envs.modea.ModeaEnv(config)

Bases: AbstractEnv

close()

Override close in your subclass to perform any necessary cleanup.

Environments will automatically close() themselves when garbage collected or when the program exits.

ensureFullLengthRepresentation(representation)

Given a (partial) representation, ensure that it is padded to become a full length customizedES representation, consisting of the required number of structure, population and parameter values. >>> ensureFullLengthRepresentation([]) [0,0,0,0,0,0,0,0,0,0,0, None,None, None,None,None,None,None,None,None,None,None,None,None,None,None] :param representation: List representation of a customizedES instance to check and pad if needed :return: Guaranteed full-length version of the representation

reset()

Reset environment

Returns

Environment state

Return type

state

step(action)

Execute environment step

Parameters

action – Action to take

Returns

  • state – Environment state

  • reward – Environment reward

  • done (bool) – Run finished flag

  • info (dict) – Additional metainfo