dacbench.envs

Subpackages

Submodules

Package Contents

Classes

LubyEnv

Environment to learn Luby Sequence

SigmoidEnv

Environment for tracing sigmoid curves

FastDownwardEnv

Environment to control Solver Heuristics of FastDownward

CMAESEnv

Environment to control the step size of CMA-ES

SGDEnv

Environment to control the learning rate of adam

OneLLEnv

Environment for (1+(lbd, lbd))-GA

ModCMAEnv

Abstract template for environments

Functions

luby_gen(i)

Generator for the Luby Sequence
class dacbench.envs.LubyEnv(config)

Bases: dacbench.AbstractEnv

Environment to learn Luby Sequence

step(self, action: int)

Execute environment step

Parameters

action (int) – action to execute

Returns

state, reward, done, info

Return type

np.array, float, bool, dict

reset(self) → List[int]

Resets env

Returns

Environment state

Return type

numpy.array

get_default_reward(self, _)
get_default_state(self, _)
close(self) → bool

Close Env

Returns

Closing confirmation

Return type

bool

render(self, mode: str = 'human') → None

Render env in human mode

Parameters

mode (str) – Execution mode

dacbench.envs.luby_gen(i)

Generator for the Luby Sequence

class dacbench.envs.SigmoidEnv(config)

Bases: dacbench.AbstractEnv

Environment for tracing sigmoid curves

_sig(self, x, scaling, inflection)

Simple sigmoid function

step(self, action: int)

Execute environment step

Parameters

action (int) – action to execute

Returns

state, reward, done, info

Return type

np.array, float, bool, dict

reset(self) → List[int]

Resets env

Returns

Environment state

Return type

numpy.array

get_default_reward(self, _)
get_default_state(self, _)
close(self) → bool

Close Env

Returns

Closing confirmation

Return type

bool

render(self, mode: str) → None

Render env in human mode

Parameters

mode (str) – Execution mode

class dacbench.envs.FastDownwardEnv(config)

Bases: dacbench.AbstractEnv

Environment to control Solver Heuristics of FastDownward

property port(self)
property argstring(self)
static _save_div(a, b)

Helper method for safe division

Parameters

  • a (list or np.array) – values to be divided

  • b (list or np.array) – values to divide by

Returns

Division result

Return type

np.array

send_msg(self, msg: bytes)

Send message and prepend the message size

Based on comment from SO see [1] [1] https://stackoverflow.com/a/17668009

Parameters

msg (bytes) – The message as byte

recv_msg(self)

Recieve a whole message. The message has to be prepended with its total size Based on comment from SO see [1]

Returns

The message as byte

Return type

bytes

recvall(self, n: int)

Given we know the size we want to recieve, we can recieve that amount of bytes. Based on comment from SO see [1]

Parameters

n (int) – Number of bytes to expect in the data

Returns

The message as byte

Return type

bytes

_process_data(self)

Split received json into state reward and done

Returns

state, reward, done

Return type

np.array, float, bool

step(self, action: Union[int, List[int]])

Environment step

Parameters

action (Union[int, List[int]]) – Parameter(s) to apply

Returns

state, reward, done, info

Return type

np.array, float, bool, dict

reset(self)

Reset environment

Returns

State after reset

Return type

np.array

kill_connection(self)

Kill the connection

close(self)

Close Env

Returns

Closing confirmation

Return type

bool

render(self, mode: str = 'human') → None

Required by gym.Env but not implemented

Parameters

mode (str) – Rendering mode

class dacbench.envs.CMAESEnv(config)

Bases: dacbench.AbstractEnv

Environment to control the step size of CMA-ES

step(self, action)

Execute environment step

Parameters

action (list) – action to execute

Returns

state, reward, done, info

Return type

np.array, float, bool, dict

reset(self)

Reset environment

Returns

Environment state

Return type

np.array

close(self)

No additional cleanup necessary

Returns

Cleanup flag

Return type

bool

render(self, mode: str = 'human')

Render env in human mode

Parameters

mode (str) – Execution mode

get_default_reward(self, _)

Compute reward

Returns

Reward

Return type

float

get_default_state(self, _)

Gather state description

Returns

Environment state

Return type

dict

class dacbench.envs.SGDEnv(config)

Bases: dacbench.AbstractEnv

Environment to control the learning rate of adam

val_model

Samuel Mueller (PhD student in our group) also uses backpack and has ran into a similar memory leak. He solved it calling this custom made RECURSIVE memory_cleanup function: # from backpack import memory_cleanup # def recursive_backpack_memory_cleanup(module: torch.nn.Module): # memory_cleanup(module) # for m in module.modules(): # memory_cleanup(m) (calling this after computing the training loss/gradients and after validation loss should suffice)

Type

TODO

get_reward(self)
get_training_reward(self)
get_validation_reward(self)
get_log_training_reward(self)
get_log_validation_reward(self)
get_log_diff_training_reward(self)
get_log_diff_validation_reward(self)
get_diff_training_reward(self)
get_diff_validation_reward(self)
get_full_training_reward(self)
property crash(self)
seed(self, seed=None, seed_action_space=False)

Set rng seed

Parameters

  • seed – seed for rng

  • seed_action_space (bool, default False) – if to seed the action space as well

step(self, action)

Execute environment step

Parameters

action (list) – action to execute

Returns

state, reward, done, info

Return type

np.array, float, bool, dict

_architecture_constructor(self, arch_str)
reset(self)

Reset environment

Returns

Environment state

Return type

np.array

set_writer(self, writer)
close(self)

No additional cleanup necessary

Returns

Cleanup flag

Return type

bool

render(self, mode: str = 'human')

Render env in human mode

Parameters

mode (str) – Execution mode

get_default_state(self, _)

Gather state description

Returns

Environment state

Return type

dict

_train_batch_(self)
train_network(self)
_get_full_training_loss(self, loader)
property current_validation_loss(self)
_get_validation_loss_(self)
_get_validation_loss(self)
_get_gradients(self)
_get_momentum(self, gradients)
get_adam_direction(self)
get_rmsprop_direction(self)
get_momentum_direction(self)
_get_loss_features(self)
_get_predictive_change_features(self, lr)
_get_alignment(self)
generate_instance_file(self, file_name, mode='test', n=100)
class dacbench.envs.OneLLEnv(config)

Bases: dacbench.AbstractEnv

Environment for (1+(lbd, lbd))-GA for both OneMax and LeadingOne problems

reset(self)

Resets env

Returns

Environment state

Return type

numpy.array

get_state(self)
get_onell_params(self, action)

Get OneLL-GA parameters (lbd1, lbd2, p and c) from an action

Returns: lbd1, lbd2, p, c
lbd1: float (will be converted to int in step())

number of mutated off-springs: in range [1,n]

lbd2: float (will be converted to int in step())

number of crossovered off-springs: in range [1,n]

p: float

mutation probability

c: float

crossover bias

step(self, action)

Execute environment step

Parameters

action (Box) – action to execute

Returns

  • state, reward, done, info

  • np.array, float, bool, dict

plot_agent_prediction(self, agent, dirname)

Plot agent progress for this particular environment

close(self) → bool

Close Env

No additional cleanup necessary

Returns

Closing confirmation

Return type

bool

class dacbench.envs.ModCMAEnv(config)

Bases: dacbench.AbstractEnv

Abstract template for environments

reset(self)

Reset environment

Returns

Environment state

Return type

state

step(self, action)

Execute environment step

Parameters

action – Action to take

Returns

  • state – Environment state

  • reward – Environment reward

  • done (bool) – Run finished flag

  • info (dict) – Additional metainfo

close(self)

Override close in your subclass to perform any necessary cleanup.

Environments will automatically close() themselves when garbage collected or when the program exits.

get_default_reward(self, *_)
get_default_state(self, *_)