mpcrl.ExperienceReplay#

class mpcrl.ExperienceReplay(iterable=(), maxlen=None, sample_size=1, include_latest=0, seed=None)[source]#

Bases: deque[ExpType]

Class for Reinforcement Learning agents’ traning to save and sample experience transitions.

The class inherits from deque, adding a couple of simple functionalities to it for sampling transitions at random from past observed data (see reset and sample).

Parameters:
iterableIterable of ExpType, optional

Initial items to be inserted in the container. By default, empty.

maxlenint, optional

Maximum length/capacity of the memory. If None, the deque has no maximum size, which is the default behaviour.

sample_sizeint or float, optional

Size (as integer, or float percentage of maxlen) of the experience replay items to draw when performing an update. By default, one item per sampling is drawn. If a float percentage, maxlen must be provided.

include_latestint or float, optional

Size (as integer, or float percentage of sample_size) dedicated to including the latest experience items. By default, 0, i.e., no last item is included.

seedNone, int, array_like of ints, SeedSequence, BitGenerator, Generator

Seed for the numpy.random.Generator used for sampling. By default, None.

Raises:
TypeError

Raises if sample_size is a float (a percentage of the maximum length), but maxlen is None, since it is impossible to compute the percentage of an unknown quantity.

Methods

append(item, /)

Add an element to the right side of the deque.

appendleft(item, /)

Add an element to the left side of the deque.

clear(/)

Remove all elements from the deque.

copy(/)

Return a shallow copy of a deque.

count(value, /)

Return number of occurrences of value.

extend(iterable, /)

Extend the right side of the deque with elements from the iterable.

extendleft(iterable, /)

Extend the left side of the deque with elements from the iterable.

index(value, [start, [stop]])

Return first index of value.

insert(index, value, /)

Insert value before index.

pop(/)

Remove and return the rightmost element.

popleft(/)

Remove and return the leftmost element.

remove(value, /)

Remove first occurrence of value.

reset([seed])

Resets the seed of the numpy.random.Generator used for sampling.

reverse(/)

Reverse IN PLACE.

rotate([n])

Rotate the deque n steps to the right.

sample()

Samples the experience memory and yields the sampled items.

Attributes

maxlen

maximum size of a deque or None if unbounded
append(item, /)#

Add an element to the right side of the deque.

appendleft(item, /)#

Add an element to the left side of the deque.

clear(/)#

Remove all elements from the deque.

copy(/)#

Return a shallow copy of a deque.

count(value, /)#

Return number of occurrences of value.

extend(iterable, /)#

Extend the right side of the deque with elements from the iterable.

extendleft(iterable, /)#

Extend the left side of the deque with elements from the iterable.

index(value[, start[, stop]])#

Return first index of value.

Raises ValueError if the value is not present.

insert(index, value, /)#

Insert value before index.

maxlen#

maximum size of a deque or None if unbounded

pop(/)#

Remove and return the rightmost element.

popleft(/)#

Remove and return the leftmost element.

remove(value, /)#

Remove first occurrence of value.

reset(seed=None)[source]#

Resets the seed of the numpy.random.Generator used for sampling.

Return type:

None

reverse(/)#

Reverse IN PLACE.

rotate(n=1, /)#

Rotate the deque n steps to the right. If n is negative, rotates left.

sample()[source]#

Samples the experience memory and yields the sampled items.

Returns:
sampleiterator of ExpType

An iterable sample is yielded.

Return type:

Iterator[TypeVar(ExpType)]