DataPipe

class datapipes.datapipe.DataPipe

Description:

This class serves as a base class for creating new Data Pipes. Each new Data Pipe should implement the store and retrieve methods. The implementation should generate reasonable keys that can be used for accessing the data. It is recommended to not interfere in the way the data is stored. For example, changing the type of the data or the format of the data. If your Data Pipe requires specific format or type, make sure you the conversion inside the Data Pipe ensuring consistency in the way tasks interact with Data Pipes. Look at Memory for sample implementation.

abstract retrieve(key) → Any

Retrieving data based on a key. The key is what is returned form store. This method should be implemented in the class inheriting DataPipe.

Parameters:

key (Any) – The key to identify the data.

Returns:

The retrieved data.

Return type:

Any

abstract store(data) → str

Storing intermediate results or needed information inside Data Pipe. This method should be implemented in the class inheriting DataPipe.

Parameters:

data (Any) – The data to be stored.

Returns:

The name of the stored data.

Return type:

str