Python API

The Rain Pyton API consists of two domains that observe the workflow graph differently, although the concepts are similar and some classes are used in both contexts.

• Code run at the client, creating sessions and task graphs, executing and querying sessions. There, the tasks are only created and declared, never actually executed.
• Python code that runs inside remote Pyhton tasks on the governors. This code has access to the actual input data, but only sees the adjacent data objects (input and output).

Client API

Client

One instance per connection to a server.

Session

One instance per a constructed graph (possibly with multiple submits). Tied to one Client.

Data objects

Tied to a Session.

Tasks

Tied to a Session.

Attributes

Input and Output

These are helper objects are used to specify task input and output attributes. In particular, specifying an Output is the preferred way to set properties of the output DataObject.

Builtin tasks and external programs

Native Rain tasks to be run at the governors.

Data instance objects

Tied to a session and a DataObject. Also used in Remote Python tasks.

Resources

Note

TODO: Describe and document task resources.

Labeled list

Remote Python tasks

API for creating routines to be run at the governors. Created by the decorating with remote (preferred) or by Remote.

Whe specifying the remote task in the client code, the relevant classes are Remote, Input, Output, RainException, RainWarning, LabeledList and the decorateor remote.

Inside the running remote task, only RainException, RainWarning, LabeledList, DataInstance and Context are relevant.

The inputs of a Remote task are arbitrary python objects containing a DataInstance in place of every DataObject, or loaded data object if autoload=True or load=True is set on the Input.

The remote should return a list, tuple or LabeledList of DataInstance (created by Context.blob()), bytes or string.