Client Interaction

As discussed in the quickstart users can interact with the worker-center network with the Executor abstraction.

This is built with lower level functions described below.

Scatter/Gather

Users rarely create RemoteData objects by hand. They are created by other client libraries or functions like gather and scatter.

distributed.client.scatter(center, data, serialize=True)[source]

Scatter data to workers

Parameters:

center:

(ip, port) tuple or Stream, or rpc object designating the Center

data: dict or iterable

either a dictionary of key: value pairs or an iterable of values

key:

if data is an iterable of values then we use the key to generate keys as key-0, key-1, key-2, ...

See also

distributed.client.gather, distributed.client._scatter, distributed.client.scatter_to_workers

Examples

>>> remote_data = scatter('127.0.0.1:8787', [1, 2, 3])  
>>> local_data = gather('127.0.0.1:8787', remote_data)  
distributed.client.gather(center, needed)[source]

Gather data from distributed workers

This operates by first asking the center who has all of the state keys and then trying those workers directly.

Keys not found on the network will not appear in the output. You should check the length of the output against the input if concerned about missing data.

Parameters:

center:

(ip, port) tuple or Stream, or rpc object designating the Center

needed: iterable

A list of required keys

Returns:

result: dict

A mapping of the given keys to data values

See also

distributed.client.scatter, distributed.client._gather, distributed.client.gather_from_workers

Examples

>>> remote_data = scatter('127.0.0.1:8787', [1, 2, 3])  
>>> local_data = gather('127.0.0.1:8787', remote_data)  
distributed.client.delete(center, keys)[source]

Delete keys from all workers

distributed.client.clear(center)[source]

Clear all data from all workers’ memory