Skip to content

Improve TCP interface for higher scales #1305

Description

@jlnav

Right now the TCP/reverse-ssh interface requires the following:

python test_1d_sampling_asktell.py --comms ssh --workers server1 server2 --worker_python /remote/path/to/python

This is obviously hard to wrangle together for more complicated server IPs, larger numbers of workers, etc.

I'm considering something like the following:

python test_1d_sampling_asktell.py --worker_file workers.yaml

AIMachine.alcf.anl:
    worker_ids: 1
    python_exe: /alcf/path/to/python
    working_dir: /home/my/dir

server2.nersc:
    worker_ids: 2-40
    python_exe: /nersc/path/to/python
    working_dir: /home

Metadata

Metadata

Assignees

Type

No type

Fields

No fields configured for issues without a type.

Projects

Status
To Do

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions