Right now the TCP/reverse-ssh interface requires the following:
python test_1d_sampling_asktell.py --comms ssh --workers server1 server2 --worker_python /remote/path/to/python
This is obviously hard to wrangle together for more complicated server IPs, larger numbers of workers, etc.
I'm considering something like the following:
python test_1d_sampling_asktell.py --worker_file workers.yaml
AIMachine.alcf.anl:
worker_ids: 1
python_exe: /alcf/path/to/python
working_dir: /home/my/dir
server2.nersc:
worker_ids: 2-40
python_exe: /nersc/path/to/python
working_dir: /home
Right now the TCP/reverse-ssh interface requires the following:
python test_1d_sampling_asktell.py --comms ssh --workers server1 server2 --worker_python /remote/path/to/pythonThis is obviously hard to wrangle together for more complicated server IPs, larger numbers of workers, etc.
I'm considering something like the following:
python test_1d_sampling_asktell.py --worker_file workers.yaml