Question

I m calling several unix commands and python on a python script from a qsub shell script, meant to run on a cluster. The trouble is that when the script executes, something seems to go awry in the shell, so that directories and files that exist are not found. For example, in the .out output files of qsub I see the following errors:

cd: /valid/dir/name: No such file or directory
python valid/script/name.py 
python: can t open file  valid/script/name.py : [Errno 2] No such file or directory

so the script cannot cd into a dir that definitely exist. Similarly, calling python on a python script that definitely exists yields an error.

any idea what might be going wrong here, or how I could try to debug this?

thanks very much.

Answer 1

the errors are pretty self explanatory. make sure /valid/dir/name is actually a directory. You can put this in your shell script to create this directory if not found.

if [ ! -d "/valid/dir/name" ];then
   mkdir -p /valid/dir/name
fi

As for Python error, because there s no "valid" directory, it gives you error.

Answer 2

Does this directory exist on every possible compute node? Remember that the script won t necessarily execute on the host you submit it from. Try adding the python equivalent of echo hostname to your script and then verifying that the directory exists on that host.

友情链接