Abort bps submission if submit-side run directory already exists.

Description

was submitting a bunch of runs at the same time and was having problems (https://lsstc.slack.com/archives/C01FBUGM2CV/p1637273611121400)  The problems stemmed from the fact that due to a setting in the submit yaml all of the runs were using the same directory for their submit-side files.  A majority of the files are uniquely named across runs, but not all of them.  

 

We could go change the files we know are not uniquely named in both the bps core and the htcondor plugin.  However, this doesn't mean all WMS plugins will always name all of their files uniquely nor that there won't be a "bug" that introduces a non-unique filename.  Also, this can mean a ridiculous number of files ending up in same directories, complications debugging a single run, etc.

 

Until there is a specific use case that requires all in one directory or always guaranteeing unique filenames (without paths) for submit files, change bps to abort submission if submit directory already exists (try to reduce race conditions).

 

 

 

Checklist

Lucidchart Diagrams

Issue Matrix

hide

Activity

Show:
Michelle Gower
December 3, 2021 at 5:28 PM

Just a few minor requests/comments. Merge approved.

Done
Pinned fields
Click on the next to a field label to start pinning.

Details

Assignee

Reporter

Labels

Reviewers

Michelle Gower

Story Points

RubinTeam

Components

Checklist

Created November 19, 2021 at 4:11 PM
Updated December 9, 2021 at 6:21 PM
Resolved December 9, 2021 at 6:10 PM