My job is pending with “ReqNodeNotAvail, Reserved for maintenance”. Can I get it to run before the scheduled maintenance?
The reason means that the walltime request of your job extends into an upcoming scheduled cluster maintenance and therefore the scheduler will not run your job.
You can get it run before the scheduled maintenance by ensuring that the job only requests an amount of time that would make it possible that it finishes before the maintenance begins. For example:
Run the command htnm
(Hours To Next Maintenance) to get the number of hours until the maintenance begins. Then, you have to guess a little based on how soon your job will start. If you think it will start nearly immediately you can probably just use the number htnm returns as the slurm -t
option for your job. Otherwise, subtract an amount you think your job might wait. Unfortunately we don’t have a good way of estimating the time until start for jobs.