HTCondor on a single machine

To test code that produces a HTCondor DAG (designed for running on a computer cluster), it is useful to have HTCondor installed on my own machine (creating a pool of one). There is a lightweight HTCondor installation called minicondor that is designed specifically for this. On Ubuntu the installation instructions can be found here.

One this is done you can submit Condor submit files or DAGs. However, due to the standard policy configurations, they won’t immediately start while your machine is in use. If you want them to start immediately you need to set various conditions by editing the /etc/condor/condor_config file. As described in the “Test-job Policy example” found by scrolling down from here, you can add the following to the end of that file:

START      = ($(START)) || Owner == "coltrane"
SUSPEND    = ($(SUSPEND)) && Owner != "coltrane"
CONTINUE   = $(CONTINUE)
PREEMPT    = ($(PREEMPT)) && Owner != "coltrane"
KILL       = $(KILL)

with "coltrane" replaced by your username. (also see the Note on the section on the START attribute for an alternative that does not set by user).

This will allow jobs to start regardless of what is happening on the machine. Only do this if you are happy for this to be the case, i.e., you have a decent amount of memory to accommodate any test jobs and have several cores available.

Leave a Reply