Enable parallelization on batch systems

Description

This class is used to parameterize scheduler options on managed high-performance computing clusters.

Usage

BatchJobsParam(workers, catch.errors = TRUE, cleanup = TRUE, 
    work.dir = getwd(), stop.on.error = TRUE, seed = NULL,
    resources = NULL, conffile = NULL, cluster.functions = NULL, 
    progressbar = TRUE, jobname = "BPJOB",
    reg.pars=list(seed=seed, work.dir=work.dir),
    conf.pars=list(conffile=conffile, cluster.functions=cluster.functions),
    submit.pars=list(resources=resources),
    ...)

Arguments

Number of workers to divide tasks (e.g., elements in the first argument of bplapply) between. On Multicore and SSH backends, this defaults to all available nodes. On managed (e.g., slurm, SGE) clusters workers defaults to NA, meaning that the number of workers equals the number of tasks. See argument n.chunks in chunk and submitJobs for more information.

Flag to determine in apply-like functions (see e.g. bplapply) whether to quit with an error as soon as one application fails or encapsulation of function calls in try blocks which triggers a resume mechanism (see bpresume). Defaults to TRUE.

BatchJobs creates temporary directories in the work.dir. If cleanup is set to TRUE (default), the directories are removed from the file systems automatically. Set this to FALSE whenever it might become necessary to utilize any special functionality provided by BatchJobs. To retrieve the registry, call loadRegistry on the temporary directory.

Directory to store temporary files. Note that this must be shared across computational nodes if you use a distributed computing backend. Default ist the current working directory of R, see getwd. Ignored when reg.pars is provided.

Stop all jobs as soon as one jobs fails (stop.on.error == TRUE) or wait for all jobs to terminate. Default is TRUE.

Set an initial seed for the RNG. See makeRegistry for more information. Default is NULL where a random seed is chosen upon initialization. Ignored when reg.pars is provided.

List of job specific resources passed to submitJobs. Default is NULL where the resources defined in the configuration are used. Ignored when submit.pars is provided.

URI to a custom BatchJobs configuration file used for execution. Default is NULL which relies on BatchJobs to handle configuration files. Ignored when conf.pars is provided.

Specify a specific cluster backend using on of the constructors provided by BatchJobs, see ClusterFunctions. Default is NULL where the default cluster functions defined in the configuration are used. Ignored when conf.pars is provided.

Suppress the progress bar used in BatchJobs and be less verbose. Default is FALSE.

Job name that is prepended to the output log and result files. Default is "BPJOB".

List of parameters passed to BatchJobs::makeRegistry(). When present, user-supplied arguments seed and work.dir to BatchJobsParam are ignored.

List of parameters passed to BatchJobs::setConfig(). When present, user-supplied arguments conffile, cluster.functions to BatchJobsParam are ignored.

List of parameters passed to BatchJobs::submitJobs. When present, user-supplied argument resources to BatchJobsParam is ignored. submitJobs parameters reg, id cannot be set.

BatchJobsParam constructor

Return an object with specified values. The object may be saved to disk or reused within a session.

Methods

The following generics are implemented and perform as documented on the corresponding help page: bpworkers, bpnworkers, bpstart, bpstop, bpisup, bpbackend, bpbackend<-

Author(s)

Michel Lang, mailto:michellang@gmail.com

See Also

getClass("BiocParallelParam") for additional parameter classes.

register for registering parameter classes for use in parallel evaluation.

Examples

p <- BatchJobsParam(progressbar=FALSE)
bplapply(1:10, sqrt, BPPARAM=p)

## Not run: 
## see vignette for additional explanation
funs <- makeClusterFunctionsSLURM("~/slurm.tmpl")
param <- BatchJobsParam(4, cluster.functions=funs)
register(param)
bplapply(1:10, function(i) sqrt)

## End(Not run)

Results

R version 3.3.1 (2016-06-21) -- "Bug in Your Hair"
Copyright (C) 2016 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(BiocParallel)
> png(filename="/home/ddbj/snapshot/RGM3/R_BC/result/BiocParallel/BatchJobsParam-class.Rd_%03d_medium.png", width=480, height=480)
> ### Name: BatchJobsParam-class
> ### Title: Enable parallelization on batch systems
> ### Aliases: BatchJobsParam-class BatchJobsParam
> ###   bpbackend,BatchJobsParam-method bpbackend<-,BatchJobsParam
> ###   bpisup,BatchJobsParam-method bpstart,BatchJobsParam-method
> ###   bpstop,BatchJobsParam-method bpworkers,BatchJobsParam-method
> ###   bpschedule,BatchJobsParam-method show,BatchJobsParam-method
> 
> ### ** Examples
> 
> p <- BatchJobsParam(progressbar=FALSE)
> bplapply(1:10, sqrt, BPPARAM=p)
[[1]]
[1] 1

[[2]]
[1] 1.414214

[[3]]
[1] 1.732051

[[4]]
[1] 2

[[5]]
[1] 2.236068

[[6]]
[1] 2.44949

[[7]]
[1] 2.645751

[[8]]
[1] 2.828427

[[9]]
[1] 3

[[10]]
[1] 3.162278

> 
> ## Not run: 
> ##D ## see vignette for additional explanation
> ##D funs <- makeClusterFunctionsSLURM("~/slurm.tmpl")
> ##D param <- BatchJobsParam(4, cluster.functions=funs)
> ##D register(param)
> ##D bplapply(1:10, function(i) sqrt)
> ## End(Not run)
> 
> 
> 
> 
> 
> dev.off()
null device 
          1 
>