forked from futureverse/future.batchtools
-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathNEWS
More file actions
141 lines (89 loc) · 4.85 KB
/
Copy pathNEWS
File metadata and controls
141 lines (89 loc) · 4.85 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
Package: future.batchtools
==========================
Version: 0.7.1 [2018-07-18]
NEW FEATURES:
o The batchtools_* backends support the handling of the standard output as
implemented in future (>= 1.9.0).
BUG FIXES:
o A bug was introduced in future.batchtools 0.7.0 that could result in "Error
in readLog(id, reg = reg) : Log file for job with id 1 not available" when
using one of the batchtools backends. It occurred when the value was
queried. It was observered using 'batchtools_torque' but not when using
'batchools_local'. This bug was missed because the 0.7.0 release was not
tested on an TORQUE/PBS HPC scheduler as it should have.
Version: 0.7.0 [2018-05-03]
NEW FEATURES:
o Argument 'workers' of future strategies may now also be a function, which
is called without argument when the future strategy is set up and used as
is. For instance, plan(callr, workers = halfCores) where
halfCores <- function() { max(1, round(availableCores() / 2)) } will use
half of the number of available cores. This is useful when using nested
future strategies with remote machines.
CODE REFACTORING:
o Preparing for futures to gather a richer set of results from batchtools
backends.
Version: 0.6.0 [2017-09-10]
NEW FEATURES:
o If the built-in attempts of batchtools for finding a default template file
fails, then system("templates", package = "future.batchtools") is searched
for template files as well. Currently, there exists a `torque.tmpl` file.
o A job's name in the scheduler is now set as the future's label (requires
batchtools 0.9.4 or newer). If no label is specified, the default job name
is controlled by batchtools.
o The period between each poll of the scheduler to check whether a future
(job) is finished or not now increases geometrically as a function of number
of polls. This lowers the load on the scheduler for long running jobs.
o The error message for expired batchtools futures now include the last few
lines of the logged output, which sometimes includes clues on why the future
expired. For instance, if a TORQUE/PBS job use more than the allocated
amount of memory it might be terminated by the scheduler leaving the message
"PBS: job killed: vmem 1234000 exceeded limit 1048576" in the output.
o print() for BatchtoolsFuture returns the object invisibly.
BUG FIXES:
o Calling future_lapply() with functions containing globals part of non-default
packages would when using batchtools futures give an error complaining that
the global is missing. This was due to updates in future (>= 1.4.0) that
broke this package.
o loggedOutput() for BatchtoolsFuture would always return NULL unless an error
had occurred.
Version: 0.5.0 [2017-06-02]
o First version submitted to CRAN.
SOFTWARE QUALITY:
o Added more tests; test coverage now at 93%.
Version: 0.4.0 [2017-05-16]
NEW FEATURES:
o Added batchtools_custom() for specifying batchtools futures using any type
of batchtools cluster functions.
o batchtools_template(pathname = NULL, type = <type>) now relies on the
batchtools package for locating the <type> template file.
o nbrOfWorkers() for batchtools futures now defaults to +Inf unless the
evaluator's 'workers' or 'cluster.functions' specify something else.
o Renamed argument 'pathname' to 'template' for batchtools_<tmpl>() functions.
BUG FIXES:
o Under plan(batchjobs_*), when being created futures would produce an error
on "all(is.finite(workers)) is not TRUE" due to an outdated sanity check.
SOFTWARE QUALITY:
o TESTS: Added test of future_lapply() for batchtools backends.
o TESTS: Added optional tests for batchjobs_* HPC schedulers listed in
environment variable 'R_FUTURE_TESTS_STRATEGIES'.
CODE REFACTORING:
o CLEANUP: Package no longer depends on R.utils.
Version: 0.3.0 [2017-03-19]
NEW FEATURES:
o The number of jobs one can add to the queues of HPC schedulers is in
principle unlimited, which is why the number of available workers for such
batchtools_* backends is reported as +Inf. However, as the number of
workers is used by future_lapply() to decide how many futures should be used
to best partition the elements, this means that future_lapply() will always
use one future per element. Because of this, it is now possible to specify
plan(batchtools_*, workers = n) where 'n' is the target number of workers.
Version: 0.2.0 [2017-02-23]
GLOBALS:
o batchtools (>= 0.9.2) now supports exporting objects with any type of names
(previously only possible if they mapped to to strictly valid filenames).
This allowed me to avoid lots of internal workaround code encoding and
decoding globals.
Version: 0.1.0 [2017-02-11]
o Package created by porting the code of future.BatchJobs. This version passes
'R CMD check --as-cran' with all OK after a minimal amount of adjustments
to the ported code.