Alternative Communications Backend for R
mirai provides an alternative communications backend for R. This functionality was developed to fulfil a request by R Core at R Project Sprint 2023.
A ‘miraiCluster’ is recognised as one of the official cluster types,
and may be created by parallel::makeCluster(type = "MIRAI")
from R 4.5 onwards. This function calls make_cluster(),
which may also be used to directly create a ‘miraiCluster’.
- Specify ‘n’ to launch nodes on the local machine.
- Specify ‘url’ for receiving connections from remote nodes.
- Optionally, specify ‘remote’ to launch remote daemons using a remote
configuration generated by
remote_config()orssh_config().
Created clusters may be used for any function in the
parallel base package such as
parallel::clusterApply() or
parallel::parLapply(), or the load-balanced versions such
as parallel::parLapplyLB().
library(parallel)
library(mirai)
cl <- makeCluster(6, type = "MIRAI")
cl
#> < miraiCluster | ID: `0` nodes: 6 active: TRUE >
parLapply(cl, iris, mean)
#> $Sepal.Length
#> [1] 5.843333
#>
#> $Sepal.Width
#> [1] 3.057333
#>
#> $Petal.Length
#> [1] 3.758
#>
#> $Petal.Width
#> [1] 1.199333
#>
#> $Species
#> [1] NAstatus() may be called on a ’miraiCluster` to query the
number of connected nodes at any time.
status(cl)
#> $connections
#> [1] 6
#>
#> $daemons
#> [1] "ipc:///tmp/bfedbd4df86f88a600784a97"
stopCluster(cl)Making a cluster specifying ‘url’ without ‘remote’ causes the shell commands for manual deployment of nodes to be printed to the console.
cl <- make_cluster(n = 2, url = host_url())
#> Shell commands for deployment on nodes:
#>
#> [1]
#> Rscript -e 'mirai::daemon("tcp://192.168.5.16:51320",dispatcher=FALSE,cleanup=FALSE,rs=c(10407,-2015301958,704158963,-654244456,-659410247,-973143258,-2088471697))'
#>
#> [2]
#> Rscript -e 'mirai::daemon("tcp://192.168.5.16:51320",dispatcher=FALSE,cleanup=FALSE,rs=c(10407,1082798196,919607134,-444405288,1518297735,1358819766,-1098042261))'
stop_cluster(cl)Foreach Support
A ‘miraiCluster’ may also be registered by doParallel
for use with the foreach
package.
Running some parallel examples for the foreach()
function:
library(doParallel)
#> Loading required package: foreach
#> Loading required package: iterators
library(foreach)
cl <- makeCluster(6, type = "MIRAI")
registerDoParallel(cl)
# normalize the rows of a matrix
m <- matrix(rnorm(9), 3, 3)
foreach(i = 1:nrow(m), .combine = rbind) %dopar%
(m[i, ] / mean(m[i, ]))
#> [,1] [,2] [,3]
#> result.1 33.042790 7.870983 -37.913773
#> result.2 2.980121 3.078128 -3.058249
#> result.3 -2.197490 -2.163921 7.361411
# simple parallel matrix multiply
a <- matrix(1:16, 4, 4)
b <- t(a)
foreach(b = iterators::iter(b, by='col'), .combine = cbind) %dopar%
(a %*% b)
#> [,1] [,2] [,3] [,4]
#> [1,] 276 304 332 360
#> [2,] 304 336 368 400
#> [3,] 332 368 404 440
#> [4,] 360 400 440 480
stopCluster(cl)