i have large rasterstack (s) following details:
class : rasterstack dimensions : 510, 1068, 544680, 19358 (nrow, ncol, ncell, nlayers) resolution : 0.08333333, 0.08333333 (x, y) extent : -141, -52, 41, 83.5 (xmin, xmax, ymin, ymax) coord. ref. : +proj=longlat +datum=nad83 +no_defs +ellps=grs80 +towgs84=0,0,0 names : jan.1961.1, jan.1961.2, jan.1961.3, jan.1961.4, jan.1961.5, jan.1961.6, jan.1961.7, jan.1961.8, jan.1961.9, jan.1961.10, jan.1961.11, jan.1961.12, jan.1961.13, jan.1961.14, jan.1961.15, ... time : 1961-01-01 - 2013-12-31 (range) doing like:
writeraster( s,"pp", overwrite=true, format="cdf", varname="p", varunit="mm", longname="totals", xname="lon", yname="lat",zname="time", zunit="numeric") takes more 2 weeks complete on computer. how can run in parallel (may via foreach loop , %dopar% command) same results shorter processing time?
sample data
s=brick(nrows=510, ncols=1068, xmn=-180, xmx=180, ymn=-90, ymx=90, crs="+proj=longlat +datum=wgs84", nl=193581) dates=seq(as.date("1961-01-01"), as.date("2013-12-31"), by="day") s<- setz(s,dates) nb: true data rasterstack not brick.
you can try code, did not tested on big dataset. , did not tested ncecat part... i'll update later, can try in meantime.
wd <- "~/bureau/tmp" # stack 16 layers nl <- 16 # 19358 s <- brick(nrows = 510, ncols = 1068, xmn = -180, xmx = 180, ymn = -90, ymx = 90, crs = "+proj=longlat +datum=wgs84", nl = nl) dates <- seq(as.date("1961-01-01"), as.date("2013-12-31"), = "day") s <- setz(s, dates) require(foreach) require(doparallel) cl <- makecluster(4) registerdoparallel(cl) tmp <- foreach(i = 1:nlayers(s)) %dopar% { r <- raster::raster(s, i) raster::writeraster(r, filename = paste0(wd, "/pp_", formatc(i, width = 6, flag = "0")), overwrite=true, format="cdf", varname="p", varunit="mm", longname="totals", xname="lon", yname="lat",zname="time", zunit="numeric") rm(r) } stopcluster(cl) ppfiles <- list.files(wd)[grep("pp_", list.files(wd))] system(paste0("ncecat ppfiles output.nc") 
Comments
Post a Comment