Memory profiling in R

• Rprof has an option memory.profiling. When TRUE the profiler also writes out information the small and large vector heap sizes, memory in nodes, and number of calls to Rf_duplicate in the interval. summaryRprof now has options to summarize this information. This is available only on Unix at the moment.
• If R is compiled with --enable-memory-profiling, the function tracemem marks an object so that a stack trace will be printed when the object is duplicated, or when it is copied by coercion or arithmetic functions. This is intended for tracking accidental copying of large objects. untracemem will untrace an object (though not all copies of it) and tracingState controls whether tracing information is printed. In the example below we see that lm does not duplicate its data argument, but that glm does, and that lm does copy the response vector.

  > data(trees)
  > tracemem(trees)
  [1] "<0x8bfff28>"
  > lm(log(Volume)~log(Height)+log(Girth),data=trees)
  
  Call:
  lm(formula = log(Volume) ~ log(Height) + log(Girth), data = trees)
  
  Coefficients:
  (Intercept)  log(Height)   log(Girth)
       -6.632        1.117        1.983
  
  >  glm(log(Volume)~log(Height)+log(Girth),data=trees)
  memtrace[0x8bfff28->0x8b6d820]: glm
  memtrace[0x8b6d820->0x89b4c10]: glm
  
  Call:  glm(formula = log(Volume) ~ log(Height) + log(Girth), data = trees)
  
  Coefficients:
  (Intercept)  log(Height)   log(Girth)
       -6.632        1.117        1.983
  
  Degrees of Freedom: 30 Total (i.e. Null);  28 Residual
  Null Deviance:      8.309
  Residual Deviance: 0.1855       AIC: -62.71
  > tracemem(trees$Volume)
  [1] "<0x895e230>"
  >  lm(Volume~Height+Girth,data=trees)
  memtrace[0x895e230->0x87a2898]: eval eval model.frame.default model.frame eval eval lm
  
  Call:
  lm(formula = Volume ~ Height + Girth, data = trees)
  
  Coefficients:
  (Intercept)       Height        Girth
     -57.9877       0.3393       4.7082
  

  tracemem cannot be used on functions, since it uses the same trace bit that trace uses, and will not work on objects such as environments that are passed by reference and not duplicated. The output for this could be made prettier and sent to a file: the main thing to decide is how to handle files when multiple objects may be being traced.
• If R is compiled with R_MEMORY_PROFILING defined, Rprofmem starts and stops a pure memory use profiler. A stack trace is sent to an output file on any large enough vector allocation (threshold set by the user) or when GetNewPage is called to expand the R heap. This profiler does not use timers and so can coexist with other forms of profiling. I do not yet have tools to summarize the output.