Paul Murrell
Department of Statistics
The University of Auckland
paul@stat.auckland.ac.nz
It has been possible for some time to read bitmap images into R and draw them, via the 'pixmap' package. However, because R's graphics engine has only had vector drawing primitives, drawing raster images via this package is performed by drawing a tiny rectangle for each pixel in the original image. This has two major problems: no interpolation occurs if the rendered size differs from the natural raster size of the image and when this sort of output is produced in a text-based format such as PostScript or PDF, the file size can get out of control. The 'EBImage' (BioConductor) could also be used to read in bitmap images (in a variety of formats), but it was only possible to view these on a package-specific "viewer", not draw them on an R graphics device.
Support has been added to the graphics engine for rendering raster images to graphics devices, via a GE_Raster(image, x, y, width, height, angle) function (with grid.raster() at the user level).
In addition, for reasons explained below, there is support for capturing raster images from graphics devices, via a GE_Cap() function (with grid.cap() at the user level).
For example, the 'graphapp' library that we use for drawing on Windows only seems to provide support for scaling images (scale_image()), but not for rotating them, and it only seems to support full transparency, not semi-transparency.
The raw X11 device (with no extensions) does not even have support for scaling images! (There is some image rotation and scaling code in src/modules/X11/rotation.c, but it is quite tightly focused on providing support for rotating text.)
XFig is a strange one where, from memory, the main file refers to an external file containing the image.
Devices can make use of these if they cannot perform the relevant operations themselves. For example, the X11 device and the Windows device make use of all three.
With raster support you get the possibility of smaller PDF files when you draw a raster image to a PDF device and you get interpolation of raster images that are drawn by R, as described in the original motivation.
You also get the ability to address individual pixels in a raster image, which is useful for things like producing HTML image maps(?)
You also get encouraged to play with raster images in R, for example, generating simple things like colour gradients for filling bars in barplots, or fake-3D bullet points for plotting symbols (plus, you can generate any gradient that you like because you can generate any matrix that you like!).
With the GE_Cap(), you also can do some interesting things like drawing vector output to a screen device (which rasterizes it, possibly including antialiasing), then reading that image back into R and doing fun things like using it as a mask on a gradient fill (e.g., filling text with a gradient fill). This might be a really good way to generate some wild gradients too(?)
All of those, along with the ability to arrange raster images using R's layout facilities, means that you start to think of R as a playground for raster image manipulation. Probably a slow one, but possibly a more convenient one to work in if you're used to R, but not so used to things like ImageMagick.
| Device | Cap | Raster | |||
|---|---|---|---|---|---|
| scaling | interpolation | rotation | alpha | ||
| Quartz | yes | yes | yes | yes | yes |
| Cairo | yes | yes | yes**** | yes | yes |
| X11 | yes | yes | yes | yes* | NRI |
| Windows | yes | yes | yes | yes* | yes** |
| NA | yes | yes | yes | yes | |
| PostScript*** | NA | yes | yes | yes | NA |
| XFig | NA | NRI | NRI | NRI | NA |
| PicTeX | NA | NRI | NRI | NRI | NA |
* Rotation of raster images in the graphics engine introduces pixels from outside the original image in the "new" regions of the rotated image and the colour of these pixels is taken from gc->fill. This is not so bad on Windows because the background can be set to "transparent" so that these introduced regions are invisible (though there will typically still be some artifacts at the edges of the rotated image because the Windows device does not support a smooth alpha channel). On the X11 device, things are worse because images must be opaque, so the best you can do is to set gc->fill to match the background.
** The Windows device can do semitransparent raster images, but ONLY if there is a constant alpha across the entire image, i.e., it CANNOT do per-pixel alpha (the Windows API AlphaBlend can, but GraphApp's bitmap structures do not support "shades of alpha" - it's either transparent or opaque). The Windows device DOES support fully transparent pixels in an image (possibly in addition to a single level of semitransparency). In other words, the image can have up to three different levels of alpha, as long as fully transparent and fully opaque are two of the three.
*** The PostScript device can only cope with small raster images (less than about 10K pixels, e.g., 100x100). NOTE that this device also relies on the graphics engine for interpolation (currently only to 72 dpi), which may radically increase the number of pixels in the image. On the other hand, interpolating to only 72 dpi may DECREASE the number of pixels for a high-res image drawn small.
**** Full interpolation support requires Cairo version >= 1.6 (otherwise interpolation is done, much more crappily, by the graphics engine)
NYI (Not Yet Implemented)
NRI (No Real Intention to implement) means that there is no plan in the immediate future to attempt to implement this feature for this device.
NA (Not Applicable) means that it is not possible to support this feature on this device.