The R Project SVN R

% File src/library/base/man/Encoding.Rd
% Part of the R package, https://www.R-project.org
% Copyright 1995-2023 R Core Team
% Distributed under GPL 2 or later

\name{Encoding}
\alias{Encoding}
\alias{Encoding<-}
\alias{enc2native}
\alias{enc2utf8}
\concept{encoding}
\title{Read or Set the Declared Encodings for a Character Vector}
\description{
  Read or set the declared encodings for a character vector.
}
\usage{
Encoding(x)

Encoding(x) <- value

enc2native(x)
enc2utf8(x)
}
\arguments{
  \item{x}{A character vector.}
  \item{value}{A character vector of positive length.}
}
\details{
  Character strings in \R can be declared to be encoded in
  \code{"latin1"} or \code{"UTF-8"} or as \code{"bytes"}.  These
  declarations can be read by \code{Encoding}, which will return a
  character vector of values \code{"latin1"}, \code{"UTF-8"}
  \code{"bytes"} or \code{"unknown"}, or set, when \code{value} is
  recycled as needed and other values are silently treated as
  \code{"unknown"}.  ASCII strings will never be marked with a declared
  encoding, since their representation is the same in all supported
  encodings.  Strings marked as \code{"bytes"} are intended to be
  non-ASCII strings which should be manipulated as bytes, and never
  converted to a character encoding (so writing them to a text file is
  supported only by \code{writeLines(useBytes = TRUE)}).
  % non-bug report PR#16327

  \code{enc2native} and \code{enc2utf8} convert elements of character
  vectors to the native encoding or UTF-8 respectively, taking any
  marked encoding into account.  They are \link{primitive} functions,
  designed to do minimal copying.

  There are other ways for character strings to acquire a declared
  encoding apart from explicitly setting it (and these have changed as
  \R has evolved).  The parser marks strings containing \samp{\\u} or
  \samp{\\U} escapes.  Functions \code{\link{scan}},
  \code{\link{read.table}}, \code{\link{readLines}}, and
  \code{\link{parse}} have an \code{encoding} argument that is used to
  declare encodings, \code{\link{iconv}} declares encodings from its
  \code{to} argument, and console input in suitable locales is also
  declared.  \code{\link{intToUtf8}} declares its output as
  \code{"UTF-8"}, and output text connections (see
  \code{\link{textConnection}}) are marked if running in a
  suitable locale.  Under some circumstances (see its help page)
  \code{\link{source}(encoding=)} will mark encodings of character
  strings it outputs.

  Most character manipulation functions will set the encoding on output
  strings if it was declared on the corresponding input.  These include
  \code{\link{chartr}}, \code{\link{strsplit}(useBytes = FALSE)},
  \code{\link{tolower}} and \code{\link{toupper}} as well as
  \code{\link{sub}(useBytes = FALSE)} and \code{\link{gsub}(useBytes =
  FALSE)}.  Note that such functions do not \emph{preserve} the
  encoding, but if they know the input encoding and that the string has
  been successfully re-encoded (to the current encoding or UTF-8), they
  mark the output.

  \code{\link{substr}} does preserve the encoding, and
  \code{\link{chartr}}, \code{\link{tolower}} and \code{\link{toupper}}
  preserve UTF-8 encoding on systems with Unicode wide characters.  With
  their \code{fixed} and \code{perl} options, \code{\link{strsplit}},
  \code{\link{sub}} and \code{gsub} will give a marked UTF-8 result if
  any of the inputs are UTF-8.

 %% \code{\link{strsplit}}, \code{\link{sub}} 
 %%  and \code{gsub} will mark their result \code{"bytes"} if it is non-ASCII
 %%  and any of the inputs are marked \code{"bytes"} or \code{useBytes = TRUE}
 %% was used.

  \code{\link{paste}} and \code{\link{sprintf}} return elements marked
  as bytes if any of the corresponding inputs is marked as bytes, and
  otherwise marked as UTF-8 if any of the inputs is marked as UTF-8.

  \code{\link{match}}, \code{\link{pmatch}}, \code{\link{charmatch}},
  \code{\link{duplicated}} and \code{\link{unique}} all match in UTF-8
  if any of the elements are marked as UTF-8.

  Changing the current encoding from a running R session may lead to
  confusion (see \code{\link{Sys.setlocale}}).

  There is some ambiguity as to what is meant by a \sQuote{Latin-1}
  locale, since some OSes (notably Windows) make use of character
  positions undefined (or used for control characters) in the ISO 8859-1
  character set.  How such characters are interpreted is
  system-dependent but as from \R 3.5.0 they are if possible interpreted
  as per Windows codepage 1252 (which Microsoft calls \sQuote{Windows
  Latin 1 (ANSI)}) when converting to e.g.\sspace{}UTF-8.
}
\value{
  A character vector.

  For \code{enc2utf8} encodings are always marked: they are for
  \code{enc2native} in UTF-8 and Latin-1 locales.
}
\examples{
## x is intended to be in latin1
x. <- x <- "fran\xE7ais"
Encoding(x.) # "unknown" (UTF-8 loc.) | "latin1" (8859-1/CP-1252 loc.) | ....
Encoding(x) <- "latin1"
x
xx <- iconv(x, "latin1", "UTF-8")
Encoding(c(x., x, xx))
c(x, xx)
xb <- xx; Encoding(xb) <- "bytes"
xb # will be encoded in hex
cat("x = ", x, ", xx = ", xx, ", xb = ", xb, "\n", sep = "")
(Ex <- Encoding(c(x.,x,xx,xb)))
stopifnot(identical(Ex, c(Encoding(x.), Encoding(x),
                          Encoding(xx), Encoding(xb))))
}
\keyword{utilities}
\keyword{character}