The R Project SVN R-packages

Private Maintainer -- TODO              -*- markdown -*-

A much smaller _public TODO_ is part of the file `./README`
Things done are moved to `./DONE-MM`

### `clusGap()`

* currently fails [in `svd()`] when `x` has any NA's

* clusGap() : now have `"original"` in addition to PCA-rotation with
   `scaleH0` string.  In addition should provide `H0gen = a RNG *function*`:
   Chris Fields (in March 2012) had proposed to alternatively allow uniform
   on a n-simplex instead of an n-cube.
    ->  `~/R/MM/Pkg-ex/cluster/clusGap-ChrisField-thoughts`

   ==> Master Thesis of Emmanuel Profumo (~/Betreute-Arbeiten/Profumo_Emmanuel/)
   has a __generalized__ clusGap()
   --> added but not yet {exported+documented} >>> R/clusGapGen.R
   NB: See more in
   ~/R/MM/Pkg-ex/cluster/clusGap/clusGapGen/            notably
   ~/R/MM/Pkg-ex/cluster/clusGap/clusGapGen/README.md
   --------------------------------------------------

* clusGap `print()` method: note that Tibshirani et al. proposed
  see their large data example (!) a different method than the one we
  implemnt currently --> provide both(!)

### `clara()`

* clara() to work with *daisy*-like distances, not just L2 & L1.

   -- Possibility: using stats package C code
       `~/R/D/r-devel/R/src/library/stats/src/distance.c`
      does not have Gower, but a few more kinds of distance methods.

* clara() - bug?: tests/clara.Rout.save.~13~ (and ~12~ , ~11~ ...)
          gives a clearly better result
          clara(ru4, k=3, met="manhattan", sampsize=4)
          than current (later) clara.Rout  files...
     -> bug since 'Mar 11 2004' (= cluster-1.[89].1 for R 1.[89]) ??
     FIXME ?!?!?!

* It should be possible to _re_start `clara()` with a **given** "best sample"

* Have introduced  as.data.frame(<silhouette>) via `S3method()` in `NAMESPACE` + 
  copy (from base) of `as.data.frame.matrix`. 
  Nicer (but less back compatible ==> need rev.dep checks!) 
  would have been to change the *class* to
  `c("silhouette", class(matrix())` such that all matrix (and array)
  methods would work with silhouette.

* `silhouette.clara()`

    - silhouette(*, full=TRUE):
     Allow option of *NOT* pass full-size length n(n-1)/2 dist object but compute
     `daisy()`-like distance *on the fly*  inside `src/sildist.c`
     ==> still needs O(n^2) CPU-effort, but not O(n^2) memory
      { which is even not *possible*, e.g., for n=70'000 }

    - Consider `silhouette(*, full = 0.50)1 to compute silhouette for a (random ?)
      subset of {0.50 * n} of the observations

*    - `plot.silhouette()` : if an observation's width is == 0., draw a small stripe
      instead of nothing at all

### `pam()` (and also `clara()`)

* clara() got metric="jaccard" (donated in 2018) ==> copy to pam(), too !!
        well: actually that is *BUGGY* 

* pam() and clara(): Should be possible to "re"start with GIVEN medoids.
             now possible for pam(), not for clara();
             --> "synchronize src/pam.c and src/clara.c in
             particular bswap() vs bswap2() !!
* pam() and clara(): With NA's, medoids "often" contain NAs even when there
             are only few NAs. ==> use modified d(.,.) which make
             NAs "bad" somehow.

* pamila(): Major smart idea: Do save the d(i,j) i=1,..k  j=1,..n
        only between *medoids* and everything else -- speedup(?) -> optional
* R/agnes.q , R/diana.q  and R/pam.q  have almost identical clause

    if(data.class(x) != "dissimilarity") {
        if(!is.numeric(x) || is.na(sizeDiss(x)))
        stop("x is not of class dissimilarity and can not be converted to this class." )
        ## convert input vector to class "dissimilarity"
        class(x) <- ..dClass
        attr(x, "Size") <- sizeDiss(x)
        attr(x, "Metric") <- "unspecified"
    }

  which can be modularized out into a NAMESPACE-local fixupDiss() function

  [ agnes() and diana() even more in common --> namespace-local functions!
    see also  "8b)" below!]

### diana()  {divisive hierarchical}:

* Should allow __early stopping__
  (for speed and size) -- simultaneously, could think of
  _`diss()` on the fly_ instead of diss() matrix,
  but see `./src/NOTES-MM`  (and `pamila()`) !


Dec. 2002:

  o clara(ruspini, 4)  BUG  in clara.c (see below)
    -- worked fine in cluster-1.5.2 (with clara.f!)
    -- gives error in   "     1.6.1 [and later]   ==== AARGH
    (the problem is *not* an integer/double one, here!)
    Status 28.Dec.2002:
     - The August-2002 fortran code doesn't seem to have a problem
     ==> ~/R/Pkgs/TMP/cluster/
     - The F2C code (called via .Fortran()) seems the same
     ==> ~/R/Pkgs/T_F2C/cluster/
     - A very slight change of the F2C code (using .C())
       has one problem but not all of the "modern" C version
     ==> ~/R/Pkgs/T_F2C-2/cluster/

    Fixed most of the above 2002-12-28 _late_ -- still one small problem!
    but it seems clear this was even in early clara.f (at least, final
    result is the same for that example)  src/clara.c << needs more

  o diana(ruspini) --> ok (again)

  o     bannerplot() is now `standalone' and has a help, man/bannerplot.Rd .
    HOWEVER it's "details" are found in  man/plot.agnes.Rd (and ???)
        instead --> centralize this info (and keep short ref.s in the man/plot.*

  o agnes() and hclust() should be merged {and based on C, not Fortran}
  o agnes() for large objects needs TWICE the time of hclust();
    both need MUCH MORE time than hcluster() in pkg 'amap',
    which is said to be the same as 'hclust' but just only
    malloc()ating the "huge" dissimilarities inside C.
    --> translate agnes, i.e.  src/twins.f  to C


July 2002:

  o Idea for new functionality :
    e.g., pamila() := PAM In Large Application
    should not *save* dissimilarities but rather re-compute them on
    the fly --> save huge storage
    ==> should give identical results but be faster for larger n,
    or at least feasible for n = 10'000 or so where it currently aint.


June 2002:

  mona() :  I think it should be possible to write an
[       as.hclust.mona()  or as.twins.mona() method
        and hence also draw a dendrogram of a mona object.

Jan. 2002:

  clusellipses() ``like part of clusplot'' for *adding* ellipses to plot;
         maybe do this with "add = TRUE, plotchar = !add, labels = 0


May 23, 2001 / Jan.2002 :
-------------------------
  I found problems with missing values /  NAs treatment :

  o Also, I'm not sure if the NAs are dealt with sensibly
    in clara() :  The result changes too much with very few NAs

  o --> look at all the  subroutine dysta*() s in  src/*.f
    Clean these up and merge in one single!
     Aug.02: partly done -- fanny() is different than others.

  In the future: When "mva" will have a C API, use  dist()'s C function!


7) Get rid of the many  \section{GENERATION}, {METHODS} and {INHERITANCE}
   sections  in man/*.Rd
   -- make sure that info is available, at least partially otherwise.

6a) The \references{} mostly contain the same things.
   man/plot.agnes.Rd has some of them nicely.
   Collect in a few places (*.Rd files), and refer to these  {partly ok}

 b) Similarly for the \section{BACKGROUND} which appears in quite a few
   *.Rd files.

    --> done partly ( ./ChangeLog 2002-01-24 )

8b) Think about "merging" the plot.agnes and plot.diana methods.

------------------------------------------------

Older TODO
==========
             (were in `README_MM` which is now eliminated)

 3) daisy() for the case of mixed variables should allow
    a weight vector (of length p = #vars) for up- or downweighing variables.
    daisy() really should accept the other methods mva's dist() does _and_
    it should use dist's C API -- but we have no C API for package code, ARRGH!

 4) Eliminate the many Fortran (g77 -Wall) warnings of the form
    >> mona.f:101: warning: `jma' might be used uninitialized in this function

---------

9) man/daisy.Rd  should mention 'Gower (1971)' ;
   mention that Kaufman & Rousseeuw *generalize* this;
   and probably show the full formula from Kauf+Rouss p.35

10) Implement the plot for "fuzzy cluster membership" of section 5.4,
    from Kauf+Rouss p. 195 ff :
    I.e. PCA of the membership matrix for the points + "the pure clusters"
   <---> Export  as.membership() and toCrisp()  in  ./R/fanny.q