The R Project SVN R

Rev

Details | Last modification | View Log | RSS feed

Rev Author Line No. Line
10347 ripley 1
/*
26855 ripley 2
  This is a version (aka dlmalloc) of malloc/free/realloc written by
3
  Doug Lea and released to the public domain.  Use, modify, and
4
  redistribute this code without permission or acknowledgement in any
5
  way you wish.  Send questions, comments, complaints, performance
6
  data, etc to dl@cs.oswego.edu
10347 ripley 7
 
26855 ripley 8
* VERSION 2.7.2 Sat Aug 17 09:07:30 2002  Doug Lea  (dl at gee)
10347 ripley 9
 
10
   Note: There may be an updated version of this malloc obtainable at
26855 ripley 11
           ftp://gee.cs.oswego.edu/pub/misc/malloc.c
10347 ripley 12
         Check before installing!
13
 
26855 ripley 14
* Quickstart
15
 
16
  This library is all in one file to simplify the most common usage:
17
  ftp it, compile it (-O), and link it into another program. All
18
  of the compile-time options default to reasonable values for use on
19
  most unix platforms. Compile -DWIN32 for reasonable defaults on windows.
20
  You might later want to step through various compile-time and dynamic
21
  tuning options.
22
 
23
  For convenience, an include file for code using this malloc is at:
24
     ftp://gee.cs.oswego.edu/pub/misc/malloc-2.7.1.h
25
  You don't really need this .h file unless you call functions not
26
  defined in your system include files.  The .h file contains only the
27
  excerpts from this file needed for using this malloc on ANSI C/C++
28
  systems, so long as you haven't changed compile-time options about
29
  naming and tuning parameters.  If you do, then you can create your
30
  own malloc.h that does include all settings by cutting at the point
31
  indicated below.
32
 
10347 ripley 33
* Why use this malloc?
34
 
35
  This is not the fastest, most space-conserving, most portable, or
36
  most tunable malloc ever written. However it is among the fastest
37
  while also being among the most space-conserving, portable and tunable.
38
  Consistent balance across these factors results in a good general-purpose
26855 ripley 39
  allocator for malloc-intensive programs.
10347 ripley 40
 
26855 ripley 41
  The main properties of the algorithms are:
42
  * For large (>= 512 bytes) requests, it is a pure best-fit allocator,
43
    with ties normally decided via FIFO (i.e. least recently used).
44
  * For small (<= 64 bytes by default) requests, it is a caching
45
    allocator, that maintains pools of quickly recycled chunks.
46
  * In between, and for combinations of large and small requests, it does
47
    the best it can trying to meet both goals at once.
48
  * For very large requests (>= 128KB by default), it relies on system
49
    memory mapping facilities, if supported.
10347 ripley 50
 
26855 ripley 51
  For a longer but slightly out of date high-level description, see
52
     http://gee.cs.oswego.edu/dl/html/malloc.html
10347 ripley 53
 
26855 ripley 54
  You may already by default be using a C library containing a malloc
55
  that is  based on some version of this malloc (for example in
56
  linux). You might still want to use the one in this file in order to
57
  customize settings or to avoid overheads associated with library
58
  versions.
10347 ripley 59
 
26855 ripley 60
* Contents, described in more detail in "description of public routines" below.
10347 ripley 61
 
26855 ripley 62
  Standard (ANSI/SVID/...)  functions:
63
    malloc(size_t n);
64
    calloc(size_t n_elements, size_t element_size);
65
    free(Void_t* p);
66
    realloc(Void_t* p, size_t n);
67
    memalign(size_t alignment, size_t n);
68
    valloc(size_t n);
69
    mallinfo()
70
    mallopt(int parameter_number, int parameter_value)
10347 ripley 71
 
26855 ripley 72
  Additional functions:
73
    independent_calloc(size_t n_elements, size_t size, Void_t* chunks[]);
74
    independent_comalloc(size_t n_elements, size_t sizes[], Void_t* chunks[]);
75
    pvalloc(size_t n);
76
    cfree(Void_t* p);
77
    malloc_trim(size_t pad);
78
    malloc_usable_size(Void_t* p);
79
    malloc_stats();
10347 ripley 80
 
26855 ripley 81
* Vital statistics:
82
 
83
  Supported pointer representation:       4 or 8 bytes
84
  Supported size_t  representation:       4 or 8 bytes 
10347 ripley 85
       Note that size_t is allowed to be 4 bytes even if pointers are 8.
26855 ripley 86
       You can adjust this by defining INTERNAL_SIZE_T
10347 ripley 87
 
26855 ripley 88
  Alignment:                              2 * sizeof(size_t) (default)
89
       (i.e., 8 byte alignment with 4byte size_t). This suffices for
90
       nearly all current machines and C compilers. However, you can
91
       define MALLOC_ALIGNMENT to be wider than this if necessary.
92
 
93
  Minimum overhead per allocated chunk:   4 or 8 bytes
94
       Each malloced chunk has a hidden word of overhead holding size
10347 ripley 95
       and status information.
96
 
97
  Minimum allocated size: 4-byte ptrs:  16 bytes    (including 4 overhead)
98
                          8-byte ptrs:  24/32 bytes (including, 4/8 overhead)
99
 
100
       When a chunk is freed, 12 (for 4byte ptrs) or 20 (for 8 byte
101
       ptrs but 4 byte size) or 24 (for 8/8) additional bytes are
26855 ripley 102
       needed; 4 (8) for a trailing size field and 8 (16) bytes for
103
       free list pointers. Thus, the minimum allocatable size is
104
       16/24/32 bytes.
10347 ripley 105
 
106
       Even a request for zero bytes (i.e., malloc(0)) returns a
107
       pointer to something of the minimum allocatable size.
108
 
26855 ripley 109
       The maximum overhead wastage (i.e., number of extra bytes
110
       allocated than were requested in malloc) is less than or equal
111
       to the minimum size, except for requests >= mmap_threshold that
112
       are serviced via mmap(), where the worst case wastage is 2 *
113
       sizeof(size_t) bytes plus the remainder from a system page (the
114
       minimal mmap unit); typically 4096 or 8192 bytes.
10347 ripley 115
 
26855 ripley 116
  Maximum allocated size:  4-byte size_t: 2^32 minus about two pages 
117
                           8-byte size_t: 2^64 minus about two pages
118
 
119
       It is assumed that (possibly signed) size_t values suffice to
10347 ripley 120
       represent chunk sizes. `Possibly signed' is due to the fact
121
       that `size_t' may be defined on a system as either a signed or
26855 ripley 122
       an unsigned type. The ISO C standard says that it must be
123
       unsigned, but a few systems are known not to adhere to this.
124
       Additionally, even when size_t is unsigned, sbrk (which is by
125
       default used to obtain memory from system) accepts signed
126
       arguments, and may not be able to handle size_t-wide arguments
127
       with negative sign bit.  Generally, values that would
128
       appear as negative after accounting for overhead and alignment
129
       are supported only via mmap(), which does not have this
130
       limitation.
10347 ripley 131
 
26855 ripley 132
       Requests for sizes outside the allowed range will perform an optional
133
       failure action and then return null. (Requests may also
134
       also fail because a system is out of memory.)
10347 ripley 135
 
26855 ripley 136
  Thread-safety: NOT thread-safe unless USE_MALLOC_LOCK defined
10347 ripley 137
 
26855 ripley 138
       When USE_MALLOC_LOCK is defined, wrappers are created to
139
       surround every public call with either a pthread mutex or
140
       a win32 spinlock (depending on WIN32). This is not
141
       especially fast, and can be a major bottleneck.
142
       It is designed only to provide minimal protection
143
       in concurrent environments, and to provide a basis for
144
       extensions.  If you are using malloc in a concurrent program,
145
       you would be far better off obtaining ptmalloc, which is
146
       derived from a version of this malloc, and is well-tuned for
147
       concurrent programs. (See http://www.malloc.de) Note that
148
       even when USE_MALLOC_LOCK is defined, you can can guarantee
149
       full thread-safety only if no threads acquire memory through 
150
       direct calls to MORECORE or other system-level allocators.
10347 ripley 151
 
26855 ripley 152
  Compliance: I believe it is compliant with the 1997 Single Unix Specification
153
       (See http://www.opennc.org). Also SVID/XPG, ANSI C, and probably 
154
       others as well.
10347 ripley 155
 
156
* Synopsis of compile-time options:
157
 
158
    People have reported using previous versions of this malloc on all
159
    versions of Unix, sometimes by tweaking some of the defines
160
    below. It has been tested most extensively on Solaris and
161
    Linux. It is also reported to work on WIN32 platforms.
26855 ripley 162
    People also report using it in stand-alone embedded systems.
10347 ripley 163
 
26855 ripley 164
    The implementation is in straight, hand-tuned ANSI C.  It is not
165
    at all modular. (Sorry!)  It uses a lot of macros.  To be at all
166
    usable, this code should be compiled using an optimizing compiler
167
    (for example gcc -O3) that can simplify expressions and control
168
    paths. (FAQ: some macros import variables as arguments rather than
169
    declare locals because people reported that some debuggers
170
    otherwise get confused.)
10347 ripley 171
 
26855 ripley 172
    OPTION                     DEFAULT VALUE
10347 ripley 173
 
26855 ripley 174
    Compilation Environment options:
10347 ripley 175
 
26855 ripley 176
    __STD_C                    derived from C compiler defines
177
    WIN32                      NOT defined
178
    HAVE_MEMCPY                defined
179
    USE_MEMCPY                 1 if HAVE_MEMCPY is defined
180
    HAVE_MMAP                  defined as 1 
181
    MMAP_CLEARS                1
182
    HAVE_MREMAP                0 unless linux defined
183
    malloc_getpagesize         derived from system #includes, or 4096 if not
184
    HAVE_USR_INCLUDE_MALLOC_H  NOT defined
185
    LACKS_UNISTD_H             NOT defined unless WIN32
186
    LACKS_SYS_PARAM_H          NOT defined unless WIN32
187
    LACKS_SYS_MMAN_H           NOT defined unless WIN32
188
    LACKS_FCNTL_H              NOT defined
189
 
190
    Changing default word sizes:
191
 
192
    INTERNAL_SIZE_T            size_t
193
    MALLOC_ALIGNMENT           2 * sizeof(INTERNAL_SIZE_T)
194
    PTR_UINT                   unsigned long
195
    CHUNK_SIZE_T               unsigned long
196
 
197
    Configuration and functionality options:
198
 
199
    USE_DL_PREFIX              NOT defined
200
    USE_PUBLIC_MALLOC_WRAPPERS NOT defined
201
    USE_MALLOC_LOCK            NOT defined
202
    DEBUG                      NOT defined
203
    REALLOC_ZERO_BYTES_FREES   NOT defined
204
    MALLOC_FAILURE_ACTION      errno = ENOMEM, if __STD_C defined, else no-op
205
    TRIM_FASTBINS              0
206
    FIRST_SORTED_BIN_SIZE      512
207
 
208
    Options for customizing MORECORE:
209
 
210
    MORECORE                   sbrk
211
    MORECORE_CONTIGUOUS        1 
212
    MORECORE_CANNOT_TRIM       NOT defined
213
    MMAP_AS_MORECORE_SIZE      (1024 * 1024) 
214
 
215
    Tuning options that are also dynamically changeable via mallopt:
216
 
217
    DEFAULT_MXFAST             64
218
    DEFAULT_TRIM_THRESHOLD     256 * 1024
219
    DEFAULT_TOP_PAD            0
220
    DEFAULT_MMAP_THRESHOLD     256 * 1024
221
    DEFAULT_MMAP_MAX           65536
222
 
223
    There are several other #defined constants and macros that you
224
    probably don't want to touch unless you are extending or adapting malloc.
10347 ripley 225
*/
226
 
26855 ripley 227
/*
228
  WIN32 sets up defaults for MS environment and compilers.
229
  Otherwise defaults are for unix.
230
*/
10347 ripley 231
 
26855 ripley 232
/* #define WIN32 */
10347 ripley 233
 
26855 ripley 234
#ifdef WIN32
10347 ripley 235
 
26855 ripley 236
#define WIN32_LEAN_AND_MEAN
237
#include <windows.h>
238
 
239
/* Win32 doesn't supply or need the following headers */
240
/*#define LACKS_UNISTD_H*/
241
#define LACKS_SYS_PARAM_H
242
#define LACKS_SYS_MMAN_H
243
 
244
/* Use the supplied emulation of sbrk */
245
#define MORECORE sbrk
246
#define MORECORE_CONTIGUOUS 0
247
#define MORECORE_FAILURE    ((void*)(-1))
248
 
249
/* Use the supplied emulation of mmap and munmap */
250
#define HAVE_MMAP 1
251
#define MUNMAP_FAILURE  (-1)
252
#define MMAP_CLEARS 1
253
 
254
/* These values don't really matter in windows mmap emulation */
255
#define MAP_PRIVATE 1
256
#define MAP_ANONYMOUS 2
257
#define PROT_READ 1
258
#define PROT_WRITE 2
259
 
260
/* Emulation functions defined at the end of this file */
261
 
262
/* If USE_MALLOC_LOCK, use supplied critical-section-based lock functions */
263
#ifdef USE_MALLOC_LOCK
264
static int slwait(int *sl);
265
static int slrelease(int *sl);
266
#endif
267
 
268
static long getpagesize(void);
269
static long getregionsize(void);
270
static void *sbrk(long size);
271
static void *mmap(void *ptr, long size, long prot, long type, long handle, long arg);
272
static long munmap(void *ptr, long size);
273
 
274
static void vminfo (unsigned long*free, unsigned long*reserved, unsigned long*committed);
26945 ripley 275
/* Commented out for R incompatibility
26855 ripley 276
static int cpuinfo (int whole, unsigned long*kernel, unsigned long*user);
277
*/
278
 
279
#endif
280
 
281
 
10347 ripley 282
 
26855 ripley 283
/*
284
  Void_t* is the pointer type that malloc should say it returns
285
*/
286
 
10347 ripley 287
#ifndef Void_t
288
#define Void_t      void
289
#endif /*Void_t*/
290
 
291
#include <stddef.h>   /* for size_t */
292
 
293
#ifdef __cplusplus
294
extern "C" {
295
#endif
296
 
26855 ripley 297
/* define LACKS_UNISTD_H if your system does not have a <unistd.h>. */
10347 ripley 298
 
26855 ripley 299
/* #define  LACKS_UNISTD_H */
10347 ripley 300
 
26855 ripley 301
#ifndef LACKS_UNISTD_H
302
#include <unistd.h>
303
#endif
10347 ripley 304
 
26855 ripley 305
/* define LACKS_SYS_PARAM_H if your system does not have a <sys/param.h>. */
10347 ripley 306
 
26855 ripley 307
/* #define  LACKS_SYS_PARAM_H */
308
 
309
 
310
#include <stdio.h>    /* needed for malloc_stats */
311
#include <errno.h>    /* needed for optional MALLOC_FAILURE_ACTION */
312
 
313
 
10347 ripley 314
/*
26855 ripley 315
  Debugging:
10347 ripley 316
 
26855 ripley 317
  Because freed chunks may be overwritten with bookkeeping fields, this
318
  malloc will often die when freed memory is overwritten by user
319
  programs.  This can be very effective (albeit in an annoying way)
320
  in helping track down dangling pointers.
10347 ripley 321
 
26855 ripley 322
  If you compile with -DDEBUG, a number of assertion checks are
323
  enabled that will catch more memory errors. You probably won't be
324
  able to make much sense of the actual assertion errors, but they
325
  should help you locate incorrectly overwritten memory.  The
326
  checking is fairly extensive, and will slow down execution
327
  noticeably. Calling malloc_stats or mallinfo with DEBUG set will
328
  attempt to check every non-mmapped allocated and free chunk in the
329
  course of computing the summmaries. (By nature, mmapped regions
330
  cannot be checked very much automatically.)
10347 ripley 331
 
26855 ripley 332
  Setting DEBUG may also be helpful if you are trying to modify
333
  this code. The assertions in the check routines spell out in more
334
  detail the assumptions and invariants underlying the algorithms.
10347 ripley 335
 
26855 ripley 336
  Setting DEBUG does NOT provide an automated mechanism for checking
337
  that all accesses to malloced memory stay within their
338
  bounds. However, there are several add-ons and adaptations of this
339
  or other mallocs available that do this.
10347 ripley 340
*/
341
 
26855 ripley 342
/* #define DEBUG 1 */
343
 
10347 ripley 344
#if DEBUG
345
#include <assert.h>
346
#else
347
#define assert(x) ((void)0)
348
#endif
349
 
26855 ripley 350
/*
351
  The unsigned integer type used for comparing any two chunk sizes.
352
  This should be at least as wide as size_t, but should not be signed.
353
*/
10347 ripley 354
 
26855 ripley 355
#ifndef CHUNK_SIZE_T
356
#define CHUNK_SIZE_T unsigned long
357
#endif
358
 
359
/* 
360
  The unsigned integer type used to hold addresses when they are are
361
  manipulated as integers. Except that it is not defined on all
362
  systems, intptr_t would suffice.
363
*/
364
#ifndef PTR_UINT
365
#define PTR_UINT unsigned long
366
#endif
367
 
368
 
10347 ripley 369
/*
370
  INTERNAL_SIZE_T is the word-size used for internal bookkeeping
26855 ripley 371
  of chunk sizes.
372
 
373
  The default version is the same as size_t.
374
 
375
  While not strictly necessary, it is best to define this as an
376
  unsigned type, even if size_t is a signed type. This may avoid some
377
  artificial size limitations on some systems.
378
 
379
  On a 64-bit machine, you may be able to reduce malloc overhead by
380
  defining INTERNAL_SIZE_T to be a 32 bit `unsigned int' at the
381
  expense of not being able to handle more than 2^32 of malloced
382
  space. If this limitation is acceptable, you are encouraged to set
383
  this unless you are on a platform requiring 16byte alignments. In
384
  this case the alignment requirements turn out to negate any
385
  potential advantages of decreasing size_t word size.
386
 
387
  Implementors: Beware of the possible combinations of:
388
     - INTERNAL_SIZE_T might be signed or unsigned, might be 32 or 64 bits,
389
       and might be the same width as int or as long
390
     - size_t might have different width and signedness as INTERNAL_SIZE_T
391
     - int and long might be 32 or 64 bits, and might be the same width
392
  To deal with this, most comparisons and difference computations
393
  among INTERNAL_SIZE_Ts should cast them to CHUNK_SIZE_T, being
394
  aware of the fact that casting an unsigned int to a wider long does
395
  not sign-extend. (This also makes checking for negative numbers
396
  awkward.) Some of these casts result in harmless compiler warnings
397
  on some systems.
10347 ripley 398
*/
399
 
400
#ifndef INTERNAL_SIZE_T
401
#define INTERNAL_SIZE_T size_t
402
#endif
403
 
26855 ripley 404
/* The corresponding word size */
405
#define SIZE_SZ                (sizeof(INTERNAL_SIZE_T))
406
 
407
 
408
 
10347 ripley 409
/*
26855 ripley 410
  MALLOC_ALIGNMENT is the minimum alignment for malloc'ed chunks.
411
  It must be a power of two at least 2 * SIZE_SZ, even on machines
412
  for which smaller alignments would suffice. It may be defined as
413
  larger than this though. Note however that code and data structures
414
  are optimized for the case of 8-byte alignment.
415
*/
416
 
417
 
418
#ifndef MALLOC_ALIGNMENT
419
#define MALLOC_ALIGNMENT       (2 * SIZE_SZ)
420
#endif
421
 
422
/* The corresponding bit mask value */
423
#define MALLOC_ALIGN_MASK      (MALLOC_ALIGNMENT - 1)
424
 
425
 
426
 
427
/*
10347 ripley 428
  REALLOC_ZERO_BYTES_FREES should be set if a call to
429
  realloc with zero bytes should be the same as a call to free.
430
  Some people think it should. Otherwise, since this malloc
431
  returns a unique pointer for malloc(0), so does realloc(p, 0).
432
*/
433
 
434
/*   #define REALLOC_ZERO_BYTES_FREES */
435
 
26855 ripley 436
/*
437
  TRIM_FASTBINS controls whether free() of a very small chunk can
438
  immediately lead to trimming. Setting to true (1) can reduce memory
439
  footprint, but will almost always slow down programs that use a lot
440
  of small chunks.
10347 ripley 441
 
26855 ripley 442
  Define this only if you are willing to give up some speed to more
443
  aggressively reduce system-level memory footprint when releasing
444
  memory in programs that use many small chunks.  You can get
445
  essentially the same effect by setting MXFAST to 0, but this can
446
  lead to even greater slowdowns in programs using many small chunks.
447
  TRIM_FASTBINS is an in-between compile-time option, that disables
448
  only those chunks bordering topmost memory from being placed in
449
  fastbins.
450
*/
451
 
452
#ifndef TRIM_FASTBINS
453
#define TRIM_FASTBINS  0
454
#endif
455
 
456
 
10347 ripley 457
/*
26855 ripley 458
  USE_DL_PREFIX will prefix all public routines with the string 'dl'.
459
  This is necessary when you only want to use this malloc in one part 
460
  of a program, using your regular system malloc elsewhere.
10347 ripley 461
*/
462
 
26855 ripley 463
/* #define USE_DL_PREFIX */
10425 ripley 464
 
465
 
466
/*
26855 ripley 467
  USE_MALLOC_LOCK causes wrapper functions to surround each
468
  callable routine with pthread mutex lock/unlock.
10425 ripley 469
 
26855 ripley 470
  USE_MALLOC_LOCK forces USE_PUBLIC_MALLOC_WRAPPERS to be defined
10425 ripley 471
*/
26855 ripley 472
 
473
 
474
/* #define USE_MALLOC_LOCK */
475
 
476
 
477
/*
478
  If USE_PUBLIC_MALLOC_WRAPPERS is defined, every public routine is
479
  actually a wrapper function that first calls MALLOC_PREACTION, then
480
  calls the internal routine, and follows it with
481
  MALLOC_POSTACTION. This is needed for locking, but you can also use
482
  this, without USE_MALLOC_LOCK, for purposes of interception,
483
  instrumentation, etc. It is a sad fact that using wrappers often
484
  noticeably degrades performance of malloc-intensive programs.
485
*/
486
 
487
#ifdef USE_MALLOC_LOCK
488
#define USE_PUBLIC_MALLOC_WRAPPERS
489
#else
490
/* #define USE_PUBLIC_MALLOC_WRAPPERS */
10347 ripley 491
#endif
492
 
493
 
26855 ripley 494
/* 
495
   Two-phase name translation.
496
   All of the actual routines are given mangled names.
497
   When wrappers are used, they become the public callable versions.
498
   When DL_PREFIX is used, the callable names are prefixed.
499
*/
500
 
501
#ifndef USE_PUBLIC_MALLOC_WRAPPERS
502
#define cALLOc      public_cALLOc
503
#define fREe        public_fREe
504
#define cFREe       public_cFREe
505
#define mALLOc      public_mALLOc
506
#define mEMALIGn    public_mEMALIGn
507
#define rEALLOc     public_rEALLOc
508
#define vALLOc      public_vALLOc
509
#define pVALLOc     public_pVALLOc
510
#define mALLINFo    public_mALLINFo
511
#define mALLOPt     public_mALLOPt
512
#define mTRIm       public_mTRIm
513
#define mSTATs      public_mSTATs
514
#define mUSABLe     public_mUSABLe
515
#define iCALLOc     public_iCALLOc
516
#define iCOMALLOc   public_iCOMALLOc
517
#endif
518
 
519
#ifdef USE_DL_PREFIX
520
#define public_cALLOc    dlcalloc
521
#define public_fREe      dlfree
522
#define public_cFREe     dlcfree
523
#define public_mALLOc    dlmalloc
524
#define public_mEMALIGn  dlmemalign
525
#define public_rEALLOc   dlrealloc
526
#define public_vALLOc    dlvalloc
527
#define public_pVALLOc   dlpvalloc
528
#define public_mALLINFo  dlmallinfo
529
#define public_mALLOPt   dlmallopt
530
#define public_mTRIm     dlmalloc_trim
531
#define public_mSTATs    dlmalloc_stats
532
#define public_mUSABLe   dlmalloc_usable_size
533
#define public_iCALLOc   dlindependent_calloc
534
#define public_iCOMALLOc dlindependent_comalloc
535
#else /* USE_DL_PREFIX */
26879 ripley 536
#define public_cALLOc    Rm_calloc
537
#define public_fREe      Rm_free
538
#define public_cFREe     Rm_cfree
539
#define public_mALLOc    Rm_malloc
540
#define public_mEMALIGn  Rm_memalign
541
#define public_rEALLOc   Rm_realloc
542
#define public_vALLOc    Rm_valloc
543
#define public_pVALLOc   Rm_pvalloc
26855 ripley 544
#define public_mALLINFo  mallinfo
545
#define public_mALLOPt   mallopt
546
#define public_mTRIm     malloc_trim
547
#define public_mSTATs    malloc_stats
548
#define public_mUSABLe   malloc_usable_size
549
#define public_iCALLOc   independent_calloc
550
#define public_iCOMALLOc independent_comalloc
551
#endif /* USE_DL_PREFIX */
552
 
553
 
10347 ripley 554
/*
555
  HAVE_MEMCPY should be defined if you are not otherwise using
556
  ANSI STD C, but still have memcpy and memset in your C library
557
  and want to use them in calloc and realloc. Otherwise simple
26855 ripley 558
  macro versions are defined below.
10347 ripley 559
 
560
  USE_MEMCPY should be defined as 1 if you actually want to
561
  have memset and memcpy called. People report that the macro
26855 ripley 562
  versions are faster than libc versions on some systems.
563
 
564
  Even if USE_MEMCPY is set to 1, loops to copy/clear small chunks
565
  (of <= 36 bytes) are manually unrolled in realloc and calloc.
10347 ripley 566
*/
567
 
568
#define HAVE_MEMCPY
569
 
570
#ifndef USE_MEMCPY
571
#ifdef HAVE_MEMCPY
572
#define USE_MEMCPY 1
573
#else
574
#define USE_MEMCPY 0
575
#endif
576
#endif
577
 
26855 ripley 578
 
29208 ripley 579
#if (defined(HAVE_MEMCPY))
26855 ripley 580
#ifdef WIN32
581
/* On Win32 memset and memcpy are already declared in windows.h */
582
#else
10347 ripley 583
void* memset(void*, int, size_t);
584
void* memcpy(void*, const void*, size_t);
585
#endif
586
#endif
587
 
26855 ripley 588
/*
589
  MALLOC_FAILURE_ACTION is the action to take before "return 0" when
590
  malloc fails to be able to return memory, either because memory is
591
  exhausted or because of illegal arguments.
592
 
593
  By default, sets errno if running on STD_C platform, else does nothing.  
594
*/
10347 ripley 595
 
26855 ripley 596
#ifndef MALLOC_FAILURE_ACTION
597
#define MALLOC_FAILURE_ACTION \
598
   errno = ENOMEM;
599
#endif
10347 ripley 600
 
26855 ripley 601
/*
602
  MORECORE-related declarations. By default, rely on sbrk
603
*/
10347 ripley 604
 
605
 
26855 ripley 606
#ifdef LACKS_UNISTD_H
607
#if !defined(__FreeBSD__) && !defined(__OpenBSD__) && !defined(__NetBSD__)
608
extern Void_t*     sbrk(ptrdiff_t);
609
#endif
610
#endif
10347 ripley 611
 
26855 ripley 612
/*
613
  MORECORE is the name of the routine to call to obtain more memory
614
  from the system.  See below for general guidance on writing
615
  alternative MORECORE functions, as well as a version for WIN32 and a
616
  sample version for pre-OSX macos.
617
*/
10347 ripley 618
 
26855 ripley 619
#ifndef MORECORE
620
#define MORECORE sbrk
621
#endif
10347 ripley 622
 
26855 ripley 623
/*
624
  MORECORE_FAILURE is the value returned upon failure of MORECORE
625
  as well as mmap. Since it cannot be an otherwise valid memory address,
626
  and must reflect values of standard sys calls, you probably ought not
627
  try to redefine it.
628
*/
629
 
630
#ifndef MORECORE_FAILURE
631
#define MORECORE_FAILURE (-1)
10347 ripley 632
#endif
633
 
26855 ripley 634
/*
635
  If MORECORE_CONTIGUOUS is true, take advantage of fact that
636
  consecutive calls to MORECORE with positive arguments always return
637
  contiguous increasing addresses.  This is true of unix sbrk.  Even
638
  if not defined, when regions happen to be contiguous, malloc will
639
  permit allocations spanning regions obtained from different
640
  calls. But defining this when applicable enables some stronger
641
  consistency checks and space efficiencies. 
642
*/
10347 ripley 643
 
26855 ripley 644
#ifndef MORECORE_CONTIGUOUS
645
#define MORECORE_CONTIGUOUS 1
646
#endif
647
 
10347 ripley 648
/*
26855 ripley 649
  Define MORECORE_CANNOT_TRIM if your version of MORECORE
650
  cannot release space back to the system when given negative
651
  arguments. This is generally necessary only if you are using
652
  a hand-crafted MORECORE function that cannot handle negative arguments.
653
*/
654
 
26945 ripley 655
/* #define MORECORE_CANNOT_TRIM */
26855 ripley 656
 
657
 
658
/*
659
  Define HAVE_MMAP as true to optionally make malloc() use mmap() to
10347 ripley 660
  allocate very large blocks.  These will be returned to the
26855 ripley 661
  operating system immediately after a free(). Also, if mmap
662
  is available, it is used as a backup strategy in cases where
663
  MORECORE fails to provide space from system.
664
 
665
  This malloc is best tuned to work with mmap for large requests.
666
  If you do not have mmap, operations involving very large chunks (1MB
667
  or so) may be slower than you'd like.
10347 ripley 668
*/
669
 
670
#ifndef HAVE_MMAP
671
#define HAVE_MMAP 1
672
#endif
673
 
26855 ripley 674
#if HAVE_MMAP
675
/* 
676
   Standard unix mmap using /dev/zero clears memory so calloc doesn't
677
   need to.
678
*/
679
 
680
#ifndef MMAP_CLEARS
681
#define MMAP_CLEARS 1
682
#endif
683
 
684
#else /* no mmap */
685
#ifndef MMAP_CLEARS
686
#define MMAP_CLEARS 0
687
#endif
688
#endif
689
 
690
 
691
/* 
692
   MMAP_AS_MORECORE_SIZE is the minimum mmap size argument to use if
693
   sbrk fails, and mmap is used as a backup (which is done only if
694
   HAVE_MMAP).  The value must be a multiple of page size.  This
695
   backup strategy generally applies only when systems have "holes" in
696
   address space, so sbrk cannot perform contiguous expansion, but
697
   there is still space available on system.  On systems for which
698
   this is known to be useful (i.e. most linux kernels), this occurs
699
   only when programs allocate huge amounts of memory.  Between this,
700
   and the fact that mmap regions tend to be limited, the size should
701
   be large, to avoid too many mmap calls and thus avoid running out
702
   of kernel resources.
703
*/
704
 
705
#ifndef MMAP_AS_MORECORE_SIZE
706
#define MMAP_AS_MORECORE_SIZE (1024 * 1024)
707
#endif
708
 
10347 ripley 709
/*
710
  Define HAVE_MREMAP to make realloc() use mremap() to re-allocate
711
  large blocks.  This is currently only possible on Linux with
712
  kernel versions newer than 1.3.77.
713
*/
714
 
715
#ifndef HAVE_MREMAP
26855 ripley 716
#ifdef linux
10347 ripley 717
#define HAVE_MREMAP 1
718
#else
719
#define HAVE_MREMAP 0
720
#endif
721
 
26855 ripley 722
#endif /* HAVE_MMAP */
10347 ripley 723
 
724
 
725
/*
26855 ripley 726
  The system page size. To the extent possible, this malloc manages
727
  memory from the system in page-size units.  Note that this value is
728
  cached during initialization into a field of malloc_state. So even
729
  if malloc_getpagesize is a function, it is only called once.
10347 ripley 730
 
26855 ripley 731
  The following mechanics for getpagesize were adapted from bsd/gnu
732
  getpagesize.h. If none of the system-probes here apply, a value of
733
  4096 is used, which should be OK: If they don't apply, then using
734
  the actual value probably doesn't impact performance.
10347 ripley 735
*/
736
 
26855 ripley 737
 
738
#ifndef malloc_getpagesize
739
 
10347 ripley 740
#ifndef LACKS_UNISTD_H
741
#  include <unistd.h>
742
#endif
743
 
744
#  ifdef _SC_PAGESIZE         /* some SVR4 systems omit an underscore */
745
#    ifndef _SC_PAGE_SIZE
746
#      define _SC_PAGE_SIZE _SC_PAGESIZE
747
#    endif
748
#  endif
26855 ripley 749
 
10347 ripley 750
#  ifdef _SC_PAGE_SIZE
751
#    define malloc_getpagesize sysconf(_SC_PAGE_SIZE)
752
#  else
753
#    if defined(BSD) || defined(DGUX) || defined(HAVE_GETPAGESIZE)
754
       extern size_t getpagesize();
755
#      define malloc_getpagesize getpagesize()
756
#    else
26855 ripley 757
#      ifdef WIN32 /* use supplied emulation of getpagesize */
758
#        define malloc_getpagesize getpagesize() 
10347 ripley 759
#      else
10425 ripley 760
#        ifndef LACKS_SYS_PARAM_H
761
#          include <sys/param.h>
762
#        endif
763
#        ifdef EXEC_PAGESIZE
764
#          define malloc_getpagesize EXEC_PAGESIZE
10347 ripley 765
#        else
10425 ripley 766
#          ifdef NBPG
767
#            ifndef CLSIZE
768
#              define malloc_getpagesize NBPG
769
#            else
770
#              define malloc_getpagesize (NBPG * CLSIZE)
771
#            endif
10347 ripley 772
#          else
10425 ripley 773
#            ifdef NBPC
774
#              define malloc_getpagesize NBPC
10347 ripley 775
#            else
10425 ripley 776
#              ifdef PAGESIZE
777
#                define malloc_getpagesize PAGESIZE
26855 ripley 778
#              else /* just guess */
779
#                define malloc_getpagesize (4096) 
10425 ripley 780
#              endif
10347 ripley 781
#            endif
782
#          endif
783
#        endif
784
#      endif
785
#    endif
786
#  endif
787
#endif
788
 
789
/*
790
  This version of malloc supports the standard SVID/XPG mallinfo
26855 ripley 791
  routine that returns a struct containing usage properties and
792
  statistics. It should work on any SVID/XPG compliant system that has
793
  a /usr/include/malloc.h defining struct mallinfo. (If you'd like to
794
  install such a thing yourself, cut out the preliminary declarations
795
  as described above and below and save them in a malloc.h file. But
796
  there's no compelling reason to bother to do this.)
10347 ripley 797
 
798
  The main declaration needed is the mallinfo struct that is returned
799
  (by-copy) by mallinfo().  The SVID/XPG malloinfo struct contains a
26855 ripley 800
  bunch of fields that are not even meaningful in this version of
801
  malloc.  These fields are are instead filled by mallinfo() with
802
  other numbers that might be of interest.
10347 ripley 803
 
804
  HAVE_USR_INCLUDE_MALLOC_H should be set if you have a
805
  /usr/include/malloc.h file that includes a declaration of struct
806
  mallinfo.  If so, it is included; else an SVID2/XPG2 compliant
807
  version is declared below.  These must be precisely the same for
26855 ripley 808
  mallinfo() to work.  The original SVID version of this struct,
809
  defined on most systems with mallinfo, declares all fields as
810
  ints. But some others define as unsigned long. If your system
811
  defines the fields using a type of different width than listed here,
812
  you must #include your system version and #define
813
  HAVE_USR_INCLUDE_MALLOC_H.
10347 ripley 814
*/
815
 
816
/* #define HAVE_USR_INCLUDE_MALLOC_H */
817
 
26855 ripley 818
#ifdef HAVE_USR_INCLUDE_MALLOC_H
10347 ripley 819
#include "/usr/include/malloc.h"
820
#else
821
 
822
/* SVID2/XPG mallinfo structure */
823
 
824
struct mallinfo {
26855 ripley 825
  int arena;    /* non-mmapped space allocated from system */
826
  int ordblks;  /* number of free chunks */
827
  int smblks;   /* number of fastbin blocks */
10347 ripley 828
  int hblks;    /* number of mmapped regions */
26855 ripley 829
  int hblkhd;   /* space in mmapped regions */
830
  int usmblks;  /* maximum total allocated space */
831
  int fsmblks;  /* space available in freed fastbin blocks */
10347 ripley 832
  int uordblks; /* total allocated space */
26855 ripley 833
  int fordblks; /* total free space */
10347 ripley 834
  int keepcost; /* top-most, releasable (via malloc_trim) space */
835
};
836
 
26855 ripley 837
/*
838
  SVID/XPG defines four standard parameter numbers for mallopt,
839
  normally defined in malloc.h.  Only one of these (M_MXFAST) is used
840
  in this malloc. The others (M_NLBLKS, M_GRAIN, M_KEEP) don't apply,
841
  so setting them has no effect. But this malloc also supports other
842
  options in mallopt described below.
843
*/
844
#endif
10347 ripley 845
 
26855 ripley 846
/* R Specific declarations here */
10347 ripley 847
 
26855 ripley 848
extern unsigned int R_max_memory;
26895 ripley 849
extern int R_Is_Running;
850
void Rf_warning(const char *, ...);
10347 ripley 851
 
852
 
26855 ripley 853
/* ---------- description of public routines ------------ */
10347 ripley 854
 
26855 ripley 855
/*
856
  malloc(size_t n)
857
  Returns a pointer to a newly allocated chunk of at least n bytes, or null
858
  if no space is available. Additionally, on failure, errno is
859
  set to ENOMEM on ANSI C systems.
10347 ripley 860
 
26855 ripley 861
  If n is zero, malloc returns a minumum-sized chunk. (The minimum
862
  size is 16 bytes on most 32bit systems, and 24 or 32 bytes on 64bit
863
  systems.)  On most systems, size_t is an unsigned type, so calls
864
  with negative arguments are interpreted as requests for huge amounts
865
  of space, which will often fail. The maximum supported value of n
866
  differs across systems, but is in all cases less than the maximum
867
  representable value of a size_t.
868
*/
869
Void_t*  public_mALLOc(size_t);
10347 ripley 870
 
871
/*
26855 ripley 872
  free(Void_t* p)
873
  Releases the chunk of memory pointed to by p, that had been previously
874
  allocated using malloc or a related routine such as realloc.
875
  It has no effect if p is null. It can have arbitrary (i.e., bad!)
876
  effects if p has already been freed.
10347 ripley 877
 
26855 ripley 878
  Unless disabled (using mallopt), freeing very large spaces will
879
  when possible, automatically trigger operations that give
880
  back unused memory to the system, thus reducing program footprint.
881
*/
882
void     public_fREe(Void_t*);
10347 ripley 883
 
26855 ripley 884
/*
885
  calloc(size_t n_elements, size_t element_size);
886
  Returns a pointer to n_elements * element_size bytes, with all locations
887
  set to zero.
888
*/
889
Void_t*  public_cALLOc(size_t, size_t);
10347 ripley 890
 
26855 ripley 891
/*
892
  realloc(Void_t* p, size_t n)
893
  Returns a pointer to a chunk of size n that contains the same data
894
  as does chunk p up to the minimum of (n, p's size) bytes, or null
895
  if no space is available. 
10347 ripley 896
 
26855 ripley 897
  The returned pointer may or may not be the same as p. The algorithm
898
  prefers extending p when possible, otherwise it employs the
899
  equivalent of a malloc-copy-free sequence.
10347 ripley 900
 
26855 ripley 901
  If p is null, realloc is equivalent to malloc.  
10347 ripley 902
 
26855 ripley 903
  If space is not available, realloc returns null, errno is set (if on
904
  ANSI) and p is NOT freed.
10347 ripley 905
 
26855 ripley 906
  if n is for fewer bytes than already held by p, the newly unused
907
  space is lopped off and freed if possible.  Unless the #define
908
  REALLOC_ZERO_BYTES_FREES is set, realloc with a size argument of
909
  zero (re)allocates a minimum-sized chunk.
910
 
911
  Large chunks that were internally obtained via mmap will always
912
  be reallocated using malloc-copy-free sequences unless
913
  the system supports MREMAP (currently only linux).
914
 
915
  The old unix realloc convention of allowing the last-free'd chunk
916
  to be used as an argument to realloc is not supported.
10347 ripley 917
*/
29208 ripley 918
 
26855 ripley 919
Void_t*  public_rEALLOc(Void_t*, size_t);
10347 ripley 920
 
26855 ripley 921
/*
922
  memalign(size_t alignment, size_t n);
923
  Returns a pointer to a newly allocated chunk of n bytes, aligned
924
  in accord with the alignment argument.
10347 ripley 925
 
26855 ripley 926
  The alignment argument should be a power of two. If the argument is
927
  not a power of two, the nearest greater power is used.
928
  8-byte alignment is guaranteed by normal malloc calls, so don't
929
  bother calling memalign with an argument of 8 or less.
930
 
931
  Overreliance on memalign is a sure way to fragment space.
932
*/
933
Void_t*  public_mEMALIGn(size_t, size_t);
10347 ripley 934
 
935
/*
26855 ripley 936
  valloc(size_t n);
937
  Equivalent to memalign(pagesize, n), where pagesize is the page
938
  size of the system. If the pagesize is unknown, 4096 is used.
939
*/
940
Void_t*  public_vALLOc(size_t);
10347 ripley 941
 
942
 
26855 ripley 943
/*
944
  mallopt(int parameter_number, int parameter_value)
945
  Sets tunable parameters The format is to provide a
946
  (parameter-number, parameter-value) pair.  mallopt then sets the
947
  corresponding parameter to the argument value if it can (i.e., so
948
  long as the value is meaningful), and returns 1 if successful else
949
  0.  SVID/XPG/ANSI defines four standard param numbers for mallopt,
950
  normally defined in malloc.h.  Only one of these (M_MXFAST) is used
951
  in this malloc. The others (M_NLBLKS, M_GRAIN, M_KEEP) don't apply,
952
  so setting them has no effect. But this malloc also supports four
953
  other options in mallopt. See below for details.  Briefly, supported
954
  parameters are as follows (listed defaults are for "typical"
955
  configurations).
10347 ripley 956
 
26855 ripley 957
  Symbol            param #   default    allowed param values
958
  M_MXFAST          1         64         0-80  (0 disables fastbins)
959
  M_TRIM_THRESHOLD -1         256*1024   any   (-1U disables trimming)
960
  M_TOP_PAD        -2         0          any  
961
  M_MMAP_THRESHOLD -3         256*1024   any   (or 0 if no MMAP support)
962
  M_MMAP_MAX       -4         65536      any   (0 disables use of mmap)
963
*/
964
int      public_mALLOPt(int, int);
10347 ripley 965
 
966
 
26855 ripley 967
/*
968
  mallinfo()
969
  Returns (by copy) a struct containing various summary statistics:
970
 
971
  arena:     current total non-mmapped bytes allocated from system 
972
  ordblks:   the number of free chunks 
973
  smblks:    the number of fastbin blocks (i.e., small chunks that
974
               have been freed but not use resused or consolidated)
975
  hblks:     current number of mmapped regions 
976
  hblkhd:    total bytes held in mmapped regions 
977
  usmblks:   the maximum total allocated space. This will be greater
978
                than current total if trimming has occurred.
979
  fsmblks:   total bytes held in fastbin blocks 
980
  uordblks:  current total allocated space (normal or mmapped)
981
  fordblks:  total free space 
982
  keepcost:  the maximum number of bytes that could ideally be released
983
               back to system via malloc_trim. ("ideally" means that
984
               it ignores page restrictions etc.)
985
 
986
  Because these fields are ints, but internal bookkeeping may
987
  be kept as longs, the reported values may wrap around zero and 
988
  thus be inaccurate.
10347 ripley 989
*/
26855 ripley 990
struct mallinfo public_mALLINFo(void);
10347 ripley 991
 
26855 ripley 992
/*
993
  independent_calloc(size_t n_elements, size_t element_size, Void_t* chunks[]);
10347 ripley 994
 
26855 ripley 995
  independent_calloc is similar to calloc, but instead of returning a
996
  single cleared space, it returns an array of pointers to n_elements
997
  independent elements that can hold contents of size elem_size, each
998
  of which starts out cleared, and can be independently freed,
999
  realloc'ed etc. The elements are guaranteed to be adjacently
1000
  allocated (this is not guaranteed to occur with multiple callocs or
1001
  mallocs), which may also improve cache locality in some
1002
  applications.
1003
 
1004
  The "chunks" argument is optional (i.e., may be null, which is
1005
  probably the most typical usage). If it is null, the returned array
1006
  is itself dynamically allocated and should also be freed when it is
1007
  no longer needed. Otherwise, the chunks array must be of at least
1008
  n_elements in length. It is filled in with the pointers to the
1009
  chunks.
1010
 
1011
  In either case, independent_calloc returns this pointer array, or
1012
  null if the allocation failed.  If n_elements is zero and "chunks"
1013
  is null, it returns a chunk representing an array with zero elements
1014
  (which should be freed if not wanted).
1015
 
1016
  Each element must be individually freed when it is no longer
1017
  needed. If you'd like to instead be able to free all at once, you
1018
  should instead use regular calloc and assign pointers into this
1019
  space to represent elements.  (In this case though, you cannot
1020
  independently free elements.)
1021
 
1022
  independent_calloc simplifies and speeds up implementations of many
1023
  kinds of pools.  It may also be useful when constructing large data
1024
  structures that initially have a fixed number of fixed-sized nodes,
1025
  but the number is not known at compile time, and some of the nodes
1026
  may later need to be freed. For example:
1027
 
1028
  struct Node { int item; struct Node* next; };
1029
 
1030
  struct Node* build_list() {
1031
    struct Node** pool;
1032
    int n = read_number_of_nodes_needed();
1033
    if (n <= 0) return 0;
1034
    pool = (struct Node**)(independent_calloc(n, sizeof(struct Node), 0);
1035
    if (pool == 0) die(); 
1036
    // organize into a linked list... 
1037
    struct Node* first = pool[0];
1038
    for (i = 0; i < n-1; ++i) 
1039
      pool[i]->next = pool[i+1];
1040
    free(pool);     // Can now free the array (or not, if it is needed later)
1041
    return first;
1042
  }
1043
*/
1044
Void_t** public_iCALLOc(size_t, size_t, Void_t**);
10347 ripley 1045
 
1046
/*
26855 ripley 1047
  independent_comalloc(size_t n_elements, size_t sizes[], Void_t* chunks[]);
10347 ripley 1048
 
26855 ripley 1049
  independent_comalloc allocates, all at once, a set of n_elements
1050
  chunks with sizes indicated in the "sizes" array.    It returns
1051
  an array of pointers to these elements, each of which can be
1052
  independently freed, realloc'ed etc. The elements are guaranteed to
1053
  be adjacently allocated (this is not guaranteed to occur with
1054
  multiple callocs or mallocs), which may also improve cache locality
1055
  in some applications.
10347 ripley 1056
 
26855 ripley 1057
  The "chunks" argument is optional (i.e., may be null). If it is null
1058
  the returned array is itself dynamically allocated and should also
1059
  be freed when it is no longer needed. Otherwise, the chunks array
1060
  must be of at least n_elements in length. It is filled in with the
1061
  pointers to the chunks.
10347 ripley 1062
 
26855 ripley 1063
  In either case, independent_comalloc returns this pointer array, or
1064
  null if the allocation failed.  If n_elements is zero and chunks is
1065
  null, it returns a chunk representing an array with zero elements
1066
  (which should be freed if not wanted).
1067
 
1068
  Each element must be individually freed when it is no longer
1069
  needed. If you'd like to instead be able to free all at once, you
1070
  should instead use a single regular malloc, and assign pointers at
1071
  particular offsets in the aggregate space. (In this case though, you 
1072
  cannot independently free elements.)
10347 ripley 1073
 
26855 ripley 1074
  independent_comallac differs from independent_calloc in that each
1075
  element may have a different size, and also that it does not
1076
  automatically clear elements.
10347 ripley 1077
 
26855 ripley 1078
  independent_comalloc can be used to speed up allocation in cases
1079
  where several structs or objects must always be allocated at the
1080
  same time.  For example:
10347 ripley 1081
 
26855 ripley 1082
  struct Head { ... }
1083
  struct Foot { ... }
10347 ripley 1084
 
26855 ripley 1085
  void send_message(char* msg) {
1086
    int msglen = strlen(msg);
1087
    size_t sizes[3] = { sizeof(struct Head), msglen, sizeof(struct Foot) };
1088
    void* chunks[3];
1089
    if (independent_comalloc(3, sizes, chunks) == 0)
1090
      die();
1091
    struct Head* head = (struct Head*)(chunks[0]);
1092
    char*        body = (char*)(chunks[1]);
1093
    struct Foot* foot = (struct Foot*)(chunks[2]);
1094
    // ...
1095
  }
10347 ripley 1096
 
26855 ripley 1097
  In general though, independent_comalloc is worth using only for
1098
  larger values of n_elements. For small values, you probably won't
1099
  detect enough difference from series of malloc calls to bother.
1100
 
1101
  Overuse of independent_comalloc can increase overall memory usage,
1102
  since it cannot reuse existing noncontiguous small chunks that
1103
  might be available for some of the elements.
10347 ripley 1104
*/
26855 ripley 1105
Void_t** public_iCOMALLOc(size_t, size_t*, Void_t**);
10347 ripley 1106
 
1107
 
26855 ripley 1108
/*
1109
  pvalloc(size_t n);
1110
  Equivalent to valloc(minimum-page-that-holds(n)), that is,
1111
  round up n to nearest pagesize.
1112
 */
1113
Void_t*  public_pVALLOc(size_t);
10347 ripley 1114
 
26855 ripley 1115
/*
1116
  cfree(Void_t* p);
1117
  Equivalent to free(p).
1118
 
1119
  cfree is needed/defined on some systems that pair it with calloc,
1120
  for odd historical reasons (such as: cfree is used in example 
1121
  code in the first edition of K&R).
1122
*/
1123
void     public_cFREe(Void_t*);
1124
 
1125
/*
1126
  malloc_trim(size_t pad);
1127
 
1128
  If possible, gives memory back to the system (via negative
1129
  arguments to sbrk) if there is unused memory at the `high' end of
1130
  the malloc pool. You can call this after freeing large blocks of
1131
  memory to potentially reduce the system-level memory requirements
1132
  of a program. However, it cannot guarantee to reduce memory. Under
1133
  some allocation patterns, some large free blocks of memory will be
1134
  locked between two used chunks, so they cannot be given back to
1135
  the system.
1136
 
1137
  The `pad' argument to malloc_trim represents the amount of free
1138
  trailing space to leave untrimmed. If this argument is zero,
1139
  only the minimum amount of memory to maintain internal data
1140
  structures will be left (one page or less). Non-zero arguments
1141
  can be supplied to maintain enough trailing space to service
1142
  future expected allocations without having to re-obtain memory
1143
  from the system.
1144
 
1145
  Malloc_trim returns 1 if it actually released any memory, else 0.
1146
  On systems that do not support "negative sbrks", it will always
1147
  rreturn 0.
1148
*/
1149
int      public_mTRIm(size_t);
10347 ripley 1150
 
1151
/*
26855 ripley 1152
  malloc_usable_size(Void_t* p);
10347 ripley 1153
 
26855 ripley 1154
  Returns the number of bytes you can actually use in
1155
  an allocated chunk, which may be more than you requested (although
1156
  often not) due to alignment and minimum size constraints.
1157
  You can use this many bytes without worrying about
1158
  overwriting other allocated objects. This is not a particularly great
1159
  programming practice. malloc_usable_size can be more useful in
1160
  debugging and assertions, for example:
10347 ripley 1161
 
26855 ripley 1162
  p = malloc(n);
1163
  assert(malloc_usable_size(p) >= 256);
1164
 
10347 ripley 1165
*/
26855 ripley 1166
size_t   public_mUSABLe(Void_t*);
10347 ripley 1167
 
26855 ripley 1168
/*
1169
  malloc_stats();
1170
  Prints on stderr the amount of space obtained from the system (both
1171
  via sbrk and mmap), the maximum amount (which may be more than
1172
  current if malloc_trim and/or munmap got called), and the current
1173
  number of bytes allocated via malloc (or realloc, etc) but not yet
1174
  freed. Note that this is the number of bytes allocated, not the
1175
  number requested. It will be larger than the number requested
1176
  because of alignment and bookkeeping overhead. Because it includes
1177
  alignment wastage as being in use, this figure may be greater than
1178
  zero even when no user-level chunks are allocated.
10347 ripley 1179
 
26855 ripley 1180
  The reported current and maximum system memory can be inaccurate if
1181
  a program makes other calls to system memory allocation functions
1182
  (normally sbrk) outside of malloc.
10347 ripley 1183
 
26855 ripley 1184
  malloc_stats prints only the most commonly interesting statistics.
1185
  More information can be obtained by calling mallinfo.
10347 ripley 1186
 
26855 ripley 1187
*/
1188
void     public_mSTATs();
1189
 
1190
/* mallopt tuning options */
1191
 
10347 ripley 1192
/*
26855 ripley 1193
  M_MXFAST is the maximum request size used for "fastbins", special bins
1194
  that hold returned chunks without consolidating their spaces. This
1195
  enables future requests for chunks of the same size to be handled
1196
  very quickly, but can increase fragmentation, and thus increase the
1197
  overall memory footprint of a program.
10347 ripley 1198
 
26855 ripley 1199
  This malloc manages fastbins very conservatively yet still
1200
  efficiently, so fragmentation is rarely a problem for values less
1201
  than or equal to the default.  The maximum supported value of MXFAST
1202
  is 80. You wouldn't want it any higher than this anyway.  Fastbins
1203
  are designed especially for use with many small structs, objects or
1204
  strings -- the default handles structs/objects/arrays with sizes up
1205
  to 16 4byte fields, or small strings representing words, tokens,
1206
  etc. Using fastbins for larger objects normally worsens
1207
  fragmentation without improving speed.
1208
 
1209
  M_MXFAST is set in REQUEST size units. It is internally used in
1210
  chunksize units, which adds padding and alignment.  You can reduce
1211
  M_MXFAST to 0 to disable all use of fastbins.  This causes the malloc
1212
  algorithm to be a closer approximation of fifo-best-fit in all cases,
1213
  not just for larger requests, but will generally cause it to be
1214
  slower.
10425 ripley 1215
*/
1216
 
1217
 
26855 ripley 1218
/* M_MXFAST is a standard SVID/XPG tuning option, usually listed in malloc.h */
1219
#ifndef M_MXFAST
1220
#define M_MXFAST            1    
1221
#endif
10425 ripley 1222
 
26855 ripley 1223
#ifndef DEFAULT_MXFAST
1224
#define DEFAULT_MXFAST     64
1225
#endif
10425 ripley 1226
 
1227
 
1228
/*
26855 ripley 1229
  M_TRIM_THRESHOLD is the maximum amount of unused top-most memory
1230
  to keep before releasing via malloc_trim in free().
10425 ripley 1231
 
26855 ripley 1232
  Automatic trimming is mainly useful in long-lived programs.
1233
  Because trimming via sbrk can be slow on some systems, and can
1234
  sometimes be wasteful (in cases where programs immediately
1235
  afterward allocate more large chunks) the value should be high
1236
  enough so that your overall system performance would improve by
1237
  releasing this much memory.
10347 ripley 1238
 
26855 ripley 1239
  The trim threshold and the mmap control parameters (see below)
1240
  can be traded off with one another. Trimming and mmapping are
1241
  two different ways of releasing unused memory back to the
1242
  system. Between these two, it is often possible to keep
1243
  system-level demands of a long-lived program down to a bare
1244
  minimum. For example, in one test suite of sessions measuring
1245
  the XF86 X server on Linux, using a trim threshold of 128K and a
1246
  mmap threshold of 192K led to near-minimal long term resource
1247
  consumption.
10347 ripley 1248
 
26855 ripley 1249
  If you are using this malloc in a long-lived program, it should
1250
  pay to experiment with these values.  As a rough guide, you
1251
  might set to a value close to the average size of a process
1252
  (program) running on your system.  Releasing this much memory
1253
  would allow such a process to run in memory.  Generally, it's
1254
  worth it to tune for trimming rather tham memory mapping when a
1255
  program undergoes phases where several large chunks are
1256
  allocated and released in ways that can reuse each other's
1257
  storage, perhaps mixed with phases where there are no such
1258
  chunks at all.  And in well-behaved long-lived programs,
1259
  controlling release of large blocks via trimming versus mapping
1260
  is usually faster.
10347 ripley 1261
 
26855 ripley 1262
  However, in most programs, these parameters serve mainly as
1263
  protection against the system-level effects of carrying around
1264
  massive amounts of unneeded memory. Since frequent calls to
1265
  sbrk, mmap, and munmap otherwise degrade performance, the default
1266
  parameters are set to relatively high values that serve only as
1267
  safeguards.
10347 ripley 1268
 
26855 ripley 1269
  The trim value must be greater than page size to have any useful
1270
  effect.  To disable trimming completely, you can set to 
1271
  (unsigned long)(-1)
10347 ripley 1272
 
26855 ripley 1273
  Trim settings interact with fastbin (MXFAST) settings: Unless
1274
  TRIM_FASTBINS is defined, automatic trimming never takes place upon
1275
  freeing a chunk with size less than or equal to MXFAST. Trimming is
1276
  instead delayed until subsequent freeing of larger chunks. However,
1277
  you can still force an attempted trim by calling malloc_trim.
10347 ripley 1278
 
26855 ripley 1279
  Also, trimming is not generally possible in cases where
1280
  the main arena is obtained via mmap.
10347 ripley 1281
 
26855 ripley 1282
  Note that the trick some people use of mallocing a huge space and
1283
  then freeing it at program startup, in an attempt to reserve system
1284
  memory, doesn't have the intended effect under automatic trimming,
1285
  since that memory will immediately be returned to the system.
1286
*/
10347 ripley 1287
 
26855 ripley 1288
#define M_TRIM_THRESHOLD       -1
10347 ripley 1289
 
26855 ripley 1290
#ifndef DEFAULT_TRIM_THRESHOLD
1291
#define DEFAULT_TRIM_THRESHOLD (256 * 1024)
10347 ripley 1292
#endif
1293
 
26855 ripley 1294
/*
1295
  M_TOP_PAD is the amount of extra `padding' space to allocate or
1296
  retain whenever sbrk is called. It is used in two ways internally:
10347 ripley 1297
 
26855 ripley 1298
  * When sbrk is called to extend the top of the arena to satisfy
1299
  a new malloc request, this much padding is added to the sbrk
1300
  request.
10347 ripley 1301
 
26855 ripley 1302
  * When malloc_trim is called automatically from free(),
1303
  it is used as the `pad' argument.
10347 ripley 1304
 
26855 ripley 1305
  In both cases, the actual amount of padding is rounded
1306
  so that the end of the arena is always a system page boundary.
10347 ripley 1307
 
26855 ripley 1308
  The main reason for using padding is to avoid calling sbrk so
1309
  often. Having even a small pad greatly reduces the likelihood
1310
  that nearly every malloc request during program start-up (or
1311
  after trimming) will invoke sbrk, which needlessly wastes
1312
  time.
10347 ripley 1313
 
26855 ripley 1314
  Automatic rounding-up to page-size units is normally sufficient
1315
  to avoid measurable overhead, so the default is 0.  However, in
1316
  systems where sbrk is relatively slow, it can pay to increase
1317
  this value, at the expense of carrying around more memory than
1318
  the program needs.
1319
*/
1320
 
1321
#define M_TOP_PAD              -2
1322
 
1323
#ifndef DEFAULT_TOP_PAD
1324
#define DEFAULT_TOP_PAD        (0)
10347 ripley 1325
#endif
1326
 
26855 ripley 1327
/*
1328
  M_MMAP_THRESHOLD is the request size threshold for using mmap()
1329
  to service a request. Requests of at least this size that cannot
1330
  be allocated using already-existing space will be serviced via mmap.
1331
  (If enough normal freed space already exists it is used instead.)
10347 ripley 1332
 
26855 ripley 1333
  Using mmap segregates relatively large chunks of memory so that
1334
  they can be individually obtained and released from the host
1335
  system. A request serviced through mmap is never reused by any
1336
  other request (at least not directly; the system may just so
1337
  happen to remap successive requests to the same locations).
10347 ripley 1338
 
26855 ripley 1339
  Segregating space in this way has the benefits that:
10347 ripley 1340
 
26855 ripley 1341
   1. Mmapped space can ALWAYS be individually released back 
1342
      to the system, which helps keep the system level memory 
1343
      demands of a long-lived program low. 
1344
   2. Mapped memory can never become `locked' between
1345
      other chunks, as can happen with normally allocated chunks, which
1346
      means that even trimming via malloc_trim would not release them.
1347
   3. On some systems with "holes" in address spaces, mmap can obtain
1348
      memory that sbrk cannot.
10347 ripley 1349
 
26855 ripley 1350
  However, it has the disadvantages that:
10347 ripley 1351
 
26855 ripley 1352
   1. The space cannot be reclaimed, consolidated, and then
1353
      used to service later requests, as happens with normal chunks.
1354
   2. It can lead to more wastage because of mmap page alignment
1355
      requirements
1356
   3. It causes malloc performance to be more dependent on host
1357
      system memory management support routines which may vary in
1358
      implementation quality and may impose arbitrary
1359
      limitations. Generally, servicing a request via normal
1360
      malloc steps is faster than going through a system's mmap.
10347 ripley 1361
 
26855 ripley 1362
  The advantages of mmap nearly always outweigh disadvantages for
1363
  "large" chunks, but the value of "large" varies across systems.  The
1364
  default is an empirically derived value that works well in most
1365
  systems.
1366
*/
1367
 
1368
#define M_MMAP_THRESHOLD      -3
1369
 
1370
#ifndef DEFAULT_MMAP_THRESHOLD
1371
#define DEFAULT_MMAP_THRESHOLD (256 * 1024)
10347 ripley 1372
#endif
1373
 
26855 ripley 1374
/*
1375
  M_MMAP_MAX is the maximum number of requests to simultaneously
1376
  service using mmap. This parameter exists because
1377
. Some systems have a limited number of internal tables for
1378
  use by mmap, and using more than a few of them may degrade
1379
  performance.
10347 ripley 1380
 
26855 ripley 1381
  The default is set to a value that serves only as a safeguard.
1382
  Setting to 0 disables use of mmap for servicing large requests.  If
1383
  HAVE_MMAP is not set, the default value is 0, and attempts to set it
1384
  to non-zero values in mallopt will fail.
1385
*/
10347 ripley 1386
 
26855 ripley 1387
#define M_MMAP_MAX             -4
1388
 
1389
#ifndef DEFAULT_MMAP_MAX
1390
#if HAVE_MMAP
1391
#define DEFAULT_MMAP_MAX       (65536)
10347 ripley 1392
#else
26855 ripley 1393
#define DEFAULT_MMAP_MAX       (0)
10347 ripley 1394
#endif
26855 ripley 1395
#endif
10347 ripley 1396
 
1397
#ifdef __cplusplus
1398
};  /* end of extern "C" */
1399
#endif
1400
 
26855 ripley 1401
/* 
1402
  ========================================================================
1403
  To make a fully customizable malloc.h header file, cut everything
1404
  above this line, put into file malloc.h, edit to suit, and #include it 
1405
  on the next line, as well as in programs that use this malloc.
1406
  ========================================================================
1407
*/
10347 ripley 1408
 
26855 ripley 1409
/* #include "malloc.h" */
10347 ripley 1410
 
26855 ripley 1411
/* --------------------- public wrappers ---------------------- */
10425 ripley 1412
 
26855 ripley 1413
#ifdef USE_PUBLIC_MALLOC_WRAPPERS
10347 ripley 1414
 
26855 ripley 1415
/* Declare all routines as internal */
1416
static Void_t*  mALLOc(size_t);
1417
static void     fREe(Void_t*);
1418
static Void_t*  rEALLOc(Void_t*, size_t);
1419
static Void_t*  mEMALIGn(size_t, size_t);
1420
static Void_t*  vALLOc(size_t);
1421
static Void_t*  pVALLOc(size_t);
1422
static Void_t*  cALLOc(size_t, size_t);
1423
static Void_t** iCALLOc(size_t, size_t, Void_t**);
1424
static Void_t** iCOMALLOc(size_t, size_t*, Void_t**);
1425
static void     cFREe(Void_t*);
1426
static int      mTRIm(size_t);
1427
static size_t   mUSABLe(Void_t*);
1428
static void     mSTATs();
1429
static int      mALLOPt(int, int);
1430
static struct mallinfo mALLINFo(void);
1431
 
10347 ripley 1432
/*
26855 ripley 1433
  MALLOC_PREACTION and MALLOC_POSTACTION should be
1434
  defined to return 0 on success, and nonzero on failure.
1435
  The return value of MALLOC_POSTACTION is currently ignored
1436
  in wrapper functions since there is no reasonable default
1437
  action to take on failure.
1438
*/
10347 ripley 1439
 
1440
 
26855 ripley 1441
#ifdef USE_MALLOC_LOCK
1442
 
10347 ripley 1443
#ifdef WIN32
1444
 
26855 ripley 1445
static int mALLOC_MUTEx;
1446
#define MALLOC_PREACTION   slwait(&mALLOC_MUTEx)
1447
#define MALLOC_POSTACTION  slrelease(&mALLOC_MUTEx)
10347 ripley 1448
 
26855 ripley 1449
#else
10347 ripley 1450
 
26855 ripley 1451
#include <pthread.h>
10347 ripley 1452
 
26855 ripley 1453
static pthread_mutex_t mALLOC_MUTEx = PTHREAD_MUTEX_INITIALIZER;
1454
 
1455
#define MALLOC_PREACTION   pthread_mutex_lock(&mALLOC_MUTEx)
1456
#define MALLOC_POSTACTION  pthread_mutex_unlock(&mALLOC_MUTEx)
1457
 
1458
#endif /* USE_MALLOC_LOCK */
1459
 
1460
#else
1461
 
1462
/* Substitute anything you like for these */
1463
 
1464
#define MALLOC_PREACTION   (0)
1465
#define MALLOC_POSTACTION  (0)
1466
 
1467
#endif
1468
 
1469
Void_t* public_mALLOc(size_t bytes) {
1470
  Void_t* m;
1471
  if (MALLOC_PREACTION != 0) {
1472
    return 0;
1473
  }
1474
  m = mALLOc(bytes);
1475
  if (MALLOC_POSTACTION != 0) {
1476
  }
1477
  return m;
10347 ripley 1478
}
1479
 
26855 ripley 1480
void public_fREe(Void_t* m) {
1481
  if (MALLOC_PREACTION != 0) {
1482
    return;
1483
  }
1484
  fREe(m);
1485
  if (MALLOC_POSTACTION != 0) {
1486
  }
1487
}
10425 ripley 1488
 
26855 ripley 1489
Void_t* public_rEALLOc(Void_t* m, size_t bytes) {
1490
  if (MALLOC_PREACTION != 0) {
1491
    return 0;
1492
  }
1493
  m = rEALLOc(m, bytes);
1494
  if (MALLOC_POSTACTION != 0) {
1495
  }
1496
  return m;
1497
}
10425 ripley 1498
 
26855 ripley 1499
Void_t* public_mEMALIGn(size_t alignment, size_t bytes) {
1500
  Void_t* m;
1501
  if (MALLOC_PREACTION != 0) {
1502
    return 0;
1503
  }
1504
  m = mEMALIGn(alignment, bytes);
1505
  if (MALLOC_POSTACTION != 0) {
1506
  }
1507
  return m;
10347 ripley 1508
}
1509
 
26855 ripley 1510
Void_t* public_vALLOc(size_t bytes) {
1511
  Void_t* m;
1512
  if (MALLOC_PREACTION != 0) {
1513
    return 0;
1514
  }
1515
  m = vALLOc(bytes);
1516
  if (MALLOC_POSTACTION != 0) {
1517
  }
1518
  return m;
1519
}
12256 pd 1520
 
26855 ripley 1521
Void_t* public_pVALLOc(size_t bytes) {
1522
  Void_t* m;
1523
  if (MALLOC_PREACTION != 0) {
1524
    return 0;
1525
  }
1526
  m = pVALLOc(bytes);
1527
  if (MALLOC_POSTACTION != 0) {
1528
  }
1529
  return m;
1530
}
10347 ripley 1531
 
26855 ripley 1532
Void_t* public_cALLOc(size_t n, size_t elem_size) {
1533
  Void_t* m;
1534
  if (MALLOC_PREACTION != 0) {
1535
    return 0;
1536
  }
1537
  m = cALLOc(n, elem_size);
1538
  if (MALLOC_POSTACTION != 0) {
1539
  }
1540
  return m;
1541
}
11007 ripley 1542
 
10347 ripley 1543
 
26855 ripley 1544
Void_t** public_iCALLOc(size_t n, size_t elem_size, Void_t** chunks) {
1545
  Void_t** m;
1546
  if (MALLOC_PREACTION != 0) {
1547
    return 0;
1548
  }
1549
  m = iCALLOc(n, elem_size, chunks);
1550
  if (MALLOC_POSTACTION != 0) {
1551
  }
1552
  return m;
1553
}
10347 ripley 1554
 
26855 ripley 1555
Void_t** public_iCOMALLOc(size_t n, size_t sizes[], Void_t** chunks) {
1556
  Void_t** m;
1557
  if (MALLOC_PREACTION != 0) {
1558
    return 0;
1559
  }
1560
  m = iCOMALLOc(n, sizes, chunks);
1561
  if (MALLOC_POSTACTION != 0) {
1562
  }
1563
  return m;
1564
}
10347 ripley 1565
 
26855 ripley 1566
void public_cFREe(Void_t* m) {
1567
  if (MALLOC_PREACTION != 0) {
1568
    return;
1569
  }
1570
  cFREe(m);
1571
  if (MALLOC_POSTACTION != 0) {
1572
  }
10347 ripley 1573
}
1574
 
26855 ripley 1575
int public_mTRIm(size_t s) {
1576
  int result;
1577
  if (MALLOC_PREACTION != 0) {
1578
    return 0;
1579
  }
1580
  result = mTRIm(s);
1581
  if (MALLOC_POSTACTION != 0) {
1582
  }
1583
  return result;
1584
}
10347 ripley 1585
 
26855 ripley 1586
size_t public_mUSABLe(Void_t* m) {
1587
  size_t result;
1588
  if (MALLOC_PREACTION != 0) {
1589
    return 0;
1590
  }
1591
  result = mUSABLe(m);
1592
  if (MALLOC_POSTACTION != 0) {
1593
  }
1594
  return result;
1595
}
1596
 
1597
void public_mSTATs() {
1598
  if (MALLOC_PREACTION != 0) {
1599
    return;
1600
  }
1601
  mSTATs();
1602
  if (MALLOC_POSTACTION != 0) {
1603
  }
1604
}
1605
 
1606
struct mallinfo public_mALLINFo() {
1607
  struct mallinfo m;
1608
  if (MALLOC_PREACTION != 0) {
1609
    struct mallinfo nm = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 };
1610
    return nm;
1611
  }
1612
  m = mALLINFo();
1613
  if (MALLOC_POSTACTION != 0) {
1614
  }
1615
  return m;
1616
}
1617
 
1618
int public_mALLOPt(int p, int v) {
1619
  int result;
1620
  if (MALLOC_PREACTION != 0) {
1621
    return 0;
1622
  }
1623
  result = mALLOPt(p, v);
1624
  if (MALLOC_POSTACTION != 0) {
1625
  }
1626
  return result;
1627
}
1628
 
1629
#endif
1630
 
1631
 
1632
 
1633
/* ------------- Optional versions of memcopy ---------------- */
1634
 
1635
 
1636
#if USE_MEMCPY
1637
 
1638
/* 
1639
  Note: memcpy is ONLY invoked with non-overlapping regions,
1640
  so the (usually slower) memmove is not needed.
1641
*/
1642
 
1643
#define MALLOC_COPY(dest, src, nbytes)  memcpy(dest, src, nbytes)
1644
#define MALLOC_ZERO(dest, nbytes)       memset(dest, 0,   nbytes)
1645
 
1646
#else /* !USE_MEMCPY */
1647
 
1648
/* Use Duff's device for good zeroing/copying performance. */
1649
 
1650
#define MALLOC_ZERO(charp, nbytes)                                            \
1651
do {                                                                          \
1652
  INTERNAL_SIZE_T* mzp = (INTERNAL_SIZE_T*)(charp);                           \
1653
  CHUNK_SIZE_T  mctmp = (nbytes)/sizeof(INTERNAL_SIZE_T);                     \
1654
  long mcn;                                                                   \
1655
  if (mctmp < 8) mcn = 0; else { mcn = (mctmp-1)/8; mctmp %= 8; }             \
1656
  switch (mctmp) {                                                            \
1657
    case 0: for(;;) { *mzp++ = 0;                                             \
1658
    case 7:           *mzp++ = 0;                                             \
1659
    case 6:           *mzp++ = 0;                                             \
1660
    case 5:           *mzp++ = 0;                                             \
1661
    case 4:           *mzp++ = 0;                                             \
1662
    case 3:           *mzp++ = 0;                                             \
1663
    case 2:           *mzp++ = 0;                                             \
1664
    case 1:           *mzp++ = 0; if(mcn <= 0) break; mcn--; }                \
1665
  }                                                                           \
1666
} while(0)
1667
 
1668
#define MALLOC_COPY(dest,src,nbytes)                                          \
1669
do {                                                                          \
1670
  INTERNAL_SIZE_T* mcsrc = (INTERNAL_SIZE_T*) src;                            \
1671
  INTERNAL_SIZE_T* mcdst = (INTERNAL_SIZE_T*) dest;                           \
1672
  CHUNK_SIZE_T  mctmp = (nbytes)/sizeof(INTERNAL_SIZE_T);                     \
1673
  long mcn;                                                                   \
1674
  if (mctmp < 8) mcn = 0; else { mcn = (mctmp-1)/8; mctmp %= 8; }             \
1675
  switch (mctmp) {                                                            \
1676
    case 0: for(;;) { *mcdst++ = *mcsrc++;                                    \
1677
    case 7:           *mcdst++ = *mcsrc++;                                    \
1678
    case 6:           *mcdst++ = *mcsrc++;                                    \
1679
    case 5:           *mcdst++ = *mcsrc++;                                    \
1680
    case 4:           *mcdst++ = *mcsrc++;                                    \
1681
    case 3:           *mcdst++ = *mcsrc++;                                    \
1682
    case 2:           *mcdst++ = *mcsrc++;                                    \
1683
    case 1:           *mcdst++ = *mcsrc++; if(mcn <= 0) break; mcn--; }       \
1684
  }                                                                           \
1685
} while(0)
1686
 
1687
#endif
1688
 
1689
/* ------------------ MMAP support ------------------  */
1690
 
1691
 
1692
#if HAVE_MMAP
1693
 
1694
#ifndef LACKS_FCNTL_H
1695
#include <fcntl.h>
1696
#endif
1697
 
1698
#ifndef LACKS_SYS_MMAN_H
1699
#include <sys/mman.h>
1700
#endif
1701
 
1702
#if !defined(MAP_ANONYMOUS) && defined(MAP_ANON)
1703
#define MAP_ANONYMOUS MAP_ANON
1704
#endif
1705
 
1706
/* 
1707
   Nearly all versions of mmap support MAP_ANONYMOUS, 
1708
   so the following is unlikely to be needed, but is
1709
   supplied just in case.
1710
*/
1711
 
1712
#ifndef MAP_ANONYMOUS
1713
 
1714
static int dev_zero_fd = -1; /* Cached file descriptor for /dev/zero. */
1715
 
1716
#define MMAP(addr, size, prot, flags) ((dev_zero_fd < 0) ? \
1717
 (dev_zero_fd = open("/dev/zero", O_RDWR), \
1718
  mmap((addr), (size), (prot), (flags), dev_zero_fd, 0)) : \
1719
   mmap((addr), (size), (prot), (flags), dev_zero_fd, 0))
1720
 
1721
#else
1722
 
1723
#define MMAP(addr, size, prot, flags) \
1724
 (mmap((addr), (size), (prot), (flags)|MAP_ANONYMOUS, -1, 0))
1725
 
1726
#endif
1727
 
1728
 
1729
#endif /* HAVE_MMAP */
1730
 
1731
 
10347 ripley 1732
/*
26855 ripley 1733
  -----------------------  Chunk representations -----------------------
10347 ripley 1734
*/
1735
 
1736
 
26855 ripley 1737
/*
1738
  This struct declaration is misleading (but accurate and necessary).
1739
  It declares a "view" into memory allowing access to necessary
1740
  fields at known offsets from a given base. See explanation below.
1741
*/
1742
 
1743
struct malloc_chunk {
1744
 
1745
  INTERNAL_SIZE_T      prev_size;  /* Size of previous chunk (if free).  */
1746
  INTERNAL_SIZE_T      size;       /* Size in bytes, including overhead. */
1747
 
1748
  struct malloc_chunk* fd;         /* double links -- used only if free. */
10347 ripley 1749
  struct malloc_chunk* bk;
1750
};
1751
 
26855 ripley 1752
 
10347 ripley 1753
typedef struct malloc_chunk* mchunkptr;
1754
 
1755
/*
1756
   malloc_chunk details:
1757
 
1758
    (The following includes lightly edited explanations by Colin Plumb.)
1759
 
1760
    Chunks of memory are maintained using a `boundary tag' method as
1761
    described in e.g., Knuth or Standish.  (See the paper by Paul
1762
    Wilson ftp://ftp.cs.utexas.edu/pub/garbage/allocsrv.ps for a
1763
    survey of such techniques.)  Sizes of free chunks are stored both
1764
    in the front of each chunk and at the end.  This makes
1765
    consolidating fragmented chunks into bigger chunks very fast.  The
1766
    size fields also hold bits representing whether chunks are free or
1767
    in use.
1768
 
1769
    An allocated chunk looks like this:
1770
 
1771
 
1772
    chunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1773
            |             Size of previous chunk, if allocated            | |
1774
            +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1775
            |             Size of chunk, in bytes                         |P|
1776
      mem-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1777
            |             User data starts here...                          .
1778
            .                                                               .
1779
            .             (malloc_usable_space() bytes)                     .
1780
            .                                                               |
1781
nextchunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1782
            |             Size of chunk                                     |
1783
            +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1784
 
1785
 
1786
    Where "chunk" is the front of the chunk for the purpose of most of
1787
    the malloc code, but "mem" is the pointer that is returned to the
1788
    user.  "Nextchunk" is the beginning of the next contiguous chunk.
1789
 
1790
    Chunks always begin on even word boundries, so the mem portion
1791
    (which is returned to the user) is also on an even word boundary, and
26855 ripley 1792
    thus at least double-word aligned.
10347 ripley 1793
 
1794
    Free chunks are stored in circular doubly-linked lists, and look like this:
1795
 
1796
    chunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1797
            |             Size of previous chunk                            |
1798
            +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1799
    `head:' |             Size of chunk, in bytes                         |P|
1800
      mem-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1801
            |             Forward pointer to next chunk in list             |
1802
            +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1803
            |             Back pointer to previous chunk in list            |
1804
            +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1805
            |             Unused space (may be 0 bytes long)                .
1806
            .                                                               .
1807
            .                                                               |
1808
nextchunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1809
    `foot:' |             Size of chunk, in bytes                           |
1810
            +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1811
 
1812
    The P (PREV_INUSE) bit, stored in the unused low-order bit of the
1813
    chunk size (which is always a multiple of two words), is an in-use
1814
    bit for the *previous* chunk.  If that bit is *clear*, then the
1815
    word before the current chunk size contains the previous chunk
1816
    size, and can be used to find the front of the previous chunk.
26855 ripley 1817
    The very first chunk allocated always has this bit set,
1818
    preventing access to non-existent (or non-owned) memory. If
1819
    prev_inuse is set for any given chunk, then you CANNOT determine
1820
    the size of the previous chunk, and might even get a memory
1821
    addressing fault when trying to do so.
10347 ripley 1822
 
1823
    Note that the `foot' of the current chunk is actually represented
26855 ripley 1824
    as the prev_size of the NEXT chunk. This makes it easier to
1825
    deal with alignments etc but can be very confusing when trying
1826
    to extend or adapt this code.
10347 ripley 1827
 
1828
    The two exceptions to all this are
1829
 
26855 ripley 1830
     1. The special chunk `top' doesn't bother using the
1831
        trailing size field since there is no next contiguous chunk
1832
        that would have to index off it. After initialization, `top'
1833
        is forced to always exist.  If it would become less than
1834
        MINSIZE bytes long, it is replenished.
10347 ripley 1835
 
1836
     2. Chunks allocated via mmap, which have the second-lowest-order
1837
        bit (IS_MMAPPED) set in their size fields.  Because they are
26855 ripley 1838
        allocated one-by-one, each must contain its own trailing size field.
10347 ripley 1839
 
26855 ripley 1840
*/
10347 ripley 1841
 
26855 ripley 1842
/*
1843
  ---------- Size and alignment checks and conversions ----------
10347 ripley 1844
*/
1845
 
26855 ripley 1846
/* conversion from malloc headers to user pointers, and back */
10347 ripley 1847
 
26855 ripley 1848
#define chunk2mem(p)   ((Void_t*)((char*)(p) + 2*SIZE_SZ))
1849
#define mem2chunk(mem) ((mchunkptr)((char*)(mem) - 2*SIZE_SZ))
10347 ripley 1850
 
26855 ripley 1851
/* The smallest possible chunk */
1852
#define MIN_CHUNK_SIZE        (sizeof(struct malloc_chunk))
10347 ripley 1853
 
26855 ripley 1854
/* The smallest size we can malloc is an aligned minimal chunk */
10347 ripley 1855
 
26855 ripley 1856
#define MINSIZE  \
1857
  (CHUNK_SIZE_T)(((MIN_CHUNK_SIZE+MALLOC_ALIGN_MASK) & ~MALLOC_ALIGN_MASK))
10347 ripley 1858
 
26855 ripley 1859
/* Check if m has acceptable alignment */
10347 ripley 1860
 
26855 ripley 1861
#define aligned_OK(m)  (((PTR_UINT)((m)) & (MALLOC_ALIGN_MASK)) == 0)
10347 ripley 1862
 
1863
 
26855 ripley 1864
/* 
1865
   Check if a request is so large that it would wrap around zero when
1866
   padded and aligned. To simplify some other code, the bound is made
1867
   low enough so that adding MINSIZE will also not wrap around sero.
1868
*/
10347 ripley 1869
 
26855 ripley 1870
#define REQUEST_OUT_OF_RANGE(req)                                 \
1871
  ((CHUNK_SIZE_T)(req) >=                                        \
1872
   (CHUNK_SIZE_T)(INTERNAL_SIZE_T)(-2 * MINSIZE))    
10347 ripley 1873
 
26855 ripley 1874
/* pad request bytes into a usable size -- internal version */
10347 ripley 1875
 
26855 ripley 1876
#define request2size(req)                                         \
1877
  (((req) + SIZE_SZ + MALLOC_ALIGN_MASK < MINSIZE)  ?             \
1878
   MINSIZE :                                                      \
1879
   ((req) + SIZE_SZ + MALLOC_ALIGN_MASK) & ~MALLOC_ALIGN_MASK)
10347 ripley 1880
 
26855 ripley 1881
/*  Same, except also perform argument check */
10347 ripley 1882
 
26855 ripley 1883
#define checked_request2size(req, sz)                             \
1884
  if (REQUEST_OUT_OF_RANGE(req)) {                                \
1885
    MALLOC_FAILURE_ACTION;                                        \
1886
    return 0;                                                     \
1887
  }                                                               \
1888
  (sz) = request2size(req);                                              
10347 ripley 1889
 
1890
/*
26855 ripley 1891
  --------------- Physical chunk operations ---------------
10347 ripley 1892
*/
1893
 
1894
 
1895
/* size field is or'ed with PREV_INUSE when previous adjacent chunk in use */
1896
#define PREV_INUSE 0x1
1897
 
26855 ripley 1898
/* extract inuse bit of previous chunk */
1899
#define prev_inuse(p)       ((p)->size & PREV_INUSE)
1900
 
1901
 
10347 ripley 1902
/* size field is or'ed with IS_MMAPPED if the chunk was obtained with mmap() */
1903
#define IS_MMAPPED 0x2
1904
 
26855 ripley 1905
/* check for mmap()'ed chunk */
1906
#define chunk_is_mmapped(p) ((p)->size & IS_MMAPPED)
10347 ripley 1907
 
26855 ripley 1908
/* 
1909
  Bits to mask off when extracting size 
1910
 
1911
  Note: IS_MMAPPED is intentionally not masked off from size field in
1912
  macros for which mmapped chunks should never be seen. This should
1913
  cause helpful core dumps to occur if it is tried by accident by
1914
  people extending or adapting this malloc.
1915
*/
10347 ripley 1916
#define SIZE_BITS (PREV_INUSE|IS_MMAPPED)
1917
 
26855 ripley 1918
/* Get size, ignoring use bits */
1919
#define chunksize(p)         ((p)->size & ~(SIZE_BITS))
10347 ripley 1920
 
26855 ripley 1921
 
10347 ripley 1922
/* Ptr to next physical malloc_chunk. */
1923
#define next_chunk(p) ((mchunkptr)( ((char*)(p)) + ((p)->size & ~PREV_INUSE) ))
1924
 
1925
/* Ptr to previous physical malloc_chunk */
26855 ripley 1926
#define prev_chunk(p) ((mchunkptr)( ((char*)(p)) - ((p)->prev_size) ))
10347 ripley 1927
 
1928
/* Treat space at ptr + offset as a chunk */
1929
#define chunk_at_offset(p, s)  ((mchunkptr)(((char*)(p)) + (s)))
1930
 
1931
/* extract p's inuse bit */
1932
#define inuse(p)\
1933
((((mchunkptr)(((char*)(p))+((p)->size & ~PREV_INUSE)))->size) & PREV_INUSE)
1934
 
26855 ripley 1935
/* set/clear chunk as being inuse without otherwise disturbing */
10347 ripley 1936
#define set_inuse(p)\
1937
((mchunkptr)(((char*)(p)) + ((p)->size & ~PREV_INUSE)))->size |= PREV_INUSE
1938
 
1939
#define clear_inuse(p)\
1940
((mchunkptr)(((char*)(p)) + ((p)->size & ~PREV_INUSE)))->size &= ~(PREV_INUSE)
1941
 
26855 ripley 1942
 
10347 ripley 1943
/* check/set/clear inuse bits in known places */
1944
#define inuse_bit_at_offset(p, s)\
1945
 (((mchunkptr)(((char*)(p)) + (s)))->size & PREV_INUSE)
1946
 
1947
#define set_inuse_bit_at_offset(p, s)\
1948
 (((mchunkptr)(((char*)(p)) + (s)))->size |= PREV_INUSE)
1949
 
1950
#define clear_inuse_bit_at_offset(p, s)\
1951
 (((mchunkptr)(((char*)(p)) + (s)))->size &= ~(PREV_INUSE))
1952
 
1953
 
26855 ripley 1954
/* Set size at head, without disturbing its use bit */
1955
#define set_head_size(p, s)  ((p)->size = (((p)->size & PREV_INUSE) | (s)))
10347 ripley 1956
 
26855 ripley 1957
/* Set size/use field */
1958
#define set_head(p, s)       ((p)->size = (s))
1959
 
1960
/* Set size at footer (only when chunk is not in use) */
1961
#define set_foot(p, s)       (((mchunkptr)((char*)(p) + (s)))->prev_size = (s))
1962
 
1963
 
10347 ripley 1964
/*
26855 ripley 1965
  -------------------- Internal data structures --------------------
1966
 
1967
   All internal state is held in an instance of malloc_state defined
1968
   below. There are no other static variables, except in two optional
1969
   cases: 
1970
   * If USE_MALLOC_LOCK is defined, the mALLOC_MUTEx declared above. 
1971
   * If HAVE_MMAP is true, but mmap doesn't support
1972
     MAP_ANONYMOUS, a dummy file descriptor for mmap.
1973
 
1974
   Beware of lots of tricks that minimize the total bookkeeping space
1975
   requirements. The result is a little over 1K bytes (for 4byte
1976
   pointers and size_t.)
10347 ripley 1977
*/
1978
 
26855 ripley 1979
/*
1980
  Bins
10347 ripley 1981
 
26855 ripley 1982
    An array of bin headers for free chunks. Each bin is doubly
1983
    linked.  The bins are approximately proportionally (log) spaced.
1984
    There are a lot of these bins (128). This may look excessive, but
1985
    works very well in practice.  Most bins hold sizes that are
1986
    unusual as malloc request sizes, but are more usual for fragments
1987
    and consolidated sets of chunks, which is what these bins hold, so
1988
    they can be found quickly.  All procedures maintain the invariant
1989
    that no consolidated chunk physically borders another one, so each
1990
    chunk in a list is known to be preceeded and followed by either
1991
    inuse chunks or the ends of memory.
10347 ripley 1992
 
26855 ripley 1993
    Chunks in bins are kept in size order, with ties going to the
1994
    approximately least recently used chunk. Ordering isn't needed
1995
    for the small bins, which all contain the same-sized chunks, but
1996
    facilitates best-fit allocation for larger chunks. These lists
1997
    are just sequential. Keeping them in order almost never requires
1998
    enough traversal to warrant using fancier ordered data
1999
    structures.  
10347 ripley 2000
 
26855 ripley 2001
    Chunks of the same size are linked with the most
2002
    recently freed at the front, and allocations are taken from the
2003
    back.  This results in LRU (FIFO) allocation order, which tends
2004
    to give each chunk an equal opportunity to be consolidated with
2005
    adjacent freed chunks, resulting in larger free chunks and less
2006
    fragmentation.
10347 ripley 2007
 
26855 ripley 2008
    To simplify use in double-linked lists, each bin header acts
2009
    as a malloc_chunk. This avoids special-casing for headers.
2010
    But to conserve space and improve locality, we allocate
2011
    only the fd/bk pointers of bins, and then use repositioning tricks
2012
    to treat these as the fields of a malloc_chunk*.  
2013
*/
10347 ripley 2014
 
26855 ripley 2015
typedef struct malloc_chunk* mbinptr;
10347 ripley 2016
 
26855 ripley 2017
/* addressing -- note that bin_at(0) does not exist */
2018
#define bin_at(m, i) ((mbinptr)((char*)&((m)->bins[(i)<<1]) - (SIZE_SZ<<1)))
10347 ripley 2019
 
26855 ripley 2020
/* analog of ++bin */
2021
#define next_bin(b)  ((mbinptr)((char*)(b) + (sizeof(mchunkptr)<<1)))
10347 ripley 2022
 
26855 ripley 2023
/* Reminders about list directionality within bins */
2024
#define first(b)     ((b)->fd)
2025
#define last(b)      ((b)->bk)
10347 ripley 2026
 
26855 ripley 2027
/* Take a chunk off a bin list */
2028
#define unlink(P, BK, FD) {                                            \
2029
  FD = P->fd;                                                          \
2030
  BK = P->bk;                                                          \
2031
  FD->bk = BK;                                                         \
2032
  BK->fd = FD;                                                         \
2033
}
10347 ripley 2034
 
2035
/*
26855 ripley 2036
  Indexing
10347 ripley 2037
 
2038
    Bins for sizes < 512 bytes contain chunks of all the same size, spaced
26855 ripley 2039
    8 bytes apart. Larger bins are approximately logarithmically spaced:
10347 ripley 2040
 
2041
    64 bins of size       8
2042
    32 bins of size      64
2043
    16 bins of size     512
2044
     8 bins of size    4096
2045
     4 bins of size   32768
2046
     2 bins of size  262144
2047
     1 bin  of size what's left
2048
 
26855 ripley 2049
    The bins top out around 1MB because we expect to service large
2050
    requests via mmap.
2051
*/
10347 ripley 2052
 
26855 ripley 2053
#define NBINS              96
2054
#define NSMALLBINS         32
2055
#define SMALLBIN_WIDTH      8
2056
#define MIN_LARGE_SIZE    256
10347 ripley 2057
 
26855 ripley 2058
#define in_smallbin_range(sz)  \
2059
  ((CHUNK_SIZE_T)(sz) < (CHUNK_SIZE_T)MIN_LARGE_SIZE)
2060
 
2061
#define smallbin_index(sz)     (((unsigned)(sz)) >> 3)
2062
 
2063
/*
2064
  Compute index for size. We expect this to be inlined when
2065
  compiled with optimization, else not, which works out well.
10347 ripley 2066
*/
26855 ripley 2067
static int largebin_index(unsigned int sz) {
2068
  unsigned int  x = sz >> SMALLBIN_WIDTH; 
2069
  unsigned int m;            /* bit position of highest set bit of m */
10347 ripley 2070
 
26855 ripley 2071
  if (x >= 0x10000) return NBINS-1;
10347 ripley 2072
 
26855 ripley 2073
  /* On intel, use BSRL instruction to find highest bit */
2074
#if defined(__GNUC__) && defined(i386)
10347 ripley 2075
 
26855 ripley 2076
  __asm__("bsrl %1,%0\n\t"
2077
          : "=r" (m) 
2078
          : "g"  (x));
10347 ripley 2079
 
26855 ripley 2080
#else
2081
  {
2082
    /*
2083
      Based on branch-free nlz algorithm in chapter 5 of Henry
2084
      S. Warren Jr's book "Hacker's Delight".
2085
    */
10347 ripley 2086
 
26855 ripley 2087
    unsigned int n = ((x - 0x100) >> 16) & 8;
2088
    x <<= n; 
2089
    m = ((x - 0x1000) >> 16) & 4;
2090
    n += m; 
2091
    x <<= m; 
2092
    m = ((x - 0x4000) >> 16) & 2;
2093
    n += m; 
2094
    x = (x << m) >> 14;
2095
    m = 13 - n + (x & ~(x>>1));
2096
  }
2097
#endif
2098
 
2099
  /* Use next 2 bits to create finer-granularity bins */
2100
  return NSMALLBINS + (m << 2) + ((sz >> (m + 6)) & 3);
2101
}
2102
 
2103
#define bin_index(sz) \
2104
 ((in_smallbin_range(sz)) ? smallbin_index(sz) : largebin_index(sz))
2105
 
10347 ripley 2106
/*
26855 ripley 2107
  FIRST_SORTED_BIN_SIZE is the chunk size corresponding to the
2108
  first bin that is maintained in sorted order. This must
2109
  be the smallest size corresponding to a given bin.
2110
 
2111
  Normally, this should be MIN_LARGE_SIZE. But you can weaken
2112
  best fit guarantees to sometimes speed up malloc by increasing value.
2113
  Doing this means that malloc may choose a chunk that is 
2114
  non-best-fitting by up to the width of the bin.
2115
 
2116
  Some useful cutoff values:
2117
      512 - all bins sorted
2118
     2560 - leaves bins <=     64 bytes wide unsorted  
2119
    12288 - leaves bins <=    512 bytes wide unsorted
2120
    65536 - leaves bins <=   4096 bytes wide unsorted
2121
   262144 - leaves bins <=  32768 bytes wide unsorted
2122
       -1 - no bins sorted (not recommended!)
10347 ripley 2123
*/
2124
 
26855 ripley 2125
#define FIRST_SORTED_BIN_SIZE MIN_LARGE_SIZE 
2126
/* #define FIRST_SORTED_BIN_SIZE 65536 */
10347 ripley 2127
 
26855 ripley 2128
/*
2129
  Unsorted chunks
10347 ripley 2130
 
26855 ripley 2131
    All remainders from chunk splits, as well as all returned chunks,
2132
    are first placed in the "unsorted" bin. They are then placed
2133
    in regular bins after malloc gives them ONE chance to be used before
2134
    binning. So, basically, the unsorted_chunks list acts as a queue,
2135
    with chunks being placed on it in free (and malloc_consolidate),
2136
    and taken off (to be either used or placed in bins) in malloc.
2137
*/
2138
 
2139
/* The otherwise unindexable 1-bin is used to hold unsorted chunks. */
2140
#define unsorted_chunks(M)          (bin_at(M, 1))
2141
 
10347 ripley 2142
/*
26855 ripley 2143
  Top
2144
 
2145
    The top-most available chunk (i.e., the one bordering the end of
2146
    available memory) is treated specially. It is never included in
2147
    any bin, is used only if no other chunk is available, and is
2148
    released back to the system if it is very large (see
2149
    M_TRIM_THRESHOLD).  Because top initially
2150
    points to its own bin with initial zero size, thus forcing
2151
    extension on the first malloc request, we avoid having any special
2152
    code in malloc to check whether it even exists yet. But we still
2153
    need to do so when getting memory from system, so we make
2154
    initial_top treat the bin as a legal but unusable chunk during the
2155
    interval between initialization and the first call to
2156
    sYSMALLOc. (This is somewhat delicate, since it relies on
2157
    the 2 preceding words to be zero during this interval as well.)
10347 ripley 2158
*/
2159
 
26855 ripley 2160
/* Conveniently, the unsorted bin can be used as dummy top on first call */
2161
#define initial_top(M)              (unsorted_chunks(M))
10347 ripley 2162
 
26855 ripley 2163
/*
2164
  Binmap
10347 ripley 2165
 
26855 ripley 2166
    To help compensate for the large number of bins, a one-level index
2167
    structure is used for bin-by-bin searching.  `binmap' is a
2168
    bitvector recording whether bins are definitely empty so they can
2169
    be skipped over during during traversals.  The bits are NOT always
2170
    cleared as soon as bins are empty, but instead only
2171
    when they are noticed to be empty during traversal in malloc.
2172
*/
10347 ripley 2173
 
26855 ripley 2174
/* Conservatively use 32 bits per map word, even if on 64bit system */
2175
#define BINMAPSHIFT      5
2176
#define BITSPERMAP       (1U << BINMAPSHIFT)
2177
#define BINMAPSIZE       (NBINS / BITSPERMAP)
10347 ripley 2178
 
26855 ripley 2179
#define idx2block(i)     ((i) >> BINMAPSHIFT)
2180
#define idx2bit(i)       ((1U << ((i) & ((1U << BINMAPSHIFT)-1))))
10347 ripley 2181
 
26855 ripley 2182
#define mark_bin(m,i)    ((m)->binmap[idx2block(i)] |=  idx2bit(i))
2183
#define unmark_bin(m,i)  ((m)->binmap[idx2block(i)] &= ~(idx2bit(i)))
2184
#define get_binmap(m,i)  ((m)->binmap[idx2block(i)] &   idx2bit(i))
10347 ripley 2185
 
26855 ripley 2186
/*
2187
  Fastbins
10347 ripley 2188
 
26855 ripley 2189
    An array of lists holding recently freed small chunks.  Fastbins
2190
    are not doubly linked.  It is faster to single-link them, and
2191
    since chunks are never removed from the middles of these lists,
2192
    double linking is not necessary. Also, unlike regular bins, they
2193
    are not even processed in FIFO order (they use faster LIFO) since
2194
    ordering doesn't much matter in the transient contexts in which
2195
    fastbins are normally used.
2196
 
2197
    Chunks in fastbins keep their inuse bit set, so they cannot
2198
    be consolidated with other free chunks. malloc_consolidate
2199
    releases all chunks in fastbins and consolidates them with
2200
    other free chunks. 
2201
*/
2202
 
2203
typedef struct malloc_chunk* mfastbinptr;
2204
 
2205
/* offset 2 to use otherwise unindexable first 2 bins */
2206
#define fastbin_index(sz)        ((((unsigned int)(sz)) >> 3) - 2)
2207
 
2208
/* The maximum fastbin request size we support */
2209
#define MAX_FAST_SIZE     80
2210
 
2211
#define NFASTBINS  (fastbin_index(request2size(MAX_FAST_SIZE))+1)
2212
 
10347 ripley 2213
/*
26855 ripley 2214
  FASTBIN_CONSOLIDATION_THRESHOLD is the size of a chunk in free()
2215
  that triggers automatic consolidation of possibly-surrounding
2216
  fastbin chunks. This is a heuristic, so the exact value should not
2217
  matter too much. It is defined at half the default trim threshold as a
2218
  compromise heuristic to only attempt consolidation if it is likely
2219
  to lead to trimming. However, it is not dynamically tunable, since
2220
  consolidation reduces fragmentation surrounding loarge chunks even 
2221
  if trimming is not used.
10347 ripley 2222
*/
2223
 
26855 ripley 2224
#define FASTBIN_CONSOLIDATION_THRESHOLD  \
2225
  ((unsigned long)(DEFAULT_TRIM_THRESHOLD) >> 1)
2226
 
10347 ripley 2227
/*
26855 ripley 2228
  Since the lowest 2 bits in max_fast don't matter in size comparisons, 
2229
  they are used as flags.
10347 ripley 2230
*/
2231
 
26855 ripley 2232
/*
2233
  ANYCHUNKS_BIT held in max_fast indicates that there may be any
2234
  freed chunks at all. It is set true when entering a chunk into any
2235
  bin.
2236
*/
10347 ripley 2237
 
26855 ripley 2238
#define ANYCHUNKS_BIT        (1U)
10347 ripley 2239
 
26855 ripley 2240
#define have_anychunks(M)     (((M)->max_fast &  ANYCHUNKS_BIT))
2241
#define set_anychunks(M)      ((M)->max_fast |=  ANYCHUNKS_BIT)
2242
#define clear_anychunks(M)    ((M)->max_fast &= ~ANYCHUNKS_BIT)
2243
 
10347 ripley 2244
/*
26855 ripley 2245
  FASTCHUNKS_BIT held in max_fast indicates that there are probably
2246
  some fastbin chunks. It is set true on entering a chunk into any
2247
  fastbin, and cleared only in malloc_consolidate.
10347 ripley 2248
*/
2249
 
26855 ripley 2250
#define FASTCHUNKS_BIT        (2U)
10347 ripley 2251
 
26855 ripley 2252
#define have_fastchunks(M)   (((M)->max_fast &  FASTCHUNKS_BIT))
2253
#define set_fastchunks(M)    ((M)->max_fast |=  (FASTCHUNKS_BIT|ANYCHUNKS_BIT))
2254
#define clear_fastchunks(M)  ((M)->max_fast &= ~(FASTCHUNKS_BIT))
10347 ripley 2255
 
26855 ripley 2256
/* 
2257
   Set value of max_fast. 
2258
   Use impossibly small value if 0.
2259
*/
2260
 
2261
#define set_max_fast(M, s) \
2262
  (M)->max_fast = (((s) == 0)? SMALLBIN_WIDTH: request2size(s)) | \
2263
  ((M)->max_fast &  (FASTCHUNKS_BIT|ANYCHUNKS_BIT))
2264
 
2265
#define get_max_fast(M) \
2266
  ((M)->max_fast & ~(FASTCHUNKS_BIT | ANYCHUNKS_BIT))
2267
 
2268
 
10347 ripley 2269
/*
26855 ripley 2270
  morecore_properties is a status word holding dynamically discovered
2271
  or controlled properties of the morecore function
10347 ripley 2272
*/
2273
 
26855 ripley 2274
#define MORECORE_CONTIGUOUS_BIT  (1U)
10347 ripley 2275
 
26855 ripley 2276
#define contiguous(M) \
2277
        (((M)->morecore_properties &  MORECORE_CONTIGUOUS_BIT))
2278
#define noncontiguous(M) \
2279
        (((M)->morecore_properties &  MORECORE_CONTIGUOUS_BIT) == 0)
2280
#define set_contiguous(M) \
2281
        ((M)->morecore_properties |=  MORECORE_CONTIGUOUS_BIT)
2282
#define set_noncontiguous(M) \
2283
        ((M)->morecore_properties &= ~MORECORE_CONTIGUOUS_BIT)
10347 ripley 2284
 
2285
 
26855 ripley 2286
/*
2287
   ----------- Internal state representation and initialization -----------
2288
*/
10347 ripley 2289
 
26855 ripley 2290
struct malloc_state {
10347 ripley 2291
 
26855 ripley 2292
  /* The maximum chunk size to be eligible for fastbin */
2293
  INTERNAL_SIZE_T  max_fast;   /* low 2 bits used as flags */
10347 ripley 2294
 
26855 ripley 2295
  /* Fastbins */
2296
  mfastbinptr      fastbins[NFASTBINS];
10347 ripley 2297
 
26855 ripley 2298
  /* Base of the topmost chunk -- not otherwise kept in a bin */
2299
  mchunkptr        top;
10347 ripley 2300
 
26855 ripley 2301
  /* The remainder from the most recent split of a small request */
2302
  mchunkptr        last_remainder;
10347 ripley 2303
 
26855 ripley 2304
  /* Normal bins packed as described above */
2305
  mchunkptr        bins[NBINS * 2];
10347 ripley 2306
 
26855 ripley 2307
  /* Bitmap of bins. Trailing zero map handles cases of largest binned size */
2308
  unsigned int     binmap[BINMAPSIZE+1];
10347 ripley 2309
 
26855 ripley 2310
  /* Tunable parameters */
2311
  CHUNK_SIZE_T     trim_threshold;
2312
  INTERNAL_SIZE_T  top_pad;
2313
  INTERNAL_SIZE_T  mmap_threshold;
10347 ripley 2314
 
26855 ripley 2315
  /* Memory map support */
2316
  int              n_mmaps;
2317
  int              n_mmaps_max;
2318
  int              max_n_mmaps;
10347 ripley 2319
 
26855 ripley 2320
  /* Cache malloc_getpagesize */
2321
  unsigned int     pagesize;    
10347 ripley 2322
 
26855 ripley 2323
  /* Track properties of MORECORE */
2324
  unsigned int     morecore_properties;
10347 ripley 2325
 
26855 ripley 2326
  /* Statistics */
2327
  INTERNAL_SIZE_T  mmapped_mem;
2328
  INTERNAL_SIZE_T  sbrked_mem;
2329
  INTERNAL_SIZE_T  max_sbrked_mem;
2330
  INTERNAL_SIZE_T  max_mmapped_mem;
2331
  INTERNAL_SIZE_T  max_total_mem;
2332
};
10347 ripley 2333
 
26855 ripley 2334
typedef struct malloc_state *mstate;
10347 ripley 2335
 
26855 ripley 2336
/* 
2337
   There is exactly one instance of this struct in this malloc.
2338
   If you are adapting this malloc in a way that does NOT use a static
2339
   malloc_state, you MUST explicitly zero-fill it before using. This
2340
   malloc relies on the property that malloc_state is initialized to
2341
   all zeroes (as is true of C statics).
2342
*/
10347 ripley 2343
 
26855 ripley 2344
static struct malloc_state av_;  /* never directly referenced */
2345
 
10347 ripley 2346
/*
26855 ripley 2347
   All uses of av_ are via get_malloc_state().
2348
   At most one "call" to get_malloc_state is made per invocation of
2349
   the public versions of malloc and free, but other routines
2350
   that in turn invoke malloc and/or free may call more then once. 
2351
   Also, it is called in check* routines if DEBUG is set.
10347 ripley 2352
*/
2353
 
26855 ripley 2354
#define get_malloc_state() (&(av_))
10347 ripley 2355
 
26855 ripley 2356
/*
2357
  Initialize a malloc_state struct.
10347 ripley 2358
 
26855 ripley 2359
  This is called only from within malloc_consolidate, which needs
2360
  be called in the same contexts anyway.  It is never called directly
2361
  outside of malloc_consolidate because some optimizing compilers try
2362
  to inline it at all call points, which turns out not to be an
2363
  optimization at all. (Inlining it in malloc_consolidate is fine though.)
2364
*/
2365
 
2366
static void malloc_init_state(mstate av)
2367
{
2368
  int     i;
2369
  mbinptr bin;
2370
 
2371
  /* Establish circular links for normal bins */
2372
  for (i = 1; i < NBINS; ++i) { 
2373
    bin = bin_at(av,i);
2374
    bin->fd = bin->bk = bin;
2375
  }
2376
 
2377
  av->top_pad        = DEFAULT_TOP_PAD;
2378
  av->n_mmaps_max    = DEFAULT_MMAP_MAX;
2379
  av->mmap_threshold = DEFAULT_MMAP_THRESHOLD;
2380
  av->trim_threshold = DEFAULT_TRIM_THRESHOLD;
2381
 
2382
#if MORECORE_CONTIGUOUS
2383
  set_contiguous(av);
2384
#else
2385
  set_noncontiguous(av);
2386
#endif
2387
 
2388
 
2389
  set_max_fast(av, DEFAULT_MXFAST);
2390
 
2391
  av->top            = initial_top(av);
2392
  av->pagesize       = malloc_getpagesize;
2393
}
2394
 
2395
/* 
2396
   Other internal utilities operating on mstates
2397
*/
2398
 
2399
static Void_t*  sYSMALLOc(INTERNAL_SIZE_T, mstate);
2400
#ifndef MORECORE_CANNOT_TRIM        
2401
static int      sYSTRIm(size_t, mstate);
2402
#endif
2403
static void     malloc_consolidate(mstate);
2404
static Void_t** iALLOc(size_t, size_t*, int, Void_t**);
2405
 
10347 ripley 2406
/*
26855 ripley 2407
  Debugging support
2408
 
10347 ripley 2409
  These routines make a number of assertions about the states
2410
  of data structures that should be true at all times. If any
2411
  are not true, it's very likely that a user program has somehow
2412
  trashed memory. (It's also possible that there is a coding error
2413
  in malloc. In which case, please report it!)
2414
*/
2415
 
26855 ripley 2416
#if ! DEBUG
2417
 
2418
#define check_chunk(P)
2419
#define check_free_chunk(P)
2420
#define check_inuse_chunk(P)
2421
#define check_remalloced_chunk(P,N)
2422
#define check_malloced_chunk(P,N)
2423
#define check_malloc_state()
2424
 
2425
#else
2426
#define check_chunk(P)              do_check_chunk(P)
2427
#define check_free_chunk(P)         do_check_free_chunk(P)
2428
#define check_inuse_chunk(P)        do_check_inuse_chunk(P)
2429
#define check_remalloced_chunk(P,N) do_check_remalloced_chunk(P,N)
2430
#define check_malloced_chunk(P,N)   do_check_malloced_chunk(P,N)
2431
#define check_malloc_state()        do_check_malloc_state()
2432
 
2433
/*
2434
  Properties of all chunks
2435
*/
2436
 
10347 ripley 2437
static void do_check_chunk(mchunkptr p)
2438
{
26855 ripley 2439
  mstate av = get_malloc_state();
2440
  CHUNK_SIZE_T  sz = chunksize(p);
2441
  /* min and max possible addresses assuming contiguous allocation */
2442
  char* max_address = (char*)(av->top) + chunksize(av->top);
2443
  char* min_address = max_address - av->sbrked_mem;
10347 ripley 2444
 
26855 ripley 2445
  if (!chunk_is_mmapped(p)) {
2446
 
2447
    /* Has legal address ... */
2448
    if (p != av->top) {
2449
      if (contiguous(av)) {
2450
        assert(((char*)p) >= min_address);
2451
        assert(((char*)p + sz) <= ((char*)(av->top)));
2452
      }
2453
    }
2454
    else {
2455
      /* top size is always at least MINSIZE */
2456
      assert((CHUNK_SIZE_T)(sz) >= MINSIZE);
2457
      /* top predecessor always marked inuse */
2458
      assert(prev_inuse(p));
2459
    }
2460
 
2461
  }
2462
  else {
2463
#if HAVE_MMAP
2464
    /* address is outside main heap  */
2465
    if (contiguous(av) && av->top != initial_top(av)) {
2466
      assert(((char*)p) < min_address || ((char*)p) > max_address);
2467
    }
2468
    /* chunk is page-aligned */
2469
    assert(((p->prev_size + sz) & (av->pagesize-1)) == 0);
2470
    /* mem is aligned */
2471
    assert(aligned_OK(chunk2mem(p)));
2472
#else
2473
    /* force an appropriate assert violation if debug set */
2474
    assert(!chunk_is_mmapped(p));
2475
#endif
2476
  }
10347 ripley 2477
}
2478
 
26855 ripley 2479
/*
2480
  Properties of free chunks
2481
*/
10347 ripley 2482
 
2483
static void do_check_free_chunk(mchunkptr p)
2484
{
26855 ripley 2485
  mstate av = get_malloc_state();
2486
 
10347 ripley 2487
  INTERNAL_SIZE_T sz = p->size & ~PREV_INUSE;
2488
  mchunkptr next = chunk_at_offset(p, sz);
2489
 
2490
  do_check_chunk(p);
2491
 
26855 ripley 2492
  /* Chunk must claim to be free ... */
10347 ripley 2493
  assert(!inuse(p));
26855 ripley 2494
  assert (!chunk_is_mmapped(p));
10347 ripley 2495
 
2496
  /* Unless a special marker, must have OK fields */
26855 ripley 2497
  if ((CHUNK_SIZE_T)(sz) >= MINSIZE)
10347 ripley 2498
  {
2499
    assert((sz & MALLOC_ALIGN_MASK) == 0);
2500
    assert(aligned_OK(chunk2mem(p)));
2501
    /* ... matching footer field */
2502
    assert(next->prev_size == sz);
2503
    /* ... and is fully consolidated */
2504
    assert(prev_inuse(p));
26855 ripley 2505
    assert (next == av->top || inuse(next));
10347 ripley 2506
 
2507
    /* ... and has minimally sane links */
2508
    assert(p->fd->bk == p);
2509
    assert(p->bk->fd == p);
2510
  }
2511
  else /* markers are always of size SIZE_SZ */
2512
    assert(sz == SIZE_SZ);
2513
}
2514
 
26855 ripley 2515
/*
2516
  Properties of inuse chunks
2517
*/
2518
 
10347 ripley 2519
static void do_check_inuse_chunk(mchunkptr p)
2520
{
26855 ripley 2521
  mstate av = get_malloc_state();
2522
  mchunkptr next;
10347 ripley 2523
  do_check_chunk(p);
2524
 
26855 ripley 2525
  if (chunk_is_mmapped(p))
2526
    return; /* mmapped chunks have no next/prev */
2527
 
10347 ripley 2528
  /* Check whether it claims to be in use ... */
2529
  assert(inuse(p));
2530
 
26855 ripley 2531
  next = next_chunk(p);
2532
 
10347 ripley 2533
  /* ... and is surrounded by OK chunks.
2534
    Since more things can be checked with free chunks than inuse ones,
2535
    if an inuse chunk borders them and debug is on, it's worth doing them.
2536
  */
26855 ripley 2537
  if (!prev_inuse(p))  {
2538
    /* Note that we cannot even look at prev unless it is not inuse */
10347 ripley 2539
    mchunkptr prv = prev_chunk(p);
2540
    assert(next_chunk(prv) == p);
2541
    do_check_free_chunk(prv);
2542
  }
26855 ripley 2543
 
2544
  if (next == av->top) {
10347 ripley 2545
    assert(prev_inuse(next));
2546
    assert(chunksize(next) >= MINSIZE);
2547
  }
2548
  else if (!inuse(next))
2549
    do_check_free_chunk(next);
2550
}
2551
 
26855 ripley 2552
/*
2553
  Properties of chunks recycled from fastbins
2554
*/
2555
 
2556
static void do_check_remalloced_chunk(mchunkptr p, INTERNAL_SIZE_T s)
10347 ripley 2557
{
2558
  INTERNAL_SIZE_T sz = p->size & ~PREV_INUSE;
2559
 
2560
  do_check_inuse_chunk(p);
2561
 
2562
  /* Legal size ... */
2563
  assert((sz & MALLOC_ALIGN_MASK) == 0);
26855 ripley 2564
  assert((CHUNK_SIZE_T)(sz) >= MINSIZE);
10347 ripley 2565
  /* ... and alignment */
2566
  assert(aligned_OK(chunk2mem(p)));
26855 ripley 2567
  /* chunk is less than MINSIZE more than request */
2568
  assert((long)(sz) - (long)(s) >= 0);
2569
  assert((long)(sz) - (long)(s + MINSIZE) < 0);
10347 ripley 2570
}
2571
 
26855 ripley 2572
/*
2573
  Properties of nonrecycled chunks at the point they are malloced
2574
*/
10347 ripley 2575
 
26855 ripley 2576
static void do_check_malloced_chunk(mchunkptr p, INTERNAL_SIZE_T s)
2577
{
2578
  /* same as recycled case ... */
2579
  do_check_remalloced_chunk(p, s);
10347 ripley 2580
 
26855 ripley 2581
  /*
2582
    ... plus,  must obey implementation invariant that prev_inuse is
2583
    always true of any allocated chunk; i.e., that each allocated
2584
    chunk borders either a previously allocated and still in-use
2585
    chunk, or the base of its memory arena. This is ensured
2586
    by making all allocations from the the `lowest' part of any found
2587
    chunk.  This does not necessarily hold however for chunks
2588
    recycled via fastbins.
2589
  */
10347 ripley 2590
 
26855 ripley 2591
  assert(prev_inuse(p));
2592
}
10347 ripley 2593
 
2594
 
2595
/*
26855 ripley 2596
  Properties of malloc_state.
10347 ripley 2597
 
26855 ripley 2598
  This may be useful for debugging malloc, as well as detecting user
2599
  programmer errors that somehow write into malloc_state.
2600
 
2601
  If you are extending or experimenting with this malloc, you can
2602
  probably figure out how to hack this routine to print out or
2603
  display chunk addresses, sizes, bins, and other instrumentation.
10347 ripley 2604
*/
2605
 
26855 ripley 2606
static void do_check_malloc_state()
2607
{
2608
  mstate av = get_malloc_state();
2609
  int i;
2610
  mchunkptr p;
2611
  mchunkptr q;
2612
  mbinptr b;
2613
  unsigned int binbit;
2614
  int empty;
2615
  unsigned int idx;
2616
  INTERNAL_SIZE_T size;
2617
  CHUNK_SIZE_T  total = 0;
2618
  int max_fast_bin;
10347 ripley 2619
 
26855 ripley 2620
  /* internal size_t must be no wider than pointer type */
2621
  assert(sizeof(INTERNAL_SIZE_T) <= sizeof(char*));
10347 ripley 2622
 
26855 ripley 2623
  /* alignment is a power of 2 */
2624
  assert((MALLOC_ALIGNMENT & (MALLOC_ALIGNMENT-1)) == 0);
10347 ripley 2625
 
26855 ripley 2626
  /* cannot run remaining checks until fully initialized */
2627
  if (av->top == 0 || av->top == initial_top(av))
2628
    return;
10347 ripley 2629
 
26855 ripley 2630
  /* pagesize is a power of 2 */
2631
  assert((av->pagesize & (av->pagesize-1)) == 0);
10347 ripley 2632
 
26855 ripley 2633
  /* properties of fastbins */
10347 ripley 2634
 
26855 ripley 2635
  /* max_fast is in allowed range */
2636
  assert(get_max_fast(av) <= request2size(MAX_FAST_SIZE));
10347 ripley 2637
 
26855 ripley 2638
  max_fast_bin = fastbin_index(av->max_fast);
10347 ripley 2639
 
26855 ripley 2640
  for (i = 0; i < NFASTBINS; ++i) {
2641
    p = av->fastbins[i];
10347 ripley 2642
 
26855 ripley 2643
    /* all bins past max_fast are empty */
2644
    if (i > max_fast_bin)
2645
      assert(p == 0);
10347 ripley 2646
 
26855 ripley 2647
    while (p != 0) {
2648
      /* each chunk claims to be inuse */
2649
      do_check_inuse_chunk(p);
2650
      total += chunksize(p);
2651
      /* chunk belongs in this bin */
2652
      assert(fastbin_index(chunksize(p)) == i);
2653
      p = p->fd;
2654
    }
2655
  }
10347 ripley 2656
 
26855 ripley 2657
  if (total != 0)
2658
    assert(have_fastchunks(av));
2659
  else if (!have_fastchunks(av))
2660
    assert(total == 0);
10347 ripley 2661
 
26855 ripley 2662
  /* check normal bins */
2663
  for (i = 1; i < NBINS; ++i) {
2664
    b = bin_at(av,i);
10347 ripley 2665
 
26855 ripley 2666
    /* binmap is accurate (except for bin 1 == unsorted_chunks) */
2667
    if (i >= 2) {
2668
      binbit = get_binmap(av,i);
2669
      empty = last(b) == b;
2670
      if (!binbit)
2671
        assert(empty);
2672
      else if (!empty)
2673
        assert(binbit);
2674
    }
10347 ripley 2675
 
26855 ripley 2676
    for (p = last(b); p != b; p = p->bk) {
2677
      /* each chunk claims to be free */
2678
      do_check_free_chunk(p);
2679
      size = chunksize(p);
2680
      total += size;
2681
      if (i >= 2) {
2682
        /* chunk belongs in bin */
2683
        idx = bin_index(size);
2684
        assert(idx == i);
2685
        /* lists are sorted */
2686
        if ((CHUNK_SIZE_T) size >= (CHUNK_SIZE_T)(FIRST_SORTED_BIN_SIZE)) {
2687
          assert(p->bk == b || 
2688
                 (CHUNK_SIZE_T)chunksize(p->bk) >= 
2689
                 (CHUNK_SIZE_T)chunksize(p));
2690
        }
2691
      }
2692
      /* chunk is followed by a legal chain of inuse chunks */
2693
      for (q = next_chunk(p);
2694
           (q != av->top && inuse(q) && 
2695
             (CHUNK_SIZE_T)(chunksize(q)) >= MINSIZE);
2696
           q = next_chunk(q))
2697
        do_check_inuse_chunk(q);
2698
    }
2699
  }
10347 ripley 2700
 
26855 ripley 2701
  /* top chunk is OK */
2702
  check_chunk(av->top);
10347 ripley 2703
 
26855 ripley 2704
  /* sanity checks for statistics */
10347 ripley 2705
 
26855 ripley 2706
  assert(total <= (CHUNK_SIZE_T)(av->max_total_mem));
2707
  assert(av->n_mmaps >= 0);
2708
  assert(av->n_mmaps <= av->max_n_mmaps);
10347 ripley 2709
 
26855 ripley 2710
  assert((CHUNK_SIZE_T)(av->sbrked_mem) <=
2711
         (CHUNK_SIZE_T)(av->max_sbrked_mem));
10347 ripley 2712
 
26855 ripley 2713
  assert((CHUNK_SIZE_T)(av->mmapped_mem) <=
2714
         (CHUNK_SIZE_T)(av->max_mmapped_mem));
2715
 
2716
  assert((CHUNK_SIZE_T)(av->max_total_mem) >=
2717
         (CHUNK_SIZE_T)(av->mmapped_mem) + (CHUNK_SIZE_T)(av->sbrked_mem));
2718
}
10347 ripley 2719
#endif
2720
 
2721
 
26855 ripley 2722
/* ----------- Routines dealing with system allocation -------------- */
10347 ripley 2723
 
26855 ripley 2724
/*
2725
  sysmalloc handles malloc cases requiring more memory from the system.
2726
  On entry, it is assumed that av->top does not have enough
2727
  space to service request for nb bytes, thus requiring that av->top
2728
  be extended or replaced.
2729
*/
10347 ripley 2730
 
26855 ripley 2731
static Void_t* sYSMALLOc(INTERNAL_SIZE_T nb, mstate av)
10347 ripley 2732
{
26855 ripley 2733
  mchunkptr       old_top;        /* incoming value of av->top */
2734
  INTERNAL_SIZE_T old_size;       /* its size */
2735
  char*           old_end;        /* its end address */
10347 ripley 2736
 
26855 ripley 2737
  long            size;           /* arg to first MORECORE or mmap call */
2738
  char*           brk;            /* return value from MORECORE */
10347 ripley 2739
 
26855 ripley 2740
  long            correction;     /* arg to 2nd MORECORE call */
2741
  char*           snd_brk;        /* 2nd return val */
10347 ripley 2742
 
26855 ripley 2743
  INTERNAL_SIZE_T front_misalign; /* unusable bytes at front of new space */
2744
  INTERNAL_SIZE_T end_misalign;   /* partial page left at end of new space */
2745
  char*           aligned_brk;    /* aligned offset into brk */
10347 ripley 2746
 
26855 ripley 2747
  mchunkptr       p;              /* the allocated/returned chunk */
2748
  mchunkptr       remainder;      /* remainder from allocation */
2749
  CHUNK_SIZE_T    remainder_size; /* its size */
10347 ripley 2750
 
26855 ripley 2751
  CHUNK_SIZE_T    sum;            /* for updating stats */
10347 ripley 2752
 
26855 ripley 2753
  size_t          pagemask  = av->pagesize - 1;
10347 ripley 2754
 
26855 ripley 2755
  /*
2756
    If there is space available in fastbins, consolidate and retry
2757
    malloc from scratch rather than getting memory from system.  This
2758
    can occur only if nb is in smallbin range so we didn't consolidate
2759
    upon entry to malloc. It is much easier to handle this case here
2760
    than in malloc proper.
2761
  */
10347 ripley 2762
 
26855 ripley 2763
  if (have_fastchunks(av)) {
2764
    assert(in_smallbin_range(nb));
2765
    malloc_consolidate(av);
2766
    return mALLOc(nb - MALLOC_ALIGN_MASK);
2767
  }
10347 ripley 2768
 
2769
 
26855 ripley 2770
#if HAVE_MMAP
10347 ripley 2771
 
26855 ripley 2772
  /*
2773
    If have mmap, and the request size meets the mmap threshold, and
2774
    the system supports mmap, and there are few enough currently
2775
    allocated mmapped regions, try to directly map this request
2776
    rather than expanding top.
2777
  */
10347 ripley 2778
 
26855 ripley 2779
  if ((CHUNK_SIZE_T)(nb) >= (CHUNK_SIZE_T)(av->mmap_threshold) &&
2780
      (av->n_mmaps < av->n_mmaps_max)) {
10347 ripley 2781
 
26855 ripley 2782
    char* mm;             /* return value from mmap call*/
10347 ripley 2783
 
26855 ripley 2784
    /*
2785
      Round up size to nearest page.  For mmapped chunks, the overhead
2786
      is one SIZE_SZ unit larger than for normal chunks, because there
2787
      is no following chunk whose prev_size field could be used.
2788
    */
2789
    size = (nb + SIZE_SZ + MALLOC_ALIGN_MASK + pagemask) & ~pagemask;
10347 ripley 2790
 
26945 ripley 2791
    /* DBT - Check that we are not about to exceed the R memory limit */
2792
    if (size + av->mmapped_mem + av->sbrked_mem > R_max_memory) {
2793
	 if(R_Is_Running) 
2794
	     Rf_warning("Reached total allocation of %dMb: see help(memory.size)", R_max_memory/1048576);
2795
	 return (void*) 0;
2796
    }
2797
 
26855 ripley 2798
    /* Don't try if size wraps around 0 */
2799
    if ((CHUNK_SIZE_T)(size) > (CHUNK_SIZE_T)(nb)) {
10347 ripley 2800
 
26855 ripley 2801
      mm = (char*)(MMAP(0, size, PROT_READ|PROT_WRITE, MAP_PRIVATE));
2802
 
2803
      if (mm != (char*)(MORECORE_FAILURE)) {
2804
 
2805
        /*
2806
          The offset to the start of the mmapped region is stored
2807
          in the prev_size field of the chunk. This allows us to adjust
2808
          returned start address to meet alignment requirements here 
2809
          and in memalign(), and still be able to compute proper
2810
          address argument for later munmap in free() and realloc().
2811
        */
2812
 
2813
        front_misalign = (INTERNAL_SIZE_T)chunk2mem(mm) & MALLOC_ALIGN_MASK;
2814
        if (front_misalign > 0) {
2815
          correction = MALLOC_ALIGNMENT - front_misalign;
2816
          p = (mchunkptr)(mm + correction);
2817
          p->prev_size = correction;
2818
          set_head(p, (size - correction) |IS_MMAPPED);
2819
        }
2820
        else {
2821
          p = (mchunkptr)mm;
2822
          p->prev_size = 0;
2823
          set_head(p, size|IS_MMAPPED);
2824
        }
2825
 
2826
        /* update statistics */
2827
 
2828
        if (++av->n_mmaps > av->max_n_mmaps) 
2829
          av->max_n_mmaps = av->n_mmaps;
2830
 
2831
        sum = av->mmapped_mem += size;
2832
        if (sum > (CHUNK_SIZE_T)(av->max_mmapped_mem)) 
2833
          av->max_mmapped_mem = sum;
2834
        sum += av->sbrked_mem;
2835
        if (sum > (CHUNK_SIZE_T)(av->max_total_mem)) 
2836
          av->max_total_mem = sum;
10347 ripley 2837
 
26855 ripley 2838
        check_chunk(p);
2839
 
2840
        return chunk2mem(p);
2841
      }
2842
    }
2843
  }
2844
#endif
10347 ripley 2845
 
26855 ripley 2846
  /* Record incoming configuration of top */
10347 ripley 2847
 
26855 ripley 2848
  old_top  = av->top;
2849
  old_size = chunksize(old_top);
2850
  old_end  = (char*)(chunk_at_offset(old_top, old_size));
10347 ripley 2851
 
26855 ripley 2852
  brk = snd_brk = (char*)(MORECORE_FAILURE); 
10347 ripley 2853
 
26855 ripley 2854
  /* 
2855
     If not the first time through, we require old_size to be
2856
     at least MINSIZE and to have prev_inuse set.
2857
  */
10347 ripley 2858
 
26855 ripley 2859
  assert((old_top == initial_top(av) && old_size == 0) || 
2860
         ((CHUNK_SIZE_T) (old_size) >= MINSIZE &&
2861
          prev_inuse(old_top)));
10347 ripley 2862
 
26855 ripley 2863
  /* Precondition: not enough current space to satisfy nb request */
2864
  assert((CHUNK_SIZE_T)(old_size) < (CHUNK_SIZE_T)(nb + MINSIZE));
10347 ripley 2865
 
26855 ripley 2866
  /* Precondition: all fastbins are consolidated */
2867
  assert(!have_fastchunks(av));
10347 ripley 2868
 
2869
 
26855 ripley 2870
  /* Request enough space for nb + pad + overhead */
10347 ripley 2871
 
26855 ripley 2872
  size = nb + av->top_pad + MINSIZE;
10347 ripley 2873
 
26855 ripley 2874
  /*
2875
    If contiguous, we can subtract out existing space that we hope to
2876
    combine with new space. We add it back later only if
2877
    we don't actually get contiguous space.
2878
  */
10347 ripley 2879
 
26855 ripley 2880
  if (contiguous(av))
2881
    size -= old_size;
10347 ripley 2882
 
26855 ripley 2883
  /*
2884
    Round to a multiple of page size.
2885
    If MORECORE is not contiguous, this ensures that we only call it
2886
    with whole-page arguments.  And if MORECORE is contiguous and
2887
    this is not first time through, this preserves page-alignment of
2888
    previous calls. Otherwise, we correct to page-align below.
2889
  */
10347 ripley 2890
 
26855 ripley 2891
  size = (size + pagemask) & ~pagemask;
10347 ripley 2892
 
26945 ripley 2893
  /* DBT - Check that we are not about to exceed the R memory limit */
2894
  if (size + av->mmapped_mem + av->sbrked_mem > R_max_memory) {
2895
	if(R_Is_Running) 
2896
	    Rf_warning("Reached total allocation of %dMb: see help(memory.size)", R_max_memory/1048576);
2897
	return (void*)-1;
2898
  }
2899
 
26855 ripley 2900
  /*
2901
    Don't try to call MORECORE if argument is so big as to appear
2902
    negative. Note that since mmap takes size_t arg, it may succeed
2903
    below even if we cannot call MORECORE.
2904
  */
10425 ripley 2905
 
26855 ripley 2906
  if (size > 0) 
2907
    brk = (char*)(MORECORE(size));
10347 ripley 2908
 
26855 ripley 2909
  /*
2910
    If have mmap, try using it as a backup when MORECORE fails or
2911
    cannot be used. This is worth doing on systems that have "holes" in
2912
    address space, so sbrk cannot extend to give contiguous space, but
2913
    space is available elsewhere.  Note that we ignore mmap max count
2914
    and threshold limits, since the space will not be used as a
2915
    segregated mmap region.
2916
  */
10347 ripley 2917
 
26855 ripley 2918
#if HAVE_MMAP
2919
  if (brk == (char*)(MORECORE_FAILURE)) {
10347 ripley 2920
 
26855 ripley 2921
    /* Cannot merge with old top, so add its size back in */
2922
    if (contiguous(av))
2923
      size = (size + old_size + pagemask) & ~pagemask;
10347 ripley 2924
 
26855 ripley 2925
    /* If we are relying on mmap as backup, then use larger units */
2926
    if ((CHUNK_SIZE_T)(size) < (CHUNK_SIZE_T)(MMAP_AS_MORECORE_SIZE))
2927
      size = MMAP_AS_MORECORE_SIZE;
10347 ripley 2928
 
26855 ripley 2929
    /* Don't try if size wraps around 0 */
2930
    if ((CHUNK_SIZE_T)(size) > (CHUNK_SIZE_T)(nb)) {
2931
 
2932
      brk = (char*)(MMAP(0, size, PROT_READ|PROT_WRITE, MAP_PRIVATE));
2933
 
2934
      if (brk != (char*)(MORECORE_FAILURE)) {
2935
 
2936
        /* We do not need, and cannot use, another sbrk call to find end */
2937
        snd_brk = brk + size;
2938
 
2939
        /* 
2940
           Record that we no longer have a contiguous sbrk region. 
2941
           After the first time mmap is used as backup, we do not
2942
           ever rely on contiguous space since this could incorrectly
2943
           bridge regions.
2944
        */
2945
        set_noncontiguous(av);
10347 ripley 2946
      }
2947
    }
2948
  }
26855 ripley 2949
#endif
10347 ripley 2950
 
26855 ripley 2951
  if (brk != (char*)(MORECORE_FAILURE)) {
2952
    av->sbrked_mem += size;
10347 ripley 2953
 
26855 ripley 2954
    /*
2955
      If MORECORE extends previous space, we can likewise extend top size.
2956
    */
2957
 
2958
    if (brk == old_end && snd_brk == (char*)(MORECORE_FAILURE)) {
2959
      set_head(old_top, (size + old_size) | PREV_INUSE);
2960
    }
10347 ripley 2961
 
26855 ripley 2962
    /*
2963
      Otherwise, make adjustments:
2964
 
2965
      * If the first time through or noncontiguous, we need to call sbrk
2966
        just to find out where the end of memory lies.
10347 ripley 2967
 
26855 ripley 2968
      * We need to ensure that all returned chunks from malloc will meet
2969
        MALLOC_ALIGNMENT
10347 ripley 2970
 
26855 ripley 2971
      * If there was an intervening foreign sbrk, we need to adjust sbrk
2972
        request size to account for fact that we will not be able to
2973
        combine new space with existing space in old_top.
10347 ripley 2974
 
26855 ripley 2975
      * Almost all systems internally allocate whole pages at a time, in
2976
        which case we might as well use the whole last page of request.
2977
        So we allocate enough more memory to hit a page boundary now,
2978
        which in turn causes future contiguous calls to page-align.
2979
    */
2980
 
2981
    else {
2982
      front_misalign = 0;
2983
      end_misalign = 0;
2984
      correction = 0;
2985
      aligned_brk = brk;
10347 ripley 2986
 
26855 ripley 2987
      /*
2988
        If MORECORE returns an address lower than we have seen before,
2989
        we know it isn't really contiguous.  This and some subsequent
2990
        checks help cope with non-conforming MORECORE functions and
2991
        the presence of "foreign" calls to MORECORE from outside of
2992
        malloc or by other threads.  We cannot guarantee to detect
2993
        these in all cases, but cope with the ones we do detect.
2994
      */
2995
      if (contiguous(av) && old_size != 0 && brk < old_end) {
2996
        set_noncontiguous(av);
2997
      }
2998
 
2999
      /* handle contiguous cases */
3000
      if (contiguous(av)) { 
10347 ripley 3001
 
26855 ripley 3002
        /* 
3003
           We can tolerate forward non-contiguities here (usually due
3004
           to foreign calls) but treat them as part of our space for
3005
           stats reporting.
3006
        */
3007
        if (old_size != 0) 
3008
          av->sbrked_mem += brk - old_end;
3009
 
3010
        /* Guarantee alignment of first new chunk made from this space */
10347 ripley 3011
 
26855 ripley 3012
        front_misalign = (INTERNAL_SIZE_T)chunk2mem(brk) & MALLOC_ALIGN_MASK;
3013
        if (front_misalign > 0) {
10347 ripley 3014
 
26855 ripley 3015
          /*
3016
            Skip over some bytes to arrive at an aligned position.
3017
            We don't need to specially mark these wasted front bytes.
3018
            They will never be accessed anyway because
3019
            prev_inuse of av->top (and any chunk created from its start)
3020
            is always true after initialization.
3021
          */
10347 ripley 3022
 
26855 ripley 3023
          correction = MALLOC_ALIGNMENT - front_misalign;
3024
          aligned_brk += correction;
3025
        }
3026
 
3027
        /*
3028
          If this isn't adjacent to existing space, then we will not
3029
          be able to merge with old_top space, so must add to 2nd request.
3030
        */
3031
 
3032
        correction += old_size;
3033
 
3034
        /* Extend the end address to hit a page boundary */
3035
        end_misalign = (INTERNAL_SIZE_T)(brk + size + correction);
3036
        correction += ((end_misalign + pagemask) & ~pagemask) - end_misalign;
3037
 
3038
        assert(correction >= 0);
3039
        snd_brk = (char*)(MORECORE(correction));
3040
 
3041
        if (snd_brk == (char*)(MORECORE_FAILURE)) {
3042
          /*
3043
            If can't allocate correction, try to at least find out current
3044
            brk.  It might be enough to proceed without failing.
3045
          */
3046
          correction = 0;
3047
          snd_brk = (char*)(MORECORE(0));
3048
        }
3049
        else if (snd_brk < brk) {
3050
          /*
3051
            If the second call gives noncontiguous space even though
3052
            it says it won't, the only course of action is to ignore
3053
            results of second call, and conservatively estimate where
3054
            the first call left us. Also set noncontiguous, so this
3055
            won't happen again, leaving at most one hole.
3056
 
3057
            Note that this check is intrinsically incomplete.  Because
3058
            MORECORE is allowed to give more space than we ask for,
3059
            there is no reliable way to detect a noncontiguity
3060
            producing a forward gap for the second call.
3061
          */
3062
          snd_brk = brk + size;
3063
          correction = 0;
3064
          set_noncontiguous(av);
3065
        }
10347 ripley 3066
 
26855 ripley 3067
      }
3068
 
3069
      /* handle non-contiguous cases */
3070
      else { 
3071
        /* MORECORE/mmap must correctly align */
3072
        assert(aligned_OK(chunk2mem(brk)));
3073
 
3074
        /* Find out current end of memory */
3075
        if (snd_brk == (char*)(MORECORE_FAILURE)) {
3076
          snd_brk = (char*)(MORECORE(0));
3077
          av->sbrked_mem += snd_brk - brk - size;
3078
        }
3079
      }
3080
 
3081
      /* Adjust top based on results of second sbrk */
3082
      if (snd_brk != (char*)(MORECORE_FAILURE)) {
3083
        av->top = (mchunkptr)aligned_brk;
3084
        set_head(av->top, (snd_brk - aligned_brk + correction) | PREV_INUSE);
3085
        av->sbrked_mem += correction;
3086
 
3087
        /*
3088
          If not the first time through, we either have a
3089
          gap due to foreign sbrk or a non-contiguous region.  Insert a
3090
          double fencepost at old_top to prevent consolidation with space
3091
          we don't own. These fenceposts are artificial chunks that are
3092
          marked as inuse and are in any case too small to use.  We need
3093
          two to make sizes and alignments work out.
3094
        */
3095
 
3096
        if (old_size != 0) {
3097
          /* 
3098
             Shrink old_top to insert fenceposts, keeping size a
3099
             multiple of MALLOC_ALIGNMENT. We know there is at least
3100
             enough space in old_top to do this.
3101
          */
3102
          old_size = (old_size - 3*SIZE_SZ) & ~MALLOC_ALIGN_MASK;
3103
          set_head(old_top, old_size | PREV_INUSE);
3104
 
3105
          /*
3106
            Note that the following assignments completely overwrite
3107
            old_top when old_size was previously MINSIZE.  This is
3108
            intentional. We need the fencepost, even if old_top otherwise gets
3109
            lost.
3110
          */
3111
          chunk_at_offset(old_top, old_size          )->size =
3112
            SIZE_SZ|PREV_INUSE;
10347 ripley 3113
 
26855 ripley 3114
          chunk_at_offset(old_top, old_size + SIZE_SZ)->size =
3115
            SIZE_SZ|PREV_INUSE;
10347 ripley 3116
 
26855 ripley 3117
          /* 
3118
             If possible, release the rest, suppressing trimming.
3119
          */
3120
          if (old_size >= MINSIZE) {
3121
            INTERNAL_SIZE_T tt = av->trim_threshold;
3122
            av->trim_threshold = (INTERNAL_SIZE_T)(-1);
3123
            fREe(chunk2mem(old_top));
3124
            av->trim_threshold = tt;
3125
          }
3126
        }
3127
      }
3128
    }
3129
 
3130
    /* Update statistics */
3131
    sum = av->sbrked_mem;
3132
    if (sum > (CHUNK_SIZE_T)(av->max_sbrked_mem))
3133
      av->max_sbrked_mem = sum;
3134
 
3135
    sum += av->mmapped_mem;
3136
    if (sum > (CHUNK_SIZE_T)(av->max_total_mem))
3137
      av->max_total_mem = sum;
10347 ripley 3138
 
26855 ripley 3139
    check_malloc_state();
3140
 
3141
    /* finally, do the allocation */
10347 ripley 3142
 
26855 ripley 3143
    p = av->top;
3144
    size = chunksize(p);
3145
 
3146
    /* check that one of the above allocation paths succeeded */
3147
    if ((CHUNK_SIZE_T)(size) >= (CHUNK_SIZE_T)(nb + MINSIZE)) {
3148
      remainder_size = size - nb;
3149
      remainder = chunk_at_offset(p, nb);
3150
      av->top = remainder;
3151
      set_head(p, nb | PREV_INUSE);
3152
      set_head(remainder, remainder_size | PREV_INUSE);
3153
      check_malloced_chunk(p, nb);
3154
      return chunk2mem(p);
3155
    }
10347 ripley 3156
 
26855 ripley 3157
  }
10347 ripley 3158
 
26855 ripley 3159
  /* catch all failure paths */
3160
  MALLOC_FAILURE_ACTION;
3161
  return 0;
3162
}
3163
 
3164
 
3165
 
3166
 
3167
#ifndef MORECORE_CANNOT_TRIM        
3168
/*
3169
  sYSTRIm is an inverse of sorts to sYSMALLOc.  It gives memory back
3170
  to the system (via negative arguments to sbrk) if there is unused
3171
  memory at the `high' end of the malloc pool. It is called
3172
  automatically by free() when top space exceeds the trim
3173
  threshold. It is also called by the public malloc_trim routine.  It
3174
  returns 1 if it actually released any memory, else 0.
10347 ripley 3175
*/
3176
 
26855 ripley 3177
static int sYSTRIm(size_t pad, mstate av)
3178
{
3179
  long  top_size;        /* Amount of top-most memory */
3180
  long  extra;           /* Amount to release */
3181
  long  released;        /* Amount actually released */
3182
  char* current_brk;     /* address returned by pre-check sbrk call */
3183
  char* new_brk;         /* address returned by post-check sbrk call */
3184
  size_t pagesz;
3185
 
3186
  pagesz = av->pagesize;
3187
  top_size = chunksize(av->top);
3188
 
3189
  /* Release in pagesize units, keeping at least one page */
3190
  extra = ((top_size - pad - MINSIZE + (pagesz-1)) / pagesz - 1) * pagesz;
3191
 
3192
  if (extra > 0) {
3193
 
3194
    /*
3195
      Only proceed if end of memory is where we last set it.
3196
      This avoids problems if there were foreign sbrk calls.
3197
    */
3198
    current_brk = (char*)(MORECORE(0));
3199
    if (current_brk == (char*)(av->top) + top_size) {
3200
 
3201
      /*
3202
        Attempt to release memory. We ignore MORECORE return value,
3203
        and instead call again to find out where new end of memory is.
3204
        This avoids problems if first call releases less than we asked,
3205
        of if failure somehow altered brk value. (We could still
3206
        encounter problems if it altered brk in some very bad way,
3207
        but the only thing we can do is adjust anyway, which will cause
3208
        some downstream failure.)
3209
      */
3210
 
3211
      MORECORE(-extra);
3212
      new_brk = (char*)(MORECORE(0));
3213
 
3214
      if (new_brk != (char*)MORECORE_FAILURE) {
3215
        released = (long)(current_brk - new_brk);
3216
 
3217
        if (released != 0) {
3218
          /* Success. Adjust top. */
3219
          av->sbrked_mem -= released;
3220
          set_head(av->top, (top_size - released) | PREV_INUSE);
3221
          check_malloc_state();
3222
          return 1;
3223
        }
3224
      }
3225
    }
3226
  }
3227
  return 0;
3228
}
3229
#endif
3230
 
3231
/*
3232
  ------------------------------ malloc ------------------------------
3233
*/
3234
 
3235
 
10347 ripley 3236
Void_t* mALLOc(size_t bytes)
3237
{
26855 ripley 3238
  mstate av = get_malloc_state();
10347 ripley 3239
 
26855 ripley 3240
  INTERNAL_SIZE_T nb;               /* normalized request size */
3241
  unsigned int    idx;              /* associated bin index */
3242
  mbinptr         bin;              /* associated bin */
3243
  mfastbinptr*    fb;               /* associated fastbin */
10347 ripley 3244
 
26855 ripley 3245
  mchunkptr       victim;           /* inspected/selected chunk */
3246
  INTERNAL_SIZE_T size;             /* its size */
3247
  int             victim_index;     /* its bin index */
10425 ripley 3248
 
26855 ripley 3249
  mchunkptr       remainder;        /* remainder from a split */
3250
  CHUNK_SIZE_T    remainder_size;   /* its size */
10425 ripley 3251
 
26855 ripley 3252
  unsigned int    block;            /* bit map traverser */
3253
  unsigned int    bit;              /* bit map traverser */
3254
  unsigned int    map;              /* current word of binmap */
10347 ripley 3255
 
26855 ripley 3256
  mchunkptr       fwd;              /* misc temp for linking */
3257
  mchunkptr       bck;              /* misc temp for linking */
10347 ripley 3258
 
26855 ripley 3259
  /*
3260
    Convert request size to internal form by adding SIZE_SZ bytes
3261
    overhead plus possibly more to obtain necessary alignment and/or
3262
    to obtain a size of at least MINSIZE, the smallest allocatable
3263
    size. Also, checked_request2size traps (returning 0) request sizes
3264
    that are so large that they wrap around zero when padded and
3265
    aligned.
3266
  */
10347 ripley 3267
 
26855 ripley 3268
  checked_request2size(bytes, nb);
10347 ripley 3269
 
26855 ripley 3270
  /*
3271
    Bypass search if no frees yet
3272
   */
3273
  if (!have_anychunks(av)) {
3274
    if (av->max_fast == 0) /* initialization check */
3275
      malloc_consolidate(av);
3276
    goto use_top;
3277
  }
3278
 
3279
  /*
3280
    If the size qualifies as a fastbin, first check corresponding bin.
3281
  */
3282
 
3283
  if ((CHUNK_SIZE_T)(nb) <= (CHUNK_SIZE_T)(av->max_fast)) { 
3284
    fb = &(av->fastbins[(fastbin_index(nb))]);
3285
    if ( (victim = *fb) != 0) {
3286
      *fb = victim->fd;
3287
      check_remalloced_chunk(victim, nb);
10347 ripley 3288
      return chunk2mem(victim);
3289
    }
3290
  }
3291
 
26855 ripley 3292
  /*
3293
    If a small request, check regular bin.  Since these "smallbins"
3294
    hold one size each, no searching within bins is necessary.
3295
    (For a large request, we need to wait until unsorted chunks are
3296
    processed to find best fit. But for small ones, fits are exact
3297
    anyway, so we can check now, which is faster.)
3298
  */
10347 ripley 3299
 
26855 ripley 3300
  if (in_smallbin_range(nb)) {
3301
    idx = smallbin_index(nb);
3302
    bin = bin_at(av,idx);
10347 ripley 3303
 
26855 ripley 3304
    if ( (victim = last(bin)) != bin) {
3305
      bck = victim->bk;
3306
      set_inuse_bit_at_offset(victim, nb);
3307
      bin->bk = bck;
3308
      bck->fd = bin;
3309
 
3310
      check_malloced_chunk(victim, nb);
3311
      return chunk2mem(victim);
10347 ripley 3312
    }
26855 ripley 3313
  }
10347 ripley 3314
 
26855 ripley 3315
  /* 
3316
     If this is a large request, consolidate fastbins before continuing.
3317
     While it might look excessive to kill all fastbins before
3318
     even seeing if there is space available, this avoids
3319
     fragmentation problems normally associated with fastbins.
3320
     Also, in practice, programs tend to have runs of either small or
3321
     large requests, but less often mixtures, so consolidation is not 
3322
     invoked all that often in most programs. And the programs that
3323
     it is called frequently in otherwise tend to fragment.
3324
  */
10347 ripley 3325
 
26855 ripley 3326
  else {
3327
    idx = largebin_index(nb);
3328
    if (have_fastchunks(av)) 
3329
      malloc_consolidate(av);
10347 ripley 3330
  }
3331
 
26855 ripley 3332
  /*
3333
    Process recently freed or remaindered chunks, taking one only if
3334
    it is exact fit, or, if this a small request, the chunk is remainder from
3335
    the most recent non-exact fit.  Place other traversed chunks in
3336
    bins.  Note that this step is the only place in any routine where
3337
    chunks are placed in bins.
3338
  */
3339
 
3340
  while ( (victim = unsorted_chunks(av)->bk) != unsorted_chunks(av)) {
3341
    bck = victim->bk;
3342
    size = chunksize(victim);
3343
 
3344
    /* 
3345
       If a small request, try to use last remainder if it is the
3346
       only chunk in unsorted bin.  This helps promote locality for
3347
       runs of consecutive small requests. This is the only
3348
       exception to best-fit, and applies only when there is
3349
       no exact fit for a small chunk.
3350
    */
3351
 
3352
    if (in_smallbin_range(nb) && 
3353
        bck == unsorted_chunks(av) &&
3354
        victim == av->last_remainder &&
3355
        (CHUNK_SIZE_T)(size) > (CHUNK_SIZE_T)(nb + MINSIZE)) {
3356
 
3357
      /* split and reattach remainder */
3358
      remainder_size = size - nb;
10347 ripley 3359
      remainder = chunk_at_offset(victim, nb);
26855 ripley 3360
      unsorted_chunks(av)->bk = unsorted_chunks(av)->fd = remainder;
3361
      av->last_remainder = remainder; 
3362
      remainder->bk = remainder->fd = unsorted_chunks(av);
3363
 
10347 ripley 3364
      set_head(victim, nb | PREV_INUSE);
3365
      set_head(remainder, remainder_size | PREV_INUSE);
3366
      set_foot(remainder, remainder_size);
26855 ripley 3367
 
10347 ripley 3368
      check_malloced_chunk(victim, nb);
3369
      return chunk2mem(victim);
3370
    }
26855 ripley 3371
 
3372
    /* remove from unsorted list */
3373
    unsorted_chunks(av)->bk = bck;
3374
    bck->fd = unsorted_chunks(av);
3375
 
3376
    /* Take now instead of binning if exact fit */
3377
 
3378
    if (size == nb) {
3379
      set_inuse_bit_at_offset(victim, size);
10347 ripley 3380
      check_malloced_chunk(victim, nb);
3381
      return chunk2mem(victim);
3382
    }
26855 ripley 3383
 
3384
    /* place chunk in bin */
3385
 
3386
    if (in_smallbin_range(size)) {
3387
      victim_index = smallbin_index(size);
3388
      bck = bin_at(av, victim_index);
3389
      fwd = bck->fd;
3390
    }
3391
    else {
3392
      victim_index = largebin_index(size);
3393
      bck = bin_at(av, victim_index);
3394
      fwd = bck->fd;
3395
 
3396
      if (fwd != bck) {
3397
        /* if smaller than smallest, place first */
3398
        if ((CHUNK_SIZE_T)(size) < (CHUNK_SIZE_T)(bck->bk->size)) {
3399
          fwd = bck;
3400
          bck = bck->bk;
3401
        }
3402
        else if ((CHUNK_SIZE_T)(size) >= 
3403
                 (CHUNK_SIZE_T)(FIRST_SORTED_BIN_SIZE)) {
3404
 
3405
          /* maintain large bins in sorted order */
3406
          size |= PREV_INUSE; /* Or with inuse bit to speed comparisons */
3407
          while ((CHUNK_SIZE_T)(size) < (CHUNK_SIZE_T)(fwd->size)) 
3408
            fwd = fwd->fd;
3409
          bck = fwd->bk;
3410
        }
3411
      }
3412
    }
3413
 
3414
    mark_bin(av, victim_index);
3415
    victim->bk = bck;
3416
    victim->fd = fwd;
3417
    fwd->bk = victim;
3418
    bck->fd = victim;
10347 ripley 3419
  }
26855 ripley 3420
 
3421
  /*
3422
    If a large request, scan through the chunks of current bin to
3423
    find one that fits.  (This will be the smallest that fits unless
3424
    FIRST_SORTED_BIN_SIZE has been changed from default.)  This is
3425
    the only step where an unbounded number of chunks might be
3426
    scanned without doing anything useful with them. However the
3427
    lists tend to be short.
3428
  */
3429
 
3430
  if (!in_smallbin_range(nb)) {
3431
    bin = bin_at(av, idx);
3432
 
3433
    for (victim = last(bin); victim != bin; victim = victim->bk) {
3434
      size = chunksize(victim);
3435
 
3436
      if ((CHUNK_SIZE_T)(size) >= (CHUNK_SIZE_T)(nb)) {
3437
        remainder_size = size - nb;
3438
        unlink(victim, bck, fwd);
3439
 
3440
        /* Exhaust */
3441
        if (remainder_size < MINSIZE)  {
3442
          set_inuse_bit_at_offset(victim, size);
3443
          check_malloced_chunk(victim, nb);
3444
          return chunk2mem(victim);
3445
        }
3446
        /* Split */
3447
        else {
3448
          remainder = chunk_at_offset(victim, nb);
3449
          unsorted_chunks(av)->bk = unsorted_chunks(av)->fd = remainder;
3450
          remainder->bk = remainder->fd = unsorted_chunks(av);
3451
          set_head(victim, nb | PREV_INUSE);
3452
          set_head(remainder, remainder_size | PREV_INUSE);
3453
          set_foot(remainder, remainder_size);
3454
          check_malloced_chunk(victim, nb);
3455
          return chunk2mem(victim);
3456
        } 
3457
      }
3458
    }    
3459
  }
10347 ripley 3460
 
3461
  /*
26855 ripley 3462
    Search for a chunk by scanning bins, starting with next largest
3463
    bin. This search is strictly by best-fit; i.e., the smallest
3464
    (with ties going to approximately the least recently used) chunk
3465
    that fits is selected.
3466
 
3467
    The bitmap avoids needing to check that most blocks are nonempty.
10347 ripley 3468
  */
26855 ripley 3469
 
3470
  ++idx;
3471
  bin = bin_at(av,idx);
3472
  block = idx2block(idx);
3473
  map = av->binmap[block];
3474
  bit = idx2bit(idx);
3475
 
3476
  for (;;) {
3477
 
3478
    /* Skip rest of block if there are no more set bits in this block.  */
3479
    if (bit > map || bit == 0) {
3480
      do {
3481
        if (++block >= BINMAPSIZE)  /* out of bins */
3482
          goto use_top;
3483
      } while ( (map = av->binmap[block]) == 0);
3484
 
3485
      bin = bin_at(av, (block << BINMAPSHIFT));
3486
      bit = 1;
3487
    }
3488
 
3489
    /* Advance to bin with set bit. There must be one. */
3490
    while ((bit & map) == 0) {
3491
      bin = next_bin(bin);
3492
      bit <<= 1;
3493
      assert(bit != 0);
3494
    }
3495
 
3496
    /* Inspect the bin. It is likely to be non-empty */
3497
    victim = last(bin);
3498
 
3499
    /*  If a false alarm (empty bin), clear the bit. */
3500
    if (victim == bin) {
3501
      av->binmap[block] = map &= ~bit; /* Write through */
3502
      bin = next_bin(bin);
3503
      bit <<= 1;
3504
    }
3505
 
3506
    else {
3507
      size = chunksize(victim);
3508
 
3509
      /*  We know the first chunk in this bin is big enough to use. */
3510
      assert((CHUNK_SIZE_T)(size) >= (CHUNK_SIZE_T)(nb));
3511
 
3512
      remainder_size = size - nb;
3513
 
3514
      /* unlink */
3515
      bck = victim->bk;
3516
      bin->bk = bck;
3517
      bck->fd = bin;
3518
 
3519
      /* Exhaust */
3520
      if (remainder_size < MINSIZE) {
3521
        set_inuse_bit_at_offset(victim, size);
3522
        check_malloced_chunk(victim, nb);
3523
        return chunk2mem(victim);
10347 ripley 3524
      }
26855 ripley 3525
 
3526
      /* Split */
3527
      else {
3528
        remainder = chunk_at_offset(victim, nb);
3529
 
3530
        unsorted_chunks(av)->bk = unsorted_chunks(av)->fd = remainder;
3531
        remainder->bk = remainder->fd = unsorted_chunks(av);
3532
        /* advertise as last remainder */
3533
        if (in_smallbin_range(nb)) 
3534
          av->last_remainder = remainder; 
3535
 
3536
        set_head(victim, nb | PREV_INUSE);
3537
        set_head(remainder, remainder_size | PREV_INUSE);
3538
        set_foot(remainder, remainder_size);
3539
        check_malloced_chunk(victim, nb);
3540
        return chunk2mem(victim);
3541
      }
10347 ripley 3542
    }
26855 ripley 3543
  }
10347 ripley 3544
 
26855 ripley 3545
  use_top:    
3546
  /*
3547
    If large enough, split off the chunk bordering the end of memory
3548
    (held in av->top). Note that this is in accord with the best-fit
3549
    search rule.  In effect, av->top is treated as larger (and thus
3550
    less well fitting) than any other available chunk since it can
3551
    be extended to be as large as necessary (up to system
3552
    limitations).
3553
 
3554
    We require that av->top always exists (i.e., has size >=
3555
    MINSIZE) after initialization, so if it would otherwise be
3556
    exhuasted by current request, it is replenished. (The main
3557
    reason for ensuring it exists is that we may need MINSIZE space
3558
    to put in fenceposts in sysmalloc.)
3559
  */
3560
 
3561
  victim = av->top;
3562
  size = chunksize(victim);
3563
 
3564
  if ((CHUNK_SIZE_T)(size) >= (CHUNK_SIZE_T)(nb + MINSIZE)) {
3565
    remainder_size = size - nb;
3566
    remainder = chunk_at_offset(victim, nb);
3567
    av->top = remainder;
3568
    set_head(victim, nb | PREV_INUSE);
3569
    set_head(remainder, remainder_size | PREV_INUSE);
3570
 
3571
    check_malloced_chunk(victim, nb);
3572
    return chunk2mem(victim);
3573
  }
3574
 
3575
  /* 
3576
     If no space in top, relay to handle system-dependent cases 
3577
  */
3578
  return sYSMALLOc(nb, av);    
3579
}
10347 ripley 3580
 
26855 ripley 3581
/*
3582
  ------------------------------ free ------------------------------
3583
*/
10347 ripley 3584
 
26855 ripley 3585
void fREe(Void_t* mem)
3586
{
3587
  mstate av = get_malloc_state();
10347 ripley 3588
 
26855 ripley 3589
  mchunkptr       p;           /* chunk corresponding to mem */
3590
  INTERNAL_SIZE_T size;        /* its size */
3591
  mfastbinptr*    fb;          /* associated fastbin */
3592
  mchunkptr       nextchunk;   /* next contiguous chunk */
3593
  INTERNAL_SIZE_T nextsize;    /* its size */
3594
  int             nextinuse;   /* true if nextchunk is used */
3595
  INTERNAL_SIZE_T prevsize;    /* size of previous contiguous chunk */
3596
  mchunkptr       bck;         /* misc temp for linking */
3597
  mchunkptr       fwd;         /* misc temp for linking */
10347 ripley 3598
 
26855 ripley 3599
  /* free(0) has no effect */
3600
  if (mem != 0) {
3601
    p = mem2chunk(mem);
3602
    size = chunksize(p);
10347 ripley 3603
 
26855 ripley 3604
    check_inuse_chunk(p);
10347 ripley 3605
 
26855 ripley 3606
    /*
3607
      If eligible, place chunk on a fastbin so it can be found
3608
      and used quickly in malloc.
3609
    */
10347 ripley 3610
 
26855 ripley 3611
    if ((CHUNK_SIZE_T)(size) <= (CHUNK_SIZE_T)(av->max_fast)
10347 ripley 3612
 
26855 ripley 3613
#if TRIM_FASTBINS
3614
        /* 
3615
           If TRIM_FASTBINS set, don't place chunks
3616
           bordering top into fastbins
3617
        */
3618
        && (chunk_at_offset(p, size) != av->top)
3619
#endif
3620
        ) {
10347 ripley 3621
 
26855 ripley 3622
      set_fastchunks(av);
3623
      fb = &(av->fastbins[fastbin_index(size)]);
3624
      p->fd = *fb;
3625
      *fb = p;
3626
    }
10347 ripley 3627
 
26855 ripley 3628
    /*
3629
       Consolidate other non-mmapped chunks as they arrive.
3630
    */
10347 ripley 3631
 
26855 ripley 3632
    else if (!chunk_is_mmapped(p)) {
3633
      set_anychunks(av);
10347 ripley 3634
 
26855 ripley 3635
      nextchunk = chunk_at_offset(p, size);
3636
      nextsize = chunksize(nextchunk);
10347 ripley 3637
 
26855 ripley 3638
      /* consolidate backward */
3639
      if (!prev_inuse(p)) {
3640
        prevsize = p->prev_size;
3641
        size += prevsize;
3642
        p = chunk_at_offset(p, -((long) prevsize));
3643
        unlink(p, bck, fwd);
3644
      }
10347 ripley 3645
 
26855 ripley 3646
      if (nextchunk != av->top) {
3647
        /* get and clear inuse bit */
3648
        nextinuse = inuse_bit_at_offset(nextchunk, nextsize);
3649
        set_head(nextchunk, nextsize);
10347 ripley 3650
 
26855 ripley 3651
        /* consolidate forward */
3652
        if (!nextinuse) {
3653
          unlink(nextchunk, bck, fwd);
3654
          size += nextsize;
3655
        }
10347 ripley 3656
 
26855 ripley 3657
        /*
3658
          Place the chunk in unsorted chunk list. Chunks are
3659
          not placed into regular bins until after they have
3660
          been given one chance to be used in malloc.
3661
        */
10347 ripley 3662
 
26855 ripley 3663
        bck = unsorted_chunks(av);
3664
        fwd = bck->fd;
3665
        p->bk = bck;
3666
        p->fd = fwd;
3667
        bck->fd = p;
3668
        fwd->bk = p;
10347 ripley 3669
 
26855 ripley 3670
        set_head(p, size | PREV_INUSE);
3671
        set_foot(p, size);
3672
 
3673
        check_free_chunk(p);
3674
      }
10347 ripley 3675
 
26855 ripley 3676
      /*
3677
         If the chunk borders the current high end of memory,
3678
         consolidate into top
3679
      */
10347 ripley 3680
 
26855 ripley 3681
      else {
3682
        size += nextsize;
3683
        set_head(p, size | PREV_INUSE);
3684
        av->top = p;
3685
        check_chunk(p);
3686
      }
10347 ripley 3687
 
26855 ripley 3688
      /*
3689
        If freeing a large space, consolidate possibly-surrounding
3690
        chunks. Then, if the total unused topmost memory exceeds trim
3691
        threshold, ask malloc_trim to reduce top.
10347 ripley 3692
 
26855 ripley 3693
        Unless max_fast is 0, we don't know if there are fastbins
3694
        bordering top, so we cannot tell for sure whether threshold
3695
        has been reached unless fastbins are consolidated.  But we
3696
        don't want to consolidate on each free.  As a compromise,
3697
        consolidation is performed if FASTBIN_CONSOLIDATION_THRESHOLD
3698
        is reached.
3699
      */
10347 ripley 3700
 
26855 ripley 3701
      if ((CHUNK_SIZE_T)(size) >= FASTBIN_CONSOLIDATION_THRESHOLD) { 
3702
        if (have_fastchunks(av)) 
3703
          malloc_consolidate(av);
10347 ripley 3704
 
26855 ripley 3705
#ifndef MORECORE_CANNOT_TRIM        
3706
        if ((CHUNK_SIZE_T)(chunksize(av->top)) >= 
3707
            (CHUNK_SIZE_T)(av->trim_threshold))
3708
          sYSTRIm(av->top_pad, av);
3709
#endif
3710
      }
10347 ripley 3711
 
26855 ripley 3712
    }
3713
    /*
3714
      If the chunk was allocated via mmap, release via munmap()
3715
      Note that if HAVE_MMAP is false but chunk_is_mmapped is
3716
      true, then user must have overwritten memory. There's nothing
3717
      we can do to catch this error unless DEBUG is set, in which case
3718
      check_inuse_chunk (above) will have triggered error.
3719
    */
10347 ripley 3720
 
26855 ripley 3721
    else {
3722
#if HAVE_MMAP
3723
      int ret;
3724
      INTERNAL_SIZE_T offset = p->prev_size;
3725
      av->n_mmaps--;
3726
      av->mmapped_mem -= (size + offset);
3727
      ret = munmap((char*)p - offset, size + offset);
3728
      /* munmap returns non-zero on failure */
3729
      assert(ret == 0);
3730
#endif
3731
    }
3732
  }
3733
}
10347 ripley 3734
 
26855 ripley 3735
/*
3736
  ------------------------- malloc_consolidate -------------------------
10347 ripley 3737
 
26855 ripley 3738
  malloc_consolidate is a specialized version of free() that tears
3739
  down chunks held in fastbins.  Free itself cannot be used for this
3740
  purpose since, among other things, it might place chunks back onto
3741
  fastbins.  So, instead, we need to use a minor variant of the same
3742
  code.
3743
 
3744
  Also, because this routine needs to be called the first time through
3745
  malloc anyway, it turns out to be the perfect place to trigger
3746
  initialization code.
10347 ripley 3747
*/
3748
 
26855 ripley 3749
static void malloc_consolidate(mstate av)
10347 ripley 3750
{
26855 ripley 3751
  mfastbinptr*    fb;                 /* current fastbin being consolidated */
3752
  mfastbinptr*    maxfb;              /* last fastbin (for loop control) */
3753
  mchunkptr       p;                  /* current chunk being consolidated */
3754
  mchunkptr       nextp;              /* next chunk to consolidate */
3755
  mchunkptr       unsorted_bin;       /* bin header */
3756
  mchunkptr       first_unsorted;     /* chunk to link to */
10347 ripley 3757
 
26855 ripley 3758
  /* These have same use as in free() */
3759
  mchunkptr       nextchunk;
3760
  INTERNAL_SIZE_T size;
3761
  INTERNAL_SIZE_T nextsize;
3762
  INTERNAL_SIZE_T prevsize;
3763
  int             nextinuse;
3764
  mchunkptr       bck;
3765
  mchunkptr       fwd;
10347 ripley 3766
 
26855 ripley 3767
  /*
3768
    If max_fast is 0, we know that av hasn't
3769
    yet been initialized, in which case do so below
3770
  */
10347 ripley 3771
 
26855 ripley 3772
  if (av->max_fast != 0) {
3773
    clear_fastchunks(av);
10347 ripley 3774
 
26855 ripley 3775
    unsorted_bin = unsorted_chunks(av);
10347 ripley 3776
 
26855 ripley 3777
    /*
3778
      Remove each chunk from fast bin and consolidate it, placing it
3779
      then in unsorted bin. Among other reasons for doing this,
3780
      placing in unsorted bin avoids needing to calculate actual bins
3781
      until malloc is sure that chunks aren't immediately going to be
3782
      reused anyway.
3783
    */
3784
 
3785
    maxfb = &(av->fastbins[fastbin_index(av->max_fast)]);
3786
    fb = &(av->fastbins[0]);
3787
    do {
3788
      if ( (p = *fb) != 0) {
3789
        *fb = 0;
3790
 
3791
        do {
3792
          check_inuse_chunk(p);
3793
          nextp = p->fd;
3794
 
3795
          /* Slightly streamlined version of consolidation code in free() */
3796
          size = p->size & ~PREV_INUSE;
3797
          nextchunk = chunk_at_offset(p, size);
3798
          nextsize = chunksize(nextchunk);
3799
 
3800
          if (!prev_inuse(p)) {
3801
            prevsize = p->prev_size;
3802
            size += prevsize;
3803
            p = chunk_at_offset(p, -((long) prevsize));
3804
            unlink(p, bck, fwd);
3805
          }
3806
 
3807
          if (nextchunk != av->top) {
3808
            nextinuse = inuse_bit_at_offset(nextchunk, nextsize);
3809
            set_head(nextchunk, nextsize);
3810
 
3811
            if (!nextinuse) {
3812
              size += nextsize;
3813
              unlink(nextchunk, bck, fwd);
3814
            }
3815
 
3816
            first_unsorted = unsorted_bin->fd;
3817
            unsorted_bin->fd = p;
3818
            first_unsorted->bk = p;
3819
 
3820
            set_head(p, size | PREV_INUSE);
3821
            p->bk = unsorted_bin;
3822
            p->fd = first_unsorted;
3823
            set_foot(p, size);
3824
          }
3825
 
3826
          else {
3827
            size += nextsize;
3828
            set_head(p, size | PREV_INUSE);
3829
            av->top = p;
3830
          }
3831
 
3832
        } while ( (p = nextp) != 0);
3833
 
3834
      }
3835
    } while (fb++ != maxfb);
10347 ripley 3836
  }
26855 ripley 3837
  else {
3838
    malloc_init_state(av);
3839
    check_malloc_state();
10347 ripley 3840
  }
3841
}
3842
 
3843
/*
26855 ripley 3844
  ------------------------------ realloc ------------------------------
10347 ripley 3845
*/
3846
 
3847
 
3848
Void_t* rEALLOc(Void_t* oldmem, size_t bytes)
3849
{
26855 ripley 3850
  mstate av = get_malloc_state();
10347 ripley 3851
 
26855 ripley 3852
  INTERNAL_SIZE_T  nb;              /* padded request size */
10347 ripley 3853
 
26855 ripley 3854
  mchunkptr        oldp;            /* chunk corresponding to oldmem */
3855
  INTERNAL_SIZE_T  oldsize;         /* its size */
10347 ripley 3856
 
26855 ripley 3857
  mchunkptr        newp;            /* chunk to return */
3858
  INTERNAL_SIZE_T  newsize;         /* its size */
3859
  Void_t*          newmem;          /* corresponding user mem */
10347 ripley 3860
 
26855 ripley 3861
  mchunkptr        next;            /* next contiguous chunk after oldp */
10347 ripley 3862
 
26855 ripley 3863
  mchunkptr        remainder;       /* extra space at end of newp */
3864
  CHUNK_SIZE_T     remainder_size;  /* its size */
10347 ripley 3865
 
26855 ripley 3866
  mchunkptr        bck;             /* misc temp for linking */
3867
  mchunkptr        fwd;             /* misc temp for linking */
10347 ripley 3868
 
26855 ripley 3869
  CHUNK_SIZE_T     copysize;        /* bytes to copy */
3870
  unsigned int     ncopies;         /* INTERNAL_SIZE_T words to copy */
3871
  INTERNAL_SIZE_T* s;               /* copy source */ 
3872
  INTERNAL_SIZE_T* d;               /* copy destination */
3873
 
3874
 
10347 ripley 3875
#ifdef REALLOC_ZERO_BYTES_FREES
26855 ripley 3876
  if (bytes == 0) {
3877
    fREe(oldmem);
3878
    return 0;
3879
  }
10347 ripley 3880
#endif
3881
 
3882
  /* realloc of null is supposed to be same as malloc */
3883
  if (oldmem == 0) return mALLOc(bytes);
3884
 
26855 ripley 3885
  checked_request2size(bytes, nb);
10347 ripley 3886
 
26855 ripley 3887
  oldp    = mem2chunk(oldmem);
3888
  oldsize = chunksize(oldp);
10347 ripley 3889
 
3890
  check_inuse_chunk(oldp);
3891
 
26855 ripley 3892
  if (!chunk_is_mmapped(oldp)) {
10347 ripley 3893
 
26855 ripley 3894
    if ((CHUNK_SIZE_T)(oldsize) >= (CHUNK_SIZE_T)(nb)) {
3895
      /* already big enough; split below */
3896
      newp = oldp;
3897
      newsize = oldsize;
3898
    }
10347 ripley 3899
 
26855 ripley 3900
    else {
3901
      next = chunk_at_offset(oldp, oldsize);
10347 ripley 3902
 
26855 ripley 3903
      /* Try to expand forward into top */
3904
      if (next == av->top &&
3905
          (CHUNK_SIZE_T)(newsize = oldsize + chunksize(next)) >=
3906
          (CHUNK_SIZE_T)(nb + MINSIZE)) {
3907
        set_head_size(oldp, nb);
3908
        av->top = chunk_at_offset(oldp, nb);
3909
        set_head(av->top, (newsize - nb) | PREV_INUSE);
3910
        return chunk2mem(oldp);
10347 ripley 3911
      }
26855 ripley 3912
 
3913
      /* Try to expand forward into next chunk;  split off remainder below */
3914
      else if (next != av->top && 
3915
               !inuse(next) &&
3916
               (CHUNK_SIZE_T)(newsize = oldsize + chunksize(next)) >=
3917
               (CHUNK_SIZE_T)(nb)) {
3918
        newp = oldp;
10347 ripley 3919
        unlink(next, bck, fwd);
3920
      }
3921
 
26855 ripley 3922
      /* allocate, copy, free */
3923
      else {
3924
        newmem = mALLOc(nb - MALLOC_ALIGN_MASK);
3925
        if (newmem == 0)
3926
          return 0; /* propagate failure */
3927
 
3928
        newp = mem2chunk(newmem);
3929
        newsize = chunksize(newp);
3930
 
3931
        /*
3932
          Avoid copy if newp is next chunk after oldp.
3933
        */
3934
        if (newp == next) {
3935
          newsize += oldsize;
3936
          newp = oldp;
3937
        }
3938
        else {
3939
          /*
3940
            Unroll copy of <= 36 bytes (72 if 8byte sizes)
3941
            We know that contents have an odd number of
3942
            INTERNAL_SIZE_T-sized words; minimally 3.
3943
          */
3944
 
3945
          copysize = oldsize - SIZE_SZ;
3946
          s = (INTERNAL_SIZE_T*)(oldmem);
3947
          d = (INTERNAL_SIZE_T*)(newmem);
3948
          ncopies = copysize / sizeof(INTERNAL_SIZE_T);
3949
          assert(ncopies >= 3);
3950
 
3951
          if (ncopies > 9)
3952
            MALLOC_COPY(d, s, copysize);
3953
 
3954
          else {
3955
            *(d+0) = *(s+0);
3956
            *(d+1) = *(s+1);
3957
            *(d+2) = *(s+2);
3958
            if (ncopies > 4) {
3959
              *(d+3) = *(s+3);
3960
              *(d+4) = *(s+4);
3961
              if (ncopies > 6) {
3962
                *(d+5) = *(s+5);
3963
                *(d+6) = *(s+6);
3964
                if (ncopies > 8) {
3965
                  *(d+7) = *(s+7);
3966
                  *(d+8) = *(s+8);
3967
                }
3968
              }
3969
            }
10347 ripley 3970
          }
26855 ripley 3971
 
3972
          fREe(oldmem);
3973
          check_inuse_chunk(newp);
3974
          return chunk2mem(newp);
10347 ripley 3975
        }
3976
      }
3977
    }
3978
 
26855 ripley 3979
    /* If possible, free extra space in old or extended chunk */
10347 ripley 3980
 
26855 ripley 3981
    assert((CHUNK_SIZE_T)(newsize) >= (CHUNK_SIZE_T)(nb));
10347 ripley 3982
 
26855 ripley 3983
    remainder_size = newsize - nb;
10347 ripley 3984
 
26855 ripley 3985
    if (remainder_size < MINSIZE) { /* not enough extra to split off */
3986
      set_head_size(newp, newsize);
3987
      set_inuse_bit_at_offset(newp, newsize);
10347 ripley 3988
    }
26855 ripley 3989
    else { /* split remainder */
3990
      remainder = chunk_at_offset(newp, nb);
3991
      set_head_size(newp, nb);
3992
      set_head(remainder, remainder_size | PREV_INUSE);
3993
      /* Mark remainder as inuse so free() won't complain */
3994
      set_inuse_bit_at_offset(remainder, remainder_size);
3995
      fREe(chunk2mem(remainder)); 
3996
    }
10347 ripley 3997
 
26855 ripley 3998
    check_inuse_chunk(newp);
3999
    return chunk2mem(newp);
10347 ripley 4000
  }
4001
 
26855 ripley 4002
  /*
4003
    Handle mmap cases
4004
  */
10347 ripley 4005
 
26855 ripley 4006
  else {
4007
#if HAVE_MMAP
10347 ripley 4008
 
26855 ripley 4009
#if HAVE_MREMAP
4010
    INTERNAL_SIZE_T offset = oldp->prev_size;
4011
    size_t pagemask = av->pagesize - 1;
4012
    char *cp;
4013
    CHUNK_SIZE_T  sum;
4014
 
4015
    /* Note the extra SIZE_SZ overhead */
4016
    newsize = (nb + offset + SIZE_SZ + pagemask) & ~pagemask;
10347 ripley 4017
 
26855 ripley 4018
    /* don't need to remap if still within same page */
4019
    if (oldsize == newsize - offset) 
4020
      return oldmem;
10347 ripley 4021
 
26855 ripley 4022
    cp = (char*)mremap((char*)oldp - offset, oldsize + offset, newsize, 1);
4023
 
4024
    if (cp != (char*)MORECORE_FAILURE) {
10347 ripley 4025
 
26855 ripley 4026
      newp = (mchunkptr)(cp + offset);
4027
      set_head(newp, (newsize - offset)|IS_MMAPPED);
4028
 
4029
      assert(aligned_OK(chunk2mem(newp)));
4030
      assert((newp->prev_size == offset));
4031
 
4032
      /* update statistics */
4033
      sum = av->mmapped_mem += newsize - oldsize;
4034
      if (sum > (CHUNK_SIZE_T)(av->max_mmapped_mem)) 
4035
        av->max_mmapped_mem = sum;
4036
      sum += av->sbrked_mem;
4037
      if (sum > (CHUNK_SIZE_T)(av->max_total_mem)) 
4038
        av->max_total_mem = sum;
4039
 
4040
      return chunk2mem(newp);
4041
    }
4042
#endif
10347 ripley 4043
 
26855 ripley 4044
    /* Note the extra SIZE_SZ overhead. */
4045
    if ((CHUNK_SIZE_T)(oldsize) >= (CHUNK_SIZE_T)(nb + SIZE_SZ)) 
4046
      newmem = oldmem; /* do nothing */
4047
    else {
4048
      /* Must alloc, copy, free. */
4049
      newmem = mALLOc(nb - MALLOC_ALIGN_MASK);
4050
      if (newmem != 0) {
4051
        MALLOC_COPY(newmem, oldmem, oldsize - 2*SIZE_SZ);
4052
        fREe(oldmem);
4053
      }
4054
    }
4055
    return newmem;
10347 ripley 4056
 
26855 ripley 4057
#else 
4058
    /* If !HAVE_MMAP, but chunk_is_mmapped, user must have overwritten mem */
4059
    check_malloc_state();
4060
    MALLOC_FAILURE_ACTION;
4061
    return 0;
4062
#endif
4063
  }
4064
}
10347 ripley 4065
 
26855 ripley 4066
/*
4067
  ------------------------------ memalign ------------------------------
10347 ripley 4068
*/
4069
 
4070
Void_t* mEMALIGn(size_t alignment, size_t bytes)
4071
{
26855 ripley 4072
  INTERNAL_SIZE_T nb;             /* padded  request size */
4073
  char*           m;              /* memory returned by malloc call */
4074
  mchunkptr       p;              /* corresponding chunk */
4075
  char*           brk;            /* alignment point within p */
4076
  mchunkptr       newp;           /* chunk to return */
4077
  INTERNAL_SIZE_T newsize;        /* its size */
4078
  INTERNAL_SIZE_T leadsize;       /* leading space before alignment point */
4079
  mchunkptr       remainder;      /* spare room at end to split off */
4080
  CHUNK_SIZE_T    remainder_size; /* its size */
4081
  INTERNAL_SIZE_T size;
10347 ripley 4082
 
4083
  /* If need less alignment than we give anyway, just relay to malloc */
4084
 
4085
  if (alignment <= MALLOC_ALIGNMENT) return mALLOc(bytes);
4086
 
4087
  /* Otherwise, ensure that it is at least a minimum chunk size */
4088
 
4089
  if (alignment <  MINSIZE) alignment = MINSIZE;
4090
 
26855 ripley 4091
  /* Make sure alignment is power of 2 (in case MINSIZE is not).  */
4092
  if ((alignment & (alignment - 1)) != 0) {
4093
    size_t a = MALLOC_ALIGNMENT * 2;
4094
    while ((CHUNK_SIZE_T)a < (CHUNK_SIZE_T)alignment) a <<= 1;
4095
    alignment = a;
4096
  }
4097
 
4098
  checked_request2size(bytes, nb);
4099
 
4100
  /*
4101
    Strategy: find a spot within that chunk that meets the alignment
4102
    request, and then possibly free the leading and trailing space.
4103
  */
4104
 
4105
 
10347 ripley 4106
  /* Call malloc with worst case padding to hit alignment. */
4107
 
4108
  m  = (char*)(mALLOc(nb + alignment + MINSIZE));
4109
 
4110
  if (m == 0) return 0; /* propagate failure */
4111
 
4112
  p = mem2chunk(m);
4113
 
26855 ripley 4114
  if ((((PTR_UINT)(m)) % alignment) != 0) { /* misaligned */
4115
 
10347 ripley 4116
    /*
26855 ripley 4117
      Find an aligned spot inside chunk.  Since we need to give back
4118
      leading space in a chunk of at least MINSIZE, if the first
4119
      calculation places us at a spot with less than MINSIZE leader,
4120
      we can move to the next aligned spot -- we've allocated enough
4121
      total room so that this is always possible.
10347 ripley 4122
    */
4123
 
26855 ripley 4124
    brk = (char*)mem2chunk((PTR_UINT)(((PTR_UINT)(m + alignment - 1)) &
4125
                           -((signed long) alignment)));
4126
    if ((CHUNK_SIZE_T)(brk - (char*)(p)) < MINSIZE)
4127
      brk += alignment;
10347 ripley 4128
 
4129
    newp = (mchunkptr)brk;
4130
    leadsize = brk - (char*)(p);
4131
    newsize = chunksize(p) - leadsize;
4132
 
26855 ripley 4133
    /* For mmapped chunks, just adjust offset */
4134
    if (chunk_is_mmapped(p)) {
10347 ripley 4135
      newp->prev_size = p->prev_size + leadsize;
4136
      set_head(newp, newsize|IS_MMAPPED);
4137
      return chunk2mem(newp);
4138
    }
4139
 
26855 ripley 4140
    /* Otherwise, give back leader, use the rest */
10347 ripley 4141
    set_head(newp, newsize | PREV_INUSE);
4142
    set_inuse_bit_at_offset(newp, newsize);
4143
    set_head_size(p, leadsize);
4144
    fREe(chunk2mem(p));
4145
    p = newp;
4146
 
26855 ripley 4147
    assert (newsize >= nb &&
4148
            (((PTR_UINT)(chunk2mem(p))) % alignment) == 0);
10347 ripley 4149
  }
4150
 
4151
  /* Also give back spare room at the end */
26855 ripley 4152
  if (!chunk_is_mmapped(p)) {
4153
    size = chunksize(p);
4154
    if ((CHUNK_SIZE_T)(size) > (CHUNK_SIZE_T)(nb + MINSIZE)) {
4155
      remainder_size = size - nb;
4156
      remainder = chunk_at_offset(p, nb);
4157
      set_head(remainder, remainder_size | PREV_INUSE);
4158
      set_head_size(p, nb);
4159
      fREe(chunk2mem(remainder));
4160
    }
10347 ripley 4161
  }
4162
 
4163
  check_inuse_chunk(p);
4164
  return chunk2mem(p);
26855 ripley 4165
}
10347 ripley 4166
 
26855 ripley 4167
/*
4168
  ------------------------------ calloc ------------------------------
4169
*/
4170
 
4171
Void_t* cALLOc(size_t n_elements, size_t elem_size)
4172
{
4173
  mchunkptr p;
4174
  CHUNK_SIZE_T  clearsize;
4175
  CHUNK_SIZE_T  nclears;
4176
  INTERNAL_SIZE_T* d;
4177
 
4178
  Void_t* mem = mALLOc(n_elements * elem_size);
4179
 
4180
  if (mem != 0) {
4181
    p = mem2chunk(mem);
4182
 
4183
    if (!chunk_is_mmapped(p))
4184
    {  
4185
      /*
4186
        Unroll clear of <= 36 bytes (72 if 8byte sizes)
4187
        We know that contents have an odd number of
4188
        INTERNAL_SIZE_T-sized words; minimally 3.
4189
      */
4190
 
4191
      d = (INTERNAL_SIZE_T*)mem;
4192
      clearsize = chunksize(p) - SIZE_SZ;
4193
      nclears = clearsize / sizeof(INTERNAL_SIZE_T);
4194
      assert(nclears >= 3);
4195
 
4196
      if (nclears > 9)
4197
        MALLOC_ZERO(d, clearsize);
4198
 
4199
      else {
4200
        *(d+0) = 0;
4201
        *(d+1) = 0;
4202
        *(d+2) = 0;
4203
        if (nclears > 4) {
4204
          *(d+3) = 0;
4205
          *(d+4) = 0;
4206
          if (nclears > 6) {
4207
            *(d+5) = 0;
4208
            *(d+6) = 0;
4209
            if (nclears > 8) {
4210
              *(d+7) = 0;
4211
              *(d+8) = 0;
4212
            }
4213
          }
4214
        }
4215
      }
4216
    }
4217
#if ! MMAP_CLEARS
4218
    else
4219
    {
4220
      d = (INTERNAL_SIZE_T*)mem;
4221
      /*
4222
        Note the additional SIZE_SZ
4223
      */
4224
      clearsize = chunksize(p) - 2*SIZE_SZ;
4225
      MALLOC_ZERO(d, clearsize);
4226
    }
4227
#endif
4228
  }
4229
  return mem;
10347 ripley 4230
}
4231
 
26855 ripley 4232
/*
4233
  ------------------------------ cfree ------------------------------
4234
*/
10347 ripley 4235
 
26855 ripley 4236
void cFREe(Void_t *mem)
4237
{
4238
  fREe(mem);
4239
}
10347 ripley 4240
 
4241
/*
26855 ripley 4242
  ------------------------- independent_calloc -------------------------
10347 ripley 4243
*/
4244
 
26855 ripley 4245
Void_t** iCALLOc(size_t n_elements, size_t elem_size, Void_t* chunks[])
10347 ripley 4246
{
26855 ripley 4247
  size_t sz = elem_size; /* serves as 1-element array */
4248
  /* opts arg of 3 means all elements are same size, and should be cleared */
4249
  return iALLOc(n_elements, &sz, 3, chunks);
10347 ripley 4250
}
4251
 
4252
/*
26855 ripley 4253
  ------------------------- independent_comalloc -------------------------
10347 ripley 4254
*/
4255
 
26855 ripley 4256
Void_t** iCOMALLOc(size_t n_elements, size_t sizes[], Void_t* chunks[])
10347 ripley 4257
{
26855 ripley 4258
  return iALLOc(n_elements, sizes, 0, chunks);
10347 ripley 4259
}
4260
 
26855 ripley 4261
 
10347 ripley 4262
/*
26855 ripley 4263
  ------------------------------ ialloc ------------------------------
4264
  ialloc provides common support for independent_X routines, handling all of
4265
  the combinations that can result.
10347 ripley 4266
 
26855 ripley 4267
  The opts arg has:
4268
    bit 0 set if all elements are same size (using sizes[0])
4269
    bit 1 set if elements should be zeroed
10347 ripley 4270
*/
4271
 
26855 ripley 4272
 
4273
static Void_t** iALLOc(size_t n_elements, 
4274
                       size_t* sizes,  
4275
                       int opts,
4276
                       Void_t* chunks[])
10347 ripley 4277
{
26855 ripley 4278
  mstate av = get_malloc_state();
4279
  INTERNAL_SIZE_T element_size;   /* chunksize of each element, if all same */
4280
  INTERNAL_SIZE_T contents_size;  /* total size of elements */
4281
  INTERNAL_SIZE_T array_size;     /* request size of pointer array */
4282
  Void_t*         mem;            /* malloced aggregate space */
4283
  mchunkptr       p;              /* corresponding chunk */
4284
  INTERNAL_SIZE_T remainder_size; /* remaining bytes while splitting */
4285
  Void_t**        marray;         /* either "chunks" or malloced ptr array */
4286
  mchunkptr       array_chunk;    /* chunk for malloced ptr array */
4287
  int             mmx;            /* to disable mmap */
4288
  INTERNAL_SIZE_T size;           
4289
  size_t          i;
10347 ripley 4290
 
26855 ripley 4291
  /* Ensure initialization */
4292
  if (av->max_fast == 0) malloc_consolidate(av);
10347 ripley 4293
 
26855 ripley 4294
  /* compute array length, if needed */
4295
  if (chunks != 0) {
4296
    if (n_elements == 0)
4297
      return chunks; /* nothing to do */
4298
    marray = chunks;
4299
    array_size = 0;
4300
  }
4301
  else {
4302
    /* if empty req, must still return chunk representing empty array */
4303
    if (n_elements == 0) 
4304
      return (Void_t**) mALLOc(0);
4305
    marray = 0;
4306
    array_size = request2size(n_elements * (sizeof(Void_t*)));
4307
  }
10425 ripley 4308
 
26855 ripley 4309
  /* compute total element size */
4310
  if (opts & 0x1) { /* all-same-size */
4311
    element_size = request2size(*sizes);
4312
    contents_size = n_elements * element_size;
4313
  }
4314
  else { /* add up all the sizes */
4315
    element_size = 0;
4316
    contents_size = 0;
4317
    for (i = 0; i != n_elements; ++i) 
4318
      contents_size += request2size(sizes[i]);     
4319
  }
10347 ripley 4320
 
26855 ripley 4321
  /* subtract out alignment bytes from total to minimize overallocation */
4322
  size = contents_size + array_size - MALLOC_ALIGN_MASK;
4323
 
4324
  /* 
4325
     Allocate the aggregate chunk.
4326
     But first disable mmap so malloc won't use it, since
4327
     we would not be able to later free/realloc space internal
4328
     to a segregated mmap region.
4329
 */
4330
  mmx = av->n_mmaps_max;   /* disable mmap */
4331
  av->n_mmaps_max = 0;
4332
  mem = mALLOc(size);
4333
  av->n_mmaps_max = mmx;   /* reset mmap */
4334
  if (mem == 0) 
10347 ripley 4335
    return 0;
4336
 
26855 ripley 4337
  p = mem2chunk(mem);
4338
  assert(!chunk_is_mmapped(p)); 
4339
  remainder_size = chunksize(p);
10347 ripley 4340
 
26855 ripley 4341
  if (opts & 0x2) {       /* optionally clear the elements */
4342
    MALLOC_ZERO(mem, remainder_size - SIZE_SZ - array_size);
4343
  }
10347 ripley 4344
 
26855 ripley 4345
  /* If not provided, allocate the pointer array as final part of chunk */
4346
  if (marray == 0) {
4347
    array_chunk = chunk_at_offset(p, contents_size);
4348
    marray = (Void_t**) (chunk2mem(array_chunk));
4349
    set_head(array_chunk, (remainder_size - contents_size) | PREV_INUSE);
4350
    remainder_size = contents_size;
4351
  }
10347 ripley 4352
 
26855 ripley 4353
  /* split out elements */
4354
  for (i = 0; ; ++i) {
4355
    marray[i] = chunk2mem(p);
4356
    if (i != n_elements-1) {
4357
      if (element_size != 0) 
4358
        size = element_size;
4359
      else
4360
        size = request2size(sizes[i]);          
4361
      remainder_size -= size;
4362
      set_head(p, size | PREV_INUSE);
4363
      p = chunk_at_offset(p, size);
4364
    }
4365
    else { /* the final element absorbs any overallocation slop */
4366
      set_head(p, remainder_size | PREV_INUSE);
4367
      break;
4368
    }
4369
  }
10347 ripley 4370
 
26855 ripley 4371
#if DEBUG
4372
  if (marray != chunks) {
4373
    /* final element must have exactly exhausted chunk */
4374
    if (element_size != 0) 
4375
      assert(remainder_size == element_size);
4376
    else
4377
      assert(remainder_size == request2size(sizes[i]));
4378
    check_inuse_chunk(mem2chunk(marray));
4379
  }
4380
 
4381
  for (i = 0; i != n_elements; ++i)
4382
    check_inuse_chunk(mem2chunk(marray[i]));
10347 ripley 4383
#endif
4384
 
26855 ripley 4385
  return marray;
10347 ripley 4386
}
4387
 
26855 ripley 4388
 
10347 ripley 4389
/*
26855 ripley 4390
  ------------------------------ valloc ------------------------------
4391
*/
10347 ripley 4392
 
26855 ripley 4393
Void_t* vALLOc(size_t bytes)
4394
{
4395
  /* Ensure initialization */
4396
  mstate av = get_malloc_state();
4397
  if (av->max_fast == 0) malloc_consolidate(av);
4398
  return mEMALIGn(av->pagesize, bytes);
4399
}
10347 ripley 4400
 
26855 ripley 4401
/*
4402
  ------------------------------ pvalloc ------------------------------
10347 ripley 4403
*/
4404
 
26855 ripley 4405
 
4406
Void_t* pVALLOc(size_t bytes)
10347 ripley 4407
{
26855 ripley 4408
  mstate av = get_malloc_state();
4409
  size_t pagesz;
4410
 
4411
  /* Ensure initialization */
4412
  if (av->max_fast == 0) malloc_consolidate(av);
4413
  pagesz = av->pagesize;
4414
  return mEMALIGn(pagesz, (bytes + pagesz - 1) & ~(pagesz - 1));
10347 ripley 4415
}
26855 ripley 4416
 
10347 ripley 4417
 
4418
/*
26855 ripley 4419
  ------------------------------ malloc_trim ------------------------------
4420
*/
10347 ripley 4421
 
26855 ripley 4422
int mTRIm(size_t pad)
4423
{
4424
  mstate av = get_malloc_state();
4425
  /* Ensure initialization/consolidation */
4426
  malloc_consolidate(av);
10347 ripley 4427
 
26855 ripley 4428
#ifndef MORECORE_CANNOT_TRIM        
4429
  return sYSTRIm(pad, av);
4430
#else
4431
  return 0;
4432
#endif
4433
}
10347 ripley 4434
 
4435
 
26855 ripley 4436
/*
4437
  ------------------------- malloc_usable_size -------------------------
10347 ripley 4438
*/
4439
 
26855 ripley 4440
size_t mUSABLe(Void_t* mem)
10347 ripley 4441
{
26855 ripley 4442
  mchunkptr p;
4443
  if (mem != 0) {
4444
    p = mem2chunk(mem);
4445
    if (chunk_is_mmapped(p))
4446
      return chunksize(p) - 2*SIZE_SZ;
4447
    else if (inuse(p))
4448
      return chunksize(p) - SIZE_SZ;
4449
  }
4450
  return 0;
4451
}
10347 ripley 4452
 
26855 ripley 4453
/*
4454
  ------------------------------ mallinfo ------------------------------
4455
*/
10347 ripley 4456
 
26855 ripley 4457
struct mallinfo mALLINFo()
4458
{
4459
  mstate av = get_malloc_state();
4460
  struct mallinfo mi;
4461
  int i;
4462
  mbinptr b;
4463
  mchunkptr p;
4464
  INTERNAL_SIZE_T avail;
4465
  INTERNAL_SIZE_T fastavail;
4466
  int nblocks;
4467
  int nfastblocks;
10347 ripley 4468
 
26855 ripley 4469
  /* Ensure initialization */
4470
  if (av->top == 0)  malloc_consolidate(av);
10347 ripley 4471
 
26855 ripley 4472
  check_malloc_state();
4473
 
4474
  /* Account for top */
4475
  avail = chunksize(av->top);
4476
  nblocks = 1;  /* top always exists */
4477
 
4478
  /* traverse fastbins */
4479
  nfastblocks = 0;
4480
  fastavail = 0;
4481
 
4482
  for (i = 0; i < NFASTBINS; ++i) {
4483
    for (p = av->fastbins[i]; p != 0; p = p->fd) {
4484
      ++nfastblocks;
4485
      fastavail += chunksize(p);
4486
    }
4487
  }
4488
 
4489
  avail += fastavail;
4490
 
4491
  /* traverse regular bins */
4492
  for (i = 1; i < NBINS; ++i) {
4493
    b = bin_at(av, i);
4494
    for (p = last(b); p != b; p = p->bk) {
4495
      ++nblocks;
4496
      avail += chunksize(p);
4497
    }
4498
  }
4499
 
4500
  mi.smblks = nfastblocks;
4501
  mi.ordblks = nblocks;
4502
  mi.fordblks = avail;
26945 ripley 4503
  mi.uordblks = (av->sbrked_mem - avail) + av->mmapped_mem;
26855 ripley 4504
  mi.arena = av->sbrked_mem;
4505
  mi.hblks = av->n_mmaps;
4506
  mi.hblkhd = av->mmapped_mem;
4507
  mi.fsmblks = fastavail;
4508
  mi.keepcost = chunksize(av->top);
4509
  mi.usmblks = av->max_total_mem;
4510
 
4511
#ifdef TRACE
4512
 
4513
#ifdef WIN32
10347 ripley 4514
  {
26855 ripley 4515
    CHUNK_SIZE_T  free, reserved, committed;
4516
    vminfo (&free, &reserved, &committed);
4517
    printf("free bytes       = %10lu\n", 
4518
            free);
4519
    printf("reserved bytes   = %10lu\n", 
4520
            reserved);
4521
    printf("committed bytes  = %10lu\n", 
4522
            committed);
4523
  }
4524
#endif
10347 ripley 4525
 
4526
 
26855 ripley 4527
  printf("max system bytes = %10lu\n",
4528
          (CHUNK_SIZE_T)(mi.usmblks));
4529
  printf("system bytes     = %10lu\n",
4530
          (CHUNK_SIZE_T)(mi.arena + mi.hblkhd));
4531
  printf("in use bytes     = %10lu\n",
4532
          (CHUNK_SIZE_T)(mi.uordblks + mi.hblkhd));
4533
#endif
10347 ripley 4534
 
26855 ripley 4535
  return mi;
4536
}
4537
 
4538
/*
4539
  ------------------------------ malloc_stats ------------------------------
4540
*/
4541
 
4542
void mSTATs()
4543
{
4544
  struct mallinfo mi = mALLINFo();
4545
 
4546
#ifdef WIN32
4547
  {
4548
    CHUNK_SIZE_T  free, reserved, committed;
4549
    vminfo (&free, &reserved, &committed);
4550
    fprintf(stderr, "free bytes       = %10lu\n", 
4551
            free);
4552
    fprintf(stderr, "reserved bytes   = %10lu\n", 
4553
            reserved);
4554
    fprintf(stderr, "committed bytes  = %10lu\n", 
4555
            committed);
4556
  }
4557
#endif
4558
 
4559
 
4560
  fprintf(stderr, "max system bytes = %10lu\n",
4561
          (CHUNK_SIZE_T)(mi.usmblks));
4562
  fprintf(stderr, "system bytes     = %10lu\n",
4563
          (CHUNK_SIZE_T)(mi.arena + mi.hblkhd));
4564
  fprintf(stderr, "in use bytes     = %10lu\n",
4565
          (CHUNK_SIZE_T)(mi.uordblks + mi.hblkhd));
4566
 
4567
#ifdef WIN32 
4568
/* Commented out for R Build Compatibility
4569
  {
4570
    CHUNK_SIZE_T  kernel, user;
4571
    if (cpuinfo (TRUE, &kernel, &user)) {
4572
      fprintf(stderr, "kernel ms        = %10lu\n", 
4573
              kernel);
4574
      fprintf(stderr, "user ms          = %10lu\n", 
4575
              user);
10347 ripley 4576
    }
4577
  }
26855 ripley 4578
 */
4579
#endif
10347 ripley 4580
}
4581
 
4582
 
4583
/*
26855 ripley 4584
  ------------------------------ mallopt ------------------------------
10347 ripley 4585
*/
4586
 
26855 ripley 4587
int mALLOPt(int param_number, int value)
10347 ripley 4588
{
26855 ripley 4589
  mstate av = get_malloc_state();
4590
  /* Ensure initialization/consolidation */
4591
  malloc_consolidate(av);
4592
 
4593
  switch(param_number) {
4594
  case M_MXFAST:
4595
    if (value >= 0 && value <= MAX_FAST_SIZE) {
4596
      set_max_fast(av, value);
4597
      return 1;
4598
    }
4599
    else
4600
      return 0;
4601
 
4602
  case M_TRIM_THRESHOLD:
4603
    av->trim_threshold = value;
4604
    return 1;
4605
 
4606
  case M_TOP_PAD:
4607
    av->top_pad = value;
4608
    return 1;
4609
 
4610
  case M_MMAP_THRESHOLD:
4611
    av->mmap_threshold = value;
4612
    return 1;
4613
 
4614
  case M_MMAP_MAX:
4615
#if !HAVE_MMAP
4616
    if (value != 0)
4617
      return 0;
4618
#endif
4619
    av->n_mmaps_max = value;
4620
    return 1;
4621
 
4622
  default:
10347 ripley 4623
    return 0;
4624
  }
4625
}
4626
 
4627
 
26855 ripley 4628
/* 
4629
  -------------------- Alternative MORECORE functions --------------------
4630
*/
10347 ripley 4631
 
4632
 
26855 ripley 4633
/*
4634
  General Requirements for MORECORE.
10347 ripley 4635
 
26855 ripley 4636
  The MORECORE function must have the following properties:
10347 ripley 4637
 
26855 ripley 4638
  If MORECORE_CONTIGUOUS is false:
4639
 
4640
    * MORECORE must allocate in multiples of pagesize. It will
4641
      only be called with arguments that are multiples of pagesize.
4642
 
4643
    * MORECORE(0) must return an address that is at least 
4644
      MALLOC_ALIGNMENT aligned. (Page-aligning always suffices.)
4645
 
4646
  else (i.e. If MORECORE_CONTIGUOUS is true):
4647
 
4648
    * Consecutive calls to MORECORE with positive arguments
4649
      return increasing addresses, indicating that space has been
4650
      contiguously extended. 
4651
 
4652
    * MORECORE need not allocate in multiples of pagesize.
4653
      Calls to MORECORE need not have args of multiples of pagesize.
4654
 
4655
    * MORECORE need not page-align.
4656
 
4657
  In either case:
4658
 
4659
    * MORECORE may allocate more memory than requested. (Or even less,
4660
      but this will generally result in a malloc failure.)
4661
 
4662
    * MORECORE must not allocate memory when given argument zero, but
4663
      instead return one past the end address of memory from previous
4664
      nonzero call. This malloc does NOT call MORECORE(0)
4665
      until at least one call with positive arguments is made, so
4666
      the initial value returned is not important.
4667
 
4668
    * Even though consecutive calls to MORECORE need not return contiguous
4669
      addresses, it must be OK for malloc'ed chunks to span multiple
4670
      regions in those cases where they do happen to be contiguous.
4671
 
4672
    * MORECORE need not handle negative arguments -- it may instead
4673
      just return MORECORE_FAILURE when given negative arguments.
4674
      Negative arguments are always multiples of pagesize. MORECORE
4675
      must not misinterpret negative args as large positive unsigned
4676
      args. You can suppress all such calls from even occurring by defining
4677
      MORECORE_CANNOT_TRIM,
4678
 
4679
  There is some variation across systems about the type of the
4680
  argument to sbrk/MORECORE. If size_t is unsigned, then it cannot
4681
  actually be size_t, because sbrk supports negative args, so it is
4682
  normally the signed type of the same width as size_t (sometimes
4683
  declared as "intptr_t", and sometimes "ptrdiff_t").  It doesn't much
4684
  matter though. Internally, we use "long" as arguments, which should
4685
  work across all reasonable possibilities.
4686
 
4687
  Additionally, if MORECORE ever returns failure for a positive
4688
  request, and HAVE_MMAP is true, then mmap is used as a noncontiguous
4689
  system allocator. This is a useful backup strategy for systems with
4690
  holes in address spaces -- in this case sbrk cannot contiguously
4691
  expand the heap, but mmap may be able to map noncontiguous space.
4692
 
4693
  If you'd like mmap to ALWAYS be used, you can define MORECORE to be
4694
  a function that always returns MORECORE_FAILURE.
4695
 
4696
  Malloc only has limited ability to detect failures of MORECORE
4697
  to supply contiguous space when it says it can. In particular,
4698
  multithreaded programs that do not use locks may result in
4699
  rece conditions across calls to MORECORE that result in gaps
4700
  that cannot be detected as such, and subsequent corruption.
4701
 
4702
  If you are using this malloc with something other than sbrk (or its
4703
  emulation) to supply memory regions, you probably want to set
4704
  MORECORE_CONTIGUOUS as false.  As an example, here is a custom
4705
  allocator kindly contributed for pre-OSX macOS.  It uses virtually
4706
  but not necessarily physically contiguous non-paged memory (locked
4707
  in, present and won't get swapped out).  You can use it by
4708
  uncommenting this section, adding some #includes, and setting up the
4709
  appropriate defines above:
4710
 
4711
      #define MORECORE osMoreCore
4712
      #define MORECORE_CONTIGUOUS 0
4713
 
4714
  There is also a shutdown routine that should somehow be called for
4715
  cleanup upon program exit.
4716
 
4717
  #define MAX_POOL_ENTRIES 100
4718
  #define MINIMUM_MORECORE_SIZE  (64 * 1024)
4719
  static int next_os_pool;
4720
  void *our_os_pools[MAX_POOL_ENTRIES];
4721
 
4722
  void *osMoreCore(int size)
10347 ripley 4723
  {
26855 ripley 4724
    void *ptr = 0;
4725
    static void *sbrk_top = 0;
4726
 
4727
    if (size > 0)
10347 ripley 4728
    {
26855 ripley 4729
      if (size < MINIMUM_MORECORE_SIZE)
4730
         size = MINIMUM_MORECORE_SIZE;
4731
      if (CurrentExecutionLevel() == kTaskLevel)
4732
         ptr = PoolAllocateResident(size + RM_PAGE_SIZE, 0);
4733
      if (ptr == 0)
4734
      {
4735
        return (void *) MORECORE_FAILURE;
4736
      }
4737
      // save ptrs so they can be freed during cleanup
4738
      our_os_pools[next_os_pool] = ptr;
4739
      next_os_pool++;
4740
      ptr = (void *) ((((CHUNK_SIZE_T) ptr) + RM_PAGE_MASK) & ~RM_PAGE_MASK);
4741
      sbrk_top = (char *) ptr + size;
4742
      return ptr;
10347 ripley 4743
    }
26855 ripley 4744
    else if (size < 0)
4745
    {
4746
      // we don't currently support shrink behavior
4747
      return (void *) MORECORE_FAILURE;
4748
    }
4749
    else
4750
    {
4751
      return sbrk_top;
4752
    }
10347 ripley 4753
  }
4754
 
26855 ripley 4755
  // cleanup any allocated memory pools
4756
  // called as last thing before shutting down driver
10347 ripley 4757
 
26855 ripley 4758
  void osCleanupMem(void)
4759
  {
4760
    void **ptr;
10347 ripley 4761
 
26855 ripley 4762
    for (ptr = our_os_pools; ptr < &our_os_pools[MAX_POOL_ENTRIES]; ptr++)
4763
      if (*ptr)
4764
      {
4765
         PoolDeallocate(*ptr);
4766
         *ptr = 0;
4767
      }
4768
  }
10347 ripley 4769
 
26855 ripley 4770
*/
10347 ripley 4771
 
4772
 
26855 ripley 4773
/* 
4774
  -------------------------------------------------------------- 
10347 ripley 4775
 
26855 ripley 4776
  Emulation of sbrk for win32. 
4777
  Donated by J. Walter <Walter@GeNeSys-e.de>.
4778
  For additional information about this code, and malloc on Win32, see 
4779
     http://www.genesys-e.de/jwalter/
10347 ripley 4780
*/
4781
 
26855 ripley 4782
 
4783
#ifdef WIN32
4784
 
4785
#ifdef _DEBUG
4786
/* #define TRACE */
4787
/* #define TRACESB */
10347 ripley 4788
#endif
26855 ripley 4789
 
4790
/* #define TRACE */
4791
/* #define TRACESB */
4792
 
4793
/* Support for USE_MALLOC_LOCK */
4794
#ifdef USE_MALLOC_LOCK
4795
 
4796
/* Wait for spin lock */
4797
static int slwait (int *sl) {
4798
    while (InterlockedCompareExchange ((void **) sl, (void *) 1, (void *) 0) != 0) 
4799
	    Sleep (0);
4800
    return 0;
10347 ripley 4801
}
4802
 
26855 ripley 4803
/* Release spin lock */
4804
static int slrelease (int *sl) {
4805
    InterlockedExchange (sl, 0);
4806
    return 0;
4807
}
10347 ripley 4808
 
26855 ripley 4809
#ifdef NEEDED
4810
/* Spin lock for emulation code */
4811
static int g_sl;
4812
#endif
4813
 
4814
#endif /* USE_MALLOC_LOCK */
4815
 
4816
/* getpagesize for windows */
4817
static long getpagesize (void) {
4818
    static long g_pagesize = 0;
4819
    if (! g_pagesize) {
4820
        SYSTEM_INFO system_info;
4821
        GetSystemInfo (&system_info);
4822
        g_pagesize = system_info.dwPageSize;
4823
    }
4824
    return g_pagesize;
10347 ripley 4825
}
26855 ripley 4826
static long getregionsize (void) {
4827
    static long g_regionsize = 0;
4828
    if (! g_regionsize) {
4829
        SYSTEM_INFO system_info;
4830
        GetSystemInfo (&system_info);
4831
        g_regionsize = system_info.dwAllocationGranularity;
4832
    }
4833
    return g_regionsize;
4834
}
10347 ripley 4835
 
26855 ripley 4836
/* A region list entry */
4837
typedef struct _region_list_entry {
4838
    void *top_allocated;
4839
    void *top_committed;
4840
    void *top_reserved;
4841
    long reserve_size;
4842
    struct _region_list_entry *previous;
4843
} region_list_entry;
10347 ripley 4844
 
26855 ripley 4845
/* Allocate and link a region entry in the region list */
4846
static int region_list_append (region_list_entry **last, void *base_reserved, long reserve_size) {
4847
#ifdef TRACESB
4848
	printf ("region_list_append %p %p %lu\n", last, base_reserved, reserve_size);
4849
#endif
4850
#ifdef DEBUG
4851
	assert(HeapValidate(GetProcessHeap(), 0, NULL));
4852
#endif
4853
    region_list_entry *next = HeapAlloc (GetProcessHeap (), 0, sizeof (region_list_entry));
4854
    if (! next)
4855
	{
4856
#ifdef TRACESB
4857
	    printf ("region_list_append HeapAlloc failed\n");
4858
#endif
4859
        return FALSE;
4860
	}
4861
    next->top_allocated = (char *) base_reserved;
4862
    next->top_committed = (char *) base_reserved;
4863
    next->top_reserved = (char *) base_reserved + reserve_size;
4864
    next->reserve_size = reserve_size;
4865
    next->previous = *last;
4866
    *last = next;
4867
    return TRUE;
4868
}
4869
/* Free and unlink the last region entry from the region list */
4870
static int region_list_remove (region_list_entry **last) {
4871
    region_list_entry *previous = (*last)->previous;
4872
#ifdef TRACESB
4873
	printf ("region_list_remove %p %p\n", last, previous);
4874
#endif
4875
#ifdef DEBUG
4876
	assert(HeapValidate(GetProcessHeap(), 0, NULL));
4877
#endif
4878
    if (! HeapFree (GetProcessHeap (), sizeof (region_list_entry), *last)) {
4879
#ifdef TRACESB
4880
	    printf ("region_list_remove HeapFree failed\n");
4881
#endif
4882
        return FALSE;
4883
	}
4884
    *last = previous;
4885
    return TRUE;
4886
}
10347 ripley 4887
 
26855 ripley 4888
#define CEIL(size,to)	(((size)+(to)-1)&~((to)-1))
4889
#define FLOOR(size,to)	((size)&~((to)-1))
10347 ripley 4890
 
26855 ripley 4891
#define SBRK_SCALE  0
4892
/* #define SBRK_SCALE  1 */
4893
/* #define SBRK_SCALE  2 */
4894
/* #define SBRK_SCALE  4  */
10347 ripley 4895
 
26855 ripley 4896
/* sbrk for windows */
4897
static void *sbrk (long size) {
4898
    static long g_pagesize, g_my_pagesize;
4899
    static long g_regionsize, g_my_regionsize;
4900
    static region_list_entry *g_last;
4901
    void *result = (void *) MORECORE_FAILURE;
10347 ripley 4902
 
26855 ripley 4903
#ifdef TRACESB
4904
    printf ("sbrk %ld\n", size);
4905
#endif
4906
#if defined (USE_MALLOC_LOCK) && defined (NEEDED)
4907
    /* Wait for spin lock */
4908
    slwait (&g_sl);
4909
#endif
4910
    /* First time initialization */
4911
    if (! g_pagesize) {
4912
        g_pagesize = getpagesize ();
4913
        g_my_pagesize = g_pagesize << SBRK_SCALE;
4914
    }
4915
    if (! g_regionsize) {
4916
        g_regionsize = getregionsize ();
4917
        g_my_regionsize = g_regionsize << SBRK_SCALE;
4918
    }
4919
    if (! g_last) {
4920
        if (! region_list_append (&g_last, 0, 0)) 
4921
           goto sbrk_exit;
4922
    }
4923
    /* Assert invariants */
4924
    assert (g_last);
4925
    assert ((char *) g_last->top_reserved - g_last->reserve_size <= (char *) g_last->top_allocated &&
4926
            g_last->top_allocated <= g_last->top_committed);
4927
    assert ((char *) g_last->top_reserved - g_last->reserve_size <= (char *) g_last->top_committed &&
4928
            g_last->top_committed <= g_last->top_reserved &&
4929
            (unsigned) g_last->top_committed % g_pagesize == 0);
4930
    assert ((unsigned) g_last->top_reserved % g_regionsize == 0);
4931
    assert ((unsigned) g_last->reserve_size % g_regionsize == 0);
4932
    /* Allocation requested? */
4933
    if (size >= 0) {
4934
        /* Allocation size is the requested size */
4935
        long allocate_size = size;
4936
        /* Compute the size to commit */
4937
        long to_commit = (char *) g_last->top_allocated + allocate_size - (char *) g_last->top_committed;
4938
        /* Do we reach the commit limit? */
4939
        if (to_commit > 0) {
4940
            /* Round size to commit */
4941
            long commit_size = CEIL (to_commit, g_my_pagesize);
4942
            /* Compute the size to reserve */
4943
            long to_reserve = (char *) g_last->top_committed + commit_size - (char *) g_last->top_reserved;
4944
            /* Do we reach the reserve limit? */
4945
            if (to_reserve > 0) {
4946
                /* Compute the remaining size to commit in the current region */
4947
                long remaining_commit_size = (char *) g_last->top_reserved - (char *) g_last->top_committed;
4948
                if (remaining_commit_size > 0) {
4949
                    /* Assert preconditions */
4950
                    assert ((unsigned) g_last->top_committed % g_pagesize == 0);
4951
                    assert (0 < remaining_commit_size && remaining_commit_size % g_pagesize == 0); {
4952
                        /* Commit this */
4953
                        void *base_committed = VirtualAlloc (g_last->top_committed, remaining_commit_size,
4954
							                                 MEM_COMMIT, PAGE_READWRITE);
4955
                        /* Check returned pointer for consistency */
4956
                        if (base_committed != g_last->top_committed)
4957
                            goto sbrk_exit;
4958
                        /* Assert postconditions */
4959
                        assert ((unsigned) base_committed % g_pagesize == 0);
4960
#ifdef TRACESB
4961
                        printf ("Commit %p %ld\n", base_committed, remaining_commit_size);
4962
#endif
4963
                        /* Adjust the regions commit top */
4964
                        g_last->top_committed = (char *) base_committed + remaining_commit_size;
4965
                    }
4966
                } {
4967
                    /* Now we are going to search and reserve. */
4968
                    int contiguous = -1;
4969
                    int found = FALSE;
4970
                    MEMORY_BASIC_INFORMATION memory_info;
4971
                    void *base_reserved;
4972
                    long reserve_size;
4973
                    do {
4974
                        /* Assume contiguous memory */
4975
                        contiguous = TRUE;
4976
                        /* Round size to reserve */
4977
                        reserve_size = CEIL (to_reserve, g_my_regionsize);
4978
                        /* Start with the current region's top */
4979
                        memory_info.BaseAddress = g_last->top_reserved;
4980
                        /* Assert preconditions */
4981
                        assert ((unsigned) memory_info.BaseAddress % g_pagesize == 0);
4982
                        assert (0 < reserve_size && reserve_size % g_regionsize == 0);
4983
                        while (VirtualQuery (memory_info.BaseAddress, &memory_info, sizeof (memory_info))) {
4984
                            /* Assert postconditions */
4985
                            assert ((unsigned) memory_info.BaseAddress % g_pagesize == 0);
4986
#ifdef TRACESB
4987
                            printf ("Query %p %ld %s\n", memory_info.BaseAddress, memory_info.RegionSize, 
4988
                                    memory_info.State == MEM_FREE ? "FREE": 
4989
                                    (memory_info.State == MEM_RESERVE ? "RESERVED":
4990
                                     (memory_info.State == MEM_COMMIT ? "COMMITTED": "?")));
4991
#endif
4992
                            /* Region is free, well aligned and big enough: we are done */
4993
                            if (memory_info.State == MEM_FREE &&
4994
                                (unsigned) memory_info.BaseAddress % g_regionsize == 0 &&
4995
                                memory_info.RegionSize >= (unsigned) reserve_size) {
4996
                                found = TRUE;
4997
                                break;
4998
                            }
4999
                            /* From now on we can't get contiguous memory! */
5000
                            contiguous = FALSE;
5001
                            /* Recompute size to reserve */
5002
                            reserve_size = CEIL (allocate_size, g_my_regionsize);
26945 ripley 5003
			    /* Wasn't free or didn't fit so move to end of block */
5004
			    memory_info.BaseAddress = (char *) memory_info.BaseAddress + memory_info.RegionSize;
5005
			    /* DBT - round up to next valid boundary */
5006
			    memory_info.BaseAddress = (void *)CEIL((unsigned long)memory_info.BaseAddress, g_regionsize);
26855 ripley 5007
                            /* Assert preconditions */
5008
                            assert ((unsigned) memory_info.BaseAddress % g_pagesize == 0);
5009
                            assert (0 < reserve_size && reserve_size % g_regionsize == 0);
5010
                        }
5011
                        /* Search failed? */
5012
                        if (! found) 
5013
                            goto sbrk_exit;
5014
                        /* Assert preconditions */
5015
                        assert ((unsigned) memory_info.BaseAddress % g_regionsize == 0);
5016
                        assert (0 < reserve_size && reserve_size % g_regionsize == 0);
5017
                        /* Try to reserve this */
5018
                        base_reserved = VirtualAlloc (memory_info.BaseAddress, reserve_size, 
5019
					                                  MEM_RESERVE, PAGE_NOACCESS);
5020
                        if (! base_reserved) {
5021
                            int rc = GetLastError ();
5022
                            if (rc != ERROR_INVALID_ADDRESS) 
5023
                                goto sbrk_exit;
5024
                        }
5025
                        /* A null pointer signals (hopefully) a race condition with another thread. */
5026
                        /* In this case, we try again. */
5027
                    } while (! base_reserved);
5028
                    /* Check returned pointer for consistency */
5029
                    if (memory_info.BaseAddress && base_reserved != memory_info.BaseAddress)
5030
                        goto sbrk_exit;
5031
                    /* Assert postconditions */
5032
                    assert ((unsigned) base_reserved % g_regionsize == 0);
5033
#ifdef TRACESB
5034
                    printf ("Reserve %p %ld\n", base_reserved, reserve_size);
5035
#endif
5036
                    /* Did we get contiguous memory? */
5037
                    if (contiguous) {
5038
                        long start_size = (char *) g_last->top_committed - (char *) g_last->top_allocated;
5039
                        /* Adjust allocation size */
5040
                        allocate_size -= start_size;
5041
                        /* Adjust the regions allocation top */
5042
                        g_last->top_allocated = g_last->top_committed;
5043
                        /* Recompute the size to commit */
5044
                        to_commit = (char *) g_last->top_allocated + allocate_size - (char *) g_last->top_committed;
5045
                        /* Round size to commit */
5046
                        commit_size = CEIL (to_commit, g_my_pagesize);
5047
                    } 
5048
                    /* Append the new region to the list */
5049
                    if (! region_list_append (&g_last, base_reserved, reserve_size))
5050
                        goto sbrk_exit;
5051
                    /* Didn't we get contiguous memory? */
5052
                    if (! contiguous) {
5053
                        /* Recompute the size to commit */
5054
                        to_commit = (char *) g_last->top_allocated + allocate_size - (char *) g_last->top_committed;
5055
                        /* Round size to commit */
5056
                        commit_size = CEIL (to_commit, g_my_pagesize);
5057
                    }
5058
                }
5059
            } 
5060
            /* Assert preconditions */
5061
            assert ((unsigned) g_last->top_committed % g_pagesize == 0);
5062
            assert (0 < commit_size && commit_size % g_pagesize == 0); {
26895 ripley 5063
		mstate av = get_malloc_state();
5064
                void *base_committed;
5065
		if (av->sbrked_mem + av->mmapped_mem + size > R_max_memory) {
5066
		    if(R_Is_Running) 
5067
			Rf_warning("Reached total allocation of %dMb: see help(memory.size)", R_max_memory/1048576);
5068
		    goto sbrk_exit;
5069
		}
26855 ripley 5070
                /* Commit this */
26895 ripley 5071
                base_committed = VirtualAlloc (g_last->top_committed, 
5072
					       commit_size, 
5073
					       MEM_COMMIT, PAGE_READWRITE);
26855 ripley 5074
                /* Check returned pointer for consistency */
5075
                if (base_committed != g_last->top_committed)
5076
                    goto sbrk_exit;
5077
                /* Assert postconditions */
5078
                assert ((unsigned) base_committed % g_pagesize == 0);
5079
#ifdef TRACESB
5080
                printf ("Commit %p %ld\n", base_committed, commit_size);
5081
#endif
5082
                /* Adjust the regions commit top */
5083
                g_last->top_committed = (char *) base_committed + commit_size;
5084
            }
5085
        } 
5086
        /* Adjust the regions allocation top */
5087
        g_last->top_allocated = (char *) g_last->top_allocated + allocate_size;
5088
        result = (char *) g_last->top_allocated - size;
5089
    /* Deallocation requested? */
5090
    } else if (size < 0) {
5091
        long deallocate_size = - size;
5092
        /* As long as we have a region to release */
5093
        while ((char *) g_last->top_allocated - deallocate_size < (char *) g_last->top_reserved - g_last->reserve_size) {
5094
            /* Get the size to release */
5095
            long release_size = g_last->reserve_size;
5096
            /* Get the base address */
5097
            void *base_reserved = (char *) g_last->top_reserved - release_size;
5098
            /* Assert preconditions */
5099
            assert ((unsigned) base_reserved % g_regionsize == 0); 
5100
            assert (0 < release_size && release_size % g_regionsize == 0); {
5101
                /* Release this */
5102
                int rc = VirtualFree (base_reserved, 0, 
5103
                                      MEM_RELEASE);
5104
                /* Check returned code for consistency */
5105
                if (! rc)
5106
                    goto sbrk_exit;
5107
#ifdef TRACESB
5108
                printf ("Release %p %ld\n", base_reserved, release_size);
5109
#endif
5110
            }
5111
            /* Adjust deallocation size */
5112
            deallocate_size -= (char *) g_last->top_allocated - (char *) base_reserved;
5113
            /* Remove the old region from the list */
5114
            if (! region_list_remove (&g_last))
5115
                goto sbrk_exit;
5116
        } {
5117
            /* Compute the size to decommit */
5118
            long to_decommit = (char *) g_last->top_committed - ((char *) g_last->top_allocated - deallocate_size);
5119
            if (to_decommit >= g_my_pagesize) {
5120
                /* Compute the size to decommit */
5121
                long decommit_size = FLOOR (to_decommit, g_my_pagesize);
5122
                /*  Compute the base address */
5123
                void *base_committed = (char *) g_last->top_committed - decommit_size;
5124
                /* Assert preconditions */
5125
                assert ((unsigned) base_committed % g_pagesize == 0);
5126
                assert (0 < decommit_size && decommit_size % g_pagesize == 0); {
5127
                    /* Decommit this */
5128
                    int rc = VirtualFree ((char *) base_committed, decommit_size, 
5129
                                          MEM_DECOMMIT);
5130
                    /* Check returned code for consistency */
5131
                    if (! rc)
5132
                        goto sbrk_exit;
5133
#ifdef TRACESB
5134
                    printf ("Decommit %p %ld\n", base_committed, decommit_size);
5135
#endif
5136
                }
5137
                /* Adjust deallocation size and regions commit and allocate top */
5138
                deallocate_size -= (char *) g_last->top_allocated - (char *) base_committed;
5139
                g_last->top_committed = base_committed;
5140
                g_last->top_allocated = base_committed;
5141
            }
5142
        }
5143
        /* Adjust regions allocate top */
5144
        g_last->top_allocated = (char *) g_last->top_allocated - deallocate_size;
5145
        /* Check for underflow */
5146
        if ((char *) g_last->top_reserved - g_last->reserve_size > (char *) g_last->top_allocated ||
5147
            g_last->top_allocated > g_last->top_committed) {
5148
            /* Adjust regions allocate top */
5149
            g_last->top_allocated = (char *) g_last->top_reserved - g_last->reserve_size;
5150
            goto sbrk_exit;
5151
        }
5152
        result = g_last->top_allocated;
5153
    }
5154
    /* Assert invariants */
5155
    assert (g_last);
5156
    assert ((char *) g_last->top_reserved - g_last->reserve_size <= (char *) g_last->top_allocated &&
5157
            g_last->top_allocated <= g_last->top_committed);
5158
    assert ((char *) g_last->top_reserved - g_last->reserve_size <= (char *) g_last->top_committed &&
5159
            g_last->top_committed <= g_last->top_reserved &&
5160
            (unsigned) g_last->top_committed % g_pagesize == 0);
5161
    assert ((unsigned) g_last->top_reserved % g_regionsize == 0);
5162
    assert ((unsigned) g_last->reserve_size % g_regionsize == 0);
10347 ripley 5163
 
26855 ripley 5164
sbrk_exit:
5165
#if defined (USE_MALLOC_LOCK) && defined (NEEDED)
5166
    /* Release spin lock */
5167
    slrelease (&g_sl);
10347 ripley 5168
#endif
26855 ripley 5169
    return result;
5170
}
5171
 
5172
/* mmap for windows */
5173
static void *mmap (void *ptr, long size, long prot, long type, long handle, long arg) {
5174
    static long g_pagesize;
5175
    static long g_regionsize;
5176
#ifdef TRACESB
5177
    printf ("mmap %ld\n", size);
10347 ripley 5178
#endif
26855 ripley 5179
#if defined (USE_MALLOC_LOCK) && defined (NEEDED)
5180
    /* Wait for spin lock */
5181
    slwait (&g_sl);
5182
#endif
5183
    /* First time initialization */
5184
    if (! g_pagesize) 
5185
        g_pagesize = getpagesize ();
5186
    if (! g_regionsize) 
5187
        g_regionsize = getregionsize ();
5188
    /* Assert preconditions */
5189
    assert ((unsigned) ptr % g_regionsize == 0);
5190
    assert (size % g_pagesize == 0);
5191
    /* Allocate this */
5192
    ptr = VirtualAlloc (ptr, size,
26895 ripley 5193
			MEM_RESERVE | MEM_COMMIT | MEM_TOP_DOWN, PAGE_READWRITE);
26855 ripley 5194
    if (! ptr) {
5195
        ptr = (void *) MORECORE_FAILURE;
5196
        goto mmap_exit;
5197
    }
5198
    /* Assert postconditions */
5199
    assert ((unsigned) ptr % g_regionsize == 0);
5200
#ifdef TRACESB
5201
    printf ("Commit %p %ld\n", ptr, size);
5202
#endif
5203
mmap_exit:
5204
#if defined (USE_MALLOC_LOCK) && defined (NEEDED)
5205
    /* Release spin lock */
5206
    slrelease (&g_sl);
5207
#endif
5208
    return ptr;
5209
}
10347 ripley 5210
 
26855 ripley 5211
/* munmap for windows */
5212
static long munmap (void *ptr, long size) {
5213
    static long g_pagesize;
5214
    static long g_regionsize;
5215
    int rc = MUNMAP_FAILURE;
5216
#ifdef TRACESB
5217
    printf ("munmap %p %ld\n", ptr, size);
5218
#endif
5219
#if defined (USE_MALLOC_LOCK) && defined (NEEDED)
5220
    /* Wait for spin lock */
5221
    slwait (&g_sl);
5222
#endif
5223
    /* First time initialization */
5224
    if (! g_pagesize) 
5225
        g_pagesize = getpagesize ();
5226
    if (! g_regionsize) 
5227
        g_regionsize = getregionsize ();
5228
    /* Assert preconditions */
5229
    assert ((unsigned) ptr % g_regionsize == 0);
5230
    assert (size % g_pagesize == 0);
5231
    /* Free this */
5232
    if (! VirtualFree (ptr, 0, 
5233
                       MEM_RELEASE))
5234
        goto munmap_exit;
5235
    rc = 0;
5236
#ifdef TRACESB
5237
    printf ("Release %p %ld\n", ptr, size);
5238
#endif
5239
munmap_exit:
5240
#if defined (USE_MALLOC_LOCK) && defined (NEEDED)
5241
    /* Release spin lock */
5242
    slrelease (&g_sl);
5243
#endif
5244
    return rc;
10347 ripley 5245
}
5246
 
26855 ripley 5247
static void vminfo (CHUNK_SIZE_T  *free, CHUNK_SIZE_T  *reserved, CHUNK_SIZE_T  *committed) {
5248
    MEMORY_BASIC_INFORMATION memory_info;
5249
    memory_info.BaseAddress = 0;
5250
    *free = *reserved = *committed = 0;
5251
    while (VirtualQuery (memory_info.BaseAddress, &memory_info, sizeof (memory_info))) {
5252
        switch (memory_info.State) {
5253
        case MEM_FREE:
5254
            *free += memory_info.RegionSize;
5255
            break;
5256
        case MEM_RESERVE:
5257
            *reserved += memory_info.RegionSize;
5258
            break;
5259
        case MEM_COMMIT:
5260
            *committed += memory_info.RegionSize;
5261
            break;
5262
        }
5263
        memory_info.BaseAddress = (char *) memory_info.BaseAddress + memory_info.RegionSize;
5264
    }
5265
}
5266
/* Commented out for R Build Compatibility
5267
static int cpuinfo (int whole, CHUNK_SIZE_T  *kernel, CHUNK_SIZE_T  *user) {
5268
    if (whole) {
5269
        __int64 creation64, exit64, kernel64, user64;
5270
        int rc = GetProcessTimes (GetCurrentProcess (), 
5271
                                  (FILETIME *) &creation64,  
5272
                                  (FILETIME *) &exit64, 
5273
                                  (FILETIME *) &kernel64, 
5274
                                  (FILETIME *) &user64);
5275
        if (! rc) {
5276
            *kernel = 0;
5277
            *user = 0;
5278
            return FALSE;
5279
        } 
5280
        *kernel = (CHUNK_SIZE_T) (kernel64 / 10000);
5281
        *user = (CHUNK_SIZE_T) (user64 / 10000);
5282
        return TRUE;
5283
    } else {
5284
        __int64 creation64, exit64, kernel64, user64;
5285
        int rc = GetThreadTimes (GetCurrentThread (), 
5286
                                 (FILETIME *) &creation64,  
5287
                                 (FILETIME *) &exit64, 
5288
                                 (FILETIME *) &kernel64, 
5289
                                 (FILETIME *) &user64);
5290
        if (! rc) {
5291
            *kernel = 0;
5292
            *user = 0;
5293
            return FALSE;
5294
        } 
5295
        *kernel = (CHUNK_SIZE_T) (kernel64 / 10000);
5296
        *user = (CHUNK_SIZE_T) (user64 / 10000);
5297
        return TRUE;
5298
    }
5299
}
5300
*/
5301
#endif /* WIN32 */
10347 ripley 5302
 
26855 ripley 5303
/* ------------------------------------------------------------
10347 ripley 5304
History:
26855 ripley 5305
    V2.7.2 Sat Aug 17 09:07:30 2002  Doug Lea  (dl at gee)
5306
      * Fix malloc_state bitmap array misdeclaration
10347 ripley 5307
 
26855 ripley 5308
    V2.7.1 Thu Jul 25 10:58:03 2002  Doug Lea  (dl at gee)
5309
      * Allow tuning of FIRST_SORTED_BIN_SIZE
5310
      * Use PTR_UINT as type for all ptr->int casts. Thanks to John Belmonte.
5311
      * Better detection and support for non-contiguousness of MORECORE. 
5312
        Thanks to Andreas Mueller, Conal Walsh, and Wolfram Gloger
5313
      * Bypass most of malloc if no frees. Thanks To Emery Berger.
5314
      * Fix freeing of old top non-contiguous chunk im sysmalloc.
5315
      * Raised default trim and map thresholds to 256K.
5316
      * Fix mmap-related #defines. Thanks to Lubos Lunak.
5317
      * Fix copy macros; added LACKS_FCNTL_H. Thanks to Neal Walfield.
5318
      * Branch-free bin calculation
5319
      * Default trim and mmap thresholds now 256K.
5320
 
5321
    V2.7.0 Sun Mar 11 14:14:06 2001  Doug Lea  (dl at gee)
5322
      * Introduce independent_comalloc and independent_calloc.
5323
        Thanks to Michael Pachos for motivation and help.
5324
      * Make optional .h file available
5325
      * Allow > 2GB requests on 32bit systems.
5326
      * new WIN32 sbrk, mmap, munmap, lock code from <Walter@GeNeSys-e.de>.
5327
        Thanks also to Andreas Mueller <a.mueller at paradatec.de>,
5328
        and Anonymous.
5329
      * Allow override of MALLOC_ALIGNMENT (Thanks to Ruud Waij for 
5330
        helping test this.)
5331
      * memalign: check alignment arg
5332
      * realloc: don't try to shift chunks backwards, since this
5333
        leads to  more fragmentation in some programs and doesn't
5334
        seem to help in any others.
5335
      * Collect all cases in malloc requiring system memory into sYSMALLOc
5336
      * Use mmap as backup to sbrk
5337
      * Place all internal state in malloc_state
5338
      * Introduce fastbins (although similar to 2.5.1)
5339
      * Many minor tunings and cosmetic improvements
5340
      * Introduce USE_PUBLIC_MALLOC_WRAPPERS, USE_MALLOC_LOCK 
5341
      * Introduce MALLOC_FAILURE_ACTION, MORECORE_CONTIGUOUS
5342
        Thanks to Tony E. Bennett <tbennett@nvidia.com> and others.
5343
      * Include errno.h to support default failure action.
5344
 
10425 ripley 5345
    V2.6.6 Sun Dec  5 07:42:19 1999  Doug Lea  (dl at gee)
5346
      * return null for negative arguments
26855 ripley 5347
      * Added Several WIN32 cleanups from Martin C. Fong <mcfong at yahoo.com>
10425 ripley 5348
         * Add 'LACKS_SYS_PARAM_H' for those systems without 'sys/param.h'
5349
          (e.g. WIN32 platforms)
26855 ripley 5350
         * Cleanup header file inclusion for WIN32 platforms
10425 ripley 5351
         * Cleanup code to avoid Microsoft Visual C++ compiler complaints
5352
         * Add 'USE_DL_PREFIX' to quickly allow co-existence with existing
5353
           memory allocation routines
5354
         * Set 'malloc_getpagesize' for WIN32 platforms (needs more work)
5355
         * Use 'assert' rather than 'ASSERT' in WIN32 code to conform to
26855 ripley 5356
           usage of 'assert' in non-WIN32 code
10425 ripley 5357
         * Improve WIN32 'sbrk()' emulation's 'findRegion()' routine to
5358
           avoid infinite loop
5359
      * Always call 'fREe()' rather than 'free()'
5360
 
10347 ripley 5361
    V2.6.5 Wed Jun 17 15:57:31 1998  Doug Lea  (dl at gee)
5362
      * Fixed ordering problem with boundary-stamping
5363
 
5364
    V2.6.3 Sun May 19 08:17:58 1996  Doug Lea  (dl at gee)
5365
      * Added pvalloc, as recommended by H.J. Liu
5366
      * Added 64bit pointer support mainly from Wolfram Gloger
5367
      * Added anonymously donated WIN32 sbrk emulation
5368
      * Malloc, calloc, getpagesize: add optimizations from Raymond Nijssen
5369
      * malloc_extend_top: fix mask error that caused wastage after
5370
        foreign sbrks
5371
      * Add linux mremap support code from HJ Liu
5372
 
5373
    V2.6.2 Tue Dec  5 06:52:55 1995  Doug Lea  (dl at gee)
5374
      * Integrated most documentation with the code.
5375
      * Add support for mmap, with help from
5376
        Wolfram Gloger (Gloger@lrz.uni-muenchen.de).
5377
      * Use last_remainder in more cases.
5378
      * Pack bins using idea from  colin@nyx10.cs.du.edu
5379
      * Use ordered bins instead of best-fit threshhold
5380
      * Eliminate block-local decls to simplify tracing and debugging.
5381
      * Support another case of realloc via move into top
5382
      * Fix error occuring when initial sbrk_base not word-aligned.
5383
      * Rely on page size for units instead of SBRK_UNIT to
5384
        avoid surprises about sbrk alignment conventions.
5385
      * Add mallinfo, mallopt. Thanks to Raymond Nijssen
5386
        (raymond@es.ele.tue.nl) for the suggestion.
5387
      * Add `pad' argument to malloc_trim and top_pad mallopt parameter.
5388
      * More precautions for cases where other routines call sbrk,
5389
        courtesy of Wolfram Gloger (Gloger@lrz.uni-muenchen.de).
5390
      * Added macros etc., allowing use in linux libc from
5391
        H.J. Lu (hjl@gnu.ai.mit.edu)
5392
      * Inverted this history list
5393
 
5394
    V2.6.1 Sat Dec  2 14:10:57 1995  Doug Lea  (dl at gee)
5395
      * Re-tuned and fixed to behave more nicely with V2.6.0 changes.
5396
      * Removed all preallocation code since under current scheme
5397
        the work required to undo bad preallocations exceeds
5398
        the work saved in good cases for most test programs.
5399
      * No longer use return list or unconsolidated bins since
5400
        no scheme using them consistently outperforms those that don't
5401
        given above changes.
5402
      * Use best fit for very large chunks to prevent some worst-cases.
5403
      * Added some support for debugging
5404
 
5405
    V2.6.0 Sat Nov  4 07:05:23 1995  Doug Lea  (dl at gee)
5406
      * Removed footers when chunks are in use. Thanks to
5407
        Paul Wilson (wilson@cs.texas.edu) for the suggestion.
5408
 
5409
    V2.5.4 Wed Nov  1 07:54:51 1995  Doug Lea  (dl at gee)
5410
      * Added malloc_trim, with help from Wolfram Gloger
5411
        (wmglo@Dent.MED.Uni-Muenchen.DE).
5412
 
5413
    V2.5.3 Tue Apr 26 10:16:01 1994  Doug Lea  (dl at g)
5414
 
5415
    V2.5.2 Tue Apr  5 16:20:40 1994  Doug Lea  (dl at g)
5416
      * realloc: try to expand in both directions
5417
      * malloc: swap order of clean-bin strategy;
5418
      * realloc: only conditionally expand backwards
5419
      * Try not to scavenge used bins
5420
      * Use bin counts as a guide to preallocation
5421
      * Occasionally bin return list chunks in first scan
5422
      * Add a few optimizations from colin@nyx10.cs.du.edu
5423
 
5424
    V2.5.1 Sat Aug 14 15:40:43 1993  Doug Lea  (dl at g)
5425
      * faster bin computation & slightly different binning
5426
      * merged all consolidations to one part of malloc proper
5427
         (eliminating old malloc_find_space & malloc_clean_bin)
5428
      * Scan 2 returns chunks (not just 1)
5429
      * Propagate failure in realloc if malloc returns 0
5430
      * Add stuff to allow compilation on non-ANSI compilers
5431
          from kpv@research.att.com
5432
 
5433
    V2.5 Sat Aug  7 07:41:59 1993  Doug Lea  (dl at g.oswego.edu)
5434
      * removed potential for odd address access in prev_chunk
5435
      * removed dependency on getpagesize.h
5436
      * misc cosmetics and a bit more internal documentation
5437
      * anticosmetics: mangled names in macros to evade debugger strangeness
5438
      * tested on sparc, hp-700, dec-mips, rs6000
5439
          with gcc & native cc (hp, dec only) allowing
5440
          Detlefs & Zorn comparison study (in SIGPLAN Notices.)
5441
 
5442
    Trial version Fri Aug 28 13:14:29 1992  Doug Lea  (dl at g.oswego.edu)
5443
      * Based loosely on libg++-1.2X malloc. (It retains some of the overall
5444
         structure of old version,  but most details differ.)
5445
 
5446
*/
5447
 
26855 ripley 5448
/* Need strdup here so we don't get calls to the wrong heap function inside the library */
26879 ripley 5449
char * Rm_strdup (const char *str)
13587 ripley 5450
{
5451
  char *newstr;
5452
 
26879 ripley 5453
  newstr = (char *) Rm_malloc (strlen (str) + 1);
13587 ripley 5454
  if (newstr)
5455
    strcpy (newstr, str);
5456
  return newstr;
5457
}