The R Project SVN R

Rev

Details | Last modification | View Log | RSS feed

Rev Author Line No. Line
40788 ripley 1
/* R changes:
2
 
3
 - use Rm_malloc, Rm_calloc, Rm_realloc, Rm_free to ensure this is only
4
   used from memory.c.
5
 
6
 - make attempt for maximum footprint to exceed limit a failure.
7
 
8
 - MinGW does have unistd and does not need VC++ pragma.
9
*/
10
 
10347 ripley 11
/*
26855 ripley 12
  This is a version (aka dlmalloc) of malloc/free/realloc written by
40788 ripley 13
  Doug Lea and released to the public domain, as explained at
14
  http://creativecommons.org/licenses/publicdomain.  Send questions,
15
  comments, complaints, performance data, etc to dl@cs.oswego.edu
10347 ripley 16
 
40788 ripley 17
* Version 2.8.3 Thu Sep 22 11:16:15 2005  Doug Lea  (dl at gee)
10347 ripley 18
 
19
   Note: There may be an updated version of this malloc obtainable at
26855 ripley 20
           ftp://gee.cs.oswego.edu/pub/misc/malloc.c
10347 ripley 21
         Check before installing!
22
 
26855 ripley 23
* Quickstart
24
 
25
  This library is all in one file to simplify the most common usage:
40788 ripley 26
  ftp it, compile it (-O3), and link it into another program. All of
27
  the compile-time options default to reasonable values for use on
28
  most platforms.  You might later want to step through various
29
  compile-time and dynamic tuning options.
26855 ripley 30
 
31
  For convenience, an include file for code using this malloc is at:
40788 ripley 32
     ftp://gee.cs.oswego.edu/pub/misc/malloc-2.8.3.h
26855 ripley 33
  You don't really need this .h file unless you call functions not
34
  defined in your system include files.  The .h file contains only the
35
  excerpts from this file needed for using this malloc on ANSI C/C++
36
  systems, so long as you haven't changed compile-time options about
37
  naming and tuning parameters.  If you do, then you can create your
38
  own malloc.h that does include all settings by cutting at the point
40788 ripley 39
  indicated below. Note that you may already by default be using a C
40
  library containing a malloc that is based on some version of this
41
  malloc (for example in linux). You might still want to use the one
42
  in this file to customize settings or to avoid overheads associated
43
  with library versions.
26855 ripley 44
 
45
* Vital statistics:
46
 
40788 ripley 47
  Supported pointer/size_t representation:       4 or 8 bytes
48
       size_t MUST be an unsigned type of the same width as
49
       pointers. (If you are using an ancient system that declares
50
       size_t as a signed type, or need it to be a different width
51
       than pointers, you can use a previous release of this malloc
52
       (e.g. 2.7.2) supporting these.)
10347 ripley 53
 
40788 ripley 54
  Alignment:                                     8 bytes (default)
55
       This suffices for nearly all current machines and C compilers.
56
       However, you can define MALLOC_ALIGNMENT to be wider than this
57
       if necessary (up to 128bytes), at the expense of using more space.
26855 ripley 58
 
40788 ripley 59
  Minimum overhead per allocated chunk:   4 or  8 bytes (if 4byte sizes)
60
                                          8 or 16 bytes (if 8byte sizes)
26855 ripley 61
       Each malloced chunk has a hidden word of overhead holding size
40788 ripley 62
       and status information, and additional cross-check word
63
       if FOOTERS is defined.
10347 ripley 64
 
40788 ripley 65
  Minimum allocated size: 4-byte ptrs:  16 bytes    (including overhead)
66
                          8-byte ptrs:  32 bytes    (including overhead)
10347 ripley 67
 
68
       Even a request for zero bytes (i.e., malloc(0)) returns a
69
       pointer to something of the minimum allocatable size.
26855 ripley 70
       The maximum overhead wastage (i.e., number of extra bytes
71
       allocated than were requested in malloc) is less than or equal
72
       to the minimum size, except for requests >= mmap_threshold that
40788 ripley 73
       are serviced via mmap(), where the worst case wastage is about
74
       32 bytes plus the remainder from a system page (the minimal
75
       mmap unit); typically 4096 or 8192 bytes.
10347 ripley 76
 
40788 ripley 77
  Security: static-safe; optionally more or less
78
       The "security" of malloc refers to the ability of malicious
79
       code to accentuate the effects of errors (for example, freeing
80
       space that is not currently malloc'ed or overwriting past the
81
       ends of chunks) in code that calls malloc.  This malloc
82
       guarantees not to modify any memory locations below the base of
83
       heap, i.e., static variables, even in the presence of usage
84
       errors.  The routines additionally detect most improper frees
85
       and reallocs.  All this holds as long as the static bookkeeping
86
       for malloc itself is not corrupted by some other means.  This
87
       is only one aspect of security -- these checks do not, and
88
       cannot, detect all possible programming errors.
26855 ripley 89
 
40788 ripley 90
       If FOOTERS is defined nonzero, then each allocated chunk
91
       carries an additional check word to verify that it was malloced
92
       from its space.  These check words are the same within each
93
       execution of a program using malloc, but differ across
94
       executions, so externally crafted fake chunks cannot be
95
       freed. This improves security by rejecting frees/reallocs that
96
       could corrupt heap memory, in addition to the checks preventing
97
       writes to statics that are always on.  This may further improve
98
       security at the expense of time and space overhead.  (Note that
99
       FOOTERS may also be worth using with MSPACES.)
10347 ripley 100
 
40788 ripley 101
       By default detected errors cause the program to abort (calling
102
       "abort()"). You can override this to instead proceed past
103
       errors by defining PROCEED_ON_ERROR.  In this case, a bad free
104
       has no effect, and a malloc that encounters a bad address
105
       caused by user overwrites will ignore the bad address by
106
       dropping pointers and indices to all known memory. This may
107
       be appropriate for programs that should continue if at all
108
       possible in the face of programming errors, although they may
109
       run out of memory because dropped memory is never reclaimed.
10347 ripley 110
 
40788 ripley 111
       If you don't like either of these options, you can define
112
       CORRUPTION_ERROR_ACTION and USAGE_ERROR_ACTION to do anything
113
       else. And if if you are sure that your program using malloc has
114
       no errors or vulnerabilities, you can define INSECURE to 1,
115
       which might (or might not) provide a small performance improvement.
10347 ripley 116
 
40788 ripley 117
  Thread-safety: NOT thread-safe unless USE_LOCKS defined
118
       When USE_LOCKS is defined, each public call to malloc, free,
119
       etc is surrounded with either a pthread mutex or a win32
120
       spinlock (depending on WIN32). This is not especially fast, and
121
       can be a major bottleneck.  It is designed only to provide
122
       minimal protection in concurrent environments, and to provide a
123
       basis for extensions.  If you are using malloc in a concurrent
124
       program, consider instead using ptmalloc, which is derived from
125
       a version of this malloc. (See http://www.malloc.de).
10347 ripley 126
 
40788 ripley 127
  System requirements: Any combination of MORECORE and/or MMAP/MUNMAP
128
       This malloc can use unix sbrk or any emulation (invoked using
129
       the CALL_MORECORE macro) and/or mmap/munmap or any emulation
130
       (invoked using CALL_MMAP/CALL_MUNMAP) to get and release system
131
       memory.  On most unix systems, it tends to work best if both
132
       MORECORE and MMAP are enabled.  On Win32, it uses emulations
133
       based on VirtualAlloc. It also uses common C library functions
134
       like memset.
135
 
136
  Compliance: I believe it is compliant with the Single Unix Specification
137
       (See http://www.unix.org). Also SVID/XPG, ANSI C, and probably
26855 ripley 138
       others as well.
10347 ripley 139
 
40788 ripley 140
* Overview of algorithms
10347 ripley 141
 
40788 ripley 142
  This is not the fastest, most space-conserving, most portable, or
143
  most tunable malloc ever written. However it is among the fastest
144
  while also being among the most space-conserving, portable and
145
  tunable.  Consistent balance across these factors results in a good
146
  general-purpose allocator for malloc-intensive programs.
10347 ripley 147
 
40788 ripley 148
  In most ways, this malloc is a best-fit allocator. Generally, it
149
  chooses the best-fitting existing chunk for a request, with ties
150
  broken in approximately least-recently-used order. (This strategy
151
  normally maintains low fragmentation.) However, for requests less
152
  than 256bytes, it deviates from best-fit when there is not an
153
  exactly fitting available chunk by preferring to use space adjacent
154
  to that used for the previous small request, as well as by breaking
155
  ties in approximately most-recently-used order. (These enhance
156
  locality of series of small allocations.)  And for very large requests
157
  (>= 256Kb by default), it relies on system memory mapping
158
  facilities, if supported.  (This helps avoid carrying around and
159
  possibly fragmenting memory used only for large chunks.)
10347 ripley 160
 
40788 ripley 161
  All operations (except malloc_stats and mallinfo) have execution
162
  times that are bounded by a constant factor of the number of bits in
163
  a size_t, not counting any clearing in calloc or copying in realloc,
164
  or actions surrounding MORECORE and MMAP that have times
165
  proportional to the number of non-contiguous regions returned by
166
  system allocation routines, which is often just 1.
10347 ripley 167
 
40788 ripley 168
  The implementation is not very modular and seriously overuses
169
  macros. Perhaps someday all C compilers will do as good a job
170
  inlining modular code as can now be done by brute-force expansion,
171
  but now, enough of them seem not to.
10347 ripley 172
 
40788 ripley 173
  Some compilers issue a lot of warnings about code that is
174
  dead/unreachable only on some platforms, and also about intentional
175
  uses of negation on unsigned types. All known cases of each can be
176
  ignored.
26855 ripley 177
 
40788 ripley 178
  For a longer but out of date high-level description, see
179
     http://gee.cs.oswego.edu/dl/html/malloc.html
26855 ripley 180
 
40788 ripley 181
* MSPACES
182
  If MSPACES is defined, then in addition to malloc, free, etc.,
183
  this file also defines mspace_malloc, mspace_free, etc. These
184
  are versions of malloc routines that take an "mspace" argument
185
  obtained using create_mspace, to control all internal bookkeeping.
186
  If ONLY_MSPACES is defined, only these versions are compiled.
187
  So if you would like to use this allocator for only some allocations,
188
  and your system malloc for others, you can compile with
189
  ONLY_MSPACES and then do something like...
190
    static mspace mymspace = create_mspace(0,0); // for example
191
    #define mymalloc(bytes)  mspace_malloc(mymspace, bytes)
26855 ripley 192
 
40788 ripley 193
  (Note: If you only need one instance of an mspace, you can instead
194
  use "USE_DL_PREFIX" to relabel the global malloc.)
26855 ripley 195
 
40788 ripley 196
  You can similarly create thread-local allocators by storing
197
  mspaces as thread-locals. For example:
198
    static __thread mspace tlms = 0;
199
    void*  tlmalloc(size_t bytes) {
200
      if (tlms == 0) tlms = create_mspace(0, 0);
201
      return mspace_malloc(tlms, bytes);
202
    }
203
    void  tlfree(void* mem) { mspace_free(tlms, mem); }
26855 ripley 204
 
40788 ripley 205
  Unless FOOTERS is defined, each mspace is completely independent.
206
  You cannot allocate from one and free to another (although
207
  conformance is only weakly checked, so usage errors are not always
208
  caught). If FOOTERS is defined, then each chunk carries around a tag
209
  indicating its originating mspace, and frees are directed to their
210
  originating spaces.
26855 ripley 211
 
40788 ripley 212
 -------------------------  Compile-time options ---------------------------
26855 ripley 213
 
40788 ripley 214
Be careful in setting #define values for numerical constants of type
215
size_t. On some systems, literal values are not automatically extended
216
to size_t precision unless they are explicitly casted.
26855 ripley 217
 
40788 ripley 218
WIN32                    default: defined if _WIN32 defined
219
  Defining WIN32 sets up defaults for MS environment and compilers.
26855 ripley 220
  Otherwise defaults are for unix.
10347 ripley 221
 
40788 ripley 222
MALLOC_ALIGNMENT         default: (size_t)8
223
  Controls the minimum alignment for malloc'ed chunks.  It must be a
224
  power of two and at least 8, even on machines for which smaller
225
  alignments would suffice. It may be defined as larger than this
226
  though. Note however that code and data structures are optimized for
227
  the case of 8-byte alignment.
10347 ripley 228
 
40788 ripley 229
MSPACES                  default: 0 (false)
230
  If true, compile in support for independent allocation spaces.
231
  This is only supported if HAVE_MMAP is true.
10347 ripley 232
 
40788 ripley 233
ONLY_MSPACES             default: 0 (false)
234
  If true, only compile in mspace versions, not regular versions.
26855 ripley 235
 
40788 ripley 236
USE_LOCKS                default: 0 (false)
237
  Causes each call to each public routine to be surrounded with
238
  pthread or WIN32 mutex lock/unlock. (If set true, this can be
239
  overridden on a per-mspace basis for mspace versions.)
26855 ripley 240
 
40788 ripley 241
FOOTERS                  default: 0
242
  If true, provide extra checking and dispatching by placing
243
  information in the footers of allocated chunks. This adds
244
  space and time overhead.
26855 ripley 245
 
40788 ripley 246
INSECURE                 default: 0
247
  If true, omit checks for usage errors and heap space overwrites.
26855 ripley 248
 
40788 ripley 249
USE_DL_PREFIX            default: NOT defined
250
  Causes compiler to prefix all public routines with the string 'dl'.
251
  This can be useful when you only want to use this malloc in one part
252
  of a program, using your regular system malloc elsewhere.
26855 ripley 253
 
40788 ripley 254
ABORT                    default: defined as abort()
255
  Defines how to abort on failed checks.  On most systems, a failed
256
  check cannot die with an "assert" or even print an informative
257
  message, because the underlying print routines in turn call malloc,
258
  which will fail again.  Generally, the best policy is to simply call
259
  abort(). It's not very useful to do more than this because many
260
  errors due to overwriting will show up as address faults (null, odd
261
  addresses etc) rather than malloc-triggered checks, so will also
262
  abort.  Also, most compilers know that abort() does not return, so
263
  can better optimize code conditionally calling it.
26855 ripley 264
 
40788 ripley 265
PROCEED_ON_ERROR           default: defined as 0 (false)
266
  Controls whether detected bad addresses cause them to bypassed
267
  rather than aborting. If set, detected bad arguments to free and
268
  realloc are ignored. And all bookkeeping information is zeroed out
269
  upon a detected overwrite of freed heap space, thus losing the
270
  ability to ever return it from malloc again, but enabling the
271
  application to proceed. If PROCEED_ON_ERROR is defined, the
272
  static variable malloc_corruption_error_count is compiled in
273
  and can be examined to see if errors have occurred. This option
274
  generates slower code than the default abort policy.
26855 ripley 275
 
40788 ripley 276
DEBUG                    default: NOT defined
277
  The DEBUG setting is mainly intended for people trying to modify
278
  this code or diagnose problems when porting to new platforms.
279
  However, it may also be able to better isolate user errors than just
280
  using runtime checks.  The assertions in the check routines spell
281
  out in more detail the assumptions and invariants underlying the
282
  algorithms.  The checking is fairly extensive, and will slow down
283
  execution noticeably. Calling malloc_stats or mallinfo with DEBUG
284
  set will attempt to check every non-mmapped allocated and free chunk
285
  in the course of computing the summaries.
26855 ripley 286
 
40788 ripley 287
ABORT_ON_ASSERT_FAILURE   default: defined as 1 (true)
288
  Debugging assertion failures can be nearly impossible if your
289
  version of the assert macro causes malloc to be called, which will
290
  lead to a cascade of further failures, blowing the runtime stack.
291
  ABORT_ON_ASSERT_FAILURE cause assertions failures to call abort(),
292
  which will usually make debugging easier.
26855 ripley 293
 
40788 ripley 294
MALLOC_FAILURE_ACTION     default: sets errno to ENOMEM, or no-op on win32
295
  The action to take before "return 0" when malloc fails to be able to
296
  return memory because there is none available.
26855 ripley 297
 
40788 ripley 298
HAVE_MORECORE             default: 1 (true) unless win32 or ONLY_MSPACES
299
  True if this system supports sbrk or an emulation of it.
26855 ripley 300
 
40788 ripley 301
MORECORE                  default: sbrk
302
  The name of the sbrk-style system routine to call to obtain more
303
  memory.  See below for guidance on writing custom MORECORE
304
  functions. The type of the argument to sbrk/MORECORE varies across
305
  systems.  It cannot be size_t, because it supports negative
306
  arguments, so it is normally the signed type of the same width as
307
  size_t (sometimes declared as "intptr_t").  It doesn't much matter
308
  though. Internally, we only call it with arguments less than half
309
  the max value of a size_t, which should work across all reasonable
310
  possibilities, although sometimes generating compiler warnings.  See
311
  near the end of this file for guidelines for creating a custom
312
  version of MORECORE.
10347 ripley 313
 
40788 ripley 314
MORECORE_CONTIGUOUS       default: 1 (true)
315
  If true, take advantage of fact that consecutive calls to MORECORE
316
  with positive arguments always return contiguous increasing
317
  addresses.  This is true of unix sbrk. It does not hurt too much to
318
  set it true anyway, since malloc copes with non-contiguities.
319
  Setting it false when definitely non-contiguous saves time
320
  and possibly wasted space it would take to discover this though.
26855 ripley 321
 
40788 ripley 322
MORECORE_CANNOT_TRIM      default: NOT defined
323
  True if MORECORE cannot release space back to the system when given
324
  negative arguments. This is generally necessary only if you are
325
  using a hand-crafted MORECORE function that cannot handle negative
326
  arguments.
10347 ripley 327
 
40788 ripley 328
HAVE_MMAP                 default: 1 (true)
329
  True if this system supports mmap or an emulation of it.  If so, and
330
  HAVE_MORECORE is not true, MMAP is used for all system
331
  allocation. If set and HAVE_MORECORE is true as well, MMAP is
332
  primarily used to directly allocate very large blocks. It is also
333
  used as a backup strategy in cases where MORECORE fails to provide
334
  space from system. Note: A single call to MUNMAP is assumed to be
335
  able to unmap memory that may have be allocated using multiple calls
336
  to MMAP, so long as they are adjacent.
10347 ripley 337
 
40788 ripley 338
HAVE_MREMAP               default: 1 on linux, else 0
339
  If true realloc() uses mremap() to re-allocate large blocks and
340
  extend or shrink allocation spaces.
10347 ripley 341
 
40788 ripley 342
MMAP_CLEARS               default: 1 on unix
343
  True if mmap clears memory so calloc doesn't need to. This is true
344
  for standard unix mmap using /dev/zero.
10347 ripley 345
 
40788 ripley 346
USE_BUILTIN_FFS            default: 0 (i.e., not used)
347
  Causes malloc to use the builtin ffs() function to compute indices.
348
  Some compilers may recognize and intrinsify ffs to be faster than the
349
  supplied C version. Also, the case of x86 using gcc is special-cased
350
  to an asm instruction, so is already as fast as it can be, and so
351
  this setting has no effect. (On most x86s, the asm version is only
352
  slightly faster than the C version.)
10347 ripley 353
 
40788 ripley 354
malloc_getpagesize         default: derive from system includes, or 4096.
355
  The system page size. To the extent possible, this malloc manages
356
  memory from the system in page-size units.  This may be (and
357
  usually is) a function rather than a constant. This is ignored
358
  if WIN32, where page size is determined using getSystemInfo during
359
  initialization.
10347 ripley 360
 
40788 ripley 361
USE_DEV_RANDOM             default: 0 (i.e., not used)
362
  Causes malloc to use /dev/random to initialize secure magic seed for
363
  stamping footers. Otherwise, the current time is used.
10347 ripley 364
 
40788 ripley 365
NO_MALLINFO                default: 0
366
  If defined, don't compile "mallinfo". This can be a simple way
367
  of dealing with mismatches between system declarations and
368
  those in this file.
26855 ripley 369
 
40788 ripley 370
MALLINFO_FIELD_TYPE        default: size_t
371
  The type of the fields in the mallinfo struct. This was originally
372
  defined as "int" in SVID etc, but is more usefully defined as
373
  size_t. The value is used only if  HAVE_USR_INCLUDE_MALLOC_H is not set
26855 ripley 374
 
40788 ripley 375
REALLOC_ZERO_BYTES_FREES    default: not defined
376
  This should be set if a call to realloc with zero bytes should 
377
  be the same as a call to free. Some people think it should. Otherwise, 
378
  since this malloc returns a unique pointer for malloc(0), so does 
379
  realloc(p, 0).
26855 ripley 380
 
40788 ripley 381
LACKS_UNISTD_H, LACKS_FCNTL_H, LACKS_SYS_PARAM_H, LACKS_SYS_MMAN_H
382
LACKS_STRINGS_H, LACKS_STRING_H, LACKS_SYS_TYPES_H,  LACKS_ERRNO_H
383
LACKS_STDLIB_H                default: NOT defined unless on WIN32
384
  Define these if your system does not have these header files.
385
  You might need to manually insert some of the declarations they provide.
26855 ripley 386
 
40788 ripley 387
DEFAULT_GRANULARITY        default: page size if MORECORE_CONTIGUOUS,
388
                                system_info.dwAllocationGranularity in WIN32,
389
                                otherwise 64K.
390
      Also settable using mallopt(M_GRANULARITY, x)
391
  The unit for allocating and deallocating memory from the system.  On
392
  most systems with contiguous MORECORE, there is no reason to
393
  make this more than a page. However, systems with MMAP tend to
394
  either require or encourage larger granularities.  You can increase
395
  this value to prevent system allocation functions to be called so
396
  often, especially if they are slow.  The value must be at least one
397
  page and must be a power of two.  Setting to 0 causes initialization
398
  to either page size or win32 region size.  (Note: In previous
399
  versions of malloc, the equivalent of this option was called
400
  "TOP_PAD")
10347 ripley 401
 
40788 ripley 402
DEFAULT_TRIM_THRESHOLD    default: 2MB
403
      Also settable using mallopt(M_TRIM_THRESHOLD, x)
404
  The maximum amount of unused top-most memory to keep before
405
  releasing via malloc_trim in free().  Automatic trimming is mainly
406
  useful in long-lived programs using contiguous MORECORE.  Because
407
  trimming via sbrk can be slow on some systems, and can sometimes be
408
  wasteful (in cases where programs immediately afterward allocate
409
  more large chunks) the value should be high enough so that your
410
  overall system performance would improve by releasing this much
411
  memory.  As a rough guide, you might set to a value close to the
412
  average size of a process (program) running on your system.
413
  Releasing this much memory would allow such a process to run in
414
  memory.  Generally, it is worth tuning trim thresholds when a
415
  program undergoes phases where several large chunks are allocated
416
  and released in ways that can reuse each other's storage, perhaps
417
  mixed with phases where there are no such chunks at all. The trim
418
  value must be greater than page size to have any useful effect.  To
419
  disable trimming completely, you can set to MAX_SIZE_T. Note that the trick
420
  some people use of mallocing a huge space and then freeing it at
421
  program startup, in an attempt to reserve system memory, doesn't
422
  have the intended effect under automatic trimming, since that memory
423
  will immediately be returned to the system.
10347 ripley 424
 
40788 ripley 425
DEFAULT_MMAP_THRESHOLD       default: 256K
426
      Also settable using mallopt(M_MMAP_THRESHOLD, x)
427
  The request size threshold for using MMAP to directly service a
428
  request. Requests of at least this size that cannot be allocated
429
  using already-existing space will be serviced via mmap.  (If enough
430
  normal freed space already exists it is used instead.)  Using mmap
431
  segregates relatively large chunks of memory so that they can be
432
  individually obtained and released from the host system. A request
433
  serviced through mmap is never reused by any other request (at least
434
  not directly; the system may just so happen to remap successive
435
  requests to the same locations).  Segregating space in this way has
436
  the benefits that: Mmapped space can always be individually released
437
  back to the system, which helps keep the system level memory demands
438
  of a long-lived program low.  Also, mapped memory doesn't become
439
  `locked' between other chunks, as can happen with normally allocated
440
  chunks, which means that even trimming via malloc_trim would not
441
  release them.  However, it has the disadvantage that the space
442
  cannot be reclaimed, consolidated, and then used to service later
443
  requests, as happens with normal chunks.  The advantages of mmap
444
  nearly always outweigh disadvantages for "large" chunks, but the
445
  value of "large" may vary across systems.  The default is an
446
  empirically derived value that works well in most systems. You can
447
  disable mmap by setting to MAX_SIZE_T.
10347 ripley 448
 
449
*/
450
 
40788 ripley 451
#ifndef WIN32
452
#ifdef _WIN32
453
#define WIN32 1
454
#endif  /* _WIN32 */
455
#endif  /* WIN32 */
456
#ifdef WIN32
457
#define WIN32_LEAN_AND_MEAN
458
#include <windows.h>
459
#define HAVE_MMAP 1
460
#define HAVE_MORECORE 0
461
/* #define LACKS_UNISTD_H */
462
#define LACKS_SYS_PARAM_H
463
#define LACKS_SYS_MMAN_H
464
#define LACKS_STRING_H
465
#define LACKS_STRINGS_H
466
#define LACKS_SYS_TYPES_H
467
#define LACKS_ERRNO_H
468
#define MALLOC_FAILURE_ACTION
469
#define MMAP_CLEARS 0 /* WINCE and some others apparently don't clear */
470
#endif  /* WIN32 */
26855 ripley 471
 
40788 ripley 472
#if defined(DARWIN) || defined(_DARWIN)
473
/* Mac OSX docs advise not to use sbrk; it seems better to use mmap */
474
#ifndef HAVE_MORECORE
475
#define HAVE_MORECORE 0
476
#define HAVE_MMAP 1
477
#endif  /* HAVE_MORECORE */
478
#endif  /* DARWIN */
10347 ripley 479
 
40788 ripley 480
#ifndef LACKS_SYS_TYPES_H
481
#include <sys/types.h>  /* For size_t */
482
#endif  /* LACKS_SYS_TYPES_H */
10347 ripley 483
 
40788 ripley 484
/* The maximum possible size_t value has all bits set */
485
#define MAX_SIZE_T           (~(size_t)0)
26855 ripley 486
 
40788 ripley 487
#ifndef ONLY_MSPACES
488
#define ONLY_MSPACES 0
489
#endif  /* ONLY_MSPACES */
490
#ifndef MSPACES
491
#if ONLY_MSPACES
492
#define MSPACES 1
493
#else   /* ONLY_MSPACES */
494
#define MSPACES 0
495
#endif  /* ONLY_MSPACES */
496
#endif  /* MSPACES */
26855 ripley 497
#ifndef MALLOC_ALIGNMENT
40788 ripley 498
#define MALLOC_ALIGNMENT ((size_t)8U)
499
#endif  /* MALLOC_ALIGNMENT */
500
#ifndef FOOTERS
501
#define FOOTERS 0
502
#endif  /* FOOTERS */
503
#ifndef ABORT
504
#define ABORT  abort()
505
#endif  /* ABORT */
506
#ifndef ABORT_ON_ASSERT_FAILURE
507
#define ABORT_ON_ASSERT_FAILURE 1
508
#endif  /* ABORT_ON_ASSERT_FAILURE */
509
#ifndef PROCEED_ON_ERROR
510
#define PROCEED_ON_ERROR 0
511
#endif  /* PROCEED_ON_ERROR */
512
#ifndef USE_LOCKS
513
#define USE_LOCKS 0
514
#endif  /* USE_LOCKS */
515
#ifndef INSECURE
516
#define INSECURE 0
517
#endif  /* INSECURE */
10347 ripley 518
#ifndef HAVE_MMAP
519
#define HAVE_MMAP 1
40788 ripley 520
#endif  /* HAVE_MMAP */
26855 ripley 521
#ifndef MMAP_CLEARS
522
#define MMAP_CLEARS 1
40788 ripley 523
#endif  /* MMAP_CLEARS */
10347 ripley 524
#ifndef HAVE_MREMAP
26855 ripley 525
#ifdef linux
10347 ripley 526
#define HAVE_MREMAP 1
40788 ripley 527
#else   /* linux */
10347 ripley 528
#define HAVE_MREMAP 0
40788 ripley 529
#endif  /* linux */
530
#endif  /* HAVE_MREMAP */
531
#ifndef MALLOC_FAILURE_ACTION
532
#define MALLOC_FAILURE_ACTION  errno = ENOMEM;
533
#endif  /* MALLOC_FAILURE_ACTION */
534
#ifndef HAVE_MORECORE
535
#if ONLY_MSPACES
536
#define HAVE_MORECORE 0
537
#else   /* ONLY_MSPACES */
538
#define HAVE_MORECORE 1
539
#endif  /* ONLY_MSPACES */
540
#endif  /* HAVE_MORECORE */
541
#if !HAVE_MORECORE
542
#define MORECORE_CONTIGUOUS 0
543
#else   /* !HAVE_MORECORE */
544
#ifndef MORECORE
545
#define MORECORE sbrk
546
#endif  /* MORECORE */
547
#ifndef MORECORE_CONTIGUOUS
548
#define MORECORE_CONTIGUOUS 1
549
#endif  /* MORECORE_CONTIGUOUS */
550
#endif  /* HAVE_MORECORE */
551
#ifndef DEFAULT_GRANULARITY
552
#if MORECORE_CONTIGUOUS
553
#define DEFAULT_GRANULARITY (0)  /* 0 means to compute in init_mparams */
554
#else   /* MORECORE_CONTIGUOUS */
555
#define DEFAULT_GRANULARITY ((size_t)64U * (size_t)1024U)
556
#endif  /* MORECORE_CONTIGUOUS */
557
#endif  /* DEFAULT_GRANULARITY */
558
#ifndef DEFAULT_TRIM_THRESHOLD
559
#ifndef MORECORE_CANNOT_TRIM
560
#define DEFAULT_TRIM_THRESHOLD ((size_t)2U * (size_t)1024U * (size_t)1024U)
561
#else   /* MORECORE_CANNOT_TRIM */
562
#define DEFAULT_TRIM_THRESHOLD MAX_SIZE_T
563
#endif  /* MORECORE_CANNOT_TRIM */
564
#endif  /* DEFAULT_TRIM_THRESHOLD */
565
#ifndef DEFAULT_MMAP_THRESHOLD
566
#if HAVE_MMAP
567
#define DEFAULT_MMAP_THRESHOLD ((size_t)256U * (size_t)1024U)
568
#else   /* HAVE_MMAP */
569
#define DEFAULT_MMAP_THRESHOLD MAX_SIZE_T
570
#endif  /* HAVE_MMAP */
571
#endif  /* DEFAULT_MMAP_THRESHOLD */
572
#ifndef USE_BUILTIN_FFS
573
#define USE_BUILTIN_FFS 0
574
#endif  /* USE_BUILTIN_FFS */
575
#ifndef USE_DEV_RANDOM
576
#define USE_DEV_RANDOM 0
577
#endif  /* USE_DEV_RANDOM */
578
#ifndef NO_MALLINFO
579
#define NO_MALLINFO 0
580
#endif  /* NO_MALLINFO */
581
#ifndef MALLINFO_FIELD_TYPE
582
#define MALLINFO_FIELD_TYPE size_t
583
#endif  /* MALLINFO_FIELD_TYPE */
10347 ripley 584
 
585
/*
40788 ripley 586
  mallopt tuning options.  SVID/XPG defines four standard parameter
587
  numbers for mallopt, normally defined in malloc.h.  None of these
588
  are used in this malloc, so setting them has no effect. But this
589
  malloc does support the following options.
10347 ripley 590
*/
591
 
40788 ripley 592
#define M_TRIM_THRESHOLD     (-1)
593
#define M_GRANULARITY        (-2)
594
#define M_MMAP_THRESHOLD     (-3)
26855 ripley 595
 
40788 ripley 596
/* ------------------------ Mallinfo declarations ------------------------ */
26855 ripley 597
 
40788 ripley 598
#if !NO_MALLINFO
10347 ripley 599
/*
600
  This version of malloc supports the standard SVID/XPG mallinfo
26855 ripley 601
  routine that returns a struct containing usage properties and
40788 ripley 602
  statistics. It should work on any system that has a
603
  /usr/include/malloc.h defining struct mallinfo.  The main
604
  declaration needed is the mallinfo struct that is returned (by-copy)
605
  by mallinfo().  The malloinfo struct contains a bunch of fields that
606
  are not even meaningful in this version of malloc.  These fields are
607
  are instead filled by mallinfo() with other numbers that might be of
608
  interest.
10347 ripley 609
 
610
  HAVE_USR_INCLUDE_MALLOC_H should be set if you have a
611
  /usr/include/malloc.h file that includes a declaration of struct
40788 ripley 612
  mallinfo.  If so, it is included; else a compliant version is
613
  declared below.  These must be precisely the same for mallinfo() to
614
  work.  The original SVID version of this struct, defined on most
615
  systems with mallinfo, declares all fields as ints. But some others
616
  define as unsigned long. If your system defines the fields using a
617
  type of different width than listed here, you MUST #include your
618
  system version and #define HAVE_USR_INCLUDE_MALLOC_H.
10347 ripley 619
*/
620
 
621
/* #define HAVE_USR_INCLUDE_MALLOC_H */
622
 
26855 ripley 623
#ifdef HAVE_USR_INCLUDE_MALLOC_H
10347 ripley 624
#include "/usr/include/malloc.h"
40788 ripley 625
#else /* HAVE_USR_INCLUDE_MALLOC_H */
10347 ripley 626
 
627
struct mallinfo {
40788 ripley 628
  MALLINFO_FIELD_TYPE arena;    /* non-mmapped space allocated from system */
629
  MALLINFO_FIELD_TYPE ordblks;  /* number of free chunks */
630
  MALLINFO_FIELD_TYPE smblks;   /* always 0 */
631
  MALLINFO_FIELD_TYPE hblks;    /* always 0 */
632
  MALLINFO_FIELD_TYPE hblkhd;   /* space in mmapped regions */
633
  MALLINFO_FIELD_TYPE usmblks;  /* maximum total allocated space */
634
  MALLINFO_FIELD_TYPE fsmblks;  /* always 0 */
635
  MALLINFO_FIELD_TYPE uordblks; /* total allocated space */
636
  MALLINFO_FIELD_TYPE fordblks; /* total free space */
637
  MALLINFO_FIELD_TYPE keepcost; /* releasable (via malloc_trim) space */
10347 ripley 638
};
639
 
40788 ripley 640
#endif /* HAVE_USR_INCLUDE_MALLOC_H */
641
#endif /* NO_MALLINFO */
10347 ripley 642
 
40788 ripley 643
#ifdef __cplusplus
644
extern "C" {
645
#endif /* __cplusplus */
646
 
26855 ripley 647
/* R Specific declarations here */
10347 ripley 648
 
50907 ripley 649
extern size_t R_max_memory;
26895 ripley 650
extern int R_Is_Running;
40788 ripley 651
static size_t R_used = 0;
10347 ripley 652
 
40788 ripley 653
#if !ONLY_MSPACES
10347 ripley 654
 
40788 ripley 655
/* ------------------- Declarations of public routines ------------------- */
10347 ripley 656
 
40788 ripley 657
#ifndef USE_DL_PREFIX
658
#define dlcalloc               Rm_calloc
659
#define dlfree                 Rm_free
660
#define dlmalloc               Rm_malloc
661
#define dlmemalign             memalign
662
#define dlrealloc              Rm_realloc
663
#define dlvalloc               valloc
664
#define dlpvalloc              pvalloc
665
#define dlmallinfo             mallinfo
666
#define dlmallopt              mallopt
667
#define dlmalloc_trim          malloc_trim
668
#define dlmalloc_stats         malloc_stats
669
#define dlmalloc_usable_size   malloc_usable_size
670
#define dlmalloc_footprint     malloc_footprint
671
#define dlmalloc_max_footprint malloc_max_footprint
672
#define dlindependent_calloc   independent_calloc
673
#define dlindependent_comalloc independent_comalloc
674
#endif /* USE_DL_PREFIX */
675
 
26855 ripley 676
/*
677
  malloc(size_t n)
40788 ripley 678
  Returns a pointer to a newly allocated chunk of at least n bytes, or
679
  null if no space is available, in which case errno is set to ENOMEM
680
  on ANSI C systems.
10347 ripley 681
 
40788 ripley 682
  If n is zero, malloc returns a minimum-sized chunk. (The minimum
683
  size is 16 bytes on most 32bit systems, and 32 bytes on 64bit
684
  systems.)  Note that size_t is an unsigned type, so calls with
685
  arguments that would be negative if signed are interpreted as
686
  requests for huge amounts of space, which will often fail. The
687
  maximum supported value of n differs across systems, but is in all
688
  cases less than the maximum representable value of a size_t.
26855 ripley 689
*/
40788 ripley 690
void* dlmalloc(size_t);
10347 ripley 691
 
692
/*
40788 ripley 693
  free(void* p)
26855 ripley 694
  Releases the chunk of memory pointed to by p, that had been previously
695
  allocated using malloc or a related routine such as realloc.
40788 ripley 696
  It has no effect if p is null. If p was not malloced or already
697
  freed, free(p) will by default cause the current program to abort.
26855 ripley 698
*/
40788 ripley 699
void  dlfree(void*);
10347 ripley 700
 
26855 ripley 701
/*
702
  calloc(size_t n_elements, size_t element_size);
703
  Returns a pointer to n_elements * element_size bytes, with all locations
704
  set to zero.
705
*/
40788 ripley 706
void* dlcalloc(size_t, size_t);
10347 ripley 707
 
26855 ripley 708
/*
40788 ripley 709
  realloc(void* p, size_t n)
26855 ripley 710
  Returns a pointer to a chunk of size n that contains the same data
711
  as does chunk p up to the minimum of (n, p's size) bytes, or null
40788 ripley 712
  if no space is available.
10347 ripley 713
 
26855 ripley 714
  The returned pointer may or may not be the same as p. The algorithm
40788 ripley 715
  prefers extending p in most cases when possible, otherwise it
716
  employs the equivalent of a malloc-copy-free sequence.
10347 ripley 717
 
40788 ripley 718
  If p is null, realloc is equivalent to malloc.
10347 ripley 719
 
26855 ripley 720
  If space is not available, realloc returns null, errno is set (if on
721
  ANSI) and p is NOT freed.
10347 ripley 722
 
26855 ripley 723
  if n is for fewer bytes than already held by p, the newly unused
40788 ripley 724
  space is lopped off and freed if possible.  realloc with a size
725
  argument of zero (re)allocates a minimum-sized chunk.
26855 ripley 726
 
727
  The old unix realloc convention of allowing the last-free'd chunk
728
  to be used as an argument to realloc is not supported.
10347 ripley 729
*/
29208 ripley 730
 
40788 ripley 731
void* dlrealloc(void*, size_t);
10347 ripley 732
 
26855 ripley 733
/*
734
  memalign(size_t alignment, size_t n);
735
  Returns a pointer to a newly allocated chunk of n bytes, aligned
736
  in accord with the alignment argument.
10347 ripley 737
 
26855 ripley 738
  The alignment argument should be a power of two. If the argument is
739
  not a power of two, the nearest greater power is used.
740
  8-byte alignment is guaranteed by normal malloc calls, so don't
741
  bother calling memalign with an argument of 8 or less.
742
 
743
  Overreliance on memalign is a sure way to fragment space.
744
*/
40788 ripley 745
void* dlmemalign(size_t, size_t);
10347 ripley 746
 
747
/*
26855 ripley 748
  valloc(size_t n);
749
  Equivalent to memalign(pagesize, n), where pagesize is the page
750
  size of the system. If the pagesize is unknown, 4096 is used.
751
*/
40788 ripley 752
void* dlvalloc(size_t);
10347 ripley 753
 
26855 ripley 754
/*
755
  mallopt(int parameter_number, int parameter_value)
756
  Sets tunable parameters The format is to provide a
757
  (parameter-number, parameter-value) pair.  mallopt then sets the
758
  corresponding parameter to the argument value if it can (i.e., so
759
  long as the value is meaningful), and returns 1 if successful else
760
  0.  SVID/XPG/ANSI defines four standard param numbers for mallopt,
40788 ripley 761
  normally defined in malloc.h.  None of these are use in this malloc,
762
  so setting them has no effect. But this malloc also supports other
763
  options in mallopt. See below for details.  Briefly, supported
26855 ripley 764
  parameters are as follows (listed defaults are for "typical"
765
  configurations).
10347 ripley 766
 
40788 ripley 767
  Symbol            param #  default    allowed param values
768
  M_TRIM_THRESHOLD     -1   2*1024*1024   any   (MAX_SIZE_T disables)
769
  M_GRANULARITY        -2     page size   any power of 2 >= page size
770
  M_MMAP_THRESHOLD     -3      256*1024   any   (or 0 if no MMAP support)
26855 ripley 771
*/
40788 ripley 772
int dlmallopt(int, int);
10347 ripley 773
 
40788 ripley 774
/*
775
  malloc_footprint();
776
  Returns the number of bytes obtained from the system.  The total
777
  number of bytes allocated by malloc, realloc etc., is less than this
778
  value. Unlike mallinfo, this function returns only a precomputed
779
  result, so can be called frequently to monitor memory consumption.
780
  Even if locks are otherwise defined, this function does not use them,
781
  so results might not be up to date.
782
*/
783
size_t dlmalloc_footprint(void);
10347 ripley 784
 
26855 ripley 785
/*
40788 ripley 786
  malloc_max_footprint();
787
  Returns the maximum number of bytes obtained from the system. This
788
  value will be greater than current footprint if deallocated space
789
  has been reclaimed by the system. The peak number of bytes allocated
790
  by malloc, realloc etc., is less than this value. Unlike mallinfo,
791
  this function returns only a precomputed result, so can be called
792
  frequently to monitor memory consumption.  Even if locks are
793
  otherwise defined, this function does not use them, so results might
794
  not be up to date.
795
*/
796
size_t dlmalloc_max_footprint(void);
797
 
798
#if !NO_MALLINFO
799
/*
26855 ripley 800
  mallinfo()
801
  Returns (by copy) a struct containing various summary statistics:
802
 
40788 ripley 803
  arena:     current total non-mmapped bytes allocated from system
804
  ordblks:   the number of free chunks
805
  smblks:    always zero.
806
  hblks:     current number of mmapped regions
807
  hblkhd:    total bytes held in mmapped regions
26855 ripley 808
  usmblks:   the maximum total allocated space. This will be greater
809
                than current total if trimming has occurred.
40788 ripley 810
  fsmblks:   always zero
26855 ripley 811
  uordblks:  current total allocated space (normal or mmapped)
40788 ripley 812
  fordblks:  total free space
26855 ripley 813
  keepcost:  the maximum number of bytes that could ideally be released
814
               back to system via malloc_trim. ("ideally" means that
815
               it ignores page restrictions etc.)
816
 
817
  Because these fields are ints, but internal bookkeeping may
40788 ripley 818
  be kept as longs, the reported values may wrap around zero and
26855 ripley 819
  thus be inaccurate.
10347 ripley 820
*/
40788 ripley 821
struct mallinfo dlmallinfo(void);
822
#endif /* NO_MALLINFO */
10347 ripley 823
 
26855 ripley 824
/*
40788 ripley 825
  independent_calloc(size_t n_elements, size_t element_size, void* chunks[]);
10347 ripley 826
 
26855 ripley 827
  independent_calloc is similar to calloc, but instead of returning a
828
  single cleared space, it returns an array of pointers to n_elements
829
  independent elements that can hold contents of size elem_size, each
830
  of which starts out cleared, and can be independently freed,
831
  realloc'ed etc. The elements are guaranteed to be adjacently
832
  allocated (this is not guaranteed to occur with multiple callocs or
833
  mallocs), which may also improve cache locality in some
834
  applications.
835
 
836
  The "chunks" argument is optional (i.e., may be null, which is
837
  probably the most typical usage). If it is null, the returned array
838
  is itself dynamically allocated and should also be freed when it is
839
  no longer needed. Otherwise, the chunks array must be of at least
840
  n_elements in length. It is filled in with the pointers to the
841
  chunks.
842
 
843
  In either case, independent_calloc returns this pointer array, or
844
  null if the allocation failed.  If n_elements is zero and "chunks"
845
  is null, it returns a chunk representing an array with zero elements
846
  (which should be freed if not wanted).
847
 
848
  Each element must be individually freed when it is no longer
849
  needed. If you'd like to instead be able to free all at once, you
850
  should instead use regular calloc and assign pointers into this
851
  space to represent elements.  (In this case though, you cannot
852
  independently free elements.)
40788 ripley 853
 
26855 ripley 854
  independent_calloc simplifies and speeds up implementations of many
855
  kinds of pools.  It may also be useful when constructing large data
856
  structures that initially have a fixed number of fixed-sized nodes,
857
  but the number is not known at compile time, and some of the nodes
858
  may later need to be freed. For example:
859
 
860
  struct Node { int item; struct Node* next; };
40788 ripley 861
 
26855 ripley 862
  struct Node* build_list() {
863
    struct Node** pool;
864
    int n = read_number_of_nodes_needed();
865
    if (n <= 0) return 0;
866
    pool = (struct Node**)(independent_calloc(n, sizeof(struct Node), 0);
40788 ripley 867
    if (pool == 0) die();
868
    // organize into a linked list...
26855 ripley 869
    struct Node* first = pool[0];
40788 ripley 870
    for (i = 0; i < n-1; ++i)
26855 ripley 871
      pool[i]->next = pool[i+1];
872
    free(pool);     // Can now free the array (or not, if it is needed later)
873
    return first;
874
  }
875
*/
40788 ripley 876
void** dlindependent_calloc(size_t, size_t, void**);
10347 ripley 877
 
878
/*
40788 ripley 879
  independent_comalloc(size_t n_elements, size_t sizes[], void* chunks[]);
10347 ripley 880
 
26855 ripley 881
  independent_comalloc allocates, all at once, a set of n_elements
882
  chunks with sizes indicated in the "sizes" array.    It returns
883
  an array of pointers to these elements, each of which can be
884
  independently freed, realloc'ed etc. The elements are guaranteed to
885
  be adjacently allocated (this is not guaranteed to occur with
886
  multiple callocs or mallocs), which may also improve cache locality
887
  in some applications.
10347 ripley 888
 
26855 ripley 889
  The "chunks" argument is optional (i.e., may be null). If it is null
890
  the returned array is itself dynamically allocated and should also
891
  be freed when it is no longer needed. Otherwise, the chunks array
892
  must be of at least n_elements in length. It is filled in with the
893
  pointers to the chunks.
10347 ripley 894
 
26855 ripley 895
  In either case, independent_comalloc returns this pointer array, or
896
  null if the allocation failed.  If n_elements is zero and chunks is
897
  null, it returns a chunk representing an array with zero elements
898
  (which should be freed if not wanted).
40788 ripley 899
 
26855 ripley 900
  Each element must be individually freed when it is no longer
901
  needed. If you'd like to instead be able to free all at once, you
902
  should instead use a single regular malloc, and assign pointers at
40788 ripley 903
  particular offsets in the aggregate space. (In this case though, you
26855 ripley 904
  cannot independently free elements.)
10347 ripley 905
 
26855 ripley 906
  independent_comallac differs from independent_calloc in that each
907
  element may have a different size, and also that it does not
908
  automatically clear elements.
10347 ripley 909
 
26855 ripley 910
  independent_comalloc can be used to speed up allocation in cases
911
  where several structs or objects must always be allocated at the
912
  same time.  For example:
10347 ripley 913
 
26855 ripley 914
  struct Head { ... }
915
  struct Foot { ... }
10347 ripley 916
 
26855 ripley 917
  void send_message(char* msg) {
918
    int msglen = strlen(msg);
919
    size_t sizes[3] = { sizeof(struct Head), msglen, sizeof(struct Foot) };
920
    void* chunks[3];
921
    if (independent_comalloc(3, sizes, chunks) == 0)
922
      die();
923
    struct Head* head = (struct Head*)(chunks[0]);
924
    char*        body = (char*)(chunks[1]);
925
    struct Foot* foot = (struct Foot*)(chunks[2]);
926
    // ...
927
  }
10347 ripley 928
 
26855 ripley 929
  In general though, independent_comalloc is worth using only for
930
  larger values of n_elements. For small values, you probably won't
931
  detect enough difference from series of malloc calls to bother.
932
 
933
  Overuse of independent_comalloc can increase overall memory usage,
934
  since it cannot reuse existing noncontiguous small chunks that
935
  might be available for some of the elements.
10347 ripley 936
*/
40788 ripley 937
void** dlindependent_comalloc(size_t, size_t*, void**);
10347 ripley 938
 
939
 
26855 ripley 940
/*
941
  pvalloc(size_t n);
942
  Equivalent to valloc(minimum-page-that-holds(n)), that is,
943
  round up n to nearest pagesize.
944
 */
40788 ripley 945
void*  dlpvalloc(size_t);
10347 ripley 946
 
26855 ripley 947
/*
40788 ripley 948
  malloc_trim(size_t pad);
26855 ripley 949
 
40788 ripley 950
  If possible, gives memory back to the system (via negative arguments
951
  to sbrk) if there is unused memory at the `high' end of the malloc
952
  pool or in unused MMAP segments. You can call this after freeing
953
  large blocks of memory to potentially reduce the system-level memory
954
  requirements of a program. However, it cannot guarantee to reduce
955
  memory. Under some allocation patterns, some large free blocks of
956
  memory will be locked between two used chunks, so they cannot be
957
  given back to the system.
26855 ripley 958
 
40788 ripley 959
  The `pad' argument to malloc_trim represents the amount of free
960
  trailing space to leave untrimmed. If this argument is zero, only
961
  the minimum amount of memory to maintain internal data structures
962
  will be left. Non-zero arguments can be supplied to maintain enough
963
  trailing space to service future expected allocations without having
964
  to re-obtain memory from the system.
26855 ripley 965
 
966
  Malloc_trim returns 1 if it actually released any memory, else 0.
967
*/
40788 ripley 968
int  dlmalloc_trim(size_t);
10347 ripley 969
 
970
/*
40788 ripley 971
  malloc_usable_size(void* p);
10347 ripley 972
 
26855 ripley 973
  Returns the number of bytes you can actually use in
974
  an allocated chunk, which may be more than you requested (although
975
  often not) due to alignment and minimum size constraints.
976
  You can use this many bytes without worrying about
977
  overwriting other allocated objects. This is not a particularly great
978
  programming practice. malloc_usable_size can be more useful in
979
  debugging and assertions, for example:
10347 ripley 980
 
26855 ripley 981
  p = malloc(n);
982
  assert(malloc_usable_size(p) >= 256);
10347 ripley 983
*/
40788 ripley 984
size_t dlmalloc_usable_size(void*);
10347 ripley 985
 
26855 ripley 986
/*
987
  malloc_stats();
988
  Prints on stderr the amount of space obtained from the system (both
989
  via sbrk and mmap), the maximum amount (which may be more than
990
  current if malloc_trim and/or munmap got called), and the current
991
  number of bytes allocated via malloc (or realloc, etc) but not yet
992
  freed. Note that this is the number of bytes allocated, not the
993
  number requested. It will be larger than the number requested
994
  because of alignment and bookkeeping overhead. Because it includes
995
  alignment wastage as being in use, this figure may be greater than
996
  zero even when no user-level chunks are allocated.
10347 ripley 997
 
26855 ripley 998
  The reported current and maximum system memory can be inaccurate if
999
  a program makes other calls to system memory allocation functions
1000
  (normally sbrk) outside of malloc.
10347 ripley 1001
 
26855 ripley 1002
  malloc_stats prints only the most commonly interesting statistics.
1003
  More information can be obtained by calling mallinfo.
1004
*/
40788 ripley 1005
void  dlmalloc_stats(void);
26855 ripley 1006
 
40788 ripley 1007
#endif /* ONLY_MSPACES */
26855 ripley 1008
 
40788 ripley 1009
#if MSPACES
1010
 
10347 ripley 1011
/*
40788 ripley 1012
  mspace is an opaque type representing an independent
1013
  region of space that supports mspace_malloc, etc.
1014
*/
1015
typedef void* mspace;
10347 ripley 1016
 
40788 ripley 1017
/*
1018
  create_mspace creates and returns a new independent space with the
1019
  given initial capacity, or, if 0, the default granularity size.  It
1020
  returns null if there is no system memory available to create the
1021
  space.  If argument locked is non-zero, the space uses a separate
1022
  lock to control access. The capacity of the space will grow
1023
  dynamically as needed to service mspace_malloc requests.  You can
1024
  control the sizes of incremental increases of this space by
1025
  compiling with a different DEFAULT_GRANULARITY or dynamically
1026
  setting with mallopt(M_GRANULARITY, value).
1027
*/
1028
mspace create_mspace(size_t capacity, int locked);
26855 ripley 1029
 
40788 ripley 1030
/*
1031
  destroy_mspace destroys the given space, and attempts to return all
1032
  of its memory back to the system, returning the total number of
1033
  bytes freed. After destruction, the results of access to all memory
1034
  used by the space become undefined.
10425 ripley 1035
*/
40788 ripley 1036
size_t destroy_mspace(mspace msp);
10425 ripley 1037
 
40788 ripley 1038
/*
1039
  create_mspace_with_base uses the memory supplied as the initial base
1040
  of a new mspace. Part (less than 128*sizeof(size_t) bytes) of this
1041
  space is used for bookkeeping, so the capacity must be at least this
1042
  large. (Otherwise 0 is returned.) When this initial space is
1043
  exhausted, additional memory will be obtained from the system.
1044
  Destroying this space will deallocate all additionally allocated
1045
  space (if possible) but not the initial base.
1046
*/
1047
mspace create_mspace_with_base(void* base, size_t capacity, int locked);
10425 ripley 1048
 
40788 ripley 1049
/*
1050
  mspace_malloc behaves as malloc, but operates within
1051
  the given space.
1052
*/
1053
void* mspace_malloc(mspace msp, size_t bytes);
10425 ripley 1054
 
40788 ripley 1055
/*
1056
  mspace_free behaves as free, but operates within
1057
  the given space.
10425 ripley 1058
 
40788 ripley 1059
  If compiled with FOOTERS==1, mspace_free is not actually needed.
1060
  free may be called instead of mspace_free because freed chunks from
1061
  any space are handled by their originating spaces.
1062
*/
1063
void mspace_free(mspace msp, void* mem);
10425 ripley 1064
 
1065
/*
40788 ripley 1066
  mspace_realloc behaves as realloc, but operates within
1067
  the given space.
10425 ripley 1068
 
40788 ripley 1069
  If compiled with FOOTERS==1, mspace_realloc is not actually
1070
  needed.  realloc may be called instead of mspace_realloc because
1071
  realloced chunks from any space are handled by their originating
1072
  spaces.
1073
*/
1074
void* mspace_realloc(mspace msp, void* mem, size_t newsize);
10347 ripley 1075
 
40788 ripley 1076
/*
1077
  mspace_calloc behaves as calloc, but operates within
1078
  the given space.
1079
*/
1080
void* mspace_calloc(mspace msp, size_t n_elements, size_t elem_size);
10347 ripley 1081
 
40788 ripley 1082
/*
1083
  mspace_memalign behaves as memalign, but operates within
1084
  the given space.
1085
*/
1086
void* mspace_memalign(mspace msp, size_t alignment, size_t bytes);
10347 ripley 1087
 
40788 ripley 1088
/*
1089
  mspace_independent_calloc behaves as independent_calloc, but
1090
  operates within the given space.
1091
*/
1092
void** mspace_independent_calloc(mspace msp, size_t n_elements,
1093
                                 size_t elem_size, void* chunks[]);
10347 ripley 1094
 
40788 ripley 1095
/*
1096
  mspace_independent_comalloc behaves as independent_comalloc, but
1097
  operates within the given space.
1098
*/
1099
void** mspace_independent_comalloc(mspace msp, size_t n_elements,
1100
                                   size_t sizes[], void* chunks[]);
10347 ripley 1101
 
40788 ripley 1102
/*
1103
  mspace_footprint() returns the number of bytes obtained from the
1104
  system for this space.
1105
*/
1106
size_t mspace_footprint(mspace msp);
10347 ripley 1107
 
40788 ripley 1108
/*
1109
  mspace_max_footprint() returns the peak number of bytes obtained from the
1110
  system for this space.
26855 ripley 1111
*/
40788 ripley 1112
size_t mspace_max_footprint(mspace msp);
10347 ripley 1113
 
1114
 
40788 ripley 1115
#if !NO_MALLINFO
1116
/*
1117
  mspace_mallinfo behaves as mallinfo, but reports properties of
1118
  the given space.
1119
*/
1120
struct mallinfo mspace_mallinfo(mspace msp);
1121
#endif /* NO_MALLINFO */
10347 ripley 1122
 
26855 ripley 1123
/*
40788 ripley 1124
  mspace_malloc_stats behaves as malloc_stats, but reports
1125
  properties of the given space.
26855 ripley 1126
*/
40788 ripley 1127
void mspace_malloc_stats(mspace msp);
26855 ripley 1128
 
1129
/*
40788 ripley 1130
  mspace_trim behaves as malloc_trim, but
1131
  operates within the given space.
26855 ripley 1132
*/
40788 ripley 1133
int mspace_trim(mspace msp, size_t pad);
26855 ripley 1134
 
1135
/*
40788 ripley 1136
  An alias for mallopt.
26855 ripley 1137
*/
40788 ripley 1138
int mspace_mallopt(int, int);
10347 ripley 1139
 
40788 ripley 1140
#endif /* MSPACES */
26855 ripley 1141
 
10347 ripley 1142
#ifdef __cplusplus
1143
};  /* end of extern "C" */
40788 ripley 1144
#endif /* __cplusplus */
10347 ripley 1145
 
40788 ripley 1146
/*
26855 ripley 1147
  ========================================================================
1148
  To make a fully customizable malloc.h header file, cut everything
40788 ripley 1149
  above this line, put into file malloc.h, edit to suit, and #include it
26855 ripley 1150
  on the next line, as well as in programs that use this malloc.
1151
  ========================================================================
1152
*/
10347 ripley 1153
 
26855 ripley 1154
/* #include "malloc.h" */
10347 ripley 1155
 
40788 ripley 1156
/*------------------------------ internal #includes ---------------------- */
10425 ripley 1157
 
40788 ripley 1158
#if 0
1159
#ifdef WIN32
1160
#pragma warning( disable : 4146 ) /* no "unsigned" warnings */
1161
#endif /* WIN32 */
1162
#endif
10347 ripley 1163
 
40788 ripley 1164
#include <stdio.h>       /* for printing in malloc_stats */
26855 ripley 1165
 
40788 ripley 1166
#ifndef LACKS_ERRNO_H
1167
#include <errno.h>       /* for MALLOC_FAILURE_ACTION */
1168
#endif /* LACKS_ERRNO_H */
1169
#if FOOTERS
1170
#include <time.h>        /* for magic initialization */
1171
#endif /* FOOTERS */
1172
#ifndef LACKS_STDLIB_H
1173
#include <stdlib.h>      /* for abort() */
1174
#endif /* LACKS_STDLIB_H */
1175
#ifdef DEBUG
1176
#if ABORT_ON_ASSERT_FAILURE
1177
#define assert(x) if(!(x)) ABORT
1178
#else /* ABORT_ON_ASSERT_FAILURE */
1179
#include <assert.h>
1180
#endif /* ABORT_ON_ASSERT_FAILURE */
1181
#else  /* DEBUG */
1182
#define assert(x)
1183
#endif /* DEBUG */
1184
#ifndef LACKS_STRING_H
1185
#include <string.h>      /* for memset etc */
1186
#endif  /* LACKS_STRING_H */
1187
#if USE_BUILTIN_FFS
1188
#ifndef LACKS_STRINGS_H
1189
#include <strings.h>     /* for ffs */
1190
#endif /* LACKS_STRINGS_H */
1191
#endif /* USE_BUILTIN_FFS */
1192
#if HAVE_MMAP
1193
#ifndef LACKS_SYS_MMAN_H
1194
#include <sys/mman.h>    /* for mmap */
1195
#endif /* LACKS_SYS_MMAN_H */
1196
#ifndef LACKS_FCNTL_H
1197
#include <fcntl.h>
1198
#endif /* LACKS_FCNTL_H */
1199
#endif /* HAVE_MMAP */
1200
#if HAVE_MORECORE
1201
#ifndef LACKS_UNISTD_H
1202
#include <unistd.h>     /* for sbrk */
1203
#else /* LACKS_UNISTD_H */
1204
#if !defined(__FreeBSD__) && !defined(__OpenBSD__) && !defined(__NetBSD__)
1205
extern void*     sbrk(ptrdiff_t);
1206
#endif /* FreeBSD etc */
1207
#endif /* LACKS_UNISTD_H */
1208
#endif /* HAVE_MMAP */
10347 ripley 1209
 
40788 ripley 1210
#ifndef WIN32
1211
#ifndef malloc_getpagesize
1212
#  ifdef _SC_PAGESIZE         /* some SVR4 systems omit an underscore */
1213
#    ifndef _SC_PAGE_SIZE
1214
#      define _SC_PAGE_SIZE _SC_PAGESIZE
1215
#    endif
1216
#  endif
1217
#  ifdef _SC_PAGE_SIZE
1218
#    define malloc_getpagesize sysconf(_SC_PAGE_SIZE)
1219
#  else
1220
#    if defined(BSD) || defined(DGUX) || defined(HAVE_GETPAGESIZE)
1221
       extern size_t getpagesize();
1222
#      define malloc_getpagesize getpagesize()
1223
#    else
1224
#      ifdef WIN32 /* use supplied emulation of getpagesize */
1225
#        define malloc_getpagesize getpagesize()
1226
#      else
1227
#        ifndef LACKS_SYS_PARAM_H
1228
#          include <sys/param.h>
1229
#        endif
1230
#        ifdef EXEC_PAGESIZE
1231
#          define malloc_getpagesize EXEC_PAGESIZE
1232
#        else
1233
#          ifdef NBPG
1234
#            ifndef CLSIZE
1235
#              define malloc_getpagesize NBPG
1236
#            else
1237
#              define malloc_getpagesize (NBPG * CLSIZE)
1238
#            endif
1239
#          else
1240
#            ifdef NBPC
1241
#              define malloc_getpagesize NBPC
1242
#            else
1243
#              ifdef PAGESIZE
1244
#                define malloc_getpagesize PAGESIZE
1245
#              else /* just guess */
1246
#                define malloc_getpagesize ((size_t)4096U)
1247
#              endif
1248
#            endif
1249
#          endif
1250
#        endif
1251
#      endif
1252
#    endif
1253
#  endif
1254
#endif
1255
#endif
10347 ripley 1256
 
40788 ripley 1257
/* ------------------- size_t and alignment properties -------------------- */
26855 ripley 1258
 
40788 ripley 1259
/* The byte and bit size of a size_t */
1260
#define SIZE_T_SIZE         (sizeof(size_t))
1261
#define SIZE_T_BITSIZE      (sizeof(size_t) << 3)
10347 ripley 1262
 
40788 ripley 1263
/* Some constants coerced to size_t */
1264
/* Annoying but necessary to avoid errors on some plaftorms */
1265
#define SIZE_T_ZERO         ((size_t)0)
1266
#define SIZE_T_ONE          ((size_t)1)
1267
#define SIZE_T_TWO          ((size_t)2)
1268
#define TWO_SIZE_T_SIZES    (SIZE_T_SIZE<<1)
1269
#define FOUR_SIZE_T_SIZES   (SIZE_T_SIZE<<2)
1270
#define SIX_SIZE_T_SIZES    (FOUR_SIZE_T_SIZES+TWO_SIZE_T_SIZES)
1271
#define HALF_MAX_SIZE_T     (MAX_SIZE_T / 2U)
10347 ripley 1272
 
40788 ripley 1273
/* The bit mask value corresponding to MALLOC_ALIGNMENT */
1274
#define CHUNK_ALIGN_MASK    (MALLOC_ALIGNMENT - SIZE_T_ONE)
10347 ripley 1275
 
40788 ripley 1276
/* True if address a has acceptable alignment */
1277
#define is_aligned(A)       (((size_t)((A)) & (CHUNK_ALIGN_MASK)) == 0)
10347 ripley 1278
 
40788 ripley 1279
/* the number of bytes to offset an address to align it */
1280
#define align_offset(A)\
1281
 ((((size_t)(A) & CHUNK_ALIGN_MASK) == 0)? 0 :\
1282
  ((MALLOC_ALIGNMENT - ((size_t)(A) & CHUNK_ALIGN_MASK)) & CHUNK_ALIGN_MASK))
26855 ripley 1283
 
40788 ripley 1284
/* -------------------------- MMAP preliminaries ------------------------- */
26855 ripley 1285
 
40788 ripley 1286
/*
1287
   If HAVE_MORECORE or HAVE_MMAP are false, we just define calls and
1288
   checks to fail so compiler optimizer can delete code rather than
1289
   using so many "#if"s.
1290
*/
26855 ripley 1291
 
1292
 
40788 ripley 1293
/* MORECORE and MMAP must return MFAIL on failure */
1294
#define MFAIL                ((void*)(MAX_SIZE_T))
1295
#define CMFAIL               ((char*)(MFAIL)) /* defined for convenience */
26855 ripley 1296
 
40788 ripley 1297
#if !HAVE_MMAP
1298
#define IS_MMAPPED_BIT       (SIZE_T_ZERO)
1299
#define USE_MMAP_BIT         (SIZE_T_ZERO)
1300
#define CALL_MMAP(s)         MFAIL
1301
#define CALL_MUNMAP(a, s)    (-1)
1302
#define DIRECT_MMAP(s)       MFAIL
26855 ripley 1303
 
40788 ripley 1304
#else /* HAVE_MMAP */
1305
#define IS_MMAPPED_BIT       (SIZE_T_ONE)
1306
#define USE_MMAP_BIT         (SIZE_T_ONE)
26855 ripley 1307
 
40788 ripley 1308
#ifndef WIN32
1309
#define CALL_MUNMAP(a, s)    munmap((a), (s))
1310
#define MMAP_PROT            (PROT_READ|PROT_WRITE)
1311
#if !defined(MAP_ANONYMOUS) && defined(MAP_ANON)
1312
#define MAP_ANONYMOUS        MAP_ANON
1313
#endif /* MAP_ANON */
1314
#ifdef MAP_ANONYMOUS
1315
#define MMAP_FLAGS           (MAP_PRIVATE|MAP_ANONYMOUS)
1316
#define CALL_MMAP(s)         mmap(0, (s), MMAP_PROT, MMAP_FLAGS, -1, 0)
1317
#else /* MAP_ANONYMOUS */
1318
/*
1319
   Nearly all versions of mmap support MAP_ANONYMOUS, so the following
1320
   is unlikely to be needed, but is supplied just in case.
1321
*/
1322
#define MMAP_FLAGS           (MAP_PRIVATE)
1323
static int dev_zero_fd = -1; /* Cached file descriptor for /dev/zero. */
1324
#define CALL_MMAP(s) ((dev_zero_fd < 0) ? \
1325
           (dev_zero_fd = open("/dev/zero", O_RDWR), \
1326
            mmap(0, (s), MMAP_PROT, MMAP_FLAGS, dev_zero_fd, 0)) : \
1327
            mmap(0, (s), MMAP_PROT, MMAP_FLAGS, dev_zero_fd, 0))
1328
#endif /* MAP_ANONYMOUS */
10347 ripley 1329
 
40788 ripley 1330
#define DIRECT_MMAP(s)       CALL_MMAP(s)
1331
#else /* WIN32 */
10425 ripley 1332
 
40788 ripley 1333
/* Win32 MMAP via VirtualAlloc */
1334
static void* win32mmap(size_t size) {
1335
  void* ptr;
1336
  /* printf("current %0.1f, asking %0.1f\n", R_used/1048576., size/1048576.);*/
1337
  if (R_used + size > R_max_memory) {
1338
      return MFAIL;
26855 ripley 1339
  }
40788 ripley 1340
  ptr = VirtualAlloc(0, size, MEM_RESERVE|MEM_COMMIT, PAGE_READWRITE);
1341
  R_used += (ptr != 0)? size: 0;
1342
  return (ptr != 0)? ptr: MFAIL;
26855 ripley 1343
}
10425 ripley 1344
 
40788 ripley 1345
/* For direct MMAP, use MEM_TOP_DOWN to minimize interference */
1346
static void* win32direct_mmap(size_t size) {
1347
  void* ptr;
1348
  /* printf("current %0.1f, asking %0.1f\n", R_used/1048576., size/1048576.);*/
1349
  if (R_used + size > R_max_memory) {
1350
      return MFAIL;
26855 ripley 1351
  }
40788 ripley 1352
  ptr = VirtualAlloc(0, size, MEM_RESERVE|MEM_COMMIT|MEM_TOP_DOWN,
1353
                           PAGE_READWRITE);
1354
  R_used += (ptr != 0)? size: 0;
1355
  return (ptr != 0)? ptr: MFAIL;
10347 ripley 1356
}
1357
 
40788 ripley 1358
/* This function supports releasing coalesed segments */
1359
static int win32munmap(void* ptr, size_t size) {
1360
  MEMORY_BASIC_INFORMATION minfo;
1361
  char* cptr = ptr;
1362
  while (size) {
1363
    if (VirtualQuery(cptr, &minfo, sizeof(minfo)) == 0)
1364
      return -1;
1365
    if (minfo.BaseAddress != cptr || minfo.AllocationBase != cptr ||
1366
        minfo.State != MEM_COMMIT || minfo.RegionSize > size)
1367
      return -1;
1368
    if (VirtualFree(cptr, 0, MEM_RELEASE) == 0)
1369
      return -1;
1370
    cptr += minfo.RegionSize;
1371
    size -= minfo.RegionSize;
1372
    /* printf("current %0.1f, releasing %0.1f\n", 
1373
       R_used/1048576., minfo.RegionSize/1048576.); */
1374
    R_used -= minfo.RegionSize;
26855 ripley 1375
  }
40788 ripley 1376
  return 0;
26855 ripley 1377
}
12256 pd 1378
 
40788 ripley 1379
#define CALL_MMAP(s)         win32mmap(s)
1380
#define CALL_MUNMAP(a, s)    win32munmap((a), (s))
1381
#define DIRECT_MMAP(s)       win32direct_mmap(s)
1382
#endif /* WIN32 */
1383
#endif /* HAVE_MMAP */
10347 ripley 1384
 
40788 ripley 1385
#if HAVE_MMAP && HAVE_MREMAP
1386
#define CALL_MREMAP(addr, osz, nsz, mv) mremap((addr), (osz), (nsz), (mv))
1387
#else  /* HAVE_MMAP && HAVE_MREMAP */
1388
#define CALL_MREMAP(addr, osz, nsz, mv) MFAIL
1389
#endif /* HAVE_MMAP && HAVE_MREMAP */
11007 ripley 1390
 
40788 ripley 1391
#if HAVE_MORECORE
1392
#define CALL_MORECORE(S)     MORECORE(S)
1393
#else  /* HAVE_MORECORE */
1394
#define CALL_MORECORE(S)     MFAIL
1395
#endif /* HAVE_MORECORE */
10347 ripley 1396
 
40788 ripley 1397
/* mstate bit set if continguous morecore disabled or failed */
1398
#define USE_NONCONTIGUOUS_BIT (4U)
10347 ripley 1399
 
40788 ripley 1400
/* segment bit set in create_mspace_with_base */
1401
#define EXTERN_BIT            (8U)
10347 ripley 1402
 
1403
 
40788 ripley 1404
/* --------------------------- Lock preliminaries ------------------------ */
10347 ripley 1405
 
40788 ripley 1406
#if USE_LOCKS
26855 ripley 1407
 
40788 ripley 1408
/*
1409
  When locks are defined, there are up to two global locks:
26855 ripley 1410
 
40788 ripley 1411
  * If HAVE_MORECORE, morecore_mutex protects sequences of calls to
1412
    MORECORE.  In many cases sys_alloc requires two calls, that should
1413
    not be interleaved with calls by other threads.  This does not
1414
    protect against direct calls to MORECORE by other threads not
1415
    using this lock, so there is still code to cope the best we can on
1416
    interference.
1417
 
1418
  * magic_init_mutex ensures that mparams.magic and other
1419
    unique mparams values are initialized only once.
1420
*/
1421
 
1422
#ifndef WIN32
1423
/* By default use posix locks */
1424
#include <pthread.h>
1425
#define MLOCK_T pthread_mutex_t
1426
#define INITIAL_LOCK(l)      pthread_mutex_init(l, NULL)
1427
#define ACQUIRE_LOCK(l)      pthread_mutex_lock(l)
1428
#define RELEASE_LOCK(l)      pthread_mutex_unlock(l)
1429
 
1430
#if HAVE_MORECORE
1431
static MLOCK_T morecore_mutex = PTHREAD_MUTEX_INITIALIZER;
1432
#endif /* HAVE_MORECORE */
1433
 
1434
static MLOCK_T magic_init_mutex = PTHREAD_MUTEX_INITIALIZER;
1435
 
1436
#else /* WIN32 */
1437
/*
1438
   Because lock-protected regions have bounded times, and there
1439
   are no recursive lock calls, we can use simple spinlocks.
1440
*/
1441
 
1442
#define MLOCK_T long
1443
static int win32_acquire_lock (MLOCK_T *sl) {
1444
  for (;;) {
1445
#ifdef InterlockedCompareExchangePointer
1446
    if (!InterlockedCompareExchange(sl, 1, 0))
1447
      return 0;
1448
#else  /* Use older void* version */
1449
    if (!InterlockedCompareExchange((void**)sl, (void*)1, (void*)0))
1450
      return 0;
1451
#endif /* InterlockedCompareExchangePointer */
1452
    Sleep (0);
26855 ripley 1453
  }
1454
}
1455
 
40788 ripley 1456
static void win32_release_lock (MLOCK_T *sl) {
1457
  InterlockedExchange (sl, 0);
26855 ripley 1458
}
1459
 
40788 ripley 1460
#define INITIAL_LOCK(l)      *(l)=0
1461
#define ACQUIRE_LOCK(l)      win32_acquire_lock(l)
1462
#define RELEASE_LOCK(l)      win32_release_lock(l)
1463
#if HAVE_MORECORE
1464
static MLOCK_T morecore_mutex;
1465
#endif /* HAVE_MORECORE */
1466
static MLOCK_T magic_init_mutex;
1467
#endif /* WIN32 */
26855 ripley 1468
 
40788 ripley 1469
#define USE_LOCK_BIT               (2U)
1470
#else  /* USE_LOCKS */
1471
#define USE_LOCK_BIT               (0U)
1472
#define INITIAL_LOCK(l)
1473
#endif /* USE_LOCKS */
26855 ripley 1474
 
40788 ripley 1475
#if USE_LOCKS && HAVE_MORECORE
1476
#define ACQUIRE_MORECORE_LOCK()    ACQUIRE_LOCK(&morecore_mutex);
1477
#define RELEASE_MORECORE_LOCK()    RELEASE_LOCK(&morecore_mutex);
1478
#else /* USE_LOCKS && HAVE_MORECORE */
1479
#define ACQUIRE_MORECORE_LOCK()
1480
#define RELEASE_MORECORE_LOCK()
1481
#endif /* USE_LOCKS && HAVE_MORECORE */
26855 ripley 1482
 
40788 ripley 1483
#if USE_LOCKS
1484
#define ACQUIRE_MAGIC_INIT_LOCK()  ACQUIRE_LOCK(&magic_init_mutex);
1485
#define RELEASE_MAGIC_INIT_LOCK()  RELEASE_LOCK(&magic_init_mutex);
1486
#else  /* USE_LOCKS */
1487
#define ACQUIRE_MAGIC_INIT_LOCK()
1488
#define RELEASE_MAGIC_INIT_LOCK()
1489
#endif /* USE_LOCKS */
26855 ripley 1490
 
1491
 
40788 ripley 1492
/* -----------------------  Chunk representations ------------------------ */
26855 ripley 1493
 
40788 ripley 1494
/*
1495
  (The following includes lightly edited explanations by Colin Plumb.)
26855 ripley 1496
 
40788 ripley 1497
  The malloc_chunk declaration below is misleading (but accurate and
1498
  necessary).  It declares a "view" into memory allowing access to
1499
  necessary fields at known offsets from a given base.
26855 ripley 1500
 
40788 ripley 1501
  Chunks of memory are maintained using a `boundary tag' method as
1502
  originally described by Knuth.  (See the paper by Paul Wilson
1503
  ftp://ftp.cs.utexas.edu/pub/garbage/allocsrv.ps for a survey of such
1504
  techniques.)  Sizes of free chunks are stored both in the front of
1505
  each chunk and at the end.  This makes consolidating fragmented
1506
  chunks into bigger chunks fast.  The head fields also hold bits
1507
  representing whether chunks are free or in use.
26855 ripley 1508
 
40788 ripley 1509
  Here are some pictures to make it clearer.  They are "exploded" to
1510
  show that the state of a chunk can be thought of as extending from
1511
  the high 31 bits of the head field of its header through the
1512
  prev_foot and PINUSE_BIT bit of the following chunk header.
26855 ripley 1513
 
40788 ripley 1514
  A chunk that's in use looks like:
26855 ripley 1515
 
40788 ripley 1516
   chunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1517
           | Size of previous chunk (if P = 1)                             |
1518
           +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1519
         +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |P|
1520
         | Size of this chunk                                         1| +-+
1521
   mem-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1522
         |                                                               |
1523
         +-                                                             -+
1524
         |                                                               |
1525
         +-                                                             -+
1526
         |                                                               :
1527
         +-      size - sizeof(size_t) available payload bytes          -+
1528
         :                                                               |
1529
 chunk-> +-                                                             -+
1530
         |                                                               |
1531
         +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1532
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |1|
1533
       | Size of next chunk (may or may not be in use)               | +-+
1534
 mem-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
26855 ripley 1535
 
40788 ripley 1536
    And if it's free, it looks like this:
26855 ripley 1537
 
40788 ripley 1538
   chunk-> +-                                                             -+
1539
           | User payload (must be in use, or we would have merged!)       |
1540
           +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1541
         +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |P|
1542
         | Size of this chunk                                         0| +-+
1543
   mem-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1544
         | Next pointer                                                  |
1545
         +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1546
         | Prev pointer                                                  |
1547
         +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1548
         |                                                               :
1549
         +-      size - sizeof(struct chunk) unused bytes               -+
1550
         :                                                               |
1551
 chunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1552
         | Size of this chunk                                            |
1553
         +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1554
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0|
1555
       | Size of next chunk (must be in use, or we would have merged)| +-+
1556
 mem-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1557
       |                                                               :
1558
       +- User payload                                                -+
1559
       :                                                               |
1560
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1561
                                                                     |0|
1562
                                                                     +-+
1563
  Note that since we always merge adjacent free chunks, the chunks
1564
  adjacent to a free chunk must be in use.
26855 ripley 1565
 
40788 ripley 1566
  Given a pointer to a chunk (which can be derived trivially from the
1567
  payload pointer) we can, in O(1) time, find out whether the adjacent
1568
  chunks are free, and if so, unlink them from the lists that they
1569
  are on and merge them with the current chunk.
26855 ripley 1570
 
40788 ripley 1571
  Chunks always begin on even word boundaries, so the mem portion
1572
  (which is returned to the user) is also on an even word boundary, and
1573
  thus at least double-word aligned.
26855 ripley 1574
 
40788 ripley 1575
  The P (PINUSE_BIT) bit, stored in the unused low-order bit of the
1576
  chunk size (which is always a multiple of two words), is an in-use
1577
  bit for the *previous* chunk.  If that bit is *clear*, then the
1578
  word before the current chunk size contains the previous chunk
1579
  size, and can be used to find the front of the previous chunk.
1580
  The very first chunk allocated always has this bit set, preventing
1581
  access to non-existent (or non-owned) memory. If pinuse is set for
1582
  any given chunk, then you CANNOT determine the size of the
1583
  previous chunk, and might even get a memory addressing fault when
1584
  trying to do so.
26855 ripley 1585
 
40788 ripley 1586
  The C (CINUSE_BIT) bit, stored in the unused second-lowest bit of
1587
  the chunk size redundantly records whether the current chunk is
1588
  inuse. This redundancy enables usage checks within free and realloc,
1589
  and reduces indirection when freeing and consolidating chunks.
26855 ripley 1590
 
40788 ripley 1591
  Each freshly allocated chunk must have both cinuse and pinuse set.
1592
  That is, each allocated chunk borders either a previously allocated
1593
  and still in-use chunk, or the base of its memory arena. This is
68538 maechler 1594
  ensured by making all allocations from the `lowest' part of any
40788 ripley 1595
  found chunk.  Further, no free chunk physically borders another one,
1596
  so each free chunk is known to be preceded and followed by either
1597
  inuse chunks or the ends of memory.
26855 ripley 1598
 
40788 ripley 1599
  Note that the `foot' of the current chunk is actually represented
1600
  as the prev_foot of the NEXT chunk. This makes it easier to
1601
  deal with alignments etc but can be very confusing when trying
1602
  to extend or adapt this code.
1603
 
1604
  The exceptions to all this are
1605
 
1606
     1. The special chunk `top' is the top-most available chunk (i.e.,
1607
        the one bordering the end of available memory). It is treated
1608
        specially.  Top is never included in any bin, is used only if
1609
        no other chunk is available, and is released back to the
1610
        system if it is very large (see M_TRIM_THRESHOLD).  In effect,
1611
        the top chunk is treated as larger (and thus less well
1612
        fitting) than any other available chunk.  The top chunk
1613
        doesn't update its trailing size field since there is no next
1614
        contiguous chunk that would have to index off it. However,
1615
        space is still allocated for it (TOP_FOOT_SIZE) to enable
1616
        separation or merging when space is extended.
1617
 
1618
     3. Chunks allocated via mmap, which have the lowest-order bit
1619
        (IS_MMAPPED_BIT) set in their prev_foot fields, and do not set
1620
        PINUSE_BIT in their head fields.  Because they are allocated
1621
        one-by-one, each must carry its own prev_foot field, which is
1622
        also used to hold the offset this chunk has within its mmapped
1623
        region, which is needed to preserve alignment. Each mmapped
1624
        chunk is trailed by the first two fields of a fake next-chunk
1625
        for sake of usage checks.
1626
 
26855 ripley 1627
*/
1628
 
40788 ripley 1629
struct malloc_chunk {
1630
  size_t               prev_foot;  /* Size of previous chunk (if free).  */
1631
  size_t               head;       /* Size and inuse bits. */
1632
  struct malloc_chunk* fd;         /* double links -- used only if free. */
1633
  struct malloc_chunk* bk;
1634
};
26855 ripley 1635
 
40788 ripley 1636
typedef struct malloc_chunk  mchunk;
1637
typedef struct malloc_chunk* mchunkptr;
1638
typedef struct malloc_chunk* sbinptr;  /* The type of bins of chunks */
1639
typedef unsigned int bindex_t;         /* Described below */
1640
typedef unsigned int binmap_t;         /* Described below */
1641
typedef unsigned int flag_t;           /* The type of various bit flag sets */
26855 ripley 1642
 
40788 ripley 1643
/* ------------------- Chunks sizes and alignments ----------------------- */
26855 ripley 1644
 
40788 ripley 1645
#define MCHUNK_SIZE         (sizeof(mchunk))
26855 ripley 1646
 
40788 ripley 1647
#if FOOTERS
1648
#define CHUNK_OVERHEAD      (TWO_SIZE_T_SIZES)
1649
#else /* FOOTERS */
1650
#define CHUNK_OVERHEAD      (SIZE_T_SIZE)
1651
#endif /* FOOTERS */
26855 ripley 1652
 
40788 ripley 1653
/* MMapped chunks need a second word of overhead ... */
1654
#define MMAP_CHUNK_OVERHEAD (TWO_SIZE_T_SIZES)
1655
/* ... and additional padding for fake next-chunk at foot */
1656
#define MMAP_FOOT_PAD       (FOUR_SIZE_T_SIZES)
26855 ripley 1657
 
40788 ripley 1658
/* The smallest size we can malloc is an aligned minimal chunk */
1659
#define MIN_CHUNK_SIZE\
1660
  ((MCHUNK_SIZE + CHUNK_ALIGN_MASK) & ~CHUNK_ALIGN_MASK)
26855 ripley 1661
 
40788 ripley 1662
/* conversion from malloc headers to user pointers, and back */
1663
#define chunk2mem(p)        ((void*)((char*)(p)       + TWO_SIZE_T_SIZES))
1664
#define mem2chunk(mem)      ((mchunkptr)((char*)(mem) - TWO_SIZE_T_SIZES))
1665
/* chunk associated with aligned address A */
1666
#define align_as_chunk(A)   (mchunkptr)((A) + align_offset(chunk2mem(A)))
26855 ripley 1667
 
40788 ripley 1668
/* Bounds on request (not chunk) sizes. */
1669
#define MAX_REQUEST         ((-MIN_CHUNK_SIZE) << 2)
1670
#define MIN_REQUEST         (MIN_CHUNK_SIZE - CHUNK_OVERHEAD - SIZE_T_ONE)
26855 ripley 1671
 
40788 ripley 1672
/* pad request bytes into a usable size */
1673
#define pad_request(req) \
1674
   (((req) + CHUNK_OVERHEAD + CHUNK_ALIGN_MASK) & ~CHUNK_ALIGN_MASK)
10347 ripley 1675
 
40788 ripley 1676
/* pad request, checking for minimum (but not maximum) */
1677
#define request2size(req) \
1678
  (((req) < MIN_REQUEST)? MIN_CHUNK_SIZE : pad_request(req))
10347 ripley 1679
 
40788 ripley 1680
 
1681
/* ------------------ Operations on head and foot fields ----------------- */
1682
 
26855 ripley 1683
/*
40788 ripley 1684
  The head field of a chunk is or'ed with PINUSE_BIT when previous
1685
  adjacent chunk in use, and or'ed with CINUSE_BIT if this chunk is in
1686
  use. If the chunk was obtained with mmap, the prev_foot field has
1687
  IS_MMAPPED_BIT set, otherwise holding the offset of the base of the
1688
  mmapped region to the base of the chunk.
26855 ripley 1689
*/
1690
 
40788 ripley 1691
#define PINUSE_BIT          (SIZE_T_ONE)
1692
#define CINUSE_BIT          (SIZE_T_TWO)
1693
#define INUSE_BITS          (PINUSE_BIT|CINUSE_BIT)
26855 ripley 1694
 
40788 ripley 1695
/* Head value for fenceposts */
1696
#define FENCEPOST_HEAD      (INUSE_BITS|SIZE_T_SIZE)
26855 ripley 1697
 
40788 ripley 1698
/* extraction of fields from head words */
1699
#define cinuse(p)           ((p)->head & CINUSE_BIT)
1700
#define pinuse(p)           ((p)->head & PINUSE_BIT)
1701
#define chunksize(p)        ((p)->head & ~(INUSE_BITS))
10347 ripley 1702
 
40788 ripley 1703
#define clear_pinuse(p)     ((p)->head &= ~PINUSE_BIT)
1704
#define clear_cinuse(p)     ((p)->head &= ~CINUSE_BIT)
26855 ripley 1705
 
40788 ripley 1706
/* Treat space at ptr +/- offset as a chunk */
1707
#define chunk_plus_offset(p, s)  ((mchunkptr)(((char*)(p)) + (s)))
1708
#define chunk_minus_offset(p, s) ((mchunkptr)(((char*)(p)) - (s)))
10347 ripley 1709
 
40788 ripley 1710
/* Ptr to next or previous physical malloc_chunk. */
1711
#define next_chunk(p) ((mchunkptr)( ((char*)(p)) + ((p)->head & ~INUSE_BITS)))
1712
#define prev_chunk(p) ((mchunkptr)( ((char*)(p)) - ((p)->prev_foot) ))
10347 ripley 1713
 
40788 ripley 1714
/* extract next chunk's pinuse bit */
1715
#define next_pinuse(p)  ((next_chunk(p)->head) & PINUSE_BIT)
10347 ripley 1716
 
40788 ripley 1717
/* Get/set size at footer */
1718
#define get_foot(p, s)  (((mchunkptr)((char*)(p) + (s)))->prev_foot)
1719
#define set_foot(p, s)  (((mchunkptr)((char*)(p) + (s)))->prev_foot = (s))
10347 ripley 1720
 
40788 ripley 1721
/* Set size, pinuse bit, and foot */
1722
#define set_size_and_pinuse_of_free_chunk(p, s)\
1723
  ((p)->head = (s|PINUSE_BIT), set_foot(p, s))
10347 ripley 1724
 
40788 ripley 1725
/* Set size, pinuse bit, foot, and clear next pinuse */
1726
#define set_free_with_pinuse(p, s, n)\
1727
  (clear_pinuse(n), set_size_and_pinuse_of_free_chunk(p, s))
10347 ripley 1728
 
40788 ripley 1729
#define is_mmapped(p)\
1730
  (!((p)->head & PINUSE_BIT) && ((p)->prev_foot & IS_MMAPPED_BIT))
10347 ripley 1731
 
40788 ripley 1732
/* Get the internal overhead associated with chunk p */
1733
#define overhead_for(p)\
1734
 (is_mmapped(p)? MMAP_CHUNK_OVERHEAD : CHUNK_OVERHEAD)
10347 ripley 1735
 
40788 ripley 1736
/* Return true if malloced space is not necessarily cleared */
1737
#if MMAP_CLEARS
1738
#define calloc_must_clear(p) (!is_mmapped(p))
1739
#else /* MMAP_CLEARS */
1740
#define calloc_must_clear(p) (1)
1741
#endif /* MMAP_CLEARS */
10347 ripley 1742
 
40788 ripley 1743
/* ---------------------- Overlaid data structures ----------------------- */
10347 ripley 1744
 
40788 ripley 1745
/*
1746
  When chunks are not in use, they are treated as nodes of either
1747
  lists or trees.
10347 ripley 1748
 
40788 ripley 1749
  "Small"  chunks are stored in circular doubly-linked lists, and look
1750
  like this:
1751
 
10347 ripley 1752
    chunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1753
            |             Size of previous chunk                            |
1754
            +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1755
    `head:' |             Size of chunk, in bytes                         |P|
1756
      mem-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1757
            |             Forward pointer to next chunk in list             |
1758
            +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1759
            |             Back pointer to previous chunk in list            |
1760
            +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1761
            |             Unused space (may be 0 bytes long)                .
1762
            .                                                               .
1763
            .                                                               |
1764
nextchunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1765
    `foot:' |             Size of chunk, in bytes                           |
1766
            +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1767
 
40788 ripley 1768
  Larger chunks are kept in a form of bitwise digital trees (aka
1769
  tries) keyed on chunksizes.  Because malloc_tree_chunks are only for
1770
  free chunks greater than 256 bytes, their size doesn't impose any
1771
  constraints on user chunk sizes.  Each node looks like:
10347 ripley 1772
 
40788 ripley 1773
    chunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1774
            |             Size of previous chunk                            |
1775
            +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1776
    `head:' |             Size of chunk, in bytes                         |P|
1777
      mem-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1778
            |             Forward pointer to next chunk of same size        |
1779
            +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1780
            |             Back pointer to previous chunk of same size       |
1781
            +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1782
            |             Pointer to left child (child[0])                  |
1783
            +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1784
            |             Pointer to right child (child[1])                 |
1785
            +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1786
            |             Pointer to parent                                 |
1787
            +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1788
            |             bin index of this chunk                           |
1789
            +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1790
            |             Unused space                                      .
1791
            .                                                               |
1792
nextchunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1793
    `foot:' |             Size of chunk, in bytes                           |
1794
            +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
10347 ripley 1795
 
40788 ripley 1796
  Each tree holding treenodes is a tree of unique chunk sizes.  Chunks
1797
  of the same size are arranged in a circularly-linked list, with only
1798
  the oldest chunk (the next to be used, in our FIFO ordering)
1799
  actually in the tree.  (Tree members are distinguished by a non-null
1800
  parent pointer.)  If a chunk with the same size an an existing node
1801
  is inserted, it is linked off the existing node using pointers that
1802
  work in the same way as fd/bk pointers of small chunks.
10347 ripley 1803
 
40788 ripley 1804
  Each tree contains a power of 2 sized range of chunk sizes (the
1805
  smallest is 0x100 <= x < 0x180), which is is divided in half at each
1806
  tree level, with the chunks in the smaller half of the range (0x100
1807
  <= x < 0x140 for the top nose) in the left subtree and the larger
1808
  half (0x140 <= x < 0x180) in the right subtree.  This is, of course,
1809
  done by inspecting individual bits.
10347 ripley 1810
 
40788 ripley 1811
  Using these rules, each node's left subtree contains all smaller
1812
  sizes than its right subtree.  However, the node at the root of each
1813
  subtree has no particular ordering relationship to either.  (The
1814
  dividing line between the subtree sizes is based on trie relation.)
1815
  If we remove the last chunk of a given size from the interior of the
1816
  tree, we need to replace it with a leaf node.  The tree ordering
1817
  rules permit a node to be replaced by any leaf below it.
10347 ripley 1818
 
40788 ripley 1819
  The smallest chunk in a tree (a common operation in a best-fit
1820
  allocator) can be found by walking a path to the leftmost leaf in
1821
  the tree.  Unlike a usual binary tree, where we follow left child
1822
  pointers until we reach a null, here we follow the right child
1823
  pointer any time the left one is null, until we reach a leaf with
1824
  both child pointers null. The smallest chunk in the tree will be
1825
  somewhere along that path.
10347 ripley 1826
 
40788 ripley 1827
  The worst case number of steps to add, find, or remove a node is
1828
  bounded by the number of bits differentiating chunks within
1829
  bins. Under current bin calculations, this ranges from 6 up to 21
1830
  (for 32 bit sizes) or up to 53 (for 64 bit sizes). The typical case
1831
  is of course much better.
10347 ripley 1832
*/
1833
 
40788 ripley 1834
struct malloc_tree_chunk {
1835
  /* The first four fields must be compatible with malloc_chunk */
1836
  size_t                    prev_foot;
1837
  size_t                    head;
1838
  struct malloc_tree_chunk* fd;
1839
  struct malloc_tree_chunk* bk;
10347 ripley 1840
 
40788 ripley 1841
  struct malloc_tree_chunk* child[2];
1842
  struct malloc_tree_chunk* parent;
1843
  bindex_t                  index;
1844
};
10347 ripley 1845
 
40788 ripley 1846
typedef struct malloc_tree_chunk  tchunk;
1847
typedef struct malloc_tree_chunk* tchunkptr;
1848
typedef struct malloc_tree_chunk* tbinptr; /* The type of bins of trees */
10347 ripley 1849
 
40788 ripley 1850
/* A little helper macro for trees */
1851
#define leftmost_child(t) ((t)->child[0] != 0? (t)->child[0] : (t)->child[1])
10347 ripley 1852
 
40788 ripley 1853
/* ----------------------------- Segments -------------------------------- */
10347 ripley 1854
 
40788 ripley 1855
/*
1856
  Each malloc space may include non-contiguous segments, held in a
1857
  list headed by an embedded malloc_segment record representing the
1858
  top-most space. Segments also include flags holding properties of
1859
  the space. Large chunks that are directly allocated by mmap are not
1860
  included in this list. They are instead independently created and
1861
  destroyed without otherwise keeping track of them.
10347 ripley 1862
 
40788 ripley 1863
  Segment management mainly comes into play for spaces allocated by
1864
  MMAP.  Any call to MMAP might or might not return memory that is
1865
  adjacent to an existing segment.  MORECORE normally contiguously
1866
  extends the current space, so this space is almost always adjacent,
1867
  which is simpler and faster to deal with. (This is why MORECORE is
1868
  used preferentially to MMAP when both are available -- see
1869
  sys_alloc.)  When allocating using MMAP, we don't use any of the
1870
  hinting mechanisms (inconsistently) supported in various
1871
  implementations of unix mmap, or distinguish reserving from
1872
  committing memory. Instead, we just ask for space, and exploit
1873
  contiguity when we get it.  It is probably possible to do
1874
  better than this on some systems, but no general scheme seems
1875
  to be significantly better.
10347 ripley 1876
 
40788 ripley 1877
  Management entails a simpler variant of the consolidation scheme
1878
  used for chunks to reduce fragmentation -- new adjacent memory is
1879
  normally prepended or appended to an existing segment. However,
1880
  there are limitations compared to chunk consolidation that mostly
1881
  reflect the fact that segment processing is relatively infrequent
1882
  (occurring only when getting memory from system) and that we
1883
  don't expect to have huge numbers of segments:
10347 ripley 1884
 
40788 ripley 1885
  * Segments are not indexed, so traversal requires linear scans.  (It
1886
    would be possible to index these, but is not worth the extra
1887
    overhead and complexity for most programs on most platforms.)
1888
  * New segments are only appended to old ones when holding top-most
1889
    memory; if they cannot be prepended to others, they are held in
1890
    different segments.
1891
 
1892
  Except for the top-most segment of an mstate, each segment record
1893
  is kept at the tail of its segment. Segments are added by pushing
1894
  segment records onto the list headed by &mstate.seg for the
1895
  containing mstate.
1896
 
1897
  Segment flags control allocation/merge/deallocation policies:
1898
  * If EXTERN_BIT set, then we did not allocate this segment,
1899
    and so should not try to deallocate or merge with others.
1900
    (This currently holds only for the initial segment passed
1901
    into create_mspace_with_base.)
1902
  * If IS_MMAPPED_BIT set, the segment may be merged with
1903
    other surrounding mmapped segments and trimmed/de-allocated
1904
    using munmap.
1905
  * If neither bit is set, then the segment was obtained using
1906
    MORECORE so can be merged with surrounding MORECORE'd segments
1907
    and deallocated/trimmed using MORECORE with negative arguments.
26855 ripley 1908
*/
10347 ripley 1909
 
40788 ripley 1910
struct malloc_segment {
1911
  char*        base;             /* base address */
1912
  size_t       size;             /* allocated size */
1913
  struct malloc_segment* next;   /* ptr to next segment */
1914
  flag_t       sflags;           /* mmap and extern flag */
1915
};
10347 ripley 1916
 
40788 ripley 1917
#define is_mmapped_segment(S)  ((S)->sflags & IS_MMAPPED_BIT)
1918
#define is_extern_segment(S)   ((S)->sflags & EXTERN_BIT)
10347 ripley 1919
 
40788 ripley 1920
typedef struct malloc_segment  msegment;
1921
typedef struct malloc_segment* msegmentptr;
10347 ripley 1922
 
40788 ripley 1923
/* ---------------------------- malloc_state ----------------------------- */
10347 ripley 1924
 
1925
/*
40788 ripley 1926
   A malloc_state holds all of the bookkeeping for a space.
1927
   The main fields are:
10347 ripley 1928
 
40788 ripley 1929
  Top
1930
    The topmost chunk of the currently active segment. Its size is
1931
    cached in topsize.  The actual size of topmost space is
1932
    topsize+TOP_FOOT_SIZE, which includes space reserved for adding
1933
    fenceposts and segment records if necessary when getting more
1934
    space from the system.  The size at which to autotrim top is
1935
    cached from mparams in trim_check, except that it is disabled if
1936
    an autotrim fails.
10347 ripley 1937
 
40788 ripley 1938
  Designated victim (dv)
1939
    This is the preferred chunk for servicing small requests that
1940
    don't have exact fits.  It is normally the chunk split off most
1941
    recently to service another small request.  Its size is cached in
1942
    dvsize. The link fields of this chunk are not maintained since it
1943
    is not kept in a bin.
10347 ripley 1944
 
40788 ripley 1945
  SmallBins
1946
    An array of bin headers for free chunks.  These bins hold chunks
1947
    with sizes less than MIN_LARGE_SIZE bytes. Each bin contains
1948
    chunks of all the same size, spaced 8 bytes apart.  To simplify
1949
    use in double-linked lists, each bin header acts as a malloc_chunk
1950
    pointing to the real first node, if it exists (else pointing to
1951
    itself).  This avoids special-casing for headers.  But to avoid
1952
    waste, we allocate only the fd/bk pointers of bins, and then use
1953
    repositioning tricks to treat these as the fields of a chunk.
26855 ripley 1954
 
40788 ripley 1955
  TreeBins
1956
    Treebins are pointers to the roots of trees holding a range of
1957
    sizes. There are 2 equally spaced treebins for each power of two
1958
    from TREE_SHIFT to TREE_SHIFT+16. The last bin holds anything
1959
    larger.
26855 ripley 1960
 
40788 ripley 1961
  Bin maps
1962
    There is one bit map for small bins ("smallmap") and one for
1963
    treebins ("treemap).  Each bin sets its bit when non-empty, and
1964
    clears the bit when empty.  Bit operations are then used to avoid
1965
    bin-by-bin searching -- nearly all "search" is done without ever
1966
    looking at bins that won't be selected.  The bit maps
1967
    conservatively use 32 bits per map word, even if on 64bit system.
1968
    For a good description of some of the bit-based techniques used
1969
    here, see Henry S. Warren Jr's book "Hacker's Delight" (and
1970
    supplement at http://hackersdelight.org/). Many of these are
1971
    intended to reduce the branchiness of paths through malloc etc, as
1972
    well as to reduce the number of memory locations read or written.
10347 ripley 1973
 
40788 ripley 1974
  Segments
1975
    A list of segments headed by an embedded malloc_segment record
1976
    representing the initial space.
10347 ripley 1977
 
40788 ripley 1978
  Address check support
1979
    The least_addr field is the least address ever obtained from
1980
    MORECORE or MMAP. Attempted frees and reallocs of any address less
1981
    than this are trapped (unless INSECURE is defined).
26855 ripley 1982
 
40788 ripley 1983
  Magic tag
1984
    A cross-check field that should always hold same value as mparams.magic.
10347 ripley 1985
 
40788 ripley 1986
  Flags
1987
    Bits recording whether to use MMAP, locks, or contiguous MORECORE
10347 ripley 1988
 
40788 ripley 1989
  Statistics
1990
    Each space keeps track of current and maximum system memory
1991
    obtained via MORECORE or MMAP.
26855 ripley 1992
 
40788 ripley 1993
  Locking
1994
    If USE_LOCKS is defined, the "mutex" lock is acquired and released
1995
    around every public call using this mspace.
1996
*/
10347 ripley 1997
 
40788 ripley 1998
/* Bin types, widths and sizes */
1999
#define NSMALLBINS        (32U)
2000
#define NTREEBINS         (32U)
2001
#define SMALLBIN_SHIFT    (3U)
2002
#define SMALLBIN_WIDTH    (SIZE_T_ONE << SMALLBIN_SHIFT)
2003
#define TREEBIN_SHIFT     (8U)
2004
#define MIN_LARGE_SIZE    (SIZE_T_ONE << TREEBIN_SHIFT)
2005
#define MAX_SMALL_SIZE    (MIN_LARGE_SIZE - SIZE_T_ONE)
2006
#define MAX_SMALL_REQUEST (MAX_SMALL_SIZE - CHUNK_ALIGN_MASK - CHUNK_OVERHEAD)
10347 ripley 2007
 
40788 ripley 2008
struct malloc_state {
2009
  binmap_t   smallmap;
2010
  binmap_t   treemap;
2011
  size_t     dvsize;
2012
  size_t     topsize;
2013
  char*      least_addr;
2014
  mchunkptr  dv;
2015
  mchunkptr  top;
2016
  size_t     trim_check;
2017
  size_t     magic;
2018
  mchunkptr  smallbins[(NSMALLBINS+1)*2];
2019
  tbinptr    treebins[NTREEBINS];
2020
  size_t     footprint;
2021
  size_t     max_footprint;
2022
  flag_t     mflags;
2023
#if USE_LOCKS
2024
  MLOCK_T    mutex;     /* locate lock among fields that rarely change */
2025
#endif /* USE_LOCKS */
2026
  msegment   seg;
2027
};
10347 ripley 2028
 
40788 ripley 2029
typedef struct malloc_state*    mstate;
10347 ripley 2030
 
40788 ripley 2031
/* ------------- Global malloc_state and malloc_params ------------------- */
10347 ripley 2032
 
40788 ripley 2033
/*
2034
  malloc_params holds global properties, including those that can be
2035
  dynamically set using mallopt. There is a single instance, mparams,
2036
  initialized in init_mparams.
2037
*/
10347 ripley 2038
 
40788 ripley 2039
struct malloc_params {
2040
  size_t magic;
2041
  size_t page_size;
2042
  size_t granularity;
2043
  size_t mmap_threshold;
2044
  size_t trim_threshold;
2045
  flag_t default_mflags;
2046
};
26855 ripley 2047
 
40788 ripley 2048
static struct malloc_params mparams;
10347 ripley 2049
 
40788 ripley 2050
/* The global malloc_state used for all non-"mspace" calls */
2051
static struct malloc_state _gm_;
2052
#define gm                 (&_gm_)
2053
#define is_global(M)       ((M) == &_gm_)
2054
#define is_initialized(M)  ((M)->top != 0)
10347 ripley 2055
 
40788 ripley 2056
/* -------------------------- system alloc setup ------------------------- */
10347 ripley 2057
 
40788 ripley 2058
/* Operations on mflags */
10347 ripley 2059
 
40788 ripley 2060
#define use_lock(M)           ((M)->mflags &   USE_LOCK_BIT)
2061
#define enable_lock(M)        ((M)->mflags |=  USE_LOCK_BIT)
2062
#define disable_lock(M)       ((M)->mflags &= ~USE_LOCK_BIT)
10347 ripley 2063
 
40788 ripley 2064
#define use_mmap(M)           ((M)->mflags &   USE_MMAP_BIT)
2065
#define enable_mmap(M)        ((M)->mflags |=  USE_MMAP_BIT)
2066
#define disable_mmap(M)       ((M)->mflags &= ~USE_MMAP_BIT)
26855 ripley 2067
 
40788 ripley 2068
#define use_noncontiguous(M)  ((M)->mflags &   USE_NONCONTIGUOUS_BIT)
2069
#define disable_contiguous(M) ((M)->mflags |=  USE_NONCONTIGUOUS_BIT)
26855 ripley 2070
 
40788 ripley 2071
#define set_lock(M,L)\
2072
 ((M)->mflags = (L)?\
2073
  ((M)->mflags | USE_LOCK_BIT) :\
2074
  ((M)->mflags & ~USE_LOCK_BIT))
26855 ripley 2075
 
40788 ripley 2076
/* page-align a size */
2077
#define page_align(S)\
2078
 (((S) + (mparams.page_size)) & ~(mparams.page_size - SIZE_T_ONE))
26855 ripley 2079
 
40788 ripley 2080
/* granularity-align a size */
2081
#define granularity_align(S)\
2082
  (((S) + (mparams.granularity)) & ~(mparams.granularity - SIZE_T_ONE))
26855 ripley 2083
 
40788 ripley 2084
#define is_page_aligned(S)\
2085
   (((size_t)(S) & (mparams.page_size - SIZE_T_ONE)) == 0)
2086
#define is_granularity_aligned(S)\
2087
   (((size_t)(S) & (mparams.granularity - SIZE_T_ONE)) == 0)
10347 ripley 2088
 
40788 ripley 2089
/*  True if segment S holds address A */
2090
#define segment_holds(S, A)\
2091
  ((char*)(A) >= S->base && (char*)(A) < S->base + S->size)
10347 ripley 2092
 
40788 ripley 2093
/* Return segment holding given address */
2094
static msegmentptr segment_holding(mstate m, char* addr) {
2095
  msegmentptr sp = &m->seg;
2096
  for (;;) {
2097
    if (addr >= sp->base && addr < sp->base + sp->size)
2098
      return sp;
2099
    if ((sp = sp->next) == 0)
2100
      return 0;
2101
  }
2102
}
10347 ripley 2103
 
40788 ripley 2104
/* Return true if segment contains a segment link */
2105
static int has_segment_link(mstate m, msegmentptr ss) {
2106
  msegmentptr sp = &m->seg;
2107
  for (;;) {
2108
    if ((char*)sp >= ss->base && (char*)sp < ss->base + ss->size)
2109
      return 1;
2110
    if ((sp = sp->next) == 0)
2111
      return 0;
2112
  }
2113
}
10347 ripley 2114
 
40788 ripley 2115
#ifndef MORECORE_CANNOT_TRIM
2116
#define should_trim(M,s)  ((s) > (M)->trim_check)
2117
#else  /* MORECORE_CANNOT_TRIM */
2118
#define should_trim(M,s)  (0)
2119
#endif /* MORECORE_CANNOT_TRIM */
10347 ripley 2120
 
40788 ripley 2121
/*
2122
  TOP_FOOT_SIZE is padding at the end of a segment, including space
2123
  that may be needed to place segment records and fenceposts when new
2124
  noncontiguous segments are added.
26855 ripley 2125
*/
40788 ripley 2126
#define TOP_FOOT_SIZE\
2127
  (align_offset(chunk2mem(0))+pad_request(sizeof(struct malloc_segment))+MIN_CHUNK_SIZE)
10347 ripley 2128
 
2129
 
40788 ripley 2130
/* -------------------------------  Hooks -------------------------------- */
10347 ripley 2131
 
2132
/*
40788 ripley 2133
  PREACTION should be defined to return 0 on success, and nonzero on
2134
  failure. If you are not using locking, you can redefine these to do
2135
  anything you like.
2136
*/
10347 ripley 2137
 
40788 ripley 2138
#if USE_LOCKS
10347 ripley 2139
 
40788 ripley 2140
/* Ensure locks are initialized */
2141
#define GLOBALLY_INITIALIZE() (mparams.page_size == 0 && init_mparams())
10347 ripley 2142
 
40788 ripley 2143
#define PREACTION(M)  ((GLOBALLY_INITIALIZE() || use_lock(M))? ACQUIRE_LOCK(&(M)->mutex) : 0)
2144
#define POSTACTION(M) { if (use_lock(M)) RELEASE_LOCK(&(M)->mutex); }
2145
#else /* USE_LOCKS */
10347 ripley 2146
 
40788 ripley 2147
#ifndef PREACTION
2148
#define PREACTION(M) (0)
2149
#endif  /* PREACTION */
10347 ripley 2150
 
40788 ripley 2151
#ifndef POSTACTION
2152
#define POSTACTION(M)
2153
#endif  /* POSTACTION */
26855 ripley 2154
 
40788 ripley 2155
#endif /* USE_LOCKS */
26855 ripley 2156
 
2157
/*
40788 ripley 2158
  CORRUPTION_ERROR_ACTION is triggered upon detected bad addresses.
2159
  USAGE_ERROR_ACTION is triggered on detected bad frees and
2160
  reallocs. The argument p is an address that might have triggered the
2161
  fault. It is ignored by the two predefined actions, but might be
2162
  useful in custom actions that try to help diagnose errors.
10347 ripley 2163
*/
2164
 
40788 ripley 2165
#if PROCEED_ON_ERROR
10347 ripley 2166
 
40788 ripley 2167
/* A count of the number of corruption errors causing resets */
2168
int malloc_corruption_error_count;
10347 ripley 2169
 
40788 ripley 2170
/* default corruption action */
2171
static void reset_on_error(mstate m);
10347 ripley 2172
 
40788 ripley 2173
#define CORRUPTION_ERROR_ACTION(m)  reset_on_error(m)
2174
#define USAGE_ERROR_ACTION(m, p)
10347 ripley 2175
 
40788 ripley 2176
#else /* PROCEED_ON_ERROR */
26855 ripley 2177
 
40788 ripley 2178
#ifndef CORRUPTION_ERROR_ACTION
2179
#define CORRUPTION_ERROR_ACTION(m) ABORT
2180
#endif /* CORRUPTION_ERROR_ACTION */
26855 ripley 2181
 
40788 ripley 2182
#ifndef USAGE_ERROR_ACTION
2183
#define USAGE_ERROR_ACTION(m,p) ABORT
2184
#endif /* USAGE_ERROR_ACTION */
26855 ripley 2185
 
40788 ripley 2186
#endif /* PROCEED_ON_ERROR */
26855 ripley 2187
 
40788 ripley 2188
/* -------------------------- Debugging setup ---------------------------- */
26855 ripley 2189
 
40788 ripley 2190
#if ! DEBUG
10347 ripley 2191
 
40788 ripley 2192
#define check_free_chunk(M,P)
2193
#define check_inuse_chunk(M,P)
2194
#define check_malloced_chunk(M,P,N)
2195
#define check_mmapped_chunk(M,P)
2196
#define check_malloc_state(M)
2197
#define check_top_chunk(M,P)
10347 ripley 2198
 
40788 ripley 2199
#else /* DEBUG */
2200
#define check_free_chunk(M,P)       do_check_free_chunk(M,P)
2201
#define check_inuse_chunk(M,P)      do_check_inuse_chunk(M,P)
2202
#define check_top_chunk(M,P)        do_check_top_chunk(M,P)
2203
#define check_malloced_chunk(M,P,N) do_check_malloced_chunk(M,P,N)
2204
#define check_mmapped_chunk(M,P)    do_check_mmapped_chunk(M,P)
2205
#define check_malloc_state(M)       do_check_malloc_state(M)
10347 ripley 2206
 
40788 ripley 2207
static void   do_check_any_chunk(mstate m, mchunkptr p);
2208
static void   do_check_top_chunk(mstate m, mchunkptr p);
2209
static void   do_check_mmapped_chunk(mstate m, mchunkptr p);
2210
static void   do_check_inuse_chunk(mstate m, mchunkptr p);
2211
static void   do_check_free_chunk(mstate m, mchunkptr p);
2212
static void   do_check_malloced_chunk(mstate m, void* mem, size_t s);
2213
static void   do_check_tree(mstate m, tchunkptr t);
2214
static void   do_check_treebin(mstate m, bindex_t i);
2215
static void   do_check_smallbin(mstate m, bindex_t i);
2216
static void   do_check_malloc_state(mstate m);
2217
static int    bin_find(mstate m, mchunkptr x);
2218
static size_t traverse_and_check(mstate m);
2219
#endif /* DEBUG */
26855 ripley 2220
 
40788 ripley 2221
/* ---------------------------- Indexing Bins ---------------------------- */
26855 ripley 2222
 
40788 ripley 2223
#define is_small(s)         (((s) >> SMALLBIN_SHIFT) < NSMALLBINS)
2224
#define small_index(s)      ((s)  >> SMALLBIN_SHIFT)
2225
#define small_index2size(i) ((i)  << SMALLBIN_SHIFT)
2226
#define MIN_SMALL_INDEX     (small_index(MIN_CHUNK_SIZE))
26855 ripley 2227
 
40788 ripley 2228
/* addressing by index. See above about smallbin repositioning */
2229
#define smallbin_at(M, i)   ((sbinptr)((char*)&((M)->smallbins[(i)<<1])))
2230
#define treebin_at(M,i)     (&((M)->treebins[i]))
10347 ripley 2231
 
40788 ripley 2232
/* assign tree index for size S to variable I */
2233
#if defined(__GNUC__) && defined(i386)
2234
#define compute_tree_index(S, I)\
2235
{\
2236
  size_t X = S >> TREEBIN_SHIFT;\
2237
  if (X == 0)\
2238
    I = 0;\
2239
  else if (X > 0xFFFF)\
2240
    I = NTREEBINS-1;\
2241
  else {\
2242
    unsigned int K;\
2243
    __asm__("bsrl %1,%0\n\t" : "=r" (K) : "rm"  (X));\
2244
    I =  (bindex_t)((K << 1) + ((S >> (K + (TREEBIN_SHIFT-1)) & 1)));\
2245
  }\
2246
}
2247
#else /* GNUC */
2248
#define compute_tree_index(S, I)\
2249
{\
2250
  size_t X = S >> TREEBIN_SHIFT;\
2251
  if (X == 0)\
2252
    I = 0;\
2253
  else if (X > 0xFFFF)\
2254
    I = NTREEBINS-1;\
2255
  else {\
2256
    unsigned int Y = (unsigned int)X;\
2257
    unsigned int N = ((Y - 0x100) >> 16) & 8;\
2258
    unsigned int K = (((Y <<= N) - 0x1000) >> 16) & 4;\
2259
    N += K;\
2260
    N += K = (((Y <<= K) - 0x4000) >> 16) & 2;\
2261
    K = 14 - N + ((Y <<= K) >> 15);\
2262
    I = (K << 1) + ((S >> (K + (TREEBIN_SHIFT-1)) & 1));\
2263
  }\
2264
}
2265
#endif /* GNUC */
10347 ripley 2266
 
40788 ripley 2267
/* Bit representing maximum resolved size in a treebin at i */
2268
#define bit_for_tree_index(i) \
2269
   (i == NTREEBINS-1)? (SIZE_T_BITSIZE-1) : (((i) >> 1) + TREEBIN_SHIFT - 2)
10347 ripley 2270
 
40788 ripley 2271
/* Shift placing maximum resolved bit in a treebin at i as sign bit */
2272
#define leftshift_for_tree_index(i) \
2273
   ((i == NTREEBINS-1)? 0 : \
2274
    ((SIZE_T_BITSIZE-SIZE_T_ONE) - (((i) >> 1) + TREEBIN_SHIFT - 2)))
10347 ripley 2275
 
40788 ripley 2276
/* The size of the smallest chunk held in bin with index i */
2277
#define minsize_for_tree_index(i) \
2278
   ((SIZE_T_ONE << (((i) >> 1) + TREEBIN_SHIFT)) |  \
2279
   (((size_t)((i) & SIZE_T_ONE)) << (((i) >> 1) + TREEBIN_SHIFT - 1)))
10347 ripley 2280
 
2281
 
40788 ripley 2282
/* ------------------------ Operations on bin maps ----------------------- */
10347 ripley 2283
 
40788 ripley 2284
/* bit corresponding to given index */
2285
#define idx2bit(i)              ((binmap_t)(1) << (i))
10347 ripley 2286
 
40788 ripley 2287
/* Mark/Clear bits with given index */
2288
#define mark_smallmap(M,i)      ((M)->smallmap |=  idx2bit(i))
2289
#define clear_smallmap(M,i)     ((M)->smallmap &= ~idx2bit(i))
2290
#define smallmap_is_marked(M,i) ((M)->smallmap &   idx2bit(i))
26855 ripley 2291
 
40788 ripley 2292
#define mark_treemap(M,i)       ((M)->treemap  |=  idx2bit(i))
2293
#define clear_treemap(M,i)      ((M)->treemap  &= ~idx2bit(i))
2294
#define treemap_is_marked(M,i)  ((M)->treemap  &   idx2bit(i))
26855 ripley 2295
 
40788 ripley 2296
/* index corresponding to given bit */
26855 ripley 2297
 
40788 ripley 2298
#if defined(__GNUC__) && defined(i386)
2299
#define compute_bit2idx(X, I)\
2300
{\
2301
  unsigned int J;\
2302
  __asm__("bsfl %1,%0\n\t" : "=r" (J) : "rm" (X));\
2303
  I = (bindex_t)J;\
2304
}
26855 ripley 2305
 
40788 ripley 2306
#else /* GNUC */
2307
#if  USE_BUILTIN_FFS
2308
#define compute_bit2idx(X, I) I = ffs(X)-1
26855 ripley 2309
 
40788 ripley 2310
#else /* USE_BUILTIN_FFS */
2311
#define compute_bit2idx(X, I)\
2312
{\
2313
  unsigned int Y = X - 1;\
2314
  unsigned int K = Y >> (16-4) & 16;\
2315
  unsigned int N = K;        Y >>= K;\
2316
  N += K = Y >> (8-3) &  8;  Y >>= K;\
2317
  N += K = Y >> (4-2) &  4;  Y >>= K;\
2318
  N += K = Y >> (2-1) &  2;  Y >>= K;\
2319
  N += K = Y >> (1-0) &  1;  Y >>= K;\
2320
  I = (bindex_t)(N + Y);\
2321
}
2322
#endif /* USE_BUILTIN_FFS */
2323
#endif /* GNUC */
26855 ripley 2324
 
40788 ripley 2325
/* isolate the least set bit of a bitmap */
2326
#define least_bit(x)         ((x) & -(x))
10347 ripley 2327
 
40788 ripley 2328
/* mask with all bits to left of least bit of x on */
2329
#define left_bits(x)         ((x<<1) | -(x<<1))
26855 ripley 2330
 
40788 ripley 2331
/* mask with all bits to left of or equal to least bit of x on */
2332
#define same_or_left_bits(x) ((x) | -(x))
10347 ripley 2333
 
2334
 
40788 ripley 2335
/* ----------------------- Runtime Check Support ------------------------- */
10347 ripley 2336
 
2337
/*
40788 ripley 2338
  For security, the main invariant is that malloc/free/etc never
2339
  writes to a static address other than malloc_state, unless static
2340
  malloc_state itself has been corrupted, which cannot occur via
2341
  malloc (because of these checks). In essence this means that we
2342
  believe all pointers, sizes, maps etc held in malloc_state, but
2343
  check all of those linked or offsetted from other embedded data
2344
  structures.  These checks are interspersed with main code in a way
2345
  that tends to minimize their run-time cost.
10347 ripley 2346
 
40788 ripley 2347
  When FOOTERS is defined, in addition to range checking, we also
2348
  verify footer fields of inuse chunks, which can be used guarantee
2349
  that the mstate controlling malloc/free is intact.  This is a
2350
  streamlined version of the approach described by William Robertson
2351
  et al in "Run-time Detection of Heap-based Overflows" LISA'03
2352
  http://www.usenix.org/events/lisa03/tech/robertson.html The footer
2353
  of an inuse chunk holds the xor of its mstate and a random seed,
2354
  that is checked upon calls to free() and realloc().  This is
2355
  (probablistically) unguessable from outside the program, but can be
2356
  computed by any code successfully malloc'ing any chunk, so does not
2357
  itself provide protection against code that has already broken
2358
  security through some other means.  Unlike Robertson et al, we
2359
  always dynamically check addresses of all offset chunks (previous,
2360
  next, etc). This turns out to be cheaper than relying on hashes.
26855 ripley 2361
*/
2362
 
40788 ripley 2363
#if !INSECURE
2364
/* Check if address a is at least as high as any from MORECORE or MMAP */
2365
#define ok_address(M, a) ((char*)(a) >= (M)->least_addr)
2366
/* Check if address of next chunk n is higher than base chunk p */
2367
#define ok_next(p, n)    ((char*)(p) < (char*)(n))
2368
/* Check if p has its cinuse bit on */
2369
#define ok_cinuse(p)     cinuse(p)
2370
/* Check if p has its pinuse bit on */
2371
#define ok_pinuse(p)     pinuse(p)
26855 ripley 2372
 
40788 ripley 2373
#else /* !INSECURE */
2374
#define ok_address(M, a) (1)
2375
#define ok_next(b, n)    (1)
2376
#define ok_cinuse(p)     (1)
2377
#define ok_pinuse(p)     (1)
2378
#endif /* !INSECURE */
26855 ripley 2379
 
40788 ripley 2380
#if (FOOTERS && !INSECURE)
2381
/* Check if (alleged) mstate m has expected magic field */
2382
#define ok_magic(M)      ((M)->magic == mparams.magic)
2383
#else  /* (FOOTERS && !INSECURE) */
2384
#define ok_magic(M)      (1)
2385
#endif /* (FOOTERS && !INSECURE) */
26855 ripley 2386
 
10347 ripley 2387
 
40788 ripley 2388
/* In gcc, use __builtin_expect to minimize impact of checks */
2389
#if !INSECURE
2390
#if defined(__GNUC__) && __GNUC__ >= 3
2391
#define RTCHECK(e)  __builtin_expect(e, 1)
2392
#else /* GNUC */
2393
#define RTCHECK(e)  (e)
2394
#endif /* GNUC */
2395
#else /* !INSECURE */
2396
#define RTCHECK(e)  (1)
2397
#endif /* !INSECURE */
10347 ripley 2398
 
40788 ripley 2399
/* macros to set up inuse chunks with or without footers */
10347 ripley 2400
 
40788 ripley 2401
#if !FOOTERS
10347 ripley 2402
 
40788 ripley 2403
#define mark_inuse_foot(M,p,s)
10347 ripley 2404
 
40788 ripley 2405
/* Set cinuse bit and pinuse bit of next chunk */
2406
#define set_inuse(M,p,s)\
2407
  ((p)->head = (((p)->head & PINUSE_BIT)|s|CINUSE_BIT),\
2408
  ((mchunkptr)(((char*)(p)) + (s)))->head |= PINUSE_BIT)
10347 ripley 2409
 
40788 ripley 2410
/* Set cinuse and pinuse of this chunk and pinuse of next chunk */
2411
#define set_inuse_and_pinuse(M,p,s)\
2412
  ((p)->head = (s|PINUSE_BIT|CINUSE_BIT),\
2413
  ((mchunkptr)(((char*)(p)) + (s)))->head |= PINUSE_BIT)
10347 ripley 2414
 
40788 ripley 2415
/* Set size, cinuse and pinuse bit of this chunk */
2416
#define set_size_and_pinuse_of_inuse_chunk(M, p, s)\
2417
  ((p)->head = (s|PINUSE_BIT|CINUSE_BIT))
10347 ripley 2418
 
40788 ripley 2419
#else /* FOOTERS */
10347 ripley 2420
 
40788 ripley 2421
/* Set foot of inuse chunk to be xor of mstate and seed */
2422
#define mark_inuse_foot(M,p,s)\
2423
  (((mchunkptr)((char*)(p) + (s)))->prev_foot = ((size_t)(M) ^ mparams.magic))
10347 ripley 2424
 
40788 ripley 2425
#define get_mstate_for(p)\
2426
  ((mstate)(((mchunkptr)((char*)(p) +\
2427
    (chunksize(p))))->prev_foot ^ mparams.magic))
10347 ripley 2428
 
40788 ripley 2429
#define set_inuse(M,p,s)\
2430
  ((p)->head = (((p)->head & PINUSE_BIT)|s|CINUSE_BIT),\
2431
  (((mchunkptr)(((char*)(p)) + (s)))->head |= PINUSE_BIT), \
2432
  mark_inuse_foot(M,p,s))
10347 ripley 2433
 
40788 ripley 2434
#define set_inuse_and_pinuse(M,p,s)\
2435
  ((p)->head = (s|PINUSE_BIT|CINUSE_BIT),\
2436
  (((mchunkptr)(((char*)(p)) + (s)))->head |= PINUSE_BIT),\
2437
 mark_inuse_foot(M,p,s))
10347 ripley 2438
 
40788 ripley 2439
#define set_size_and_pinuse_of_inuse_chunk(M, p, s)\
2440
  ((p)->head = (s|PINUSE_BIT|CINUSE_BIT),\
2441
  mark_inuse_foot(M, p, s))
10347 ripley 2442
 
40788 ripley 2443
#endif /* !FOOTERS */
10347 ripley 2444
 
40788 ripley 2445
/* ---------------------------- setting mparams -------------------------- */
10347 ripley 2446
 
40788 ripley 2447
/* Initialize mparams */
2448
static int init_mparams(void) {
2449
  if (mparams.page_size == 0) {
2450
    size_t s;
10347 ripley 2451
 
40788 ripley 2452
    mparams.mmap_threshold = DEFAULT_MMAP_THRESHOLD;
2453
    mparams.trim_threshold = DEFAULT_TRIM_THRESHOLD;
2454
#if MORECORE_CONTIGUOUS
2455
    mparams.default_mflags = USE_LOCK_BIT|USE_MMAP_BIT;
2456
#else  /* MORECORE_CONTIGUOUS */
2457
    mparams.default_mflags = USE_LOCK_BIT|USE_MMAP_BIT|USE_NONCONTIGUOUS_BIT;
2458
#endif /* MORECORE_CONTIGUOUS */
10347 ripley 2459
 
40788 ripley 2460
#if (FOOTERS && !INSECURE)
2461
    {
2462
#if USE_DEV_RANDOM
2463
      int fd;
2464
      unsigned char buf[sizeof(size_t)];
2465
      /* Try to use /dev/urandom, else fall back on using time */
2466
      if ((fd = open("/dev/urandom", O_RDONLY)) >= 0 &&
2467
          read(fd, buf, sizeof(buf)) == sizeof(buf)) {
2468
        s = *((size_t *) buf);
2469
        close(fd);
2470
      }
2471
      else
2472
#endif /* USE_DEV_RANDOM */
2473
        s = (size_t)(time(0) ^ (size_t)0x55555555U);
10347 ripley 2474
 
40788 ripley 2475
      s |= (size_t)8U;    /* ensure nonzero */
2476
      s &= ~(size_t)7U;   /* improve chances of fault for bad values */
26855 ripley 2477
 
40788 ripley 2478
    }
2479
#else /* (FOOTERS && !INSECURE) */
2480
    s = (size_t)0x58585858U;
2481
#endif /* (FOOTERS && !INSECURE) */
2482
    ACQUIRE_MAGIC_INIT_LOCK();
2483
    if (mparams.magic == 0) {
2484
      mparams.magic = s;
2485
      /* Set up lock for main malloc area */
2486
      INITIAL_LOCK(&gm->mutex);
2487
      gm->mflags = mparams.default_mflags;
2488
    }
2489
    RELEASE_MAGIC_INIT_LOCK();
10347 ripley 2490
 
40788 ripley 2491
#ifndef WIN32
2492
    mparams.page_size = malloc_getpagesize;
2493
    mparams.granularity = ((DEFAULT_GRANULARITY != 0)?
2494
                           DEFAULT_GRANULARITY : mparams.page_size);
2495
#else /* WIN32 */
2496
    {
2497
      SYSTEM_INFO system_info;
2498
      GetSystemInfo(&system_info);
2499
      mparams.page_size = system_info.dwPageSize;
2500
      mparams.granularity = system_info.dwAllocationGranularity;
2501
    }
2502
#endif /* WIN32 */
10347 ripley 2503
 
40788 ripley 2504
    /* Sanity-check configuration:
2505
       size_t must be unsigned and as wide as pointer type.
2506
       ints must be at least 4 bytes.
2507
       alignment must be at least 8.
2508
       Alignment, min chunk size, and page size must all be powers of 2.
2509
    */
2510
    if ((sizeof(size_t) != sizeof(char*)) ||
2511
        (MAX_SIZE_T < MIN_CHUNK_SIZE)  ||
2512
        (sizeof(int) < 4)  ||
2513
        (MALLOC_ALIGNMENT < (size_t)8U) ||
2514
        ((MALLOC_ALIGNMENT    & (MALLOC_ALIGNMENT-SIZE_T_ONE))    != 0) ||
2515
        ((MCHUNK_SIZE         & (MCHUNK_SIZE-SIZE_T_ONE))         != 0) ||
2516
        ((mparams.granularity & (mparams.granularity-SIZE_T_ONE)) != 0) ||
2517
        ((mparams.page_size   & (mparams.page_size-SIZE_T_ONE))   != 0))
2518
      ABORT;
2519
  }
2520
  return 0;
2521
}
10347 ripley 2522
 
40801 ripley 2523
#if 0
40788 ripley 2524
/* support for mallopt */
2525
static int change_mparam(int param_number, int value) {
2526
  size_t val = (size_t)value;
2527
  init_mparams();
2528
  switch(param_number) {
2529
  case M_TRIM_THRESHOLD:
2530
    mparams.trim_threshold = val;
2531
    return 1;
2532
  case M_GRANULARITY:
2533
    if (val >= mparams.page_size && ((val & (val-1)) == 0)) {
2534
      mparams.granularity = val;
2535
      return 1;
2536
    }
2537
    else
2538
      return 0;
2539
  case M_MMAP_THRESHOLD:
2540
    mparams.mmap_threshold = val;
2541
    return 1;
2542
  default:
2543
    return 0;
26855 ripley 2544
  }
40788 ripley 2545
}
40801 ripley 2546
#endif
26855 ripley 2547
 
40788 ripley 2548
#if DEBUG
2549
/* ------------------------- Debugging Support --------------------------- */
26855 ripley 2550
 
40788 ripley 2551
/* Check properties of any chunk, whether free, inuse, mmapped etc  */
2552
static void do_check_any_chunk(mstate m, mchunkptr p) {
2553
  assert((is_aligned(chunk2mem(p))) || (p->head == FENCEPOST_HEAD));
2554
  assert(ok_address(m, p));
2555
}
26855 ripley 2556
 
40788 ripley 2557
/* Check properties of top chunk */
2558
static void do_check_top_chunk(mstate m, mchunkptr p) {
2559
  msegmentptr sp = segment_holding(m, (char*)p);
2560
  size_t  sz = chunksize(p);
2561
  assert(sp != 0);
2562
  assert((is_aligned(chunk2mem(p))) || (p->head == FENCEPOST_HEAD));
2563
  assert(ok_address(m, p));
2564
  assert(sz == m->topsize);
2565
  assert(sz > 0);
2566
  assert(sz == ((sp->base + sp->size) - (char*)p) - TOP_FOOT_SIZE);
2567
  assert(pinuse(p));
2568
  assert(!next_pinuse(p));
2569
}
26855 ripley 2570
 
40788 ripley 2571
/* Check properties of (inuse) mmapped chunks */
2572
static void do_check_mmapped_chunk(mstate m, mchunkptr p) {
2573
  size_t  sz = chunksize(p);
2574
  size_t len = (sz + (p->prev_foot & ~IS_MMAPPED_BIT) + MMAP_FOOT_PAD);
2575
  assert(is_mmapped(p));
2576
  assert(use_mmap(m));
2577
  assert((is_aligned(chunk2mem(p))) || (p->head == FENCEPOST_HEAD));
2578
  assert(ok_address(m, p));
2579
  assert(!is_small(sz));
2580
  assert((len & (mparams.page_size-SIZE_T_ONE)) == 0);
2581
  assert(chunk_plus_offset(p, sz)->head == FENCEPOST_HEAD);
2582
  assert(chunk_plus_offset(p, sz+SIZE_T_SIZE)->head == 0);
2583
}
26855 ripley 2584
 
40788 ripley 2585
/* Check properties of inuse chunks */
2586
static void do_check_inuse_chunk(mstate m, mchunkptr p) {
2587
  do_check_any_chunk(m, p);
2588
  assert(cinuse(p));
2589
  assert(next_pinuse(p));
2590
  /* If not pinuse and not mmapped, previous chunk has OK offset */
2591
  assert(is_mmapped(p) || pinuse(p) || next_chunk(prev_chunk(p)) == p);
2592
  if (is_mmapped(p))
2593
    do_check_mmapped_chunk(m, p);
26855 ripley 2594
}
2595
 
40788 ripley 2596
/* Check properties of free chunks */
2597
static void do_check_free_chunk(mstate m, mchunkptr p) {
2598
  size_t sz = p->head & ~(PINUSE_BIT|CINUSE_BIT);
2599
  mchunkptr next = chunk_plus_offset(p, sz);
2600
  do_check_any_chunk(m, p);
2601
  assert(!cinuse(p));
2602
  assert(!next_pinuse(p));
2603
  assert (!is_mmapped(p));
2604
  if (p != m->dv && p != m->top) {
2605
    if (sz >= MIN_CHUNK_SIZE) {
2606
      assert((sz & CHUNK_ALIGN_MASK) == 0);
2607
      assert(is_aligned(chunk2mem(p)));
2608
      assert(next->prev_foot == sz);
2609
      assert(pinuse(p));
2610
      assert (next == m->top || cinuse(next));
2611
      assert(p->fd->bk == p);
2612
      assert(p->bk->fd == p);
2613
    }
2614
    else  /* markers are always of size SIZE_T_SIZE */
2615
      assert(sz == SIZE_T_SIZE);
2616
  }
2617
}
26855 ripley 2618
 
40788 ripley 2619
/* Check properties of malloced chunks at the point they are malloced */
2620
static void do_check_malloced_chunk(mstate m, void* mem, size_t s) {
2621
  if (mem != 0) {
2622
    mchunkptr p = mem2chunk(mem);
2623
    size_t sz = p->head & ~(PINUSE_BIT|CINUSE_BIT);
2624
    do_check_inuse_chunk(m, p);
2625
    assert((sz & CHUNK_ALIGN_MASK) == 0);
2626
    assert(sz >= MIN_CHUNK_SIZE);
2627
    assert(sz >= s);
2628
    /* unless mmapped, size is less than MIN_CHUNK_SIZE more than request */
2629
    assert(is_mmapped(p) || sz < (s + MIN_CHUNK_SIZE));
2630
  }
2631
}
26855 ripley 2632
 
40788 ripley 2633
/* Check a tree and its subtrees.  */
2634
static void do_check_tree(mstate m, tchunkptr t) {
2635
  tchunkptr head = 0;
2636
  tchunkptr u = t;
2637
  bindex_t tindex = t->index;
2638
  size_t tsize = chunksize(t);
2639
  bindex_t idx;
2640
  compute_tree_index(tsize, idx);
2641
  assert(tindex == idx);
2642
  assert(tsize >= MIN_LARGE_SIZE);
2643
  assert(tsize >= minsize_for_tree_index(idx));
2644
  assert((idx == NTREEBINS-1) || (tsize < minsize_for_tree_index((idx+1))));
26855 ripley 2645
 
40788 ripley 2646
  do { /* traverse through chain of same-sized nodes */
2647
    do_check_any_chunk(m, ((mchunkptr)u));
2648
    assert(u->index == tindex);
2649
    assert(chunksize(u) == tsize);
2650
    assert(!cinuse(u));
2651
    assert(!next_pinuse(u));
2652
    assert(u->fd->bk == u);
2653
    assert(u->bk->fd == u);
2654
    if (u->parent == 0) {
2655
      assert(u->child[0] == 0);
2656
      assert(u->child[1] == 0);
2657
    }
2658
    else {
2659
      assert(head == 0); /* only one node on chain has parent */
2660
      head = u;
2661
      assert(u->parent != u);
2662
      assert (u->parent->child[0] == u ||
2663
              u->parent->child[1] == u ||
2664
              *((tbinptr*)(u->parent)) == u);
2665
      if (u->child[0] != 0) {
2666
        assert(u->child[0]->parent == u);
2667
        assert(u->child[0] != u);
2668
        do_check_tree(m, u->child[0]);
2669
      }
2670
      if (u->child[1] != 0) {
2671
        assert(u->child[1]->parent == u);
2672
        assert(u->child[1] != u);
2673
        do_check_tree(m, u->child[1]);
2674
      }
2675
      if (u->child[0] != 0 && u->child[1] != 0) {
2676
        assert(chunksize(u->child[0]) < chunksize(u->child[1]));
2677
      }
2678
    }
2679
    u = u->fd;
2680
  } while (u != t);
2681
  assert(head != 0);
2682
}
10347 ripley 2683
 
40788 ripley 2684
/*  Check all the chunks in a treebin.  */
2685
static void do_check_treebin(mstate m, bindex_t i) {
2686
  tbinptr* tb = treebin_at(m, i);
2687
  tchunkptr t = *tb;
2688
  int empty = (m->treemap & (1U << i)) == 0;
2689
  if (t == 0)
2690
    assert(empty);
2691
  if (!empty)
2692
    do_check_tree(m, t);
2693
}
26855 ripley 2694
 
40788 ripley 2695
/*  Check all the chunks in a smallbin.  */
2696
static void do_check_smallbin(mstate m, bindex_t i) {
2697
  sbinptr b = smallbin_at(m, i);
2698
  mchunkptr p = b->bk;
2699
  unsigned int empty = (m->smallmap & (1U << i)) == 0;
2700
  if (p == b)
2701
    assert(empty);
2702
  if (!empty) {
2703
    for (; p != b; p = p->bk) {
2704
      size_t size = chunksize(p);
2705
      mchunkptr q;
2706
      /* each chunk claims to be free */
2707
      do_check_free_chunk(m, p);
2708
      /* chunk belongs in bin */
2709
      assert(small_index(size) == i);
2710
      assert(p->bk == b || chunksize(p->bk) == chunksize(p));
2711
      /* chunk is followed by an inuse chunk */
2712
      q = next_chunk(p);
2713
      if (q->head != FENCEPOST_HEAD)
2714
        do_check_inuse_chunk(m, q);
2715
    }
2716
  }
2717
}
26855 ripley 2718
 
40788 ripley 2719
/* Find x in a bin. Used in other check functions. */
2720
static int bin_find(mstate m, mchunkptr x) {
2721
  size_t size = chunksize(x);
2722
  if (is_small(size)) {
2723
    bindex_t sidx = small_index(size);
2724
    sbinptr b = smallbin_at(m, sidx);
2725
    if (smallmap_is_marked(m, sidx)) {
2726
      mchunkptr p = b;
2727
      do {
2728
        if (p == x)
2729
          return 1;
2730
      } while ((p = p->fd) != b);
26855 ripley 2731
    }
2732
  }
2733
  else {
40788 ripley 2734
    bindex_t tidx;
2735
    compute_tree_index(size, tidx);
2736
    if (treemap_is_marked(m, tidx)) {
2737
      tchunkptr t = *treebin_at(m, tidx);
2738
      size_t sizebits = size << leftshift_for_tree_index(tidx);
2739
      while (t != 0 && chunksize(t) != size) {
2740
        t = t->child[(sizebits >> (SIZE_T_BITSIZE-SIZE_T_ONE)) & 1];
2741
        sizebits <<= 1;
2742
      }
2743
      if (t != 0) {
2744
        tchunkptr u = t;
2745
        do {
2746
          if (u == (tchunkptr)x)
2747
            return 1;
2748
        } while ((u = u->fd) != t);
2749
      }
26855 ripley 2750
    }
2751
  }
40788 ripley 2752
  return 0;
10347 ripley 2753
}
2754
 
40788 ripley 2755
/* Traverse each chunk and check it; return total */
2756
static size_t traverse_and_check(mstate m) {
2757
  size_t sum = 0;
2758
  if (is_initialized(m)) {
2759
    msegmentptr s = &m->seg;
2760
    sum += m->topsize + TOP_FOOT_SIZE;
2761
    while (s != 0) {
2762
      mchunkptr q = align_as_chunk(s->base);
2763
      mchunkptr lastq = 0;
2764
      assert(pinuse(q));
2765
      while (segment_holds(s, q) &&
2766
             q != m->top && q->head != FENCEPOST_HEAD) {
2767
        sum += chunksize(q);
2768
        if (cinuse(q)) {
2769
          assert(!bin_find(m, q));
2770
          do_check_inuse_chunk(m, q);
2771
        }
2772
        else {
2773
          assert(q == m->dv || bin_find(m, q));
2774
          assert(lastq == 0 || cinuse(lastq)); /* Not 2 consecutive free */
2775
          do_check_free_chunk(m, q);
2776
        }
2777
        lastq = q;
2778
        q = next_chunk(q);
2779
      }
2780
      s = s->next;
2781
    }
2782
  }
2783
  return sum;
2784
}
10347 ripley 2785
 
40788 ripley 2786
/* Check all properties of malloc_state. */
2787
static void do_check_malloc_state(mstate m) {
2788
  bindex_t i;
2789
  size_t total;
2790
  /* check bins */
2791
  for (i = 0; i < NSMALLBINS; ++i)
2792
    do_check_smallbin(m, i);
2793
  for (i = 0; i < NTREEBINS; ++i)
2794
    do_check_treebin(m, i);
26855 ripley 2795
 
40788 ripley 2796
  if (m->dvsize != 0) { /* check dv chunk */
2797
    do_check_any_chunk(m, m->dv);
2798
    assert(m->dvsize == chunksize(m->dv));
2799
    assert(m->dvsize >= MIN_CHUNK_SIZE);
2800
    assert(bin_find(m, m->dv) == 0);
2801
  }
10347 ripley 2802
 
40788 ripley 2803
  if (m->top != 0) {   /* check top chunk */
2804
    do_check_top_chunk(m, m->top);
2805
    assert(m->topsize == chunksize(m->top));
2806
    assert(m->topsize > 0);
2807
    assert(bin_find(m, m->top) == 0);
2808
  }
10347 ripley 2809
 
40788 ripley 2810
  total = traverse_and_check(m);
2811
  assert(total <= m->footprint);
2812
  assert(m->footprint <= m->max_footprint);
2813
}
2814
#endif /* DEBUG */
10347 ripley 2815
 
40788 ripley 2816
/* ----------------------------- statistics ------------------------------ */
10347 ripley 2817
 
40788 ripley 2818
#if !NO_MALLINFO
2819
static struct mallinfo internal_mallinfo(mstate m) {
2820
  struct mallinfo nm = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 };
2821
  if (!PREACTION(m)) {
2822
    check_malloc_state(m);
2823
    if (is_initialized(m)) {
2824
      size_t nfree = SIZE_T_ONE; /* top always free */
2825
      size_t mfree = m->topsize + TOP_FOOT_SIZE;
2826
      size_t sum = mfree;
2827
      msegmentptr s = &m->seg;
2828
      while (s != 0) {
2829
        mchunkptr q = align_as_chunk(s->base);
2830
        while (segment_holds(s, q) &&
2831
               q != m->top && q->head != FENCEPOST_HEAD) {
2832
          size_t sz = chunksize(q);
2833
          sum += sz;
2834
          if (!cinuse(q)) {
2835
            mfree += sz;
2836
            ++nfree;
2837
          }
2838
          q = next_chunk(q);
2839
        }
2840
        s = s->next;
2841
      }
2842
 
2843
      nm.arena    = sum;
2844
      nm.ordblks  = nfree;
2845
      nm.hblkhd   = m->footprint - sum;
2846
      nm.usmblks  = m->max_footprint;
2847
      nm.uordblks = m->footprint - mfree;
2848
      nm.fordblks = mfree;
2849
      nm.keepcost = m->topsize;
2850
    }
2851
 
2852
    POSTACTION(m);
10347 ripley 2853
  }
40788 ripley 2854
  return nm;
10347 ripley 2855
}
40788 ripley 2856
#endif /* !NO_MALLINFO */
10347 ripley 2857
 
40801 ripley 2858
#if 0
40788 ripley 2859
static void internal_malloc_stats(mstate m) {
2860
  if (!PREACTION(m)) {
2861
    size_t maxfp = 0;
2862
    size_t fp = 0;
2863
    size_t used = 0;
2864
    check_malloc_state(m);
2865
    if (is_initialized(m)) {
2866
      msegmentptr s = &m->seg;
2867
      maxfp = m->max_footprint;
2868
      fp = m->footprint;
2869
      used = fp - (m->topsize + TOP_FOOT_SIZE);
26855 ripley 2870
 
40788 ripley 2871
      while (s != 0) {
2872
        mchunkptr q = align_as_chunk(s->base);
2873
        while (segment_holds(s, q) &&
2874
               q != m->top && q->head != FENCEPOST_HEAD) {
2875
          if (!cinuse(q))
2876
            used -= chunksize(q);
2877
          q = next_chunk(q);
2878
        }
2879
        s = s->next;
2880
      }
2881
    }
10347 ripley 2882
 
40788 ripley 2883
    fprintf(stderr, "max system bytes = %10lu\n", (unsigned long)(maxfp));
2884
    fprintf(stderr, "system bytes     = %10lu\n", (unsigned long)(fp));
2885
    fprintf(stderr, "in use bytes     = %10lu\n", (unsigned long)(used));
26855 ripley 2886
 
40788 ripley 2887
    POSTACTION(m);
10347 ripley 2888
  }
2889
}
40801 ripley 2890
#endif
10347 ripley 2891
 
40788 ripley 2892
/* ----------------------- Operations on smallbins ----------------------- */
2893
 
26855 ripley 2894
/*
40788 ripley 2895
  Various forms of linking and unlinking are defined as macros.  Even
2896
  the ones for trees, which are very long but have very short typical
2897
  paths.  This is ugly but reduces reliance on inlining support of
2898
  compilers.
26855 ripley 2899
*/
2900
 
40788 ripley 2901
/* Link a free chunk into a smallbin  */
2902
#define insert_small_chunk(M, P, S) {\
2903
  bindex_t I  = small_index(S);\
2904
  mchunkptr B = smallbin_at(M, I);\
2905
  mchunkptr F = B;\
2906
  assert(S >= MIN_CHUNK_SIZE);\
2907
  if (!smallmap_is_marked(M, I))\
2908
    mark_smallmap(M, I);\
2909
  else if (RTCHECK(ok_address(M, B->fd)))\
2910
    F = B->fd;\
2911
  else {\
2912
    CORRUPTION_ERROR_ACTION(M);\
2913
  }\
2914
  B->fd = P;\
2915
  F->bk = P;\
2916
  P->fd = F;\
2917
  P->bk = B;\
2918
}
10347 ripley 2919
 
40788 ripley 2920
/* Unlink a chunk from a smallbin  */
2921
#define unlink_small_chunk(M, P, S) {\
2922
  mchunkptr F = P->fd;\
2923
  mchunkptr B = P->bk;\
2924
  bindex_t I = small_index(S);\
2925
  assert(P != B);\
2926
  assert(P != F);\
2927
  assert(chunksize(P) == small_index2size(I));\
2928
  if (F == B)\
2929
    clear_smallmap(M, I);\
2930
  else if (RTCHECK((F == smallbin_at(M,I) || ok_address(M, F)) &&\
2931
                   (B == smallbin_at(M,I) || ok_address(M, B)))) {\
2932
    F->bk = B;\
2933
    B->fd = F;\
2934
  }\
2935
  else {\
2936
    CORRUPTION_ERROR_ACTION(M);\
2937
  }\
2938
}
10347 ripley 2939
 
40788 ripley 2940
/* Unlink the first chunk from a smallbin */
2941
#define unlink_first_small_chunk(M, B, P, I) {\
2942
  mchunkptr F = P->fd;\
2943
  assert(P != B);\
2944
  assert(P != F);\
2945
  assert(chunksize(P) == small_index2size(I));\
2946
  if (B == F)\
2947
    clear_smallmap(M, I);\
2948
  else if (RTCHECK(ok_address(M, F))) {\
2949
    B->fd = F;\
2950
    F->bk = B;\
2951
  }\
2952
  else {\
2953
    CORRUPTION_ERROR_ACTION(M);\
2954
  }\
10347 ripley 2955
}
2956
 
40788 ripley 2957
/* Replace dv node, binning the old one */
2958
/* Used only when dvsize known to be small */
2959
#define replace_dv(M, P, S) {\
2960
  size_t DVS = M->dvsize;\
2961
  if (DVS != 0) {\
2962
    mchunkptr DV = M->dv;\
2963
    assert(is_small(DVS));\
2964
    insert_small_chunk(M, DV, DVS);\
2965
  }\
2966
  M->dvsize = S;\
2967
  M->dv = P;\
2968
}
10347 ripley 2969
 
40788 ripley 2970
/* ------------------------- Operations on trees ------------------------- */
10347 ripley 2971
 
40788 ripley 2972
/* Insert chunk into tree */
2973
#define insert_large_chunk(M, X, S) {\
2974
  tbinptr* H;\
2975
  bindex_t I;\
2976
  compute_tree_index(S, I);\
2977
  H = treebin_at(M, I);\
2978
  X->index = I;\
2979
  X->child[0] = X->child[1] = 0;\
2980
  if (!treemap_is_marked(M, I)) {\
2981
    mark_treemap(M, I);\
2982
    *H = X;\
2983
    X->parent = (tchunkptr)H;\
2984
    X->fd = X->bk = X;\
2985
  }\
2986
  else {\
2987
    tchunkptr T = *H;\
2988
    size_t K = S << leftshift_for_tree_index(I);\
2989
    for (;;) {\
2990
      if (chunksize(T) != S) {\
2991
        tchunkptr* C = &(T->child[(K >> (SIZE_T_BITSIZE-SIZE_T_ONE)) & 1]);\
2992
        K <<= 1;\
2993
        if (*C != 0)\
2994
          T = *C;\
2995
        else if (RTCHECK(ok_address(M, C))) {\
2996
          *C = X;\
2997
          X->parent = T;\
2998
          X->fd = X->bk = X;\
2999
          break;\
3000
        }\
3001
        else {\
3002
          CORRUPTION_ERROR_ACTION(M);\
3003
          break;\
3004
        }\
3005
      }\
3006
      else {\
3007
        tchunkptr F = T->fd;\
3008
        if (RTCHECK(ok_address(M, T) && ok_address(M, F))) {\
3009
          T->fd = F->bk = X;\
3010
          X->fd = F;\
3011
          X->bk = T;\
3012
          X->parent = 0;\
3013
          break;\
3014
        }\
3015
        else {\
3016
          CORRUPTION_ERROR_ACTION(M);\
3017
          break;\
3018
        }\
3019
      }\
3020
    }\
3021
  }\
26855 ripley 3022
}
10347 ripley 3023
 
3024
/*
40788 ripley 3025
  Unlink steps:
10347 ripley 3026
 
40788 ripley 3027
  1. If x is a chained node, unlink it from its same-sized fd/bk links
3028
     and choose its bk node as its replacement.
3029
  2. If x was the last node of its size, but not a leaf node, it must
3030
     be replaced with a leaf node (not merely one with an open left or
3031
     right), to make sure that lefts and rights of descendents
3032
     correspond properly to bit masks.  We use the rightmost descendent
3033
     of x.  We could use any other leaf, but this is easy to locate and
3034
     tends to counteract removal of leftmosts elsewhere, and so keeps
3035
     paths shorter than minimally guaranteed.  This doesn't loop much
3036
     because on average a node in a tree is near the bottom.
3037
  3. If x is the base of a chain (i.e., has parent links) relink
3038
     x's parent and children to x's replacement (or null if none).
10347 ripley 3039
*/
3040
 
40788 ripley 3041
#define unlink_large_chunk(M, X) {\
3042
  tchunkptr XP = X->parent;\
3043
  tchunkptr R;\
3044
  if (X->bk != X) {\
3045
    tchunkptr F = X->fd;\
3046
    R = X->bk;\
3047
    if (RTCHECK(ok_address(M, F))) {\
3048
      F->bk = R;\
3049
      R->fd = F;\
3050
    }\
3051
    else {\
3052
      CORRUPTION_ERROR_ACTION(M);\
3053
    }\
3054
  }\
3055
  else {\
3056
    tchunkptr* RP;\
3057
    if (((R = *(RP = &(X->child[1]))) != 0) ||\
3058
        ((R = *(RP = &(X->child[0]))) != 0)) {\
3059
      tchunkptr* CP;\
3060
      while ((*(CP = &(R->child[1])) != 0) ||\
3061
             (*(CP = &(R->child[0])) != 0)) {\
3062
        R = *(RP = CP);\
3063
      }\
3064
      if (RTCHECK(ok_address(M, RP)))\
3065
        *RP = 0;\
3066
      else {\
3067
        CORRUPTION_ERROR_ACTION(M);\
3068
      }\
3069
    }\
3070
  }\
3071
  if (XP != 0) {\
3072
    tbinptr* H = treebin_at(M, X->index);\
3073
    if (X == *H) {\
3074
      if ((*H = R) == 0) \
3075
        clear_treemap(M, X->index);\
3076
    }\
3077
    else if (RTCHECK(ok_address(M, XP))) {\
3078
      if (XP->child[0] == X) \
3079
        XP->child[0] = R;\
3080
      else \
3081
        XP->child[1] = R;\
3082
    }\
3083
    else\
3084
      CORRUPTION_ERROR_ACTION(M);\
3085
    if (R != 0) {\
3086
      if (RTCHECK(ok_address(M, R))) {\
3087
        tchunkptr C0, C1;\
3088
        R->parent = XP;\
3089
        if ((C0 = X->child[0]) != 0) {\
3090
          if (RTCHECK(ok_address(M, C0))) {\
3091
            R->child[0] = C0;\
3092
            C0->parent = R;\
3093
          }\
3094
          else\
3095
            CORRUPTION_ERROR_ACTION(M);\
3096
        }\
3097
        if ((C1 = X->child[1]) != 0) {\
3098
          if (RTCHECK(ok_address(M, C1))) {\
3099
            R->child[1] = C1;\
3100
            C1->parent = R;\
3101
          }\
3102
          else\
3103
            CORRUPTION_ERROR_ACTION(M);\
3104
        }\
3105
      }\
3106
      else\
3107
        CORRUPTION_ERROR_ACTION(M);\
3108
    }\
3109
  }\
3110
}
10347 ripley 3111
 
40788 ripley 3112
/* Relays to large vs small bin operations */
10347 ripley 3113
 
40788 ripley 3114
#define insert_chunk(M, P, S)\
3115
  if (is_small(S)) insert_small_chunk(M, P, S)\
3116
  else { tchunkptr TP = (tchunkptr)(P); insert_large_chunk(M, TP, S); }
10347 ripley 3117
 
40788 ripley 3118
#define unlink_chunk(M, P, S)\
3119
  if (is_small(S)) unlink_small_chunk(M, P, S)\
3120
  else { tchunkptr TP = (tchunkptr)(P); unlink_large_chunk(M, TP); }
10347 ripley 3121
 
3122
 
40788 ripley 3123
/* Relays to internal calls to malloc/free from realloc, memalign etc */
10347 ripley 3124
 
40788 ripley 3125
#if ONLY_MSPACES
3126
#define internal_malloc(m, b) mspace_malloc(m, b)
3127
#define internal_free(m, mem) mspace_free(m,mem);
3128
#else /* ONLY_MSPACES */
3129
#if MSPACES
3130
#define internal_malloc(m, b)\
3131
   (m == gm)? dlmalloc(b) : mspace_malloc(m, b)
3132
#define internal_free(m, mem)\
3133
   if (m == gm) dlfree(mem); else mspace_free(m,mem);
3134
#else /* MSPACES */
3135
#define internal_malloc(m, b) dlmalloc(b)
3136
#define internal_free(m, mem) dlfree(mem)
3137
#endif /* MSPACES */
3138
#endif /* ONLY_MSPACES */
10347 ripley 3139
 
40788 ripley 3140
/* -----------------------  Direct-mmapping chunks ----------------------- */
10347 ripley 3141
 
40788 ripley 3142
/*
3143
  Directly mmapped chunks are set up with an offset to the start of
3144
  the mmapped region stored in the prev_foot field of the chunk. This
3145
  allows reconstruction of the required argument to MUNMAP when freed,
3146
  and also allows adjustment of the returned chunk to meet alignment
3147
  requirements (especially in memalign).  There is also enough space
3148
  allocated to hold a fake next chunk of size SIZE_T_SIZE to maintain
3149
  the PINUSE bit so frees can be checked.
3150
*/
10347 ripley 3151
 
40788 ripley 3152
/* Malloc using mmap */
3153
static void* mmap_alloc(mstate m, size_t nb) {
3154
  size_t mmsize = granularity_align(nb + SIX_SIZE_T_SIZES + CHUNK_ALIGN_MASK);
3155
  if (mmsize > nb) {     /* Check for wrap around 0 */
3156
    char* mm = (char*)(DIRECT_MMAP(mmsize));
3157
    if (mm != CMFAIL) {
3158
      size_t offset = align_offset(chunk2mem(mm));
3159
      size_t psize = mmsize - offset - MMAP_FOOT_PAD;
3160
      mchunkptr p = (mchunkptr)(mm + offset);
3161
      p->prev_foot = offset | IS_MMAPPED_BIT;
3162
      (p)->head = (psize|CINUSE_BIT);
3163
      mark_inuse_foot(m, p, psize);
3164
      chunk_plus_offset(p, psize)->head = FENCEPOST_HEAD;
3165
      chunk_plus_offset(p, psize+SIZE_T_SIZE)->head = 0;
10347 ripley 3166
 
40788 ripley 3167
      if (mm < m->least_addr)
3168
        m->least_addr = mm;
3169
      if ((m->footprint += mmsize) > m->max_footprint)
3170
        m->max_footprint = m->footprint;
3171
      assert(is_aligned(chunk2mem(p)));
3172
      check_mmapped_chunk(m, p);
3173
      return chunk2mem(p);
26855 ripley 3174
    }
3175
  }
40788 ripley 3176
  return 0;
3177
}
10347 ripley 3178
 
40788 ripley 3179
/* Realloc using mmap */
3180
static mchunkptr mmap_resize(mstate m, mchunkptr oldp, size_t nb) {
3181
  size_t oldsize = chunksize(oldp);
3182
  if (is_small(nb)) /* Can't shrink mmap regions below small size */
3183
    return 0;
3184
  /* Keep old chunk if big enough but not too big */
3185
  if (oldsize >= nb + SIZE_T_SIZE &&
3186
      (oldsize - nb) <= (mparams.granularity << 1))
3187
    return oldp;
3188
  else {
3189
    size_t offset = oldp->prev_foot & ~IS_MMAPPED_BIT;
3190
    size_t oldmmsize = oldsize + offset + MMAP_FOOT_PAD;
3191
    size_t newmmsize = granularity_align(nb + SIX_SIZE_T_SIZES +
3192
                                         CHUNK_ALIGN_MASK);
3193
    char* cp = (char*)CALL_MREMAP((char*)oldp - offset,
3194
                                  oldmmsize, newmmsize, 1);
3195
    if (cp != CMFAIL) {
3196
      mchunkptr newp = (mchunkptr)(cp + offset);
3197
      size_t psize = newmmsize - offset - MMAP_FOOT_PAD;
3198
      newp->head = (psize|CINUSE_BIT);
3199
      mark_inuse_foot(m, newp, psize);
3200
      chunk_plus_offset(newp, psize)->head = FENCEPOST_HEAD;
3201
      chunk_plus_offset(newp, psize+SIZE_T_SIZE)->head = 0;
10347 ripley 3202
 
40788 ripley 3203
      if (cp < m->least_addr)
3204
        m->least_addr = cp;
3205
      if ((m->footprint += newmmsize - oldmmsize) > m->max_footprint)
3206
        m->max_footprint = m->footprint;
3207
      check_mmapped_chunk(m, newp);
3208
      return newp;
26855 ripley 3209
    }
3210
  }
40788 ripley 3211
  return 0;
3212
}
10347 ripley 3213
 
40788 ripley 3214
/* -------------------------- mspace management -------------------------- */
10347 ripley 3215
 
40788 ripley 3216
/* Initialize top chunk and its size */
3217
static void init_top(mstate m, mchunkptr p, size_t psize) {
3218
  /* Ensure alignment */
3219
  size_t offset = align_offset(chunk2mem(p));
3220
  p = (mchunkptr)((char*)p + offset);
3221
  psize -= offset;
10347 ripley 3222
 
40788 ripley 3223
  m->top = p;
3224
  m->topsize = psize;
3225
  p->head = psize | PINUSE_BIT;
3226
  /* set size of fake trailing chunk holding overhead space only once */
3227
  chunk_plus_offset(p, psize)->head = TOP_FOOT_SIZE;
3228
  m->trim_check = mparams.trim_threshold; /* reset on each update */
3229
}
10347 ripley 3230
 
40788 ripley 3231
/* Initialize bins for a new mstate that is otherwise zeroed out */
3232
static void init_bins(mstate m) {
3233
  /* Establish circular links for smallbins */
3234
  bindex_t i;
3235
  for (i = 0; i < NSMALLBINS; ++i) {
3236
    sbinptr bin = smallbin_at(m,i);
3237
    bin->fd = bin->bk = bin;
3238
  }
3239
}
10347 ripley 3240
 
40788 ripley 3241
#if PROCEED_ON_ERROR
26855 ripley 3242
 
40788 ripley 3243
/* default corruption action */
3244
static void reset_on_error(mstate m) {
3245
  int i;
3246
  ++malloc_corruption_error_count;
3247
  /* Reinitialize fields to forget about all memory */
3248
  m->smallbins = m->treebins = 0;
3249
  m->dvsize = m->topsize = 0;
3250
  m->seg.base = 0;
3251
  m->seg.size = 0;
3252
  m->seg.next = 0;
3253
  m->top = m->dv = 0;
3254
  for (i = 0; i < NTREEBINS; ++i)
3255
    *treebin_at(m, i) = 0;
3256
  init_bins(m);
26855 ripley 3257
}
40788 ripley 3258
#endif /* PROCEED_ON_ERROR */
10347 ripley 3259
 
40788 ripley 3260
/* Allocate chunk and prepend remainder with chunk in successor base. */
3261
static void* prepend_alloc(mstate m, char* newbase, char* oldbase,
3262
                           size_t nb) {
3263
  mchunkptr p = align_as_chunk(newbase);
3264
  mchunkptr oldfirst = align_as_chunk(oldbase);
3265
  size_t psize = (char*)oldfirst - (char*)p;
3266
  mchunkptr q = chunk_plus_offset(p, nb);
3267
  size_t qsize = psize - nb;
3268
  set_size_and_pinuse_of_inuse_chunk(m, p, nb);
10347 ripley 3269
 
40788 ripley 3270
  assert((char*)oldfirst > (char*)q);
3271
  assert(pinuse(oldfirst));
3272
  assert(qsize >= MIN_CHUNK_SIZE);
10347 ripley 3273
 
40788 ripley 3274
  /* consolidate remainder with first chunk of old base */
3275
  if (oldfirst == m->top) {
3276
    size_t tsize = m->topsize += qsize;
3277
    m->top = q;
3278
    q->head = tsize | PINUSE_BIT;
3279
    check_top_chunk(m, q);
3280
  }
3281
  else if (oldfirst == m->dv) {
3282
    size_t dsize = m->dvsize += qsize;
3283
    m->dv = q;
3284
    set_size_and_pinuse_of_free_chunk(q, dsize);
3285
  }
3286
  else {
3287
    if (!cinuse(oldfirst)) {
3288
      size_t nsize = chunksize(oldfirst);
3289
      unlink_chunk(m, oldfirst, nsize);
3290
      oldfirst = chunk_plus_offset(oldfirst, nsize);
3291
      qsize += nsize;
3292
    }
3293
    set_free_with_pinuse(q, qsize, oldfirst);
3294
    insert_chunk(m, q, qsize);
3295
    check_free_chunk(m, q);
3296
  }
10347 ripley 3297
 
40788 ripley 3298
  check_malloced_chunk(m, chunk2mem(p), nb);
3299
  return chunk2mem(p);
3300
}
10347 ripley 3301
 
3302
 
40788 ripley 3303
/* Add a segment to hold a new noncontiguous region */
3304
static void add_segment(mstate m, char* tbase, size_t tsize, flag_t mmapped) {
3305
  /* Determine locations and sizes of segment, fenceposts, old top */
3306
  char* old_top = (char*)m->top;
3307
  msegmentptr oldsp = segment_holding(m, old_top);
3308
  char* old_end = oldsp->base + oldsp->size;
3309
  size_t ssize = pad_request(sizeof(struct malloc_segment));
3310
  char* rawsp = old_end - (ssize + FOUR_SIZE_T_SIZES + CHUNK_ALIGN_MASK);
3311
  size_t offset = align_offset(chunk2mem(rawsp));
3312
  char* asp = rawsp + offset;
3313
  char* csp = (asp < (old_top + MIN_CHUNK_SIZE))? old_top : asp;
3314
  mchunkptr sp = (mchunkptr)csp;
3315
  msegmentptr ss = (msegmentptr)(chunk2mem(sp));
3316
  mchunkptr tnext = chunk_plus_offset(sp, ssize);
3317
  mchunkptr p = tnext;
3318
  int nfences = 0;
10347 ripley 3319
 
40788 ripley 3320
  /* reset top to new space */
3321
  init_top(m, (mchunkptr)tbase, tsize - TOP_FOOT_SIZE);
10347 ripley 3322
 
40788 ripley 3323
  /* Set up segment record */
3324
  assert(is_aligned(ss));
3325
  set_size_and_pinuse_of_inuse_chunk(m, sp, ssize);
3326
  *ss = m->seg; /* Push current record */
3327
  m->seg.base = tbase;
3328
  m->seg.size = tsize;
3329
  m->seg.sflags = mmapped;
3330
  m->seg.next = ss;
10347 ripley 3331
 
40788 ripley 3332
  /* Insert trailing fenceposts */
3333
  for (;;) {
3334
    mchunkptr nextp = chunk_plus_offset(p, SIZE_T_SIZE);
3335
    p->head = FENCEPOST_HEAD;
3336
    ++nfences;
3337
    if ((char*)(&(nextp->head)) < old_end)
3338
      p = nextp;
3339
    else
3340
      break;
3341
  }
3342
  assert(nfences >= 2);
10347 ripley 3343
 
40788 ripley 3344
  /* Insert the rest of old top into a bin as an ordinary free chunk */
3345
  if (csp != old_top) {
3346
    mchunkptr q = (mchunkptr)old_top;
3347
    size_t psize = csp - old_top;
3348
    mchunkptr tn = chunk_plus_offset(q, psize);
3349
    set_free_with_pinuse(q, psize, tn);
3350
    insert_chunk(m, q, psize);
3351
  }
10347 ripley 3352
 
40788 ripley 3353
  check_top_chunk(m, m->top);
3354
}
10347 ripley 3355
 
40788 ripley 3356
/* -------------------------- System allocation -------------------------- */
10347 ripley 3357
 
40788 ripley 3358
/* Get memory from system using MORECORE or MMAP */
3359
static void* sys_alloc(mstate m, size_t nb) {
3360
  char* tbase = CMFAIL;
3361
  size_t tsize = 0;
3362
  flag_t mmap_flag = 0;
10347 ripley 3363
 
40788 ripley 3364
  init_mparams();
10347 ripley 3365
 
40788 ripley 3366
  /* Directly map large chunks */
3367
  if (use_mmap(m) && nb >= mparams.mmap_threshold) {
3368
    void* mem = mmap_alloc(m, nb);
3369
    if (mem != 0)
3370
      return mem;
3371
  }
3372
 
26855 ripley 3373
  /*
40788 ripley 3374
    Try getting memory in any of three ways (in most-preferred to
3375
    least-preferred order):
3376
    1. A call to MORECORE that can normally contiguously extend memory.
3377
       (disabled if not MORECORE_CONTIGUOUS or not HAVE_MORECORE or
3378
       or main space is mmapped or a previous contiguous call failed)
3379
    2. A call to MMAP new space (disabled if not HAVE_MMAP).
3380
       Note that under the default settings, if MORECORE is unable to
3381
       fulfill a request, and HAVE_MMAP is true, then mmap is
3382
       used as a noncontiguous system allocator. This is a useful backup
3383
       strategy for systems with holes in address spaces -- in this case
3384
       sbrk cannot contiguously expand the heap, but mmap may be able to
3385
       find space.
3386
    3. A call to MORECORE that cannot usually contiguously extend memory.
3387
       (disabled if not HAVE_MORECORE)
26855 ripley 3388
  */
10347 ripley 3389
 
40788 ripley 3390
  if (MORECORE_CONTIGUOUS && !use_noncontiguous(m)) {
3391
    char* br = CMFAIL;
3392
    msegmentptr ss = (m->top == 0)? 0 : segment_holding(m, (char*)m->top);
3393
    size_t asize = 0;
3394
    ACQUIRE_MORECORE_LOCK();
10347 ripley 3395
 
40788 ripley 3396
    if (ss == 0) {  /* First time through or recovery */
3397
      char* base = (char*)CALL_MORECORE(0);
3398
      if (base != CMFAIL) {
3399
        asize = granularity_align(nb + TOP_FOOT_SIZE + SIZE_T_ONE);
3400
        /* Adjust to end on a page boundary */
3401
        if (!is_page_aligned(base))
3402
          asize += (page_align((size_t)base) - (size_t)base);
3403
        /* Can't call MORECORE if size is negative when treated as signed */
3404
        if (asize < HALF_MAX_SIZE_T &&
3405
            (br = (char*)(CALL_MORECORE(asize))) == base) {
3406
          tbase = base;
3407
          tsize = asize;
3408
        }
3409
      }
26945 ripley 3410
    }
40788 ripley 3411
    else {
3412
      /* Subtract out existing available top space from MORECORE request. */
3413
      asize = granularity_align(nb - m->topsize + TOP_FOOT_SIZE + SIZE_T_ONE);
3414
      /* Use mem here only if it did continuously extend old space */
3415
      if (asize < HALF_MAX_SIZE_T &&
3416
          (br = (char*)(CALL_MORECORE(asize))) == ss->base+ss->size) {
3417
        tbase = br;
3418
        tsize = asize;
3419
      }
3420
    }
26945 ripley 3421
 
40788 ripley 3422
    if (tbase == CMFAIL) {    /* Cope with partial failure */
3423
      if (br != CMFAIL) {    /* Try to use/extend the space we did get */
3424
        if (asize < HALF_MAX_SIZE_T &&
3425
            asize < nb + TOP_FOOT_SIZE + SIZE_T_ONE) {
3426
          size_t esize = granularity_align(nb + TOP_FOOT_SIZE + SIZE_T_ONE - asize);
3427
          if (esize < HALF_MAX_SIZE_T) {
3428
            char* end = (char*)CALL_MORECORE(esize);
3429
            if (end != CMFAIL)
3430
              asize += esize;
3431
            else {            /* Can't use; try to release */
3432
#if 0 /* warning: statement with no effect */
3433
              CALL_MORECORE(-asize);
3434
#endif
3435
              br = CMFAIL;
3436
            }
3437
          }
26855 ripley 3438
        }
3439
      }
40788 ripley 3440
      if (br != CMFAIL) {    /* Use the space we did get */
3441
        tbase = br;
3442
        tsize = asize;
3443
      }
3444
      else
3445
        disable_contiguous(m); /* Don't try contiguous path in the future */
26855 ripley 3446
    }
10347 ripley 3447
 
40788 ripley 3448
    RELEASE_MORECORE_LOCK();
26945 ripley 3449
  }
3450
 
40788 ripley 3451
  if (HAVE_MMAP && tbase == CMFAIL) {  /* Try MMAP */
3452
    size_t req = nb + TOP_FOOT_SIZE + SIZE_T_ONE;
3453
    size_t rsize = granularity_align(req);
3454
    if (rsize > nb) { /* Fail if wraps around zero */
3455
      char* mp = (char*)(CALL_MMAP(rsize));
3456
      if (mp != CMFAIL) {
3457
        tbase = mp;
3458
        tsize = rsize;
3459
        mmap_flag = IS_MMAPPED_BIT;
10347 ripley 3460
      }
3461
    }
3462
  }
3463
 
40788 ripley 3464
  if (HAVE_MORECORE && tbase == CMFAIL) { /* Try noncontiguous MORECORE */
3465
    size_t asize = granularity_align(nb + TOP_FOOT_SIZE + SIZE_T_ONE);
3466
    if (asize < HALF_MAX_SIZE_T) {
3467
      char* br = CMFAIL;
3468
      char* end = CMFAIL;
3469
      ACQUIRE_MORECORE_LOCK();
3470
      br = (char*)(CALL_MORECORE(asize));
3471
      end = (char*)(CALL_MORECORE(0));
3472
      RELEASE_MORECORE_LOCK();
3473
      if (br != CMFAIL && end != CMFAIL && br < end) {
3474
        size_t ssize = end - br;
3475
        if (ssize > nb + TOP_FOOT_SIZE) {
3476
          tbase = br;
3477
          tsize = ssize;
3478
        }
3479
      }
26855 ripley 3480
    }
40788 ripley 3481
  }
10347 ripley 3482
 
40788 ripley 3483
  if (tbase != CMFAIL) {
10347 ripley 3484
 
40788 ripley 3485
    if ((m->footprint += tsize) > m->max_footprint)
3486
      m->max_footprint = m->footprint;
10347 ripley 3487
 
40788 ripley 3488
    if (!is_initialized(m)) { /* first-time initialization */
3489
      m->seg.base = m->least_addr = tbase;
3490
      m->seg.size = tsize;
3491
      m->seg.sflags = mmap_flag;
3492
      m->magic = mparams.magic;
3493
      init_bins(m);
3494
      if (is_global(m)) 
3495
        init_top(m, (mchunkptr)tbase, tsize - TOP_FOOT_SIZE);
3496
      else {
3497
        /* Offset top by embedded malloc_state */
3498
        mchunkptr mn = next_chunk(mem2chunk(m));
3499
        init_top(m, mn, (size_t)((tbase + tsize) - (char*)mn) -TOP_FOOT_SIZE);
3500
      }
3501
    }
10347 ripley 3502
 
26855 ripley 3503
    else {
40788 ripley 3504
      /* Try to merge with an existing segment */
3505
      msegmentptr sp = &m->seg;
3506
      while (sp != 0 && tbase != sp->base + sp->size)
3507
        sp = sp->next;
3508
      if (sp != 0 &&
3509
          !is_extern_segment(sp) &&
3510
          (sp->sflags & IS_MMAPPED_BIT) == mmap_flag &&
3511
          segment_holds(sp, m->top)) { /* append */
3512
        sp->size += tsize;
3513
        init_top(m, m->top, m->topsize + tsize);
26855 ripley 3514
      }
40788 ripley 3515
      else {
3516
        if (tbase < m->least_addr)
3517
          m->least_addr = tbase;
3518
        sp = &m->seg;
3519
        while (sp != 0 && sp->base != tbase + tsize)
3520
          sp = sp->next;
3521
        if (sp != 0 &&
3522
            !is_extern_segment(sp) &&
3523
            (sp->sflags & IS_MMAPPED_BIT) == mmap_flag) {
3524
          char* oldbase = sp->base;
3525
          sp->base = tbase;
3526
          sp->size += tsize;
3527
          return prepend_alloc(m, tbase, oldbase, nb);
26855 ripley 3528
        }
40788 ripley 3529
        else
3530
          add_segment(m, tbase, tsize, mmap_flag);
26855 ripley 3531
      }
3532
    }
10347 ripley 3533
 
40788 ripley 3534
    if (nb < m->topsize) { /* Allocate from new or extended top space */
3535
      size_t rsize = m->topsize -= nb;
3536
      mchunkptr p = m->top;
3537
      mchunkptr r = m->top = chunk_plus_offset(p, nb);
3538
      r->head = rsize | PINUSE_BIT;
3539
      set_size_and_pinuse_of_inuse_chunk(m, p, nb);
3540
      check_top_chunk(m, m->top);
3541
      check_malloced_chunk(m, chunk2mem(p), nb);
26855 ripley 3542
      return chunk2mem(p);
3543
    }
3544
  }
10347 ripley 3545
 
26855 ripley 3546
  MALLOC_FAILURE_ACTION;
3547
  return 0;
3548
}
3549
 
40788 ripley 3550
/* -----------------------  system deallocation -------------------------- */
26855 ripley 3551
 
40788 ripley 3552
/* Unmap and unlink any mmapped segments that don't contain used chunks */
3553
static size_t release_unused_segments(mstate m) {
3554
  size_t released = 0;
3555
  msegmentptr pred = &m->seg;
3556
  msegmentptr sp = pred->next;
3557
  while (sp != 0) {
3558
    char* base = sp->base;
3559
    size_t size = sp->size;
3560
    msegmentptr next = sp->next;
3561
    if (is_mmapped_segment(sp) && !is_extern_segment(sp)) {
3562
      mchunkptr p = align_as_chunk(base);
3563
      size_t psize = chunksize(p);
3564
      /* Can unmap if first chunk holds entire segment and not pinned */
3565
      if (!cinuse(p) && (char*)p + psize >= base + size - TOP_FOOT_SIZE) {
3566
        tchunkptr tp = (tchunkptr)p;
3567
        assert(segment_holds(sp, (char*)sp));
3568
        if (p == m->dv) {
3569
          m->dv = 0;
3570
          m->dvsize = 0;
26855 ripley 3571
        }
40788 ripley 3572
        else {
3573
          unlink_large_chunk(m, tp);
3574
        }
3575
        if (CALL_MUNMAP(base, size) == 0) {
3576
          released += size;
3577
          m->footprint -= size;
3578
          /* unlink obsoleted record */
3579
          sp = pred;
3580
          sp->next = next;
3581
        }
3582
        else { /* back out if cannot unmap */
3583
          insert_large_chunk(m, tp, psize);
3584
        }
26855 ripley 3585
      }
3586
    }
40788 ripley 3587
    pred = sp;
3588
    sp = next;
26855 ripley 3589
  }
40788 ripley 3590
  return released;
26855 ripley 3591
}
3592
 
40788 ripley 3593
static int sys_trim(mstate m, size_t pad) {
3594
  size_t released = 0;
3595
  if (pad < MAX_REQUEST && is_initialized(m)) {
3596
    pad += TOP_FOOT_SIZE; /* ensure enough room for segment overhead */
26855 ripley 3597
 
40788 ripley 3598
    if (m->topsize > pad) {
3599
      /* Shrink top space in granularity-size units, keeping at least one */
3600
      size_t unit = mparams.granularity;
3601
      size_t extra = ((m->topsize - pad + (unit - SIZE_T_ONE)) / unit -
3602
                      SIZE_T_ONE) * unit;
3603
      msegmentptr sp = segment_holding(m, (char*)m->top);
26855 ripley 3604
 
40788 ripley 3605
      if (!is_extern_segment(sp)) {
3606
        if (is_mmapped_segment(sp)) {
3607
          if (HAVE_MMAP &&
3608
              sp->size >= extra &&
3609
              !has_segment_link(m, sp)) { /* can't shrink if pinned */
3610
            size_t newsize = sp->size - extra;
3611
            /* Prefer mremap, fall back to munmap */
3612
            if ((CALL_MREMAP(sp->base, sp->size, newsize, 0) != MFAIL) ||
3613
                (CALL_MUNMAP(sp->base + newsize, extra) == 0)) {
3614
              released = extra;
3615
            }
3616
          }
3617
        }
3618
        else if (HAVE_MORECORE) {
3619
          if (extra >= HALF_MAX_SIZE_T) /* Avoid wrapping negative */
3620
            extra = (HALF_MAX_SIZE_T) + SIZE_T_ONE - unit;
3621
          ACQUIRE_MORECORE_LOCK();
3622
          {
3623
            /* Make sure end of memory is where we last set it. */
3624
            char* old_br = (char*)(CALL_MORECORE(0));
3625
            if (old_br == sp->base + sp->size) {
3626
              char* rel_br = (char*)(CALL_MORECORE(-extra));
3627
              char* new_br = (char*)(CALL_MORECORE(0));
3628
              if (rel_br != CMFAIL && new_br < old_br)
3629
                released = old_br - new_br;
3630
            }
3631
          }
3632
          RELEASE_MORECORE_LOCK();
3633
        }
3634
      }
10347 ripley 3635
 
40788 ripley 3636
      if (released != 0) {
3637
        sp->size -= released;
3638
        m->footprint -= released;
3639
        init_top(m, m->top, m->topsize - released);
3640
        check_top_chunk(m, m->top);
3641
      }
3642
    }
10347 ripley 3643
 
40788 ripley 3644
    /* Unmap any unused mmapped segments */
3645
    if (HAVE_MMAP) 
3646
      released += release_unused_segments(m);
10425 ripley 3647
 
40788 ripley 3648
    /* On failure, disable autotrim to avoid repeated failed future calls */
3649
    if (released == 0)
3650
      m->trim_check = MAX_SIZE_T;
3651
  }
10425 ripley 3652
 
40788 ripley 3653
  return (released != 0)? 1 : 0;
3654
}
10347 ripley 3655
 
40788 ripley 3656
/* ---------------------------- malloc support --------------------------- */
10347 ripley 3657
 
40788 ripley 3658
/* allocate a large request from the best fitting chunk in a treebin */
3659
static void* tmalloc_large(mstate m, size_t nb) {
3660
  tchunkptr v = 0;
3661
  size_t rsize = -nb; /* Unsigned negation */
3662
  tchunkptr t;
3663
  bindex_t idx;
3664
  compute_tree_index(nb, idx);
10347 ripley 3665
 
40788 ripley 3666
  if ((t = *treebin_at(m, idx)) != 0) {
3667
    /* Traverse tree for this bin looking for node with size == nb */
3668
    size_t sizebits = nb << leftshift_for_tree_index(idx);
3669
    tchunkptr rst = 0;  /* The deepest untaken right subtree */
3670
    for (;;) {
3671
      tchunkptr rt;
3672
      size_t trem = chunksize(t) - nb;
3673
      if (trem < rsize) {
3674
        v = t;
3675
        if ((rsize = trem) == 0)
3676
          break;
3677
      }
3678
      rt = t->child[1];
3679
      t = t->child[(sizebits >> (SIZE_T_BITSIZE-SIZE_T_ONE)) & 1];
3680
      if (rt != 0 && rt != t)
3681
        rst = rt;
3682
      if (t == 0) {
3683
        t = rst; /* set t to least subtree holding sizes > nb */
3684
        break;
3685
      }
3686
      sizebits <<= 1;
3687
    }
26855 ripley 3688
  }
3689
 
40788 ripley 3690
  if (t == 0 && v == 0) { /* set t to root of next non-empty treebin */
3691
    binmap_t leftbits = left_bits(idx2bit(idx)) & m->treemap;
3692
    if (leftbits != 0) {
3693
      bindex_t i;
3694
      binmap_t leastbit = least_bit(leftbits);
3695
      compute_bit2idx(leastbit, i);
3696
      t = *treebin_at(m, i);
10347 ripley 3697
    }
3698
  }
3699
 
40788 ripley 3700
  while (t != 0) { /* find smallest of tree or subtree */
3701
    size_t trem = chunksize(t) - nb;
3702
    if (trem < rsize) {
3703
      rsize = trem;
3704
      v = t;
10347 ripley 3705
    }
40788 ripley 3706
    t = leftmost_child(t);
26855 ripley 3707
  }
10347 ripley 3708
 
40788 ripley 3709
  /*  If dv is a better fit, return 0 so malloc will use it */
3710
  if (v != 0 && rsize < (size_t)(m->dvsize - nb)) {
3711
    if (RTCHECK(ok_address(m, v))) { /* split */
3712
      mchunkptr r = chunk_plus_offset(v, nb);
3713
      assert(chunksize(v) == rsize + nb);
3714
      if (RTCHECK(ok_next(v, r))) {
3715
        unlink_large_chunk(m, v);
3716
        if (rsize < MIN_CHUNK_SIZE)
3717
          set_inuse_and_pinuse(m, v, (rsize + nb));
3718
        else {
3719
          set_size_and_pinuse_of_inuse_chunk(m, v, nb);
3720
          set_size_and_pinuse_of_free_chunk(r, rsize);
3721
          insert_chunk(m, r, rsize);
26855 ripley 3722
        }
40788 ripley 3723
        return chunk2mem(v);
26855 ripley 3724
      }
3725
    }
40788 ripley 3726
    CORRUPTION_ERROR_ACTION(m);
10347 ripley 3727
  }
40788 ripley 3728
  return 0;
26855 ripley 3729
}
10347 ripley 3730
 
40788 ripley 3731
/* allocate a small request from the best fitting chunk in a treebin */
3732
static void* tmalloc_small(mstate m, size_t nb) {
3733
  tchunkptr t, v;
3734
  size_t rsize;
3735
  bindex_t i;
3736
  binmap_t leastbit = least_bit(m->treemap);
3737
  compute_bit2idx(leastbit, i);
10347 ripley 3738
 
40788 ripley 3739
  v = t = *treebin_at(m, i);
3740
  rsize = chunksize(t) - nb;
10347 ripley 3741
 
40788 ripley 3742
  while ((t = leftmost_child(t)) != 0) {
3743
    size_t trem = chunksize(t) - nb;
3744
    if (trem < rsize) {
3745
      rsize = trem;
3746
      v = t;
26855 ripley 3747
    }
40788 ripley 3748
  }
10347 ripley 3749
 
40788 ripley 3750
  if (RTCHECK(ok_address(m, v))) {
3751
    mchunkptr r = chunk_plus_offset(v, nb);
3752
    assert(chunksize(v) == rsize + nb);
3753
    if (RTCHECK(ok_next(v, r))) {
3754
      unlink_large_chunk(m, v);
3755
      if (rsize < MIN_CHUNK_SIZE)
3756
        set_inuse_and_pinuse(m, v, (rsize + nb));
26855 ripley 3757
      else {
40788 ripley 3758
        set_size_and_pinuse_of_inuse_chunk(m, v, nb);
3759
        set_size_and_pinuse_of_free_chunk(r, rsize);
3760
        replace_dv(m, r, rsize);
26855 ripley 3761
      }
40788 ripley 3762
      return chunk2mem(v);
26855 ripley 3763
    }
3764
  }
10347 ripley 3765
 
40788 ripley 3766
  CORRUPTION_ERROR_ACTION(m);
3767
  return 0;
10347 ripley 3768
}
3769
 
40788 ripley 3770
/* --------------------------- realloc support --------------------------- */
10347 ripley 3771
 
40788 ripley 3772
static void* internal_realloc(mstate m, void* oldmem, size_t bytes) {
3773
  if (bytes >= MAX_REQUEST) {
3774
    MALLOC_FAILURE_ACTION;
26855 ripley 3775
    return 0;
3776
  }
40788 ripley 3777
  if (!PREACTION(m)) {
3778
    mchunkptr oldp = mem2chunk(oldmem);
3779
    size_t oldsize = chunksize(oldp);
3780
    mchunkptr next = chunk_plus_offset(oldp, oldsize);
3781
    mchunkptr newp = 0;
3782
    void* extra = 0;
10347 ripley 3783
 
40788 ripley 3784
    /* Try to either shrink or extend into top. Else malloc-copy-free */
10347 ripley 3785
 
40788 ripley 3786
    if (RTCHECK(ok_address(m, oldp) && ok_cinuse(oldp) &&
3787
                ok_next(oldp, next) && ok_pinuse(next))) {
3788
      size_t nb = request2size(bytes);
3789
      if (is_mmapped(oldp))
3790
        newp = mmap_resize(m, oldp, nb);
3791
      else if (oldsize >= nb) { /* already big enough */
3792
        size_t rsize = oldsize - nb;
3793
        newp = oldp;
3794
        if (rsize >= MIN_CHUNK_SIZE) {
3795
          mchunkptr remainder = chunk_plus_offset(newp, nb);
3796
          set_inuse(m, newp, nb);
3797
          set_inuse(m, remainder, rsize);
3798
          extra = chunk2mem(remainder);
3799
        }
10347 ripley 3800
      }
40788 ripley 3801
      else if (next == m->top && oldsize + m->topsize > nb) {
3802
        /* Expand into top */
3803
        size_t newsize = oldsize + m->topsize;
3804
        size_t newtopsize = newsize - nb;
3805
        mchunkptr newtop = chunk_plus_offset(oldp, nb);
3806
        set_inuse(m, oldp, nb);
3807
        newtop->head = newtopsize |PINUSE_BIT;
3808
        m->top = newtop;
3809
        m->topsize = newtopsize;
26855 ripley 3810
        newp = oldp;
10347 ripley 3811
      }
3812
    }
40788 ripley 3813
    else {
3814
      USAGE_ERROR_ACTION(m, oldmem);
3815
      POSTACTION(m);
3816
      return 0;
10347 ripley 3817
    }
3818
 
40788 ripley 3819
    POSTACTION(m);
10347 ripley 3820
 
40788 ripley 3821
    if (newp != 0) {
3822
      if (extra != 0) {
3823
        internal_free(m, extra);
3824
      }
3825
      check_inuse_chunk(m, newp);
26855 ripley 3826
      return chunk2mem(newp);
3827
    }
3828
    else {
40788 ripley 3829
      void* newmem = internal_malloc(m, bytes);
26855 ripley 3830
      if (newmem != 0) {
40788 ripley 3831
        size_t oc = oldsize - overhead_for(oldp);
3832
        memcpy(newmem, oldmem, (oc < bytes)? oc : bytes);
3833
        internal_free(m, oldmem);
26855 ripley 3834
      }
40788 ripley 3835
      return newmem;
26855 ripley 3836
    }
3837
  }
40788 ripley 3838
  return 0;
26855 ripley 3839
}
10347 ripley 3840
 
40788 ripley 3841
/* --------------------------- memalign support -------------------------- */
10347 ripley 3842
 
40801 ripley 3843
#if 0
40788 ripley 3844
static void* internal_memalign(mstate m, size_t alignment, size_t bytes) {
3845
  if (alignment <= MALLOC_ALIGNMENT)    /* Can just use malloc */
3846
    return internal_malloc(m, bytes);
3847
  if (alignment <  MIN_CHUNK_SIZE) /* must be at least a minimum chunk size */
3848
    alignment = MIN_CHUNK_SIZE;
3849
  if ((alignment & (alignment-SIZE_T_ONE)) != 0) {/* Ensure a power of 2 */
3850
    size_t a = MALLOC_ALIGNMENT << 1;
3851
    while (a < alignment) a <<= 1;
26855 ripley 3852
    alignment = a;
3853
  }
40788 ripley 3854
 
3855
  if (bytes >= MAX_REQUEST - alignment) {
3856
    if (m != 0)  { /* Test isn't needed but avoids compiler warning */
3857
      MALLOC_FAILURE_ACTION;
10347 ripley 3858
    }
3859
  }
40788 ripley 3860
  else {
3861
    size_t nb = request2size(bytes);
3862
    size_t req = nb + alignment + MIN_CHUNK_SIZE - CHUNK_OVERHEAD;
3863
    char* mem = (char*)internal_malloc(m, req);
3864
    if (mem != 0) {
3865
      void* leader = 0;
3866
      void* trailer = 0;
3867
      mchunkptr p = mem2chunk(mem);
10347 ripley 3868
 
40788 ripley 3869
      if (PREACTION(m)) return 0;
3870
      if ((((size_t)(mem)) % alignment) != 0) { /* misaligned */
3871
        /*
3872
          Find an aligned spot inside chunk.  Since we need to give
3873
          back leading space in a chunk of at least MIN_CHUNK_SIZE, if
3874
          the first calculation places us at a spot with less than
3875
          MIN_CHUNK_SIZE leader, we can move to the next aligned spot.
3876
          We've allocated enough total room so that this is always
3877
          possible.
3878
        */
3879
        char* br = (char*)mem2chunk((size_t)(((size_t)(mem +
3880
                                                       alignment -
3881
                                                       SIZE_T_ONE)) &
3882
                                             -alignment));
3883
        char* pos = ((size_t)(br - (char*)(p)) >= MIN_CHUNK_SIZE)?
3884
          br : br+alignment;
3885
        mchunkptr newp = (mchunkptr)pos;
3886
        size_t leadsize = pos - (char*)(p);
3887
        size_t newsize = chunksize(p) - leadsize;
10347 ripley 3888
 
40788 ripley 3889
        if (is_mmapped(p)) { /* For mmapped chunks, just adjust offset */
3890
          newp->prev_foot = p->prev_foot + leadsize;
3891
          newp->head = (newsize|CINUSE_BIT);
3892
        }
3893
        else { /* Otherwise, give back leader, use the rest */
3894
          set_inuse(m, newp, newsize);
3895
          set_inuse(m, p, leadsize);
3896
          leader = chunk2mem(p);
3897
        }
3898
        p = newp;
3899
      }
10347 ripley 3900
 
40788 ripley 3901
      /* Give back spare room at the end */
3902
      if (!is_mmapped(p)) {
3903
        size_t size = chunksize(p);
3904
        if (size > nb + MIN_CHUNK_SIZE) {
3905
          size_t remainder_size = size - nb;
3906
          mchunkptr remainder = chunk_plus_offset(p, nb);
3907
          set_inuse(m, p, nb);
3908
          set_inuse(m, remainder, remainder_size);
3909
          trailer = chunk2mem(remainder);
26855 ripley 3910
        }
3911
      }
40788 ripley 3912
 
3913
      assert (chunksize(p) >= nb);
3914
      assert((((size_t)(chunk2mem(p))) % alignment) == 0);
3915
      check_inuse_chunk(m, p);
3916
      POSTACTION(m);
3917
      if (leader != 0) {
3918
        internal_free(m, leader);
3919
      }
3920
      if (trailer != 0) {
3921
        internal_free(m, trailer);
3922
      }
3923
      return chunk2mem(p);
26855 ripley 3924
    }
3925
  }
40788 ripley 3926
  return 0;
10347 ripley 3927
}
3928
 
40788 ripley 3929
/* ------------------------ comalloc/coalloc support --------------------- */
10347 ripley 3930
 
40788 ripley 3931
static void** ialloc(mstate m,
3932
                     size_t n_elements,
3933
                     size_t* sizes,
3934
                     int opts,
3935
                     void* chunks[]) {
3936
  /*
3937
    This provides common support for independent_X routines, handling
3938
    all of the combinations that can result.
10347 ripley 3939
 
40788 ripley 3940
    The opts arg has:
26855 ripley 3941
    bit 0 set if all elements are same size (using sizes[0])
3942
    bit 1 set if elements should be zeroed
40788 ripley 3943
  */
10347 ripley 3944
 
40788 ripley 3945
  size_t    element_size;   /* chunksize of each element, if all same */
3946
  size_t    contents_size;  /* total size of elements */
3947
  size_t    array_size;     /* request size of pointer array */
3948
  void*     mem;            /* malloced aggregate space */
3949
  mchunkptr p;              /* corresponding chunk */
3950
  size_t    remainder_size; /* remaining bytes while splitting */
3951
  void**    marray;         /* either "chunks" or malloced ptr array */
3952
  mchunkptr array_chunk;    /* chunk for malloced ptr array */
3953
  flag_t    was_enabled;    /* to disable mmap */
3954
  size_t    size;
3955
  size_t    i;
26855 ripley 3956
 
3957
  /* compute array length, if needed */
3958
  if (chunks != 0) {
3959
    if (n_elements == 0)
3960
      return chunks; /* nothing to do */
3961
    marray = chunks;
3962
    array_size = 0;
3963
  }
3964
  else {
3965
    /* if empty req, must still return chunk representing empty array */
40788 ripley 3966
    if (n_elements == 0)
3967
      return (void**)internal_malloc(m, 0);
26855 ripley 3968
    marray = 0;
40788 ripley 3969
    array_size = request2size(n_elements * (sizeof(void*)));
26855 ripley 3970
  }
10425 ripley 3971
 
26855 ripley 3972
  /* compute total element size */
3973
  if (opts & 0x1) { /* all-same-size */
3974
    element_size = request2size(*sizes);
3975
    contents_size = n_elements * element_size;
3976
  }
3977
  else { /* add up all the sizes */
3978
    element_size = 0;
3979
    contents_size = 0;
40788 ripley 3980
    for (i = 0; i != n_elements; ++i)
3981
      contents_size += request2size(sizes[i]);
26855 ripley 3982
  }
10347 ripley 3983
 
40788 ripley 3984
  size = contents_size + array_size;
3985
 
3986
  /*
3987
     Allocate the aggregate chunk.  First disable direct-mmapping so
3988
     malloc won't use it, since we would not be able to later
3989
     free/realloc space internal to a segregated mmap region.
3990
  */
3991
  was_enabled = use_mmap(m);
3992
  disable_mmap(m);
3993
  mem = internal_malloc(m, size - CHUNK_OVERHEAD);
3994
  if (was_enabled)
3995
    enable_mmap(m);
3996
  if (mem == 0)
10347 ripley 3997
    return 0;
3998
 
40788 ripley 3999
  if (PREACTION(m)) return 0;
26855 ripley 4000
  p = mem2chunk(mem);
4001
  remainder_size = chunksize(p);
10347 ripley 4002
 
40788 ripley 4003
  assert(!is_mmapped(p));
4004
 
26855 ripley 4005
  if (opts & 0x2) {       /* optionally clear the elements */
40788 ripley 4006
    memset((size_t*)mem, 0, remainder_size - SIZE_T_SIZE - array_size);
26855 ripley 4007
  }
10347 ripley 4008
 
26855 ripley 4009
  /* If not provided, allocate the pointer array as final part of chunk */
4010
  if (marray == 0) {
40788 ripley 4011
    size_t  array_chunk_size;
4012
    array_chunk = chunk_plus_offset(p, contents_size);
4013
    array_chunk_size = remainder_size - contents_size;
4014
    marray = (void**) (chunk2mem(array_chunk));
4015
    set_size_and_pinuse_of_inuse_chunk(m, array_chunk, array_chunk_size);
26855 ripley 4016
    remainder_size = contents_size;
4017
  }
10347 ripley 4018
 
26855 ripley 4019
  /* split out elements */
4020
  for (i = 0; ; ++i) {
4021
    marray[i] = chunk2mem(p);
4022
    if (i != n_elements-1) {
40788 ripley 4023
      if (element_size != 0)
26855 ripley 4024
        size = element_size;
4025
      else
40788 ripley 4026
        size = request2size(sizes[i]);
26855 ripley 4027
      remainder_size -= size;
40788 ripley 4028
      set_size_and_pinuse_of_inuse_chunk(m, p, size);
4029
      p = chunk_plus_offset(p, size);
26855 ripley 4030
    }
4031
    else { /* the final element absorbs any overallocation slop */
40788 ripley 4032
      set_size_and_pinuse_of_inuse_chunk(m, p, remainder_size);
26855 ripley 4033
      break;
4034
    }
4035
  }
10347 ripley 4036
 
26855 ripley 4037
#if DEBUG
4038
  if (marray != chunks) {
4039
    /* final element must have exactly exhausted chunk */
40788 ripley 4040
    if (element_size != 0) {
26855 ripley 4041
      assert(remainder_size == element_size);
40788 ripley 4042
    }
4043
    else {
26855 ripley 4044
      assert(remainder_size == request2size(sizes[i]));
40788 ripley 4045
    }
4046
    check_inuse_chunk(m, mem2chunk(marray));
26855 ripley 4047
  }
4048
  for (i = 0; i != n_elements; ++i)
40788 ripley 4049
    check_inuse_chunk(m, mem2chunk(marray[i]));
10347 ripley 4050
 
40788 ripley 4051
#endif /* DEBUG */
4052
 
4053
  POSTACTION(m);
26855 ripley 4054
  return marray;
10347 ripley 4055
}
40801 ripley 4056
#endif
10347 ripley 4057
 
40788 ripley 4058
/* -------------------------- public routines ---------------------------- */
10347 ripley 4059
 
40788 ripley 4060
#if !ONLY_MSPACES
4061
 
4062
void* dlmalloc(size_t bytes) {
4063
  /*
4064
     Basic algorithm:
4065
     If a small request (< 256 bytes minus per-chunk overhead):
4066
       1. If one exists, use a remainderless chunk in associated smallbin.
4067
          (Remainderless means that there are too few excess bytes to
4068
          represent as a chunk.)
4069
       2. If it is big enough, use the dv chunk, which is normally the
4070
          chunk adjacent to the one used for the most recent small request.
4071
       3. If one exists, split the smallest available chunk in a bin,
4072
          saving remainder in dv.
4073
       4. If it is big enough, use the top chunk.
4074
       5. If available, get memory from system and use it
4075
     Otherwise, for a large request:
4076
       1. Find the smallest available binned chunk that fits, and use it
4077
          if it is better fitting than dv chunk, splitting if necessary.
4078
       2. If better fitting than any binned chunk, use the dv chunk.
4079
       3. If it is big enough, use the top chunk.
4080
       4. If request size >= mmap threshold, try to directly mmap this chunk.
4081
       5. If available, get memory from system and use it
4082
 
4083
     The ugly goto's here ensure that postaction occurs along all paths.
4084
  */
4085
 
4086
  if (!PREACTION(gm)) {
4087
    void* mem;
4088
    size_t nb;
4089
    if (bytes <= MAX_SMALL_REQUEST) {
4090
      bindex_t idx;
4091
      binmap_t smallbits;
4092
      nb = (bytes < MIN_REQUEST)? MIN_CHUNK_SIZE : pad_request(bytes);
4093
      idx = small_index(nb);
4094
      smallbits = gm->smallmap >> idx;
4095
 
4096
      if ((smallbits & 0x3U) != 0) { /* Remainderless fit to a smallbin. */
4097
        mchunkptr b, p;
4098
        idx += ~smallbits & 1;       /* Uses next bin if idx empty */
4099
        b = smallbin_at(gm, idx);
4100
        p = b->fd;
4101
        assert(chunksize(p) == small_index2size(idx));
4102
        unlink_first_small_chunk(gm, b, p, idx);
4103
        set_inuse_and_pinuse(gm, p, small_index2size(idx));
4104
        mem = chunk2mem(p);
4105
        check_malloced_chunk(gm, mem, nb);
4106
        goto postaction;
4107
      }
4108
 
4109
      else if (nb > gm->dvsize) {
4110
        if (smallbits != 0) { /* Use chunk in next nonempty smallbin */
4111
          mchunkptr b, p, r;
4112
          size_t rsize;
4113
          bindex_t i;
4114
          binmap_t leftbits = (smallbits << idx) & left_bits(idx2bit(idx));
4115
          binmap_t leastbit = least_bit(leftbits);
4116
          compute_bit2idx(leastbit, i);
4117
          b = smallbin_at(gm, i);
4118
          p = b->fd;
4119
          assert(chunksize(p) == small_index2size(i));
4120
          unlink_first_small_chunk(gm, b, p, i);
4121
          rsize = small_index2size(i) - nb;
4122
          /* Fit here cannot be remainderless if 4byte sizes */
4123
          if (SIZE_T_SIZE != 4 && rsize < MIN_CHUNK_SIZE)
4124
            set_inuse_and_pinuse(gm, p, small_index2size(i));
4125
          else {
4126
            set_size_and_pinuse_of_inuse_chunk(gm, p, nb);
4127
            r = chunk_plus_offset(p, nb);
4128
            set_size_and_pinuse_of_free_chunk(r, rsize);
4129
            replace_dv(gm, r, rsize);
4130
          }
4131
          mem = chunk2mem(p);
4132
          check_malloced_chunk(gm, mem, nb);
4133
          goto postaction;
4134
        }
4135
 
4136
        else if (gm->treemap != 0 && (mem = tmalloc_small(gm, nb)) != 0) {
4137
          check_malloced_chunk(gm, mem, nb);
4138
          goto postaction;
4139
        }
4140
      }
4141
    }
4142
    else if (bytes >= MAX_REQUEST)
4143
      nb = MAX_SIZE_T; /* Too big to allocate. Force failure (in sys alloc) */
4144
    else {
4145
      nb = pad_request(bytes);
4146
      if (gm->treemap != 0 && (mem = tmalloc_large(gm, nb)) != 0) {
4147
        check_malloced_chunk(gm, mem, nb);
4148
        goto postaction;
4149
      }
4150
    }
4151
 
4152
    if (nb <= gm->dvsize) {
4153
      size_t rsize = gm->dvsize - nb;
4154
      mchunkptr p = gm->dv;
4155
      if (rsize >= MIN_CHUNK_SIZE) { /* split dv */
4156
        mchunkptr r = gm->dv = chunk_plus_offset(p, nb);
4157
        gm->dvsize = rsize;
4158
        set_size_and_pinuse_of_free_chunk(r, rsize);
4159
        set_size_and_pinuse_of_inuse_chunk(gm, p, nb);
4160
      }
4161
      else { /* exhaust dv */
4162
        size_t dvs = gm->dvsize;
4163
        gm->dvsize = 0;
4164
        gm->dv = 0;
4165
        set_inuse_and_pinuse(gm, p, dvs);
4166
      }
4167
      mem = chunk2mem(p);
4168
      check_malloced_chunk(gm, mem, nb);
4169
      goto postaction;
4170
    }
4171
 
4172
    else if (nb < gm->topsize) { /* Split top */
4173
      size_t rsize = gm->topsize -= nb;
4174
      mchunkptr p = gm->top;
4175
      mchunkptr r = gm->top = chunk_plus_offset(p, nb);
4176
      r->head = rsize | PINUSE_BIT;
4177
      set_size_and_pinuse_of_inuse_chunk(gm, p, nb);
4178
      mem = chunk2mem(p);
4179
      check_top_chunk(gm, gm->top);
4180
      check_malloced_chunk(gm, mem, nb);
4181
      goto postaction;
4182
    }
4183
 
4184
    mem = sys_alloc(gm, nb);
4185
 
4186
  postaction:
4187
    POSTACTION(gm);
4188
    return mem;
4189
  }
4190
 
4191
  return 0;
26855 ripley 4192
}
10347 ripley 4193
 
40788 ripley 4194
void dlfree(void* mem) {
4195
  /*
4196
     Consolidate freed chunks with preceeding or succeeding bordering
4197
     free chunks, if they exist, and then place in a bin.  Intermixed
4198
     with special cases for top, dv, mmapped chunks, and usage errors.
4199
  */
10347 ripley 4200
 
40788 ripley 4201
  if (mem != 0) {
4202
    mchunkptr p  = mem2chunk(mem);
4203
#if FOOTERS
4204
    mstate fm = get_mstate_for(p);
4205
    if (!ok_magic(fm)) {
4206
      USAGE_ERROR_ACTION(fm, p);
4207
      return;
4208
    }
4209
#else /* FOOTERS */
4210
#define fm gm
4211
#endif /* FOOTERS */
4212
    if (!PREACTION(fm)) {
4213
      check_inuse_chunk(fm, p);
4214
      if (RTCHECK(ok_address(fm, p) && ok_cinuse(p))) {
4215
        size_t psize = chunksize(p);
4216
        mchunkptr next = chunk_plus_offset(p, psize);
4217
        if (!pinuse(p)) {
4218
          size_t prevsize = p->prev_foot;
4219
          if ((prevsize & IS_MMAPPED_BIT) != 0) {
4220
            prevsize &= ~IS_MMAPPED_BIT;
4221
            psize += prevsize + MMAP_FOOT_PAD;
4222
            if (CALL_MUNMAP((char*)p - prevsize, psize) == 0)
4223
              fm->footprint -= psize;
4224
            goto postaction;
4225
          }
4226
          else {
4227
            mchunkptr prev = chunk_minus_offset(p, prevsize);
4228
            psize += prevsize;
4229
            p = prev;
4230
            if (RTCHECK(ok_address(fm, prev))) { /* consolidate backward */
4231
              if (p != fm->dv) {
4232
                unlink_chunk(fm, p, prevsize);
4233
              }
4234
              else if ((next->head & INUSE_BITS) == INUSE_BITS) {
4235
                fm->dvsize = psize;
4236
                set_free_with_pinuse(p, psize, next);
4237
                goto postaction;
4238
              }
4239
            }
4240
            else
4241
              goto erroraction;
4242
          }
4243
        }
26855 ripley 4244
 
40788 ripley 4245
        if (RTCHECK(ok_next(p, next) && ok_pinuse(next))) {
4246
          if (!cinuse(next)) {  /* consolidate forward */
4247
            if (next == fm->top) {
4248
              size_t tsize = fm->topsize += psize;
4249
              fm->top = p;
4250
              p->head = tsize | PINUSE_BIT;
4251
              if (p == fm->dv) {
4252
                fm->dv = 0;
4253
                fm->dvsize = 0;
4254
              }
4255
              if (should_trim(fm, tsize))
4256
                sys_trim(fm, 0);
4257
              goto postaction;
4258
            }
4259
            else if (next == fm->dv) {
4260
              size_t dsize = fm->dvsize += psize;
4261
              fm->dv = p;
4262
              set_size_and_pinuse_of_free_chunk(p, dsize);
4263
              goto postaction;
4264
            }
4265
            else {
4266
              size_t nsize = chunksize(next);
4267
              psize += nsize;
4268
              unlink_chunk(fm, next, nsize);
4269
              set_size_and_pinuse_of_free_chunk(p, psize);
4270
              if (p == fm->dv) {
4271
                fm->dvsize = psize;
4272
                goto postaction;
4273
              }
4274
            }
4275
          }
4276
          else
4277
            set_free_with_pinuse(p, psize, next);
4278
          insert_chunk(fm, p, psize);
4279
          check_free_chunk(fm, p);
4280
          goto postaction;
4281
        }
4282
      }
4283
    erroraction:
4284
      USAGE_ERROR_ACTION(fm, p);
4285
    postaction:
4286
      POSTACTION(fm);
4287
    }
4288
  }
4289
#if !FOOTERS
4290
#undef fm
4291
#endif /* FOOTERS */
4292
}
4293
 
4294
void* dlcalloc(size_t n_elements, size_t elem_size) {
4295
  void* mem;
4296
  size_t req = 0;
4297
  if (n_elements != 0) {
4298
    req = n_elements * elem_size;
4299
    if (((n_elements | elem_size) & ~(size_t)0xffff) &&
4300
        (req / n_elements != elem_size))
4301
      req = MAX_SIZE_T; /* force downstream failure on overflow */
4302
  }
4303
  mem = dlmalloc(req);
4304
  if (mem != 0 && calloc_must_clear(mem2chunk(mem)))
4305
    memset(mem, 0, req);
4306
  return mem;
4307
}
4308
 
4309
void* dlrealloc(void* oldmem, size_t bytes) {
4310
  if (oldmem == 0)
4311
    return dlmalloc(bytes);
4312
#ifdef REALLOC_ZERO_BYTES_FREES
4313
  if (bytes == 0) {
4314
    dlfree(oldmem);
4315
    return 0;
4316
  }
4317
#endif /* REALLOC_ZERO_BYTES_FREES */
4318
  else {
4319
#if ! FOOTERS
4320
    mstate m = gm;
4321
#else /* FOOTERS */
4322
    mstate m = get_mstate_for(mem2chunk(oldmem));
4323
    if (!ok_magic(m)) {
4324
      USAGE_ERROR_ACTION(m, oldmem);
4325
      return 0;
4326
    }
4327
#endif /* FOOTERS */
4328
    return internal_realloc(m, oldmem, bytes);
4329
  }
4330
}
4331
 
40801 ripley 4332
#if 0
40788 ripley 4333
void* dlmemalign(size_t alignment, size_t bytes) {
4334
  return internal_memalign(gm, alignment, bytes);
4335
}
4336
 
4337
void** dlindependent_calloc(size_t n_elements, size_t elem_size,
4338
                                 void* chunks[]) {
4339
  size_t sz = elem_size; /* serves as 1-element array */
4340
  return ialloc(gm, n_elements, &sz, 3, chunks);
4341
}
4342
 
4343
void** dlindependent_comalloc(size_t n_elements, size_t sizes[],
4344
                                   void* chunks[]) {
4345
  return ialloc(gm, n_elements, sizes, 0, chunks);
4346
}
4347
 
4348
void* dlvalloc(size_t bytes) {
26855 ripley 4349
  size_t pagesz;
40788 ripley 4350
  init_mparams();
4351
  pagesz = mparams.page_size;
4352
  return dlmemalign(pagesz, bytes);
4353
}
26855 ripley 4354
 
40788 ripley 4355
void* dlpvalloc(size_t bytes) {
4356
  size_t pagesz;
4357
  init_mparams();
4358
  pagesz = mparams.page_size;
4359
  return dlmemalign(pagesz, (bytes + pagesz - SIZE_T_ONE) & ~(pagesz - SIZE_T_ONE));
10347 ripley 4360
}
4361
 
40788 ripley 4362
int dlmalloc_trim(size_t pad) {
4363
  int result = 0;
4364
  if (!PREACTION(gm)) {
4365
    result = sys_trim(gm, pad);
4366
    POSTACTION(gm);
4367
  }
4368
  return result;
4369
}
10347 ripley 4370
 
40788 ripley 4371
size_t dlmalloc_footprint(void) {
4372
  return gm->footprint;
4373
}
10347 ripley 4374
 
40788 ripley 4375
size_t dlmalloc_max_footprint(void) {
4376
  return gm->max_footprint;
26855 ripley 4377
}
40801 ripley 4378
#endif
10347 ripley 4379
 
40788 ripley 4380
#if !NO_MALLINFO
4381
struct mallinfo dlmallinfo(void) {
4382
  return internal_mallinfo(gm);
4383
}
4384
#endif /* NO_MALLINFO */
10347 ripley 4385
 
40801 ripley 4386
#if 0
40788 ripley 4387
void dlmalloc_stats() {
4388
  internal_malloc_stats(gm);
4389
}
10347 ripley 4390
 
40788 ripley 4391
size_t dlmalloc_usable_size(void* mem) {
26855 ripley 4392
  if (mem != 0) {
40788 ripley 4393
    mchunkptr p = mem2chunk(mem);
4394
    if (cinuse(p))
4395
      return chunksize(p) - overhead_for(p);
26855 ripley 4396
  }
4397
  return 0;
4398
}
10347 ripley 4399
 
40788 ripley 4400
int dlmallopt(int param_number, int value) {
4401
  return change_mparam(param_number, value);
4402
}
40801 ripley 4403
#endif
10347 ripley 4404
 
40788 ripley 4405
#endif /* !ONLY_MSPACES */
10347 ripley 4406
 
40788 ripley 4407
/* ----------------------------- user mspaces ---------------------------- */
10347 ripley 4408
 
40788 ripley 4409
#if MSPACES
26855 ripley 4410
 
40788 ripley 4411
static mstate init_user_mstate(char* tbase, size_t tsize) {
4412
  size_t msize = pad_request(sizeof(struct malloc_state));
4413
  mchunkptr mn;
4414
  mchunkptr msp = align_as_chunk(tbase);
4415
  mstate m = (mstate)(chunk2mem(msp));
4416
  memset(m, 0, msize);
4417
  INITIAL_LOCK(&m->mutex);
4418
  msp->head = (msize|PINUSE_BIT|CINUSE_BIT);
4419
  m->seg.base = m->least_addr = tbase;
4420
  m->seg.size = m->footprint = m->max_footprint = tsize;
4421
  m->magic = mparams.magic;
4422
  m->mflags = mparams.default_mflags;
4423
  disable_contiguous(m);
4424
  init_bins(m);
4425
  mn = next_chunk(mem2chunk(m));
4426
  init_top(m, mn, (size_t)((tbase + tsize) - (char*)mn) - TOP_FOOT_SIZE);
4427
  check_top_chunk(m, m->top);
4428
  return m;
4429
}
26855 ripley 4430
 
40788 ripley 4431
mspace create_mspace(size_t capacity, int locked) {
4432
  mstate m = 0;
4433
  size_t msize = pad_request(sizeof(struct malloc_state));
4434
  init_mparams(); /* Ensure pagesize etc initialized */
26855 ripley 4435
 
40788 ripley 4436
  if (capacity < (size_t) -(msize + TOP_FOOT_SIZE + mparams.page_size)) {
4437
    size_t rs = ((capacity == 0)? mparams.granularity :
4438
                 (capacity + TOP_FOOT_SIZE + msize));
4439
    size_t tsize = granularity_align(rs);
4440
    char* tbase = (char*)(CALL_MMAP(tsize));
4441
    if (tbase != CMFAIL) {
4442
      m = init_user_mstate(tbase, tsize);
4443
      m->seg.sflags = IS_MMAPPED_BIT;
4444
      set_lock(m, locked);
26855 ripley 4445
    }
4446
  }
40788 ripley 4447
  return (mspace)m;
4448
}
26855 ripley 4449
 
40788 ripley 4450
mspace create_mspace_with_base(void* base, size_t capacity, int locked) {
4451
  mstate m = 0;
4452
  size_t msize = pad_request(sizeof(struct malloc_state));
4453
  init_mparams(); /* Ensure pagesize etc initialized */
26855 ripley 4454
 
40788 ripley 4455
  if (capacity > msize + TOP_FOOT_SIZE &&
4456
      capacity < (size_t) -(msize + TOP_FOOT_SIZE + mparams.page_size)) {
4457
    m = init_user_mstate((char*)base, capacity);
4458
    m->seg.sflags = EXTERN_BIT;
4459
    set_lock(m, locked);
26855 ripley 4460
  }
40788 ripley 4461
  return (mspace)m;
4462
}
26855 ripley 4463
 
40788 ripley 4464
size_t destroy_mspace(mspace msp) {
4465
  size_t freed = 0;
4466
  mstate ms = (mstate)msp;
4467
  if (ok_magic(ms)) {
4468
    msegmentptr sp = &ms->seg;
4469
    while (sp != 0) {
4470
      char* base = sp->base;
4471
      size_t size = sp->size;
4472
      flag_t flag = sp->sflags;
4473
      sp = sp->next;
4474
      if ((flag & IS_MMAPPED_BIT) && !(flag & EXTERN_BIT) &&
4475
          CALL_MUNMAP(base, size) == 0)
4476
        freed += size;
4477
    }
26855 ripley 4478
  }
40788 ripley 4479
  else {
4480
    USAGE_ERROR_ACTION(ms,ms);
4481
  }
4482
  return freed;
26855 ripley 4483
}
4484
 
4485
/*
40788 ripley 4486
  mspace versions of routines are near-clones of the global
4487
  versions. This is not so nice but better than the alternatives.
26855 ripley 4488
*/
4489
 
4490
 
40788 ripley 4491
void* mspace_malloc(mspace msp, size_t bytes) {
4492
  mstate ms = (mstate)msp;
4493
  if (!ok_magic(ms)) {
4494
    USAGE_ERROR_ACTION(ms,ms);
4495
    return 0;
26855 ripley 4496
  }
40788 ripley 4497
  if (!PREACTION(ms)) {
4498
    void* mem;
4499
    size_t nb;
4500
    if (bytes <= MAX_SMALL_REQUEST) {
4501
      bindex_t idx;
4502
      binmap_t smallbits;
4503
      nb = (bytes < MIN_REQUEST)? MIN_CHUNK_SIZE : pad_request(bytes);
4504
      idx = small_index(nb);
4505
      smallbits = ms->smallmap >> idx;
26855 ripley 4506
 
40788 ripley 4507
      if ((smallbits & 0x3U) != 0) { /* Remainderless fit to a smallbin. */
4508
        mchunkptr b, p;
4509
        idx += ~smallbits & 1;       /* Uses next bin if idx empty */
4510
        b = smallbin_at(ms, idx);
4511
        p = b->fd;
4512
        assert(chunksize(p) == small_index2size(idx));
4513
        unlink_first_small_chunk(ms, b, p, idx);
4514
        set_inuse_and_pinuse(ms, p, small_index2size(idx));
4515
        mem = chunk2mem(p);
4516
        check_malloced_chunk(ms, mem, nb);
4517
        goto postaction;
4518
      }
26855 ripley 4519
 
40788 ripley 4520
      else if (nb > ms->dvsize) {
4521
        if (smallbits != 0) { /* Use chunk in next nonempty smallbin */
4522
          mchunkptr b, p, r;
4523
          size_t rsize;
4524
          bindex_t i;
4525
          binmap_t leftbits = (smallbits << idx) & left_bits(idx2bit(idx));
4526
          binmap_t leastbit = least_bit(leftbits);
4527
          compute_bit2idx(leastbit, i);
4528
          b = smallbin_at(ms, i);
4529
          p = b->fd;
4530
          assert(chunksize(p) == small_index2size(i));
4531
          unlink_first_small_chunk(ms, b, p, i);
4532
          rsize = small_index2size(i) - nb;
4533
          /* Fit here cannot be remainderless if 4byte sizes */
4534
          if (SIZE_T_SIZE != 4 && rsize < MIN_CHUNK_SIZE)
4535
            set_inuse_and_pinuse(ms, p, small_index2size(i));
4536
          else {
4537
            set_size_and_pinuse_of_inuse_chunk(ms, p, nb);
4538
            r = chunk_plus_offset(p, nb);
4539
            set_size_and_pinuse_of_free_chunk(r, rsize);
4540
            replace_dv(ms, r, rsize);
4541
          }
4542
          mem = chunk2mem(p);
4543
          check_malloced_chunk(ms, mem, nb);
4544
          goto postaction;
4545
        }
26855 ripley 4546
 
40788 ripley 4547
        else if (ms->treemap != 0 && (mem = tmalloc_small(ms, nb)) != 0) {
4548
          check_malloced_chunk(ms, mem, nb);
4549
          goto postaction;
4550
        }
4551
      }
10347 ripley 4552
    }
40788 ripley 4553
    else if (bytes >= MAX_REQUEST)
4554
      nb = MAX_SIZE_T; /* Too big to allocate. Force failure (in sys alloc) */
4555
    else {
4556
      nb = pad_request(bytes);
4557
      if (ms->treemap != 0 && (mem = tmalloc_large(ms, nb)) != 0) {
4558
        check_malloced_chunk(ms, mem, nb);
4559
        goto postaction;
4560
      }
4561
    }
10347 ripley 4562
 
40788 ripley 4563
    if (nb <= ms->dvsize) {
4564
      size_t rsize = ms->dvsize - nb;
4565
      mchunkptr p = ms->dv;
4566
      if (rsize >= MIN_CHUNK_SIZE) { /* split dv */
4567
        mchunkptr r = ms->dv = chunk_plus_offset(p, nb);
4568
        ms->dvsize = rsize;
4569
        set_size_and_pinuse_of_free_chunk(r, rsize);
4570
        set_size_and_pinuse_of_inuse_chunk(ms, p, nb);
4571
      }
4572
      else { /* exhaust dv */
4573
        size_t dvs = ms->dvsize;
4574
        ms->dvsize = 0;
4575
        ms->dv = 0;
4576
        set_inuse_and_pinuse(ms, p, dvs);
4577
      }
4578
      mem = chunk2mem(p);
4579
      check_malloced_chunk(ms, mem, nb);
4580
      goto postaction;
4581
    }
10347 ripley 4582
 
40788 ripley 4583
    else if (nb < ms->topsize) { /* Split top */
4584
      size_t rsize = ms->topsize -= nb;
4585
      mchunkptr p = ms->top;
4586
      mchunkptr r = ms->top = chunk_plus_offset(p, nb);
4587
      r->head = rsize | PINUSE_BIT;
4588
      set_size_and_pinuse_of_inuse_chunk(ms, p, nb);
4589
      mem = chunk2mem(p);
4590
      check_top_chunk(ms, ms->top);
4591
      check_malloced_chunk(ms, mem, nb);
4592
      goto postaction;
4593
    }
10347 ripley 4594
 
40788 ripley 4595
    mem = sys_alloc(ms, nb);
26855 ripley 4596
 
40788 ripley 4597
  postaction:
4598
    POSTACTION(ms);
4599
    return mem;
4600
  }
4601
 
4602
  return 0;
4603
}
4604
 
4605
void mspace_free(mspace msp, void* mem) {
4606
  if (mem != 0) {
4607
    mchunkptr p  = mem2chunk(mem);
4608
#if FOOTERS
4609
    mstate fm = get_mstate_for(p);
4610
#else /* FOOTERS */
4611
    mstate fm = (mstate)msp;
4612
#endif /* FOOTERS */
4613
    if (!ok_magic(fm)) {
4614
      USAGE_ERROR_ACTION(fm, p);
4615
      return;
26855 ripley 4616
    }
40788 ripley 4617
    if (!PREACTION(fm)) {
4618
      check_inuse_chunk(fm, p);
4619
      if (RTCHECK(ok_address(fm, p) && ok_cinuse(p))) {
4620
        size_t psize = chunksize(p);
4621
        mchunkptr next = chunk_plus_offset(p, psize);
4622
        if (!pinuse(p)) {
4623
          size_t prevsize = p->prev_foot;
4624
          if ((prevsize & IS_MMAPPED_BIT) != 0) {
4625
            prevsize &= ~IS_MMAPPED_BIT;
4626
            psize += prevsize + MMAP_FOOT_PAD;
4627
            if (CALL_MUNMAP((char*)p - prevsize, psize) == 0)
4628
              fm->footprint -= psize;
4629
            goto postaction;
4630
          }
4631
          else {
4632
            mchunkptr prev = chunk_minus_offset(p, prevsize);
4633
            psize += prevsize;
4634
            p = prev;
4635
            if (RTCHECK(ok_address(fm, prev))) { /* consolidate backward */
4636
              if (p != fm->dv) {
4637
                unlink_chunk(fm, p, prevsize);
4638
              }
4639
              else if ((next->head & INUSE_BITS) == INUSE_BITS) {
4640
                fm->dvsize = psize;
4641
                set_free_with_pinuse(p, psize, next);
4642
                goto postaction;
4643
              }
4644
            }
4645
            else
4646
              goto erroraction;
4647
          }
4648
        }
26855 ripley 4649
 
40788 ripley 4650
        if (RTCHECK(ok_next(p, next) && ok_pinuse(next))) {
4651
          if (!cinuse(next)) {  /* consolidate forward */
4652
            if (next == fm->top) {
4653
              size_t tsize = fm->topsize += psize;
4654
              fm->top = p;
4655
              p->head = tsize | PINUSE_BIT;
4656
              if (p == fm->dv) {
4657
                fm->dv = 0;
4658
                fm->dvsize = 0;
4659
              }
4660
              if (should_trim(fm, tsize))
4661
                sys_trim(fm, 0);
4662
              goto postaction;
4663
            }
4664
            else if (next == fm->dv) {
4665
              size_t dsize = fm->dvsize += psize;
4666
              fm->dv = p;
4667
              set_size_and_pinuse_of_free_chunk(p, dsize);
4668
              goto postaction;
4669
            }
4670
            else {
4671
              size_t nsize = chunksize(next);
4672
              psize += nsize;
4673
              unlink_chunk(fm, next, nsize);
4674
              set_size_and_pinuse_of_free_chunk(p, psize);
4675
              if (p == fm->dv) {
4676
                fm->dvsize = psize;
4677
                goto postaction;
4678
              }
4679
            }
4680
          }
4681
          else
4682
            set_free_with_pinuse(p, psize, next);
4683
          insert_chunk(fm, p, psize);
4684
          check_free_chunk(fm, p);
4685
          goto postaction;
4686
        }
4687
      }
4688
    erroraction:
4689
      USAGE_ERROR_ACTION(fm, p);
4690
    postaction:
4691
      POSTACTION(fm);
4692
    }
4693
  }
4694
}
26855 ripley 4695
 
40788 ripley 4696
void* mspace_calloc(mspace msp, size_t n_elements, size_t elem_size) {
4697
  void* mem;
4698
  size_t req = 0;
4699
  mstate ms = (mstate)msp;
4700
  if (!ok_magic(ms)) {
4701
    USAGE_ERROR_ACTION(ms,ms);
4702
    return 0;
4703
  }
4704
  if (n_elements != 0) {
4705
    req = n_elements * elem_size;
4706
    if (((n_elements | elem_size) & ~(size_t)0xffff) &&
4707
        (req / n_elements != elem_size))
4708
      req = MAX_SIZE_T; /* force downstream failure on overflow */
4709
  }
4710
  mem = internal_malloc(ms, req);
4711
  if (mem != 0 && calloc_must_clear(mem2chunk(mem)))
4712
    memset(mem, 0, req);
4713
  return mem;
4714
}
26855 ripley 4715
 
40788 ripley 4716
void* mspace_realloc(mspace msp, void* oldmem, size_t bytes) {
4717
  if (oldmem == 0)
4718
    return mspace_malloc(msp, bytes);
4719
#ifdef REALLOC_ZERO_BYTES_FREES
4720
  if (bytes == 0) {
4721
    mspace_free(msp, oldmem);
4722
    return 0;
4723
  }
4724
#endif /* REALLOC_ZERO_BYTES_FREES */
4725
  else {
4726
#if FOOTERS
4727
    mchunkptr p  = mem2chunk(oldmem);
4728
    mstate ms = get_mstate_for(p);
4729
#else /* FOOTERS */
4730
    mstate ms = (mstate)msp;
4731
#endif /* FOOTERS */
4732
    if (!ok_magic(ms)) {
4733
      USAGE_ERROR_ACTION(ms,ms);
26855 ripley 4734
      return 0;
40788 ripley 4735
    }
4736
    return internal_realloc(ms, oldmem, bytes);
4737
  }
4738
}
26855 ripley 4739
 
40788 ripley 4740
void* mspace_memalign(mspace msp, size_t alignment, size_t bytes) {
4741
  mstate ms = (mstate)msp;
4742
  if (!ok_magic(ms)) {
4743
    USAGE_ERROR_ACTION(ms,ms);
10347 ripley 4744
    return 0;
4745
  }
40788 ripley 4746
  return internal_memalign(ms, alignment, bytes);
10347 ripley 4747
}
4748
 
40788 ripley 4749
void** mspace_independent_calloc(mspace msp, size_t n_elements,
4750
                                 size_t elem_size, void* chunks[]) {
4751
  size_t sz = elem_size; /* serves as 1-element array */
4752
  mstate ms = (mstate)msp;
4753
  if (!ok_magic(ms)) {
4754
    USAGE_ERROR_ACTION(ms,ms);
4755
    return 0;
4756
  }
4757
  return ialloc(ms, n_elements, &sz, 3, chunks);
4758
}
10347 ripley 4759
 
40788 ripley 4760
void** mspace_independent_comalloc(mspace msp, size_t n_elements,
4761
                                   size_t sizes[], void* chunks[]) {
4762
  mstate ms = (mstate)msp;
4763
  if (!ok_magic(ms)) {
4764
    USAGE_ERROR_ACTION(ms,ms);
4765
    return 0;
4766
  }
4767
  return ialloc(ms, n_elements, sizes, 0, chunks);
4768
}
10347 ripley 4769
 
40788 ripley 4770
int mspace_trim(mspace msp, size_t pad) {
4771
  int result = 0;
4772
  mstate ms = (mstate)msp;
4773
  if (ok_magic(ms)) {
4774
    if (!PREACTION(ms)) {
4775
      result = sys_trim(ms, pad);
4776
      POSTACTION(ms);
4777
    }
4778
  }
4779
  else {
4780
    USAGE_ERROR_ACTION(ms,ms);
4781
  }
4782
  return result;
4783
}
10347 ripley 4784
 
40788 ripley 4785
void mspace_malloc_stats(mspace msp) {
4786
  mstate ms = (mstate)msp;
4787
  if (ok_magic(ms)) {
4788
    internal_malloc_stats(ms);
4789
  }
4790
  else {
4791
    USAGE_ERROR_ACTION(ms,ms);
4792
  }
4793
}
10347 ripley 4794
 
40788 ripley 4795
size_t mspace_footprint(mspace msp) {
4796
  size_t result;
4797
  mstate ms = (mstate)msp;
4798
  if (ok_magic(ms)) {
4799
    result = ms->footprint;
4800
  }
4801
  USAGE_ERROR_ACTION(ms,ms);
4802
  return result;
4803
}
10347 ripley 4804
 
26855 ripley 4805
 
40788 ripley 4806
size_t mspace_max_footprint(mspace msp) {
4807
  size_t result;
4808
  mstate ms = (mstate)msp;
4809
  if (ok_magic(ms)) {
4810
    result = ms->max_footprint;
4811
  }
4812
  USAGE_ERROR_ACTION(ms,ms);
4813
  return result;
4814
}
26855 ripley 4815
 
4816
 
40788 ripley 4817
#if !NO_MALLINFO
4818
struct mallinfo mspace_mallinfo(mspace msp) {
4819
  mstate ms = (mstate)msp;
4820
  if (!ok_magic(ms)) {
4821
    USAGE_ERROR_ACTION(ms,ms);
4822
  }
4823
  return internal_mallinfo(ms);
4824
}
4825
#endif /* NO_MALLINFO */
26855 ripley 4826
 
40788 ripley 4827
int mspace_mallopt(int param_number, int value) {
4828
  return change_mparam(param_number, value);
4829
}
26855 ripley 4830
 
40788 ripley 4831
#endif /* MSPACES */
26855 ripley 4832
 
40788 ripley 4833
/* -------------------- Alternative MORECORE functions ------------------- */
26855 ripley 4834
 
40788 ripley 4835
/*
4836
  Guidelines for creating a custom version of MORECORE:
26855 ripley 4837
 
40788 ripley 4838
  * For best performance, MORECORE should allocate in multiples of pagesize.
4839
  * MORECORE may allocate more memory than requested. (Or even less,
4840
      but this will usually result in a malloc failure.)
4841
  * MORECORE must not allocate memory when given argument zero, but
26855 ripley 4842
      instead return one past the end address of memory from previous
40788 ripley 4843
      nonzero call.
4844
  * For best performance, consecutive calls to MORECORE with positive
4845
      arguments should return increasing addresses, indicating that
4846
      space has been contiguously extended.
4847
  * Even though consecutive calls to MORECORE need not return contiguous
26855 ripley 4848
      addresses, it must be OK for malloc'ed chunks to span multiple
4849
      regions in those cases where they do happen to be contiguous.
40788 ripley 4850
  * MORECORE need not handle negative arguments -- it may instead
4851
      just return MFAIL when given negative arguments.
26855 ripley 4852
      Negative arguments are always multiples of pagesize. MORECORE
4853
      must not misinterpret negative args as large positive unsigned
4854
      args. You can suppress all such calls from even occurring by defining
4855
      MORECORE_CANNOT_TRIM,
4856
 
40788 ripley 4857
  As an example alternative MORECORE, here is a custom allocator
4858
  kindly contributed for pre-OSX macOS.  It uses virtually but not
4859
  necessarily physically contiguous non-paged memory (locked in,
4860
  present and won't get swapped out).  You can use it by uncommenting
4861
  this section, adding some #includes, and setting up the appropriate
4862
  defines above:
26855 ripley 4863
 
4864
      #define MORECORE osMoreCore
4865
 
4866
  There is also a shutdown routine that should somehow be called for
4867
  cleanup upon program exit.
4868
 
4869
  #define MAX_POOL_ENTRIES 100
40788 ripley 4870
  #define MINIMUM_MORECORE_SIZE  (64 * 1024U)
26855 ripley 4871
  static int next_os_pool;
4872
  void *our_os_pools[MAX_POOL_ENTRIES];
4873
 
4874
  void *osMoreCore(int size)
10347 ripley 4875
  {
26855 ripley 4876
    void *ptr = 0;
4877
    static void *sbrk_top = 0;
4878
 
4879
    if (size > 0)
10347 ripley 4880
    {
26855 ripley 4881
      if (size < MINIMUM_MORECORE_SIZE)
4882
         size = MINIMUM_MORECORE_SIZE;
4883
      if (CurrentExecutionLevel() == kTaskLevel)
4884
         ptr = PoolAllocateResident(size + RM_PAGE_SIZE, 0);
4885
      if (ptr == 0)
4886
      {
40788 ripley 4887
        return (void *) MFAIL;
26855 ripley 4888
      }
4889
      // save ptrs so they can be freed during cleanup
4890
      our_os_pools[next_os_pool] = ptr;
4891
      next_os_pool++;
40788 ripley 4892
      ptr = (void *) ((((size_t) ptr) + RM_PAGE_MASK) & ~RM_PAGE_MASK);
26855 ripley 4893
      sbrk_top = (char *) ptr + size;
4894
      return ptr;
10347 ripley 4895
    }
26855 ripley 4896
    else if (size < 0)
4897
    {
4898
      // we don't currently support shrink behavior
40788 ripley 4899
      return (void *) MFAIL;
26855 ripley 4900
    }
4901
    else
4902
    {
4903
      return sbrk_top;
4904
    }
10347 ripley 4905
  }
4906
 
26855 ripley 4907
  // cleanup any allocated memory pools
4908
  // called as last thing before shutting down driver
10347 ripley 4909
 
26855 ripley 4910
  void osCleanupMem(void)
4911
  {
4912
    void **ptr;
10347 ripley 4913
 
26855 ripley 4914
    for (ptr = our_os_pools; ptr < &our_os_pools[MAX_POOL_ENTRIES]; ptr++)
4915
      if (*ptr)
4916
      {
4917
         PoolDeallocate(*ptr);
4918
         *ptr = 0;
4919
      }
4920
  }
10347 ripley 4921
 
26855 ripley 4922
*/
10347 ripley 4923
 
4924
 
40788 ripley 4925
/* -----------------------------------------------------------------------
4926
History:
4927
    V2.8.3 Thu Sep 22 11:16:32 2005  Doug Lea  (dl at gee)
4928
      * Add max_footprint functions
4929
      * Ensure all appropriate literals are size_t
4930
      * Fix conditional compilation problem for some #define settings
4931
      * Avoid concatenating segments with the one provided
4932
        in create_mspace_with_base
4933
      * Rename some variables to avoid compiler shadowing warnings
4934
      * Use explicit lock initialization.
4935
      * Better handling of sbrk interference.
4936
      * Simplify and fix segment insertion, trimming and mspace_destroy
4937
      * Reinstate REALLOC_ZERO_BYTES_FREES option from 2.7.x
4938
      * Thanks especially to Dennis Flanagan for help on these.
10347 ripley 4939
 
40788 ripley 4940
    V2.8.2 Sun Jun 12 16:01:10 2005  Doug Lea  (dl at gee)
4941
      * Fix memalign brace error.
10347 ripley 4942
 
40788 ripley 4943
    V2.8.1 Wed Jun  8 16:11:46 2005  Doug Lea  (dl at gee)
4944
      * Fix improper #endif nesting in C++
4945
      * Add explicit casts needed for C++
26855 ripley 4946
 
40788 ripley 4947
    V2.8.0 Mon May 30 14:09:02 2005  Doug Lea  (dl at gee)
4948
      * Use trees for large bins
4949
      * Support mspaces
4950
      * Use segments to unify sbrk-based and mmap-based system allocation,
4951
        removing need for emulation on most platforms without sbrk.
4952
      * Default safety checks
4953
      * Optional footer checks. Thanks to William Robertson for the idea.
4954
      * Internal code refactoring
4955
      * Incorporate suggestions and platform-specific changes.
4956
        Thanks to Dennis Flanagan, Colin Plumb, Niall Douglas,
4957
        Aaron Bachmann,  Emery Berger, and others.
4958
      * Speed up non-fastbin processing enough to remove fastbins.
4959
      * Remove useless cfree() to avoid conflicts with other apps.
4960
      * Remove internal memcpy, memset. Compilers handle builtins better.
4961
      * Remove some options that no one ever used and rename others.
26855 ripley 4962
 
4963
    V2.7.2 Sat Aug 17 09:07:30 2002  Doug Lea  (dl at gee)
4964
      * Fix malloc_state bitmap array misdeclaration
10347 ripley 4965
 
26855 ripley 4966
    V2.7.1 Thu Jul 25 10:58:03 2002  Doug Lea  (dl at gee)
4967
      * Allow tuning of FIRST_SORTED_BIN_SIZE
4968
      * Use PTR_UINT as type for all ptr->int casts. Thanks to John Belmonte.
40788 ripley 4969
      * Better detection and support for non-contiguousness of MORECORE.
26855 ripley 4970
        Thanks to Andreas Mueller, Conal Walsh, and Wolfram Gloger
4971
      * Bypass most of malloc if no frees. Thanks To Emery Berger.
4972
      * Fix freeing of old top non-contiguous chunk im sysmalloc.
4973
      * Raised default trim and map thresholds to 256K.
4974
      * Fix mmap-related #defines. Thanks to Lubos Lunak.
4975
      * Fix copy macros; added LACKS_FCNTL_H. Thanks to Neal Walfield.
4976
      * Branch-free bin calculation
4977
      * Default trim and mmap thresholds now 256K.
4978
 
4979
    V2.7.0 Sun Mar 11 14:14:06 2001  Doug Lea  (dl at gee)
4980
      * Introduce independent_comalloc and independent_calloc.
4981
        Thanks to Michael Pachos for motivation and help.
4982
      * Make optional .h file available
4983
      * Allow > 2GB requests on 32bit systems.
4984
      * new WIN32 sbrk, mmap, munmap, lock code from <Walter@GeNeSys-e.de>.
4985
        Thanks also to Andreas Mueller <a.mueller at paradatec.de>,
4986
        and Anonymous.
40788 ripley 4987
      * Allow override of MALLOC_ALIGNMENT (Thanks to Ruud Waij for
26855 ripley 4988
        helping test this.)
4989
      * memalign: check alignment arg
4990
      * realloc: don't try to shift chunks backwards, since this
4991
        leads to  more fragmentation in some programs and doesn't
4992
        seem to help in any others.
40788 ripley 4993
      * Collect all cases in malloc requiring system memory into sysmalloc
26855 ripley 4994
      * Use mmap as backup to sbrk
4995
      * Place all internal state in malloc_state
4996
      * Introduce fastbins (although similar to 2.5.1)
4997
      * Many minor tunings and cosmetic improvements
40788 ripley 4998
      * Introduce USE_PUBLIC_MALLOC_WRAPPERS, USE_MALLOC_LOCK
26855 ripley 4999
      * Introduce MALLOC_FAILURE_ACTION, MORECORE_CONTIGUOUS
5000
        Thanks to Tony E. Bennett <tbennett@nvidia.com> and others.
5001
      * Include errno.h to support default failure action.
5002
 
10425 ripley 5003
    V2.6.6 Sun Dec  5 07:42:19 1999  Doug Lea  (dl at gee)
5004
      * return null for negative arguments
26855 ripley 5005
      * Added Several WIN32 cleanups from Martin C. Fong <mcfong at yahoo.com>
10425 ripley 5006
         * Add 'LACKS_SYS_PARAM_H' for those systems without 'sys/param.h'
5007
          (e.g. WIN32 platforms)
26855 ripley 5008
         * Cleanup header file inclusion for WIN32 platforms
10425 ripley 5009
         * Cleanup code to avoid Microsoft Visual C++ compiler complaints
5010
         * Add 'USE_DL_PREFIX' to quickly allow co-existence with existing
5011
           memory allocation routines
5012
         * Set 'malloc_getpagesize' for WIN32 platforms (needs more work)
5013
         * Use 'assert' rather than 'ASSERT' in WIN32 code to conform to
26855 ripley 5014
           usage of 'assert' in non-WIN32 code
10425 ripley 5015
         * Improve WIN32 'sbrk()' emulation's 'findRegion()' routine to
5016
           avoid infinite loop
5017
      * Always call 'fREe()' rather than 'free()'
5018
 
10347 ripley 5019
    V2.6.5 Wed Jun 17 15:57:31 1998  Doug Lea  (dl at gee)
5020
      * Fixed ordering problem with boundary-stamping
5021
 
5022
    V2.6.3 Sun May 19 08:17:58 1996  Doug Lea  (dl at gee)
5023
      * Added pvalloc, as recommended by H.J. Liu
5024
      * Added 64bit pointer support mainly from Wolfram Gloger
5025
      * Added anonymously donated WIN32 sbrk emulation
5026
      * Malloc, calloc, getpagesize: add optimizations from Raymond Nijssen
5027
      * malloc_extend_top: fix mask error that caused wastage after
5028
        foreign sbrks
5029
      * Add linux mremap support code from HJ Liu
5030
 
5031
    V2.6.2 Tue Dec  5 06:52:55 1995  Doug Lea  (dl at gee)
5032
      * Integrated most documentation with the code.
5033
      * Add support for mmap, with help from
5034
        Wolfram Gloger (Gloger@lrz.uni-muenchen.de).
5035
      * Use last_remainder in more cases.
5036
      * Pack bins using idea from  colin@nyx10.cs.du.edu
5037
      * Use ordered bins instead of best-fit threshhold
5038
      * Eliminate block-local decls to simplify tracing and debugging.
5039
      * Support another case of realloc via move into top
5040
      * Fix error occuring when initial sbrk_base not word-aligned.
5041
      * Rely on page size for units instead of SBRK_UNIT to
5042
        avoid surprises about sbrk alignment conventions.
5043
      * Add mallinfo, mallopt. Thanks to Raymond Nijssen
5044
        (raymond@es.ele.tue.nl) for the suggestion.
5045
      * Add `pad' argument to malloc_trim and top_pad mallopt parameter.
5046
      * More precautions for cases where other routines call sbrk,
5047
        courtesy of Wolfram Gloger (Gloger@lrz.uni-muenchen.de).
5048
      * Added macros etc., allowing use in linux libc from
5049
        H.J. Lu (hjl@gnu.ai.mit.edu)
5050
      * Inverted this history list
5051
 
5052
    V2.6.1 Sat Dec  2 14:10:57 1995  Doug Lea  (dl at gee)
5053
      * Re-tuned and fixed to behave more nicely with V2.6.0 changes.
5054
      * Removed all preallocation code since under current scheme
5055
        the work required to undo bad preallocations exceeds
5056
        the work saved in good cases for most test programs.
5057
      * No longer use return list or unconsolidated bins since
5058
        no scheme using them consistently outperforms those that don't
5059
        given above changes.
5060
      * Use best fit for very large chunks to prevent some worst-cases.
5061
      * Added some support for debugging
5062
 
5063
    V2.6.0 Sat Nov  4 07:05:23 1995  Doug Lea  (dl at gee)
5064
      * Removed footers when chunks are in use. Thanks to
5065
        Paul Wilson (wilson@cs.texas.edu) for the suggestion.
5066
 
5067
    V2.5.4 Wed Nov  1 07:54:51 1995  Doug Lea  (dl at gee)
5068
      * Added malloc_trim, with help from Wolfram Gloger
5069
        (wmglo@Dent.MED.Uni-Muenchen.DE).
5070
 
5071
    V2.5.3 Tue Apr 26 10:16:01 1994  Doug Lea  (dl at g)
5072
 
5073
    V2.5.2 Tue Apr  5 16:20:40 1994  Doug Lea  (dl at g)
5074
      * realloc: try to expand in both directions
5075
      * malloc: swap order of clean-bin strategy;
5076
      * realloc: only conditionally expand backwards
5077
      * Try not to scavenge used bins
5078
      * Use bin counts as a guide to preallocation
5079
      * Occasionally bin return list chunks in first scan
5080
      * Add a few optimizations from colin@nyx10.cs.du.edu
5081
 
5082
    V2.5.1 Sat Aug 14 15:40:43 1993  Doug Lea  (dl at g)
5083
      * faster bin computation & slightly different binning
5084
      * merged all consolidations to one part of malloc proper
5085
         (eliminating old malloc_find_space & malloc_clean_bin)
5086
      * Scan 2 returns chunks (not just 1)
5087
      * Propagate failure in realloc if malloc returns 0
5088
      * Add stuff to allow compilation on non-ANSI compilers
5089
          from kpv@research.att.com
5090
 
5091
    V2.5 Sat Aug  7 07:41:59 1993  Doug Lea  (dl at g.oswego.edu)
5092
      * removed potential for odd address access in prev_chunk
5093
      * removed dependency on getpagesize.h
5094
      * misc cosmetics and a bit more internal documentation
5095
      * anticosmetics: mangled names in macros to evade debugger strangeness
5096
      * tested on sparc, hp-700, dec-mips, rs6000
5097
          with gcc & native cc (hp, dec only) allowing
5098
          Detlefs & Zorn comparison study (in SIGPLAN Notices.)
5099
 
5100
    Trial version Fri Aug 28 13:14:29 1992  Doug Lea  (dl at g.oswego.edu)
5101
      * Based loosely on libg++-1.2X malloc. (It retains some of the overall
5102
         structure of old version,  but most details differ.)
40788 ripley 5103
 
10347 ripley 5104
*/