| 
									
										
										
										
											2002-04-28 04:11:46 +00:00
										 |  |  | /* The PyMem_ family:  low-level memory allocation interfaces.
 | 
					
						
							|  |  |  |    See objimpl.h for the PyObject_ memory family. | 
					
						
							|  |  |  | */ | 
					
						
							| 
									
										
										
										
											2000-07-31 22:19:30 +00:00
										 |  |  | 
 | 
					
						
							|  |  |  | #ifndef Py_PYMEM_H
 | 
					
						
							|  |  |  | #define Py_PYMEM_H
 | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | #include "pyport.h"
 | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | #ifdef __cplusplus
 | 
					
						
							|  |  |  | extern "C" { | 
					
						
							|  |  |  | #endif
 | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | /* BEWARE:
 | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2002-04-28 04:11:46 +00:00
										 |  |  |    Each interface exports both functions and macros.  Extension modules should | 
					
						
							|  |  |  |    use the functions, to ensure binary compatibility across Python versions. | 
					
						
							|  |  |  |    Because the Python implementation is free to change internal details, and | 
					
						
							|  |  |  |    the macros may (or may not) expose details for speed, if you do use the | 
					
						
							|  |  |  |    macros you must recompile your extensions with each Python release. | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |    Never mix calls to PyMem_ with calls to the platform malloc/realloc/ | 
					
						
							|  |  |  |    calloc/free.  For example, on Windows different DLLs may end up using | 
					
						
							|  |  |  |    different heaps, and if you use PyMem_Malloc you'll get the memory from the | 
					
						
							|  |  |  |    heap used by the Python DLL; it could be a disaster if you free()'ed that | 
					
						
							|  |  |  |    directly in your own extension.  Using PyMem_Free instead ensures Python | 
					
						
							|  |  |  |    can return the memory to the proper heap.  As another example, in | 
					
						
							|  |  |  |    PYMALLOC_DEBUG mode, Python wraps all calls to all PyMem_ and PyObject_ | 
					
						
							|  |  |  |    memory functions in special debugging wrappers that add additional | 
					
						
							|  |  |  |    debugging info to dynamic memory blocks.  The system routines have no idea | 
					
						
							|  |  |  |    what to do with that stuff, and the Python wrappers have no idea what to do | 
					
						
							|  |  |  |    with raw blocks obtained directly by the system routines then. | 
					
						
							|  |  |  | */ | 
					
						
							| 
									
										
										
										
											2000-07-31 22:19:30 +00:00
										 |  |  | 
 | 
					
						
							|  |  |  | /*
 | 
					
						
							|  |  |  |  * Raw memory interface | 
					
						
							|  |  |  |  * ==================== | 
					
						
							|  |  |  |  */ | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2002-04-28 04:11:46 +00:00
										 |  |  | /* Functions
 | 
					
						
							| 
									
										
										
										
											2000-07-31 22:19:30 +00:00
										 |  |  | 
 | 
					
						
							| 
									
										
										
										
											2002-04-28 04:11:46 +00:00
										 |  |  |    Functions supplying platform-independent semantics for malloc/realloc/ | 
					
						
							|  |  |  |    free.  These functions make sure that allocating 0 bytes returns a distinct | 
					
						
							| 
									
										
											  
											
												First stab at rationalizing the PyMem_ API.  Mixing PyObject_xyz with
PyMem_{Del, DEL} doesn't work yet (compilation problems).
pyport.h:  _PyMem_EXTRA is gone.
pmem.h:  Repaired comments.  PyMem_{Malloc, MALLOC} and
PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when
asking for 0 bytes, and when passing a NULL pointer to the latter.
object.c:  PyMem_{Malloc, Realloc} just call their macro versions
now, since the latter take care of the x-platform 0 and NULL stuff
by themselves now.
pypcre.c, grow_stack():  So sue me.  On two lines, this called
PyMem_RESIZE to grow a "const" area.  It's not legit to realloc a
const area, so the compiler warned given the new expansion of
PyMem_RESIZE.  It would have gotten the same warning before if it
had used PyMem_Resize() instead; the older macro version, but not the
function version, silently cast away the constness.  IMO that was a wrong
thing to do, and the docs say the macro versions of PyMem_xyz are
deprecated anyway.  If somebody else is resizing const areas with the
macro spelling, they'll get a warning when they recompile now too.
											
										 
											2002-04-12 07:22:56 +00:00
										 |  |  |    non-NULL pointer (whenever possible -- if we're flat out of memory, NULL | 
					
						
							|  |  |  |    may be returned), even if the platform malloc and realloc don't. | 
					
						
							|  |  |  |    Returned pointers must be checked for NULL explicitly.  No action is | 
					
						
							| 
									
										
										
										
											2002-04-28 04:11:46 +00:00
										 |  |  |    performed on failure (no exception is set, no warning is printed, etc). | 
					
						
							|  |  |  | */ | 
					
						
							| 
									
										
											  
											
												First stab at rationalizing the PyMem_ API.  Mixing PyObject_xyz with
PyMem_{Del, DEL} doesn't work yet (compilation problems).
pyport.h:  _PyMem_EXTRA is gone.
pmem.h:  Repaired comments.  PyMem_{Malloc, MALLOC} and
PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when
asking for 0 bytes, and when passing a NULL pointer to the latter.
object.c:  PyMem_{Malloc, Realloc} just call their macro versions
now, since the latter take care of the x-platform 0 and NULL stuff
by themselves now.
pypcre.c, grow_stack():  So sue me.  On two lines, this called
PyMem_RESIZE to grow a "const" area.  It's not legit to realloc a
const area, so the compiler warned given the new expansion of
PyMem_RESIZE.  It would have gotten the same warning before if it
had used PyMem_Resize() instead; the older macro version, but not the
function version, silently cast away the constness.  IMO that was a wrong
thing to do, and the docs say the macro versions of PyMem_xyz are
deprecated anyway.  If somebody else is resizing const areas with the
macro spelling, they'll get a warning when they recompile now too.
											
										 
											2002-04-12 07:22:56 +00:00
										 |  |  | 
 | 
					
						
							| 
									
										
										
										
											2002-08-12 07:21:58 +00:00
										 |  |  | PyAPI_FUNC(void *) PyMem_Malloc(size_t); | 
					
						
							|  |  |  | PyAPI_FUNC(void *) PyMem_Realloc(void *, size_t); | 
					
						
							|  |  |  | PyAPI_FUNC(void) PyMem_Free(void *); | 
					
						
							| 
									
										
										
										
											2000-07-31 22:19:30 +00:00
										 |  |  | 
 | 
					
						
							|  |  |  | /* Starting from Python 1.6, the wrappers Py_{Malloc,Realloc,Free} are
 | 
					
						
							|  |  |  |    no longer supported. They used to call PyErr_NoMemory() on failure. */ | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
											  
											
												First stab at rationalizing the PyMem_ API.  Mixing PyObject_xyz with
PyMem_{Del, DEL} doesn't work yet (compilation problems).
pyport.h:  _PyMem_EXTRA is gone.
pmem.h:  Repaired comments.  PyMem_{Malloc, MALLOC} and
PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when
asking for 0 bytes, and when passing a NULL pointer to the latter.
object.c:  PyMem_{Malloc, Realloc} just call their macro versions
now, since the latter take care of the x-platform 0 and NULL stuff
by themselves now.
pypcre.c, grow_stack():  So sue me.  On two lines, this called
PyMem_RESIZE to grow a "const" area.  It's not legit to realloc a
const area, so the compiler warned given the new expansion of
PyMem_RESIZE.  It would have gotten the same warning before if it
had used PyMem_Resize() instead; the older macro version, but not the
function version, silently cast away the constness.  IMO that was a wrong
thing to do, and the docs say the macro versions of PyMem_xyz are
deprecated anyway.  If somebody else is resizing const areas with the
macro spelling, they'll get a warning when they recompile now too.
											
										 
											2002-04-12 07:22:56 +00:00
										 |  |  | /* Macros. */ | 
					
						
							| 
									
										
										
										
											2002-04-22 02:33:27 +00:00
										 |  |  | #ifdef PYMALLOC_DEBUG
 | 
					
						
							|  |  |  | /* Redirect all memory operations to Python's debugging allocator. */ | 
					
						
							|  |  |  | #define PyMem_MALLOC		PyObject_MALLOC
 | 
					
						
							|  |  |  | #define PyMem_REALLOC		PyObject_REALLOC
 | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | #else	/* ! PYMALLOC_DEBUG */
 | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2002-11-23 09:13:40 +00:00
										 |  |  | /* PyMem_MALLOC(0) means malloc(1). Some systems would return NULL
 | 
					
						
							|  |  |  |    for malloc(0), which would be treated as an error. Some platforms | 
					
						
							|  |  |  |    would return a pointer with no memory behind it, which would break | 
					
						
							|  |  |  |    pymalloc. To solve these problems, allocate an extra byte. */ | 
					
						
							| 
									
										
											  
											
												First stab at rationalizing the PyMem_ API.  Mixing PyObject_xyz with
PyMem_{Del, DEL} doesn't work yet (compilation problems).
pyport.h:  _PyMem_EXTRA is gone.
pmem.h:  Repaired comments.  PyMem_{Malloc, MALLOC} and
PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when
asking for 0 bytes, and when passing a NULL pointer to the latter.
object.c:  PyMem_{Malloc, Realloc} just call their macro versions
now, since the latter take care of the x-platform 0 and NULL stuff
by themselves now.
pypcre.c, grow_stack():  So sue me.  On two lines, this called
PyMem_RESIZE to grow a "const" area.  It's not legit to realloc a
const area, so the compiler warned given the new expansion of
PyMem_RESIZE.  It would have gotten the same warning before if it
had used PyMem_Resize() instead; the older macro version, but not the
function version, silently cast away the constness.  IMO that was a wrong
thing to do, and the docs say the macro versions of PyMem_xyz are
deprecated anyway.  If somebody else is resizing const areas with the
macro spelling, they'll get a warning when they recompile now too.
											
										 
											2002-04-12 07:22:56 +00:00
										 |  |  | #define PyMem_MALLOC(n)         malloc((n) ? (n) : 1)
 | 
					
						
							|  |  |  | #define PyMem_REALLOC(p, n)     realloc((p), (n) ? (n) : 1)
 | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2002-04-22 02:33:27 +00:00
										 |  |  | #endif	/* PYMALLOC_DEBUG */
 | 
					
						
							| 
									
										
											  
											
												First stab at rationalizing the PyMem_ API.  Mixing PyObject_xyz with
PyMem_{Del, DEL} doesn't work yet (compilation problems).
pyport.h:  _PyMem_EXTRA is gone.
pmem.h:  Repaired comments.  PyMem_{Malloc, MALLOC} and
PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when
asking for 0 bytes, and when passing a NULL pointer to the latter.
object.c:  PyMem_{Malloc, Realloc} just call their macro versions
now, since the latter take care of the x-platform 0 and NULL stuff
by themselves now.
pypcre.c, grow_stack():  So sue me.  On two lines, this called
PyMem_RESIZE to grow a "const" area.  It's not legit to realloc a
const area, so the compiler warned given the new expansion of
PyMem_RESIZE.  It would have gotten the same warning before if it
had used PyMem_Resize() instead; the older macro version, but not the
function version, silently cast away the constness.  IMO that was a wrong
thing to do, and the docs say the macro versions of PyMem_xyz are
deprecated anyway.  If somebody else is resizing const areas with the
macro spelling, they'll get a warning when they recompile now too.
											
										 
											2002-04-12 07:22:56 +00:00
										 |  |  | 
 | 
					
						
							| 
									
										
										
										
											2002-04-28 04:11:46 +00:00
										 |  |  | /* In order to avoid breaking old code mixing PyObject_{New, NEW} with
 | 
					
						
							|  |  |  |    PyMem_{Del, DEL} and PyMem_{Free, FREE}, the PyMem "release memory" | 
					
						
							|  |  |  |    functions have to be redirected to the object deallocator. */ | 
					
						
							|  |  |  | #define PyMem_FREE           	PyObject_FREE
 | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2000-07-31 22:19:30 +00:00
										 |  |  | /*
 | 
					
						
							|  |  |  |  * Type-oriented memory interface | 
					
						
							|  |  |  |  * ============================== | 
					
						
							| 
									
										
											  
											
												First stab at rationalizing the PyMem_ API.  Mixing PyObject_xyz with
PyMem_{Del, DEL} doesn't work yet (compilation problems).
pyport.h:  _PyMem_EXTRA is gone.
pmem.h:  Repaired comments.  PyMem_{Malloc, MALLOC} and
PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when
asking for 0 bytes, and when passing a NULL pointer to the latter.
object.c:  PyMem_{Malloc, Realloc} just call their macro versions
now, since the latter take care of the x-platform 0 and NULL stuff
by themselves now.
pypcre.c, grow_stack():  So sue me.  On two lines, this called
PyMem_RESIZE to grow a "const" area.  It's not legit to realloc a
const area, so the compiler warned given the new expansion of
PyMem_RESIZE.  It would have gotten the same warning before if it
had used PyMem_Resize() instead; the older macro version, but not the
function version, silently cast away the constness.  IMO that was a wrong
thing to do, and the docs say the macro versions of PyMem_xyz are
deprecated anyway.  If somebody else is resizing const areas with the
macro spelling, they'll get a warning when they recompile now too.
											
										 
											2002-04-12 07:22:56 +00:00
										 |  |  |  * | 
					
						
							|  |  |  |  * These are carried along for historical reasons.  There's rarely a good | 
					
						
							| 
									
										
										
										
											2002-04-28 04:11:46 +00:00
										 |  |  |  * reason to use them anymore (you can just as easily do the multiply and | 
					
						
							|  |  |  |  * cast yourself). | 
					
						
							| 
									
										
										
										
											2000-07-31 22:19:30 +00:00
										 |  |  |  */ | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | #define PyMem_New(type, n) \
 | 
					
						
							|  |  |  | 	( (type *) PyMem_Malloc((n) * sizeof(type)) ) | 
					
						
							|  |  |  | #define PyMem_NEW(type, n) \
 | 
					
						
							| 
									
										
											  
											
												First stab at rationalizing the PyMem_ API.  Mixing PyObject_xyz with
PyMem_{Del, DEL} doesn't work yet (compilation problems).
pyport.h:  _PyMem_EXTRA is gone.
pmem.h:  Repaired comments.  PyMem_{Malloc, MALLOC} and
PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when
asking for 0 bytes, and when passing a NULL pointer to the latter.
object.c:  PyMem_{Malloc, Realloc} just call their macro versions
now, since the latter take care of the x-platform 0 and NULL stuff
by themselves now.
pypcre.c, grow_stack():  So sue me.  On two lines, this called
PyMem_RESIZE to grow a "const" area.  It's not legit to realloc a
const area, so the compiler warned given the new expansion of
PyMem_RESIZE.  It would have gotten the same warning before if it
had used PyMem_Resize() instead; the older macro version, but not the
function version, silently cast away the constness.  IMO that was a wrong
thing to do, and the docs say the macro versions of PyMem_xyz are
deprecated anyway.  If somebody else is resizing const areas with the
macro spelling, they'll get a warning when they recompile now too.
											
										 
											2002-04-12 07:22:56 +00:00
										 |  |  | 	( (type *) PyMem_MALLOC((n) * sizeof(type)) ) | 
					
						
							| 
									
										
										
										
											2002-04-28 04:11:46 +00:00
										 |  |  | 
 | 
					
						
							|  |  |  | #define PyMem_Resize(p, type, n) \
 | 
					
						
							|  |  |  | 	( (p) = (type *) PyMem_Realloc((p), (n) * sizeof(type)) ) | 
					
						
							| 
									
										
											  
											
												First stab at rationalizing the PyMem_ API.  Mixing PyObject_xyz with
PyMem_{Del, DEL} doesn't work yet (compilation problems).
pyport.h:  _PyMem_EXTRA is gone.
pmem.h:  Repaired comments.  PyMem_{Malloc, MALLOC} and
PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when
asking for 0 bytes, and when passing a NULL pointer to the latter.
object.c:  PyMem_{Malloc, Realloc} just call their macro versions
now, since the latter take care of the x-platform 0 and NULL stuff
by themselves now.
pypcre.c, grow_stack():  So sue me.  On two lines, this called
PyMem_RESIZE to grow a "const" area.  It's not legit to realloc a
const area, so the compiler warned given the new expansion of
PyMem_RESIZE.  It would have gotten the same warning before if it
had used PyMem_Resize() instead; the older macro version, but not the
function version, silently cast away the constness.  IMO that was a wrong
thing to do, and the docs say the macro versions of PyMem_xyz are
deprecated anyway.  If somebody else is resizing const areas with the
macro spelling, they'll get a warning when they recompile now too.
											
										 
											2002-04-12 07:22:56 +00:00
										 |  |  | #define PyMem_RESIZE(p, type, n) \
 | 
					
						
							|  |  |  | 	( (p) = (type *) PyMem_REALLOC((p), (n) * sizeof(type)) ) | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2002-04-28 04:11:46 +00:00
										 |  |  | /* In order to avoid breaking old code mixing PyObject_{New, NEW} with
 | 
					
						
							|  |  |  |    PyMem_{Del, DEL} and PyMem_{Free, FREE}, the PyMem "release memory" | 
					
						
							|  |  |  |    functions have to be redirected to the object deallocator. */ | 
					
						
							|  |  |  | #define PyMem_Del		PyObject_Free
 | 
					
						
							|  |  |  | #define PyMem_DEL		PyObject_FREE
 | 
					
						
							| 
									
										
											  
											
												Give Python a debug-mode pymalloc, much as sketched on Python-Dev.
When WITH_PYMALLOC is defined, define PYMALLOC_DEBUG to enable the debug
allocator.  This can be done independent of build type (release or debug).
A debug build automatically defines PYMALLOC_DEBUG when pymalloc is
enabled.  It's a detected error to define PYMALLOC_DEBUG when pymalloc
isn't enabled.
Two debugging entry points defined only under PYMALLOC_DEBUG:
+ _PyMalloc_DebugCheckAddress(const void *p) can be used (e.g., from gdb)
  to sanity-check a memory block obtained from pymalloc.  It sprays
  info to stderr (see next) and dies via Py_FatalError if the block is
  detectably damaged.
+ _PyMalloc_DebugDumpAddress(const void *p) can be used to spray info
  about a debug memory block to stderr.
A tiny start at implementing "API family" checks isn't good for
anything yet.
_PyMalloc_DebugRealloc() has been optimized to do little when the new
size is <= old size.  However, if the new size is larger, it really
can't call the underlying realloc() routine without either violating its
contract, or knowing something non-trivial about how the underlying
realloc() works.  A memcpy is always done in this case.
This was a disaster for (and only) one of the std tests:  test_bufio
creates single text file lines up to a million characters long.  On
Windows, fileobject.c's get_line() uses the horridly funky
getline_via_fgets(), which keeps growing and growing a string object
hoping to find a newline.  It grew the string object 1000 bytes each
time, so for a million-character string it took approximately forever
(I gave up after a few minutes).
So, also:
fileobject.c, getline_via_fgets():  When a single line is outrageously
long, grow the string object at a mildly exponential rate, instead of
just 1000 bytes at a time.
That's enough so that a debug-build test_bufio finishes in about 5 seconds
on my Win98SE box.  I'm curious to try this on Win2K, because it has very
different memory behavior than Win9X, and test_bufio always took a factor
of 10 longer to complete on Win2K.  It *could* be that the endless
reallocs were simply killing it on Win2K even in the release build.
											
										 
											2002-03-23 10:03:50 +00:00
										 |  |  | 
 | 
					
						
							| 
									
										
										
										
											2000-07-31 22:19:30 +00:00
										 |  |  | #ifdef __cplusplus
 | 
					
						
							|  |  |  | } | 
					
						
							|  |  |  | #endif
 | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | #endif /* !Py_PYMEM_H */
 |