This seems to resolve the issue. But i am not quite sure if this is the right way.
Notes:
- i am doing the check twice at this moment, only the slaballoc one should be required.
- in _slaballoc size is rewritten and it might be better to use a separate local variable instead. This would help with bug tracing. Changing the size that was requested makes it harder to track what is happening, and prevents later checks if so required. (BTW: there are multiple (other) locations where the requested size is rewritten as well).
- in _slaballoc size is not checked for overflow (i think), before it is rewritten (line 922).
With this patch in place the results resemble the Linux Results exactly.
```
diff --git a/lib/libc/stdlib/nmalloc.c b/lib/libc/stdlib/nmalloc.c
index b39aaf301..d9bc90fb8 100644
--- a/lib/libc/stdlib/nmalloc.c
+++ b/lib/libc/stdlib/nmalloc.c
@ -753,6 +753,8 @ zoneindex(size_t *bytes, size_t *chunking)
return(0);
}
#define MUL_NO_OVERFLOW (1UL << (sizeof(size_t) * 4))
/*
* malloc() - call internal slab allocator
*/
@ -761,6 +763,11 @ __malloc(size_t size)
{
void *ptr;
+ if ((size >= MUL_NO_OVERFLOW ) || (SIZE_MAX < size)) {
+ errno = ENOMEM;
+ return(NULL);
+ }
+
ptr = _slaballoc(size, 0);
if (ptr == NULL)
errno = ENOMEM;
@ -769,8 +776,6 @ __malloc(size_t size)
return(ptr);
}
#define MUL_NO_OVERFLOW (1UL << (sizeof(size_t) * 4))
/*
* calloc() - call internal slab allocator
*/
@ -982,6 +987,9 @ _slaballoc(size_t size, int flags)
bigalloc_t big;
bigalloc_t *bigp;
+ if ((size >= MUL_NO_OVERFLOW ) || (SIZE_MAX < size) ) {
+ return(NULL);
+ }
/*
* Page-align and cache-color in case of virtually indexed
* physically tagged L1 caches (aka SandyBridge). No sweat
@ -989,7 +997,8 @ _slaballoc(size_t size, int flags)
*
* (don't count as excess).
/
- size = (size + PAGE_MASK) & ~(size_t)PAGE_MASK;
+ size = (size + PAGE_MASK) & ~(size_t)PAGE_MASK; / Note: Changing size, without checking overflow.
+ also might be better to use a different variable instead of the original request size */
/*
* If we have overflown above when rounding to the page
```
Updated by