linux/sbrk is wrong

linux/sbrk is wrong

[Szabolcs Nagy]

[-- Attachment #1: Type: text/plain, Size: 248 bytes --]

linux brk syscall returns the end pointer
so if it fails then the old one
if it succeeds then the new one

the current implementation of sbrk returns
the end of the allocated area instead of
the begining

i attached a patch that fixes brk and sbrk

[-- Attachment #2: sbrk.diff --]
[-- Type: text/x-diff, Size: 647 bytes --]

diff --git a/src/linux/brk.c b/src/linux/brk.c
index 9f63c5a..c261518 100644
--- a/src/linux/brk.c
+++ b/src/linux/brk.c
@@ -2,5 +2,5 @@
 
 int brk(void *end)
 {
-	return -(syscall(SYS_brk, end) == -1);
+	return -((void *)syscall(SYS_brk, end) != end);
 }
diff --git a/src/linux/sbrk.c b/src/linux/sbrk.c
index b2943a9..478a35e 100644
--- a/src/linux/sbrk.c
+++ b/src/linux/sbrk.c
@@ -3,5 +3,11 @@
 
 void *sbrk(ptrdiff_t inc)
 {
-	return (void *)syscall(SYS_brk, syscall(SYS_brk, 0)+inc);
+	void *p = (void *)syscall(SYS_brk, 0);
+
+	if (inc == 0)
+		return p;
+	if ((void *)syscall(SYS_brk, p+inc) == p+inc)
+		return p;
+	return (void *)-1;
 }

Re: linux/sbrk is wrong

[Rich Felker]

On Sun, May 22, 2011 at 04:55:55PM +0200, Szabolcs Nagy wrote:
> linux brk syscall returns the end pointer
> so if it fails then the old one
> if it succeeds then the new one
> 
> the current implementation of sbrk returns
> the end of the allocated area instead of
> the begining

IMO there's no way for correct code to use these functions anyway, but
I'll fix them anyway.

>  int brk(void *end)
>  {
> -	return -(syscall(SYS_brk, end) == -1);
> +	return -((void *)syscall(SYS_brk, end) != end);
>  }

This looks fine.

>  void *sbrk(ptrdiff_t inc)
>  {
> -	return (void *)syscall(SYS_brk, syscall(SYS_brk, 0)+inc);
> +	void *p = (void *)syscall(SYS_brk, 0);
> +
> +	if (inc == 0)
> +		return p;
> +	if ((void *)syscall(SYS_brk, p+inc) == p+inc)

This line is invalid C. p+inc is not defined since p has type void *.
The most correct would probably be to use uintptr_t internally
everywhere and just cast to void * in the return statement.

Rich