Fun with big widths and precisions in printf

[Alexander Cherepanov <ch3root@openwall.com>]

Hi!

Some fun with big values for field widths and precisions in *printf 
functions.

1. POSIX requires *printf functions to set errno in case of errors. It's 
not always done in musl.

2. Field width[1] and precision[2] are positive numbers and are 
extracted from the format string as int by the getint function[3]. It 
doesn't check for overflows while neither C11 nor POSIX set any limits 
for these numbers. Overflows in getint are bad as it's UB itself plus it 
gives wrong results -- either negative numbers or positive but smaller 
than original. Both cases lead to wrong behavior in printf.

Negative widths are caught while positive overflown values are still a 
problem. For precisions both cases are a problem.

 From C11 POV most of these problems cannot occur. E.g., a field with 
width greater than INT_MAX leads to a return value greater than INT_MAX. 
printf cannot return such values because it has int as its return type. 
Thus, this is UB in user code and implementations don't have to care 
about it. But there is one case where this is still a problem -- 
overflown positive precision for %s:

   printf("%.4294967296s", "abc");

In musl it prints empty string while having to print "abc".

OTOH POSIX requires *printf functions to fail with errno=EOVERFLOW if 
the value to be returned is greater than INT_MAX. This makes the problem 
wider.

[1] http://git.musl-libc.org/cgit/musl/tree/src/stdio/vfprintf.c#n491
[2] http://git.musl-libc.org/cgit/musl/tree/src/stdio/vfprintf.c#n505
[3] http://git.musl-libc.org/cgit/musl/tree/src/stdio/vfprintf.c#n438

3. When width is an asterisk and its value is taken from an argument, 
the value could initially be negative. Processing of this case[4] 
overflows on INT_MIN with the effect similar to the previous case (UB 
plus wrong result).

[4] http://git.musl-libc.org/cgit/musl/tree/src/stdio/vfprintf.c#n502

Sample code:

----------------------------------------------------------------------

#include <limits.h>
#include <stdio.h>
#include <errno.h>

int main()
{
   int res;


   /* errno is not always set */

   errno = 0;
   res = printf("%2147483648d", 1);
   fprintf(stderr, "res = %d, errno = %d\n", res, errno);
   /* stderr output: res = -1, errno = 0
      errno should be set */


   /* Integer overflow while extracting field width */

   errno = 0;
   res = printf("%4294967296d", 1);
   fprintf(stderr, "res = %d, errno = %d\n", res, errno);
   /* stderr output: res = 1, errno = 0
      res = 1 is wrong, the call should fail
      Overflow, 4294967296 treated as 0 */


   /* Overflow while extracting precision */

   errno = 0;
   res = printf("%.2147483648d", 1);
   fprintf(stderr, "res = %d, errno = %d\n", res, errno);
   /* stderr output: res = 1, errno = 0
      res = 1 is wrong, the call should fail
      Overflow, 2147483648 treated as -2147483648 */

   errno = 0;
   res = printf("%.4294967296d", 1);
   fprintf(stderr, "res = %d, errno = %d\n", res, errno);
   /* stderr output: res = 1, errno = 0
      res = 1 is wrong, the call should fail
      Overflow, 4294967296 treated as 0 */

   errno = 0;
   res = printf("%.4294967296s", "abc");
   fprintf(stderr, "res = %d, errno = %d\n", res, errno);
   /* stderr output: res = 0, errno = 0
      res = 0 is wrong, should be 3
      Overflow, 4294967296 treated as 0 */


   /* Overflow while taking width from an argument */

   errno = 0;
   res = printf("%*d", INT_MIN, 1);
   fprintf(stderr, "res = %d, errno = %d\n", res, errno);
   /* stderr output: res = 1, errno = 0
      res = 1 is wrong, the call should fail
      Overflow, -(-2147483648) = -2147483648 */
}

----------------------------------------------------------------------

-- 
Alexander Cherepanov