dc hitting a compiler bug, or undefined behavior

[Denys Vlasenko]

On Sun, Mar 30, 2014 at 11:18 AM, Lauri Kasanen <curaga at operamail.com> wrote:
> Hi,
>
> I'm seeing busybox dc acting funny when compiled with some versions of
> gcc. This is on busybox git. The x86 binary busybox_unstripped and
> config are attached.
>
> gcc 4.2.2 - ok
> gcc 4.7.2:
> nc 10 1 add p
> 2.738e+93
>
> So either bb is hitting a compiler bug,

Looks like a compiler bug:

        d = strtod(argument, &end);
        if (end != argument && *end == '\0') {
                push(d);

is compiled to this:

   0x08049490 <+27>:    call   0x80488b0 <strtod at plt>
   0x08049495 <+32>:    mov    -0x14(%ebp),%eax
   0x08049498 <+35>:    pop    %ecx
   0x08049499 <+36>:    pop    %ebx
   0x0804949a <+37>:    cmp    %esi,%eax
   0x0804949c <+39>:    je     0x80494b1 <stack_machine+60>
   0x0804949e <+41>:    cmpb   $0x0,(%eax)
   0x080494a1 <+44>:    jne    0x80494b5 <stack_machine+64>
   0x080494a3 <+46>:    push   %eax
   0x080494a4 <+47>:    push   %eax
   0x080494a5 <+48>:    fstpl  (%esp)  <============ HERE
   0x080494a8 <+51>:    call   0x8048f10 <push>

Note how push(double a) argument gets passed on stack.
But push() starts with this:

Dump of assembler code for function push:
=> 0x08048f10 <+0>:    fldl   (%edi)

which reads argument from some bogus location instead
of stack:

(gdb) nexti  <=========== EXECUTE fldl
0x08048f12 in push ()
(gdb) p $st0
$1 = 2.0554264135011661055418432229752339e-314

That's not 10!
10 exists on stack all right:

(gdb) p *(double*)($esp+4)
$5 = 10

the program just doesn't fetch it correctly.