[PATCH] Fix ioctl() wrappers for 64-bit systems

[Denys Vlasenko]

On Sunday 20 April 2008 21:08, Nate Case wrote:
> On Sun, 2008-04-20 at 16:30 +0200, Denys Vlasenko wrote:
> > Exactly. It means that compiler doesn't see it too,
> > and because of that it handles ioctl as variadic funtion.
> > 
> > When it sees int parameter, it pushes int on the stack, not long.
> > 
> > Headers need to be fixed to contain correct prototype.
> 
> I wanted to believe this, so I tried just inserting the correct
> prototype directly in my .c file as a test.  However, it still failed.
>
> Eventually I figured out that the key to why TCSETS fails but TCGETS
> succeeds is that TCSETS is actually > 2**31.  On my system TCSETS is
> 0x803c7414 which would become a negative number when cast as a 32-bit
> signed integer (-2143521772).
> 
> The ppc64 __ioctl() function in glibc which tries to catch TCSETS
> requests would do a compare via the switch statement code snippet I
> posted earlier.  The comparison in the switch statement then fails to
> catch the TCSETS request since it's a u64 comparison.  In other words,
> the busybox wrapper is passing -2143521772 to ioctl().  It looks like
> the right hex values / bits when doing the debug prints but it's
> deceiving since we get bit by the comparison.
> 
> Because this comparison fails, __tcgetattr() never gets called to fix
> the arg passed to the syscall (explaining why it was the arg that got
> messed up and not the request field itself).

Can you test attached patch?
--
vda
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 5.patch
Type: text/x-diff
Size: 3649 bytes
Desc: not available
Url : http://lists.busybox.net/pipermail/busybox/attachments/20080421/f905619b/attachment.bin