[PATCH] Bionic lacks transparent LFS migrations; provide a workaround

[Rich Felker]

On Mon, Apr 27, 2015 at 08:14:19PM -0400, Matt Whitlock wrote:
> On Sunday, 26 April 2015, at 10:28 pm, Rich Felker wrote:
> > On Sat, Apr 25, 2015 at 04:06:23PM -0400, Matt Whitlock wrote:
> > > In short, Bionic has most of the syscall-level support introduced by
> > > _LARGEFILE64_SOURCE in Glibc, but they did not implement the
> > > transparent migration introduced by _FILE_OFFSET_BITS==64. This is
> > > easy to implement atop Bionic, however, as it is simply a matter of
> > > mapping the relevant 32-bit types and functions to their 64-bit
> > > equivalents. (See feature_test_macros(7).)
> > 
> > This really can and should be fixed at the header level for Bionic.
> > Nobody should be writing foo64 in sources except as a transparent
> > workaround for broken systems. While you could do the workaround in
> > Busybox, I think it would be much more useful as a patch to the Bionic
> > headers that anyone can apply to the installed headers to fix them,
> > and a patch to send upstream to get the source of the problem fixed.
> 
> The Bionic headers aren't broken. _FILE_OFFSET_BITS is a convenience
> provided by Glibc, but it is not specified by any standard.

The standard specifies off_t not off64_t or any functions with names
ending in 64. The portable way to ensure you're getting 64-bit off_t
is to use the confstr(3) function or getconf(1) command to lookup
CFLAGS etc. for one of the compilation environments that offers 64-bit
off_t. That would be _CS_POSIX_V7_ILP32_OFFBIG_CFLAGS,
_CS_POSIX_V7_LP64_OFF64_CFLAGS etc.

With that said, this is not why I consider Bionic broken. The reason I
consider it broken is that 32-bit off_t is NOT USABLE on Linux and has
not been for a long time. Using the syscalls that have 32-bit off_t
also gives you 32-bit ino_t, and inode numbers do not fit in 32 bits
anymore, resulting in spurious EOVERFLOW on stat(2), etc.

All this is ignoring the much more basic issue that we live in a world
where files larger than 2GB are common, making 32-bit off_t unsuitable
for real-world work.

The only reason glibc has 32-bit off_t in the "non-suffixed" functions
is a historical mistake. Bionic's copying of this mistake long after
it was fixed by adding the new syscalls was a major failure.

> If you think it's a bad idea to map off_t to off64_t within BusyBox,

I don't think it's a _bad_ idea as long as the ugliness is isolated.

> then another option would be to define a bb_off_t type that is
> equivalent to off_t when compiling without _LARGEFILE64_SOURCE or
> else equivalent to off64_t when compiling with _LARGEFILE64_SOURCE.

That would be uglier and less isolated IMO.

What I do think is a bad idea is providing a hackish environment where
only some things work correctly with 64-bit off_t and others silently
fail or even misinterpret their arguments and blow up. If that can be
avoided and the ugliness can be contained in a pit that's isolated to
Bionic, I'm not opposed to it.

Rich