[PATCH] Updates on busybox POSIX compliance

[Rob Landley]

On Thursday 18 June 2009 09:35:32 Mike Frysinger wrote:
> On Thursday 18 June 2009 07:33:48 David Krakov wrote:
> > Table can be updated only manually - documenting differences between
> > POSIX and busybox behaviour on options that do exist.
> > It can serve as a mini-TODO file.
> > The only easy way I see to go with tests is to check for full
> > compliance by comparing with GNU (which is not strictly compliant
> > itself).
> >
> > I ran the test suite - there are some FAILs:
> >  * Is it up to date?
>
> unfortunately, some tests may require some options enabled/disabled.  it
> may or may not be documented.  this is something that should be fixed (i.e.
> XFAIL if a test requires an option but said option is disabled).  this is
> what we've been doing in the hush tests.

I've had infrastructure for that toybox for over a year.  I have a big todo 
item to port as much of the toybox infrastructure as I can back into BusyBox, 
it's just not at the top of my todo list yet and it's a big thing I'll have to 
do in stanges (and several bits of which are kind of intrusive requiring 
documentation, explanation of the rationale, and probably extensive 
discussion).

First question: is the current test suite more closely maintained than the 
documentation directory?  (Is there somebody specific caring for it, or just 
"when the maintainer has spare time"?)

> >  * I've seen some tests compare with GNU tools output - does it
> > require a specific version of GNU utilities?
>
> we only care about the latest versions.  that means some tests may have
> been written for older versions and so fail with latest ones.

An implementation is not a standard, especially where different gnu tool 
versions give different results.

> or they're
> written for  the latest and so it is expected they'll fail on older ones. 
> we should document in the test the last known tested GNU version, but we
> should always be targeting the latest GNU releases.

We should not be testing against gnu tools.  The test suite should run and 
produce usable results on a system that _only_ has busybox installed.

> > Though there are tests that can be used for this objective, the test
> > suite is lacking a simple way to mark existing tests with some kind of
> > a flag like POSIXTEST. How do you propose to do that?
>
> i think we should split the tests so that POSIX and GNU conformance is kept
> separate.

Annotating existing tests which what context they test is useful.  Splitting 
the test suite into separate scripts means each one now gets half as much 
attention.  (Proposing we do twice as much testing when we haven't got 
remotely thorough _existing_ tests seems kind of utopian.)

Rob
-- 
Latency is more important than throughput. It's that simple. - Linus Torvalds