XZ embedded bug unpacking linux-3.8.tar.xz (was: Re: tar: short read on linux-3.8.tar.xz)

[Matias A. Fonzo]

El Thu, 28 Feb 2013 08:05:39 +0100
Denys Vlasenko <vda.linux at googlemail.com> escribió:
> On Thu, Feb 28, 2013 at 1:34 AM, Matias A. Fonzo <selk at dragora.org>
> wrote:
> > Hello Denys,
> >
> > El Wed, 27 Feb 2013 17:59:25 +0100
> > Denys Vlasenko <vda.linux at googlemail.com> escribió:
> >> On Mon, Feb 25, 2013 at 7:20 PM, Matias A. Fonzo <selk at dragora.org>
> >> wrote:
> >> > Can be lzip considered for inclusion in busybox?:
> >> >
> >> > [1] http://lzip.nongnu.org
> >> > [2] http://en.wikipedia.org/wiki/Lzip
> >> > [3]
> >> > http://lists.busybox.net/pipermail/busybox/2012-December/078750.html
> >> > [4] http://ur1.ca/810mp
> >>
> >> Matias, sure, this can be done.
> >
> > Great.
> >
> >> But bbox already has *two* LZMA decompressors.
> >> Feels wrong, isn't it?
> >>
> >> In the long run it would be a nightmare to have two
> >> or more LZMA (de)compressors in common use on Linux.
> >
> > Why?. *.lzma are deprecated some time ago
> 
> Because someone submitted the code:
> 
> commit c1d69906a0c5f28f3d84c14afb3b74c8f19f81c1
> Author: Rob Landley <rob at landley.net>
> Date:   Fri Jan 20 18:28:50 2006 +0000
> 
>     Patch from Aurelien Jacobs to add unlzma.  (A new decompression
> type, see www.7-zip.org)
> 
> 

Maybe is time to remove unlzma from busybox.

> 
> >> What happened between lzip and xz? Are they incompatible?
> >> On what level? File format, or compression stream format too?
> >
> > In both levels. In brief:
> >
> > Xz uses the method from LZMA SDK (By Igor Pavlov).
> >
> > Lzip uses a simplified version of the LZMA algorithm.
> >
> > About the file format, the first four bytes of a .lz file, says:
> >
> >   LZIP
> >
> > While the first four bytes of a .xz file contains a reference to
> > "7zip". That is because Igor Pavlov (a Windows developer) was
> > involved in the development of xz. To think in xz as an intent to
> > introduced the Windows philosophy into Unix-like systems is: valid.
> 
> I don't really see advantages of one format over the other.
> 

http://lists.gnu.org/archive/html/automake/2011-12/msg00077.html
http://lzip.nongnu.org/benchmark.txt

> 
> > Lzip was created before than xz. As a software distributor, I
> > remember the existence of a project called "Tukaani" (a GNU/Linux
> > distribution based on Slackware Linux featuring xz), in that time
> > xz was in *beta state*. Meanwhile, Lzip's author was publishing and
> > polishing stable versions of a LZMA implementation. Tukaani was in
> > a lethargy, when Slackware decided to incorporate xz to distribute
> > their packages; this example was followed by Fedora and other
> > distributions, then Tukaani comes up to restart his activities
> > (focusing only in the xz) -- After several months (if not years) of
> > the xz creation, they announce the stable version...
> 
> So basically, lzip lost the race wrt adoption. xz is used more widely.
> Kernel tarballs are .xz, not .lz.

Basically, I'm saying that some upstreams projects have been taking the
wrong decision from the beginning to distribute packages / sources in a
LZMA implementation presented as beta (worse was with lzma-alone).

> What I'm saying is that bbox project would like to have is (ideally)
> _one_ LZMA decoder. Unpacking the compressed stream from two formats
> isn't a terribly difficult thing.
>
> But can lzip decompress unxz *stream*?

No, it should be?.