XZ embedded bug unpacking linux-3.8.tar.xz (was: Re: tar: short read on linux-3.8.tar.xz)

[Matias A. Fonzo]

Hello Denys,

El Wed, 27 Feb 2013 17:59:25 +0100
Denys Vlasenko <vda.linux at googlemail.com> escribió:
> On Mon, Feb 25, 2013 at 7:20 PM, Matias A. Fonzo <selk at dragora.org>
> wrote:
> > Can be lzip considered for inclusion in busybox?:
> >
> > [1] http://lzip.nongnu.org
> > [2] http://en.wikipedia.org/wiki/Lzip
> > [3]
> > http://lists.busybox.net/pipermail/busybox/2012-December/078750.html
> > [4] http://ur1.ca/810mp
> 
> Matias, sure, this can be done.

Great.
 
> But bbox already has *two* LZMA decompressors.
> Feels wrong, isn't it?
> 
> In the long run it would be a nightmare to have two
> or more LZMA (de)compressors in common use on Linux.

Why?. *.lzma are deprecated some time ago, because does not provide a
proper header -- impeding the (correct) recognition of lzma-alone
files, and that it lacked integrity checking (happily producing
corrupted output without a warning).

> What happened between lzip and xz? Are they incompatible?
> On what level? File format, or compression stream format too?

In both levels. In brief:

Xz uses the method from LZMA SDK (By Igor Pavlov).

Lzip uses a simplified version of the LZMA algorithm.

About the file format, the first four bytes of a .lz file, says:

  LZIP

While the first four bytes of a .xz file contains a reference to
"7zip". That is because Igor Pavlov (a Windows developer) was involved
in the development of xz. To think in xz as an intent to introduced the
Windows philosophy into Unix-like systems is: valid.

http://tukaani.org/xz/xz-file-format.txt
http://lzip.nongnu.org/manual/lzip_manual.html

What happened between lzip and xz?. Well..

Lzip was created before than xz. As a software distributor, I remember
the existence of a project called "Tukaani" (a GNU/Linux distribution
based on Slackware Linux featuring xz), in that time xz was in *beta
state*. Meanwhile, Lzip's author was publishing and polishing stable
versions of a LZMA implementation. Tukaani was in a lethargy, when
Slackware decided to incorporate xz to distribute their packages; this
example was followed by Fedora and other distributions, then Tukaani
comes up to restart his activities (focusing only in the xz) -- After
several months (if not years) of the xz creation, they announce the
stable version...