[PATCH] Add a gzip fastpath for the xmalloc readers, v2

[Denys Vlasenko]

On Fri, Nov 28, 2014 at 10:44 AM, Lauri Kasanen <curaga at operamail.com> wrote:
> v2: Add missing check on open
>
> The performance and number of processes for a "depmod -a" with gzipped
> modules was abysmal. This patch adds a fast path without fork for well-
> behaved gzip files, benefiting all users of xmalloc_open_zipped_read_close.
>
> "modinfo radeon.ko.gz", a single-file reader, got 30% faster.
>
> "depmod -a", which used to fork over 800 times, got 20% faster. And of course
> a whole lot less processes -> much saved RAM.
>
> function                                             old     new   delta
> inflate_get_next_window                                -    1877   +1877
> xmalloc_unpack_gz                                      -     356    +356
> check_header_gzip                                      -     298    +298
> xmalloc_inflate_unzip_internal                         -     223    +223
> xmalloc_open_zipped_read_close                        73     176    +103
> inflate_init                                           -      97     +97
> inflate_store_unused                                   -      35     +35
> unpack_gz_stream                                     567     299    -268
> inflate_unzip_internal                              2304     172   -2132
> ------------------------------------------------------------------------------
> (add/remove: 6/0 grow/shrink: 1/2 up/down: 2989/-2400)        Total: 589 bytes


This feels somewhat big.

Looking at the code, you have significant code duplication.

I think a following approach can work:

* Extend transformer_aux_data_t so that it can specify
  "I want decompressed data to go into a mem buffer".
  Say, size_t aux->mem_output_size. If >0, it's the maximum
  amount of bytes you allow to decompress into it
  (0 means "decompress into dst_fd").
  The result is char *aux->mem_output_buf.

* Pass aux pointer into inflate_unzip_internal().

* There, modify a decompression loop so that it stores
  result in aux->mem_output_buf if aux->mem_output_size >0
  (xreallocing it so that it grows with decompression)

* In xmalloc_open_zipped_read, you only need to construct
  a suitable aux, call unpack_gz_stream(), and get
  aux->mem_output_buf as your result buffer.

This way, you don't need xmalloc_unpack_gz()
and xmalloc_inflate_unzip_internal().

The special-casing of GZ in xmalloc_open_zipped_read looks ugly,
but demanding a generic mechanism for in-memory unpacking
is too much for one patch...