[PATCH] introduce CONFIG_BUFFER_SIZE()

[Alexander Shishkin]

On Sun, Aug 01, 2010 at 01:05:53 +0200, Denys Vlasenko wrote:
> On Saturday 31 July 2010 12:50, Alexander Shishkin wrote:
> > On Fri, Jul 30, 2010 at 03:26:25 +0200, Denys Vlasenko wrote:
> > > On Tuesday 27 July 2010 16:21, Alexander Shishkin wrote:
> > > > Since "sizeof buffer" won't work with dynamically allocated buffers,
> > > > and we still sometimes need to know the size of the whole buffer, we
> > > > need a way to always obtain said size.
> > > 
> > > In almost all cases, allocating on heap is the best anyway.
> > > Maybe we should just gradually move to it and get rid of this
> > > three-way configurability? There is a bug in fdisk somewhere which
> > > only triggers with on-stack allocation IIRC, that's the price
> > > for having a rarely tested config...
> > > 
> > > Do I miss why on-stack alloc may be better that malloc
> > > for some users? Is it NOMMU?
> > 
> > Well, on-stack allocation is cheaper since it doesn't require any extra
> > calls in userspace.
> 
> Yes. But this seems to be the only positive side.
> 
> Whereas negative sides are the following:
> 
> There are limits on how much you can allocate. Even on MMU machines
> like x86 the limit is 8 mbytes (nscd had bugs where that was a problem);

That's per-process and adjustable. And I can hardly think up a situation
(apart from a bug), when busybox might get remotely close to that.

> on NOMMU it is much smaller: the stack can't be grown there,
> therefore it is preallocated, and preallocating a megabyte of stack
> for every process is not practical.

Be that as it may, the whole memory allocation business on nommu machines
is tricky and perhaps it shouldn't use the same strategies as the normal
machines capable of virtual memory.

> Stack can grow, but it never shrinks; if you run something deeply

It gets deallocated when busybox exits. And correct me if I'm wrong,
but I think quite some of busybox applets are of get-in-quickly-do-your-
business-and-leave persuasion, so that in most cases one shouldn't worry
about busybox leaking stack.

> recursive and you momentarily ate a megabyte of stack, all these
> dirtied pages will remain in your process until it exits,
> even though you are likely won't ever use them again.
> Try this: run busybox top, press 's' several times until
> you sorted processes by decreased stack size. Result on my machine:
> 
>   PID   VSZ VSZRW   RSS (SHR) DIRTY (SHR)*STACK COMMAND
>  2287 90604 25936 54468 12004 34844     0  1920 kmail
> 
> This is it. kmail ran something deeply recursive
> (or called a function with ridiculously huge on-stack
> objects). Now it is permanently saddled with nerly 2 megs
> of dirty anonymous pages.
> 
> malloc has none of these problems.
> 
> > In a situation with heavy memory pressure, you might 
> > well end up having malloc() returning NULL, which will lead to busybox
> > exiting, whereas with onstack allocation you it will trigger OOM killer
> > which is likely to kill something else.
> 
> And having a random other application killed is better because ... ?

It's obviously badly written and needs fixing and it's standing in the way
of me trying to do my stuff by means of busybox.

> > Other than that, I would actually vouch for BSS allocation, which also
> > has the benefit of being done only once when the binary is loaded.
> 
> bss pages are populated when any byte in them is modified.
> Which means that with data elements less than a page you end up using
> more memory than you need.
> 
> bss "allocation" can't be deallocated and can't be used in recursive
> functions.

Again, for certain applets in busybox this might be considered a bonus. But
the recursion is a problem, yes.

Regards,
--
Alex