Puppet

There is a known (albeit poorly understood) bug in MRI’s GC/stack frame handling that can cause extreme memory leakage.

Ruby manages a number of special “global” variables that are actually scope limited transitory thread local state accessors; for example $_ (the last line read with gets) $~,$1,$2,… (the details of the last regular expression match).

These are normally stored in the stack frame of the routine that triggered them (so if you do “Bob’s your uncle” =~ /(.*)’s/ in a routine you will subsequently see $1 == “Bob” until the next match is performed, but the routine that called you will not see a change in $1 on return).

A problem arises when a routine that does not have its own stack frame (because of a compiler optimization) performs an operation that creates one or more of these variables. In this case MRI dynamically creates a stack frame but (due to the bug) neither cleans it up on exit nor properly manages for the GC to collect later. This is worse than a normal memory leak (due to careless object reference management for example) in that the orphaned chunks apparently can not be moved. Thus the heap grows increasingly fragmented causing even more memory to be wasted; hundreds of megabytes can be consumed in a few minutes, only to be released when the program exits.

The simplest characterization of the sort of routines that trigger the optimization is “routines with no local variable assignments”. Characterizing the operations that will trigger the bug is a bit harder (see the attached files for a few examples that explore the boundary).

We have around ten routines that meet the definition, several of which can be shown to exhibit the problem.

The following files show cases that do (and do not) cause explosive memory consumption. They are all as simple as possible to demonstrate the point; their behavior under MRI 1.8.6 can be inferred from their names.