Indirect Calling of Nested Functions on GCC Without Executable Stack

Martin Uecker, 2026-08-29

Introduction

We discussed last time how one can use nested functions on GCC 17 and Clang for callbacks with requiring an executable stack. But what if ones needs to support older versions of GCC? Of course, one can simply accept an executable stack (it is not quite as terrible as some people claim), but it is also possible to avoid this with a hack.

GCC: Nested Functions and Trampolines

Let's discuss first how GCC supports taking the address of a nested function. Our toy example without the use of the new macros is shown below (Godbolt Example). 


	typedef int cb_f(int y);

	int baz(cb_f p, int x)
	{
		return p(x);
	}

	int foo(int k)
	{
		int bar(int x) { return k + x; }
    		return baz(bar, 2 * k);
	}

On x86_64, the generated assembly is the following. 


bar.0:
        movl    %edi, %eax
        addl    (%r10), %eax
        ret
foo:
        subq    $56, %rsp
        leaq    64(%rsp), %rax
        movq    %rax, 32(%rsp)
        movl    %edi, (%rsp)
        leaq    4(%rsp), %rax
        movw    $-17591, 4(%rsp)
        movabsq $bar.0, %rcx
        movq    %rcx, 6(%rsp)
        movw    $-17847, 14(%rsp)
        movq    %rsp, 16(%rsp)
        movl    $-1864106167, 24(%rsp)
        addl    %edi, %edi
        call    *%rax
        addq    $56, %rsp
        ret

This code places a trampoline on the stack and immediately invokes it via the inlined baz function. The trampoline is a short code sequence that loads the static frame register to a structure on the stack that contains the captured variables from the parent function and then jumps to the local function. If one translates the constants -17591, -17847, and -1864106167 back to assembly instructions one obtains the following x86_64 code. 


        movq	$bar.0, r11
        movq    $frame, r10
	jump	*r11

Both, the static chain and the code address are immediate constants used by move instructions in the code of the trampoline. Instead of using the address of the trampoline to call the function, we can extract the code address and the static chain from the trampoline and use them with the __builtin_call_with_static_chain built-in function to call the local function directly. For example, using noplate's peek and array_slice macros, this could be done in the following way for x64_64 (and a large memory model). 


	unsigned char (*tramp)[24] = (void*)bar;
	void *code = peek(uint64_t, &array_slice(tramp, 2, 10));
	void *chain = peek(uint64_t, &array_slice(tramp, 12, 20));

These pointers are then exactly the same information that can be obtained on the yet to-be-released GCC 17 with the new built-ins __builtin_call_static_chain and __builtin_call_code_adress and can be used to call the local function with as discussed previously. 


	__builtin_call_with_static_chain(((typeof(bar)*)code)(arg), chain);

Thus, we can use reading of these two pointer values from a trampoline as a fallback mechanism in older versions of GCC. The downsides are that a trampoline is still created, the compiler can still not devirtualize the indirect call, and the stack will still be marked executable. So what was gained? Since we never actually invoke the trampoline, we can make the stack non-executable again with the following command, which at least addresses the security concerns of this feature. 


	patchelf --clear-execstack program

This idea is implemented in my experimental library, noplate, where a wide pointer is constructed from the code address and the static chain.

Trampolines as Function Descriptors

There is another idea I find worth exploring: One could also use the trampoline itself as a function descriptor. Instead of extracting the static chain and code pointer where the trampoline is created we just pass on the address of the trampoline as usually. But everywhere where we might call the trampoline, we first check whether the pointer points to a trampoline, and then extract code address and static chain to call the nested function directly using __builtin_call_with_static_chain. In some sense we could say that instead of invoking the trampoline, we are interpreting the code of the trampoline at the call site using a super simple interpreter that only can interpret this specific code sequence and that is so simple that it can be inlined (Godbolt Example).

Literature