Using GCC's Nested Functions with Wide Pointers and no Trampolines II

Martin Uecker, 2026-07-14

Introduction

When the address of a nested function is taken, GCC creates at trampoline at run-time. This trampoline is placed on the stack, which means that the stack has to be executable. This is problematic because a non-executable stack is an important security feature. For some time now, GCC has the option to place the trampoline on the heap, which is better but still not ideal as the allocation has a higher cost and the trampoline could leak when longjmp is used.

Previously, I pointed out how this could be avoided with a new wide pointer type, and also talked about a preliminary patch to GCC that implements some core functionality that would be necessary for this.

Here, I want to give two updates.

GCC 16: Nested Functions Without Capture

First, GCC 16 was released. In GCC 16 it is guaranteed that a nested function that does not access variables of a parent function will not require a trampoline. In practice, this was already ensured when optimizing, but now it is also ensured when not optimizing and documented. This also means that you can safely return such a function from its parent and that you will get a warning when trying to return a function that does access the local context (at least in simple cases where this is obvious to the compiler) as in the following example (Godbolt Example) 


	typedef int cb_f(int);

	cb_f *foo(int x)
	{  
	  	int worker(int y)
	    	{
        		return x + y;	// capture
    		}

   		return worker;
	}

	cb_f *bar(int _x)
	{
		static int x;
		x = _x;

		int worker(int y)
		{
			return x + y;	// no capture
		}

		return worker;
	}

Nested function that do not access variables of the parent function can still access static variables, named constants, or types (if not variably modified). This is useful for writing callback functions (Godbolt Example). 


	typedef int cb_f(void *, int y);

	int process(cb_f *cb, void *data)
	{
		return cb(data, 5);
	}

	int foo(int x)
	{
		struct { int x; } data = { .x = x };

		int worker(void *_data, int y)
		{
			typeof(data) *data = _data;
			return data->x + y;
		}

		return process(worker, &data);
	}

Still, the true power of nested functions is being able to directly access the variables of the parent function. Also, the code above seems like we just simulate such functions by manually adding some boilerplate code - the compiler should certainly be able to help us with this. This is what my second update is about.

GCC 17: Nested Functions with Capture

Recently, a version of my patch was merged into the development branch of GCC that implements the two new built-in functions that can be used together with the existing built-in __builtin_call_with_static_chain to write this example (Godbolt Example) without requiring trampolines. 


	typedef int cb_f(int y);

	static int process(cb_f *cb, void *data)
	{
		return __builtin_call_with_static_chain(cb(5), data);
	}

	int main()
	{
		int x = 7; 
    
		int worker(int y)
    		{
			return x + y;
    		}

		return process(__builtin_call_code_address(worker),
			       __builtin_call_static_chain(worker));
	}

In fact, with trampolines out of the picture, all that GCC does under the hood is to automatically transform this into the example above! The only difference to the manually written version that the data pointer is passed via a special register. This register is used by many other programming language for passing the static chain and therefor already reserved for this use by the ABI.

Even without proper language support, we can still make this example a bit nicer by defining a generic wide pointer type and wrapping the built-ins via macros. (Godbolt Example) 


	#define wide(T) struct wide_##T { typeof(T) *code; void *chain; }

	typedef int cb_f(int y);

	int baz(wide(cb_f) p, int x)
	{
		return CALL(p, (x));
	}

	int foo(int k)
	{
		int bar(int x) { return k + x; }
    		return baz(CLOSURE(cb_f, bar), 2 * k);
	}

The compiler will happily optimize this to a single assembly instruction that implements a multiplication by three.


	foo:
        	lea     eax, [rdi+rdi*2]
        	ret

Outlook

While there are compilers other than GCC that support nested functions, this feature is not really portable. In the future, I will discuss how one can use nested functions with both clang and GCC and how one can, with a hack, avoid creating trampolines even on older versions of GCC.

Literature