If you need to expose a number of functions in C++ to an interface, template metaprogramming may be of tremendous help. This post outlines one such case. Here is the entire working code example (GCC’s optimizer is good enough to optimize all of the program away, that’s why the compiled version is blank!)
- The problem
- Possible approaches
- Solution using TMP
After having had the pleasure of programming in a variety of languages in very different projects, I can safely say that C++ is the one that strikes me the most.
This post will be about one of my favorite practical examples of the usefulness of template metaprogramming; if you’ve never heard of that before, I suggest reading up on the topic or watching these videos if you have some C++ experience already (part1, part2)
Imagine the following: you’re the author of an application server, written in C++, whose behaviour is supposed to be customized by its users via scripts. You’ve decided for a scripting language (that could be LUA, Ruby, …). Naturally you will want to expose a set of functions and data specific to your application to that scripting language, so your users can fully leverage the awesome stuff you’ve been working on for so long.
Depending on the actual scripting language, the way to do just that varies. Let’s focus on functions for now and assume you can register a function with the following C interface with the
The call to actually register this function could look something like this:
And the actual implementation of that function:
That doesn’t look too bad for now - but now imagine there are about 300 functions you’d like to register this way. You’d need to write 300 wrapper functions (just like
MyFunction), possibly list them in a static array together with their unique string identifiers, eventually registering them one by one at runtime in a loop. This is not only tedious and boring but also terribly error-prone.
Here’s a number of possible solutions you may come up with.
Instead of writing one wrapper function for each function you’d like to expose, you register a single wrapper function that contains a massive
switch-statement, like this:
This doesn’t solve the problem; you just moved all the annoying wrapper logic somewhere else and added some overhead with the
switch-logic along the way.
The “hacker” way
Again, you only write a single wrapper function, in which you will take the parameters and push them onto the stack one by one (according to the calling convention in place), taking the corresponding function address (that you picked from a table using the string identifier as an index) and calling it. Obviously it only works if the data types passed through
params are so trivial that they need no conversion logic, so you also need not be aware of the implementation function’s signature. It also requires that the
params array is (null)-terminated.
This is plain bad (but still fun to implement, nevertheless) and due to the very platform-dependent nature of this, there’s not much point in providing a code example. You’ll probably have to fiddle with assembly and you will lose all of the safety a static function call provides as well.
Solution using TMP
And here’s how to do it the most elegant way (in my opinion): by creating a set of generic function templates that the compiler will use to instantiate a wrapper function for each script function you’d like to expose.
The entire solution is, due to the nature of templates in C++, rather verbose, so I will try to break it down to the most essential parts (without explaining all of the intricate details).
First off, we will need some way to reflect on our code - more specifically, the script functions we’d like to expose. To instantiate a wrapper function, we need the following information about it: its function address, its parameter types as well as its return value type. Here’s the structures and helper templates that will be used to capture this information:
Note that all of this data will exist only at compile-time; it will be instantiated so the compiler can use it in the wrapper function template later on.
There may be other or even better ways to do it, but I decided to encode the above-mentioned information as simple character values.
TypeChar is the template that maps a type to a (unique) character that we can use later on. Every possible type that you are using in your script functions should have a
TypeChar mapping. Provided above are examples for
TypeString is an array of
TypeChar - given a parameter pack
Types, it will determine the character for each of the types provided.
ScriptIdentity holds all the information we need:
types will contain the characters corresponding to the parameter types of the function,
ret is a single character indicating the return type, and
numargs is simply the number of parameters. The constructor of
ScriptIdentity is templated so to allow extracting the parameter types and return type of a function passed to it as an argument.
ScriptFunction adds some meta info about the function to it all that we will provide ourselves - for now, this is just a descriptive name.
Given this implementation, we can declare an array referencing all our script functions:
The glorious prospect: the only thing that will be necessary to create wrapper functions for any (or at least, the large majority) of your script functions will be to add them to the array above (and recompile). Isn’t that exciting?
Given the structures defined in the previous section, we can now write a templated function that will be the basis for the compiler to generate a wrapper for each function we’d like to expose to the script engine. The signature of this function needs to be exactly the same as the one of
MyFunction declared in the previous post.
Inside it, we need to perform the following essential steps:
- Transform all the parameters we receive from the script engine from
const data_type*to the types we need to call the script function
- Call the script function with these transformed values
- Receive the return value of this call and transform it to
Let’s start off by defining a function template for converting a single parameter from the
data_type array to a given type
R. It can look like this:
Most likely, you will want to specialize this template for various
R in case the conversion takes more than just a simple
With this building block in place, we can move on to implementing the centerpiece: a templated function performing the call to the script function. Since the call will involve passing an arbitrary number of arguments of arbitrary types, we will be employing a variadic template. Variadic templates are one of the major new features of modern C++ you have to know!
dispatch_ struct will be instantiated with
I, which is the index of the next argument we will extract, and
F which is the index into the constant expression
functions array we declared earlier indicating which script function we will be calling in the end. The
dispatch function contained within makes use of
R which is the return type of the script function and
Args, which is the parameter pack we are going to build by processing each argument recursively.
Args will be empty: while
I is greater than zero, we will extract a single argument using
I by one and instantiate
dispatch_ again until we have extracted all arguments and
I reaches zero. A specialization for
dispatch_ exists for this particular case, effectively terminating the recursion; a pattern which is commonly used when programming with templates. In it, the actual call to the script function is performed and the return value of it given back to the caller.
Note that the arguments are extracted in reverse order: we start using the last (“rightmost”) one and stop with the first (“leftmost”). The script function is then, however, called with the arguments in the expected order.
CharType is a template which is essentially
TypeChar with its mapping reversed: it is being used to map the argument type (which we encoded as a character) back into an actual type which will be forwarded to
(side note: one major improvement I’d like to see in this implementation is replacing the ellipsis in the
reinterpret_cast with a proper type. I believe it should be possible to instantiate this function type using the information we encoded in the
ScriptFunction structure, but I haven’t attempted it yet)
Now we can wrap it up and specify a function having the
MyFunction signature which I mentioned initially: quite simple now!
You may want to implement a template similar to
extract which is performing a proper conversion from the type of
result_type if necessary.
A gotcha, then: this will fail to compile if the script function’s return type is
void. You can specialize this template with the help of
enable_if / SFINAE to work around this.
I mentioned we’d end up only having to add a single line per script function to the
functions array and recompile to be done. We didn’t achieve this yet! For automatically instantiating all the wrappers, we will let the compiler create another array containing them.
We are using
integer_sequence (C++14) to generate a series of indices based on the number of entries in
functions, turning the sequence into a parameter pack
Is and expanding it into a static array containing the wrapped functions. To retrieve this array at runtime (to register the contained wrapped functions with the script engine), you’d call
wrappers(functions_seq()). It should be possible to implement this array without using a function returning it too if necessary.
This is it! Here is a full working implementation of everything described. It should compile fine using any C++14 compiler (I tried Clang 3.9). If you can suggest any improvements or have questions/comments, feel free to let me know on the GitHub repository. Thanks for reading!