Tuesday, August 23, 2005

Variadic functions and non-POD arguments

This is regarding an issue that I came across a few days back and I really liked this one as I helped the guy understand a few concepts involved better and in the process learning myself. This in heart involved variadic functions. I quote the problem below:


There is a class.(I would not provide the code as it would make the post long, its easily understandable in words and if you feel like then you try experimenting with some code on your own). The various members it has are:

1. A constructor with no arguments.
2. A copy constructor.
3. A destructor.
4. An overloaded (++) operator. (not that significant as far as the solution to the issue is concerned)
5. A private data member. (not that significant as far as the solution to the issue is concerned)

We create an object of this and use printf statement once. In the printf I pass the object of this class. Printf and scanf are functions of truely a different breed. You must have heard about variadic functions. They can take any number of different types of arguments that's their speciality. What happens during this whole process in main - constructor is called twice (2nd one being a call to the copy constructor) and the destructor is called just once!!! Is there some kind of memory leak? What happened, whats the reason for this awkward behaviour?

Short answer:

The behaviour in the given scenario is undefined. What it means is you got these results on some compiler - you would get different results on some other and there is no guarantee as such that application would not crash. I tried running the code of my machine with VC++ 6.0 and it worked fine. That is what is undefined behaviour... we never know.


We will look forward to the solution with a very generic approach. We would not restrict the reasoning to just printf. Since they are examples of variadic functions so lets start with variadic functions in general. To know more about variadic functions - I would suggest you to visit this -
Variadic Functions : gnu documentation
and Variadic functions : informit article.

There are various restrictions imposed whenever you try using variadic functions in C++. I would try stating the scenarios making for this undefined behaviour below and put quotes from these links that I find significant.

1. Simply because the rules say (Hehe :-)), you cannot use non-POD types in the variable argument list of a variadic function. That would result in an undefined behaviour. It's about time I gave a definition to non-POD type or rather POD type? Here's what they are (I quote the definition from the C++ FAQ Lite - a collection by Marshall Cline). POD type = Plain Old Data type.

A POD type is a C++ type that has an equivalent in C, and that uses the same rules as C uses for initialization, copying, layout, and addressing. As an example, the C declaration struct Fred x; does not initialize the members of the Fred variable x. To make this same behavior happen in C++, Fred would need to not have any constructors. Similarly to make the C++ version of copying the same as the C version, the C++ Fred must not have overloaded the assignment operator. To make sure the other rules match, the C++ version must not have virtual functions, base classes, non-static members that are private or protected, or a destructor. It can, however, have static data members, static member functions, and non-static non-virtual member functions. The actual definition of a POD type is recursive and gets a little gnarly.

Here's a slightly simplified definition of POD: a POD type's non-static data members must be public and can be of any of these types: bool, any numeric type including the various char variants, any enumeration type, any data-pointer type (that is, any type convertible to void*), any pointer-to-function type, or any POD type, including arrays of any of these. Note: data-pointers and pointers-to-function are okay, but pointers-to-member are not. Also note that references are not allowed. In addition, a POD type can't have constructors, virtual functions, base classes, or an overloaded assignment operator.

2. C++ has two mechanisms of passing arguments - by value and by reference compared to just pass by value as in C. Now, here's the ambiguity that creeps in, in these 2 cases is that when you make the calls by either of these they are same syntax from caller point! So, the compiler (and the variadic function) is not able to distinguish if that is intended to be passed by value or by reference, if a copy is to be made or not. This might be fixed by forcing one single argument passing option i.e. it always happens by pass-by-value (can't choose by-reference as we need to keep them working for C as well). Calling conventions (point 3 below) make this more complicated as you will see.

3. The function calling conventions, we would be considering are __stdcall and __cdecl. You could read up on them on MSDN with a simple search. With __cdecl, the caller has the responsibility to clean up the stack and with __stdcall the callee has this responsibility. Now, variadic functions impose certain basic rules as stated in point (1) above due to which _stdcall would not work. __cdecl would possibly give similar behaviours but still in case of non-POD types, its a general behaviour that the callee (I dont have proofs to back this but you can take my word - or you could try doing some research with the assembly code) calls the respective destructors or atleast that it's not standardized behaviour so you cannot rule it out. Now, here's the problem - the function doesnt know the number and types of the arguments and hence this would be a step that would cause issues..dreading issues. So, it simply assumes what I have stated in point (2) that the variables being passed are POD types (againts the non-POD types that are actually being passed) and the destructor doesn't (or better say, may not) get called! This is a problem.

4. There are certain restrictions regarding variadic arguments/functions that simply are rules to reject any possibilities:

    4.1) A function that accepts a variable number of arguments must be declared with a prototype that says so. You write the fixed arguments as usual, and then tack on `...' to indicate the possibility of additional arguments. The syntax of ISO C requires at least one fixed argument before the `...'.

    4.2) For some C compilers, the last required argument must not be declared register in the function definition. Furthermore, this argument's type must be self-promoting: that is, the default promotions must not change its type. This rules out array and function types, as well as float, char (whether signed or not) and short int (whether signed or not). This is actually an ISO C requirement.

    4.3) The parameter parmN is the identifier of the rightmost parameter in the variable parameter list of the function definition (the one just before the ...). If the parameter parmN is declared with a function, array, or reference type, or with a type that is not compatible with the type that results when passing an argument for which there is no parameter, the behavior is undefined. (C++ ANSI standard, 18.7.3)

    4.4) Lastly, when there is no parameter for a given argument, the argument is passed in such a way that the receiving function can obtain the value of the argument by invoking va_arg (18.7). ...if the argument does not have arithmetic, enumeration, pointer, pointer to member, or class type, the program is ill-formed. If the argument has a non-POD class type, the behavior is undefined. (ANSI C++

Apart from all that, probably, there might be a way to implement variadics such that they just work but the points above are useful to know when writing variadics according to the current set of rules about when they work and when they don't.


So, now I infer that variadic functions are quite specific to C. You cannot use non-POD C++ types with them. For manipulating non-POD types, you should use the C++ techniques. You are quite restricted to using variadics in C++ but still they are sometimes needed and they should be carefully used with POD types only. The article links (the two that I have above) illustrate the points further in detail.

Here are a few alternatives (one of the below or they together with each other) to these functions in C++:

1. Functions with Default Arguments
2. USage of Function Templates
3. Packing Function Arguments in a Container
4. Overloading operator <<

You could also refer to the codeguru thread that is the base for this post - that helped me digest the above as a rule - Destruction (variadic woes).

P.S. - Please report comments, corrections, and suggestions at this blog. I am learning.


TooTall said...

Dear abnegator
Thanks! Very helpful. MFC and wxWidgets string objects implement Format with the ability to pass CString objects or wxString objects unadorned on the arg list.

This is MAGIC. During compilation apparently the sMyString gets converted into (LPCTSTR) sMyString, but you cannot single step it in the debugger because it happens at compile time.

How does it know which conversion operator to call?

Anyway, thanks again

abnegator said...

Hello Tim,

Sorry for a rather late response. Just noticed this recently.

I don't think what you say is reliable. It would still be undefined behavior as the rules apply to variadic functions in general and CString::Format falls into the same category.

It is merely chance, that the code might work but it is dangerous. The CString object layout might favor the interpretation of the object correctly but those are internal details that can change.

If you intend to use CString object (or any other C++ objects) to pass into Format (or any other variadic function), you should do the appropriate conversion and pass that into it instead.