Handling dynamic array allocations

One of the recent questions on the forums, that I answered to recently, was related to using std::auto_ptr<> with allocations from array form of new. The need to know was - if the following would work?

        std::auto_ptr<int> ptr(new int[100]);

if not then why not?

The answer was simple if one knows what std::auto_ptr<> is good for, what its deleter is. std::auto_ptr<> uses delete operator to destruct owned pointer (pointing to a dynamic allocation made via new operator). So, it comes down to mixing of constructs like array form of new and simple delete. That is undefined behaviour as per the standards. You can read up more about these operators here - Free New, Delete Malloc

By design, the standard auto_ptr<> smart pointer is not suitable for dynamic array allocations. There can be many alternatives though. Few of those are as listed below:

• Avoid self memory management and instead use std::vector<> / std::deque<> for dynamic arrays.
• Use boost::scoped_array<>/boost::shared_array<>/boost::shared_ptr<> with a custom deleter (delete[]) passed in.
• Write your own smart pointer : auto_ptr_array<> (same implementation as std::auto_ptr<>) just that it uses array form of delete instead of delete.

For the second point above - you can read more about custom deleters and shared_ptr<> here - Custom deleters with smart pointers

A boost::shared_array<>/boost::shared_ptr<> might not be the right choice if you don't want the reference count overhead of shared ownership which really should not be required when talking about auto_ptr<>s but if that's not an issue to worry about, it should work fine.

Another confusion arose, why doesn't std::auto_ptr<> do a delete[] instead of delete? Well, again, because it is not intended for array allocations. It would hence not work for singular objects created on the free store. This is because of the way delete[] might be implemented by the compiler.

There can be many ways in which delete[] could be implemented. Two of those are explained here - How does delete[] know how many elements to destroy? It is important for delete[] to know how many elements to destroy particularly since it is used with non-POD types as well and there, the destructor calls are necessary to be made for each of the to-be-destructed elements of the array.

The first method listed in the C++ FAQ Lite uses an extra allocation to remember the size you ask for at the runtime, if you used delete[] for single allocations, the compiler might try to interpret the block before the pointer you call it on as being size but since that was not allocated in the first place (new[] was not used!) - expect anything to happen, a crash or any other form of runtime error (depending upon how that error is translated on a particular system).

For the second, where array length association might not exist, it could cause anything depending upon a number of factors about the compiler implementation. One such possibility is, it could leave the variable uninitialized (yeah, its a bad thing, may not actually be happening, but who knows for sure about all compilers? We are just talking of possibilities here) - in which case the size could be anything - even 2 which goes beyond the buffer you allocated and again can cause any fault to the system/system free store. If the size is initialized to 0, it could leave the memory untouched and hence lost resulting in a leak. Memory leaks can be really dangerous particularly on systems that don't reclaim the leaked blocks by a process.

So, when the standard says, its undefined behaviour, it can result in anything. Worse could happen and thinking about these issues beyond the statement of "undefined behaviour" is rather silly. There can be many such "interesting" stories that one can write up. The key is, use std::auto_ptr<> what it is suited for, use delete/delete[] what they are suited for. Mixing them is really dangerous.

There is a peculiarity to note though. If you do the allocation for the single element using array form of new - you could use delete[] for that single element. That is:

        int * ptr = new int[1];
        delete[] ptr; //OK

Well, not really a peculiarity, but yeah, something to take note of.

Forum reference : What's wrong with std::auto_ptr<int> ptr(new int[100]);?

Custom deleters with smart pointers

There is just one smart pointer as part of the standard C++ library as of now (excluding the tr1 and proposals). That is the std::auto_ptr.

It is crucial to know that you can only use it for allocations made via the new operator. It neither handles, the allocations made via the array form of new or malloc or any other allocation routine provided, for that matter.

How can you extend it to be able to use it with those? Basically, you can't. There are an option though - you lay down your own version of auto_ptrs specific to those which have the delete call replaced by free/delete[]/or the relevant deleter. This will ask for writing down multiple classes for them and using them as appropriate. There is another way to extend it but before that let's look at how boost::shared_ptr handles it.

Boost shared_ptr has a concept for a custom deleter. The deleter would operate on the pointer (when the reference count drops to 0 - remembers the ownership is shared!) and the effect should be that of freeing the resource. The following is an illustration:

[CODE]

template<typename T>
struct mallocDeleter
{
    void operator() (T*& ptr)
    {
        if (ptr)
        {
            free(ptr);
            ptr=NULL;
        }
    }
};
void somefunction()
{
    MyType* ptr = getAllocatedPointer();//returns pointer to memory allocated by malloc
    shared_ptr<MyType, mallocDeleter<MyType> >object(ptr, mallocDeleter());
}

//scope ends shared_ptr's destructor gets called that then called mallocDeleter(ptr) - The default deleter is "delete".

Notice, how this expresses the flexibility. You could have acquired any other resource and would have wrapped it inside a shared_ptr and would provide the deleter to free that resource. For example, a File Handle - on which the custom deleter calls close (for C++ fstream objects, you don't need that - they already exploit RAII). Or, a db connection handle, for which the deleter would handle closing it. Just about anything that is manually allocated! Amazing how the ability to provide the deleter makes this smartness so generic.

I wonder why the auto_ptr or the boost scoped_ptr don't provide this ability. RAII is truely magic and takes care of resource leak issues so well.

Cheers!

Free New, Delete Malloc

On numerous ocassions have I found people making mistakes in usage of the memory allocation routines acceptable to C++. Primarily between new, delete and malloc, free (and not to mention the array forms of new and delete).

A mistake can be very dangerous from a memory leak to undefined behavior that may make your computer dissappear! :D Of course, I am kidding but it is still dangerous.

Take a look at this FAQ entry that I posted on Codeguru regarding the topic - C++ Memory Management : What is the difference between malloc/free and new/delete?

Hope you find it useful.

Take care!

auto_ptr and their usability with STL containers

Aaaahh... Smart pointers..aren't they one of the best things that has happened to C++. They are also sometimes called Managed pointers. I am not going to talk about them right from introduction, so if you are unaware what they are and what are their uses, then try figuring them out through the web-searches or @ codeguru... my fav place as far as C++ is concerned.

Ok..Ok..not ready to search for them...fine.. I will try giving a very brief description. Remember when you use pointers in C++ with dynamic memory management, it's such a headache to get them going and finish them off once you are done? If you did a new or new[], you need to have a corresponding delete or delete[], so that the allocated memory is freed up and the destructor for the new-ed object is called. Also, when you are done with the delete/delete[], freed-up the memory, you need to cautiously set the pointer to NULL, so that you won't try dereferencing it after the de-allocation assuming it points to a valid object. (Recall Dangling pointers. They could be disastrous if dereferenced!)

But just imagine that you could use pointers in the same way you used to do and forget about the headache of the deleting (and optionally NULLing it)? How would that be? Wouldn't that be cool and care-free? ...and that is what is made possible by these managed pointers/smart pointers. They are really smart and they exist in wide varieties.

For now, I would limit my post to just one of that category which is the standard auto_ptr (read auto-pointers). They are a part of C++. They own the dynamically allocated object and then perform the automatic cleanup whenever the object goes out of scope or is not needed. Now, if at some stage you want the manual ownership back they have this function member - release(), make a call to it and all the headache is back yours.

[CODE]
//Example showing a bad situation with naked pointers
void MyFunction()
{
        MyClass* ptr( new MyClass);
        /*.some more code.*/
        delete ptr;
        ptr=NULL;
}

In this scenario, just in case of some exception it's possible that somehow the delete ptr; did not happen...Disasterous..a memory leak! There's more - suppose, in the destructor of MyClass, you call for some resource release like close database connections, releasing file handles or release semaphores, locks etc. All resources are blocked and there's no way out. There could be many more aweful situations.

Now, lets use an auto_ptr and see how it makes life easy:

[CODE]
//Situation above tackled using auto_ptr
void MyFunction()
{
        auto_ptr<MyClass> ptr(new MyClass);
        /*.some more code.*/
}

There is no need to explicitly call a delete; as soon as ptr goes out of scope the destructor of MyClass is called and the object is freed. You should note that auto_ptr only handles allocations with new and not those done with new[] or malloc or any other memory allocation routine for that matter. For those, refer to the following posts - Handling dynamic array allocations, Custom deleters with smart pointers, Free New, Delete Malloc

You can even transfer the ownership of an auto_ptr to another auto_ptr, to another function or free/reset it to use it again.

Here's an example showing the transfer of ownership from one auto_ptr to another.

[CODE]
        //Example showing transfer of ownership.
        auto_ptr<MyClass> autoptr1(new MyClass);
        auto_ptr<MyClass> autoptr2;
        autoptr2 = autoptr1;
        //Now autoptr2 owns the pointer & autoptr1 doesn't
        autoptr1->MyFunction(); // ERROR !!!
        //autoptr1, now is a null pointer

However, one very interesting and important fact about auto_ptr's is that their copies are not equivalent as is the case with naked pointers.

• Consider STL containers or some generic algorithms/functions like sort. There is no restriction by the standard on the vendors of the STL implementers that the containers won't make a copy of the members being added into it (and probably do nothing with it!). In fact, for example, the implementations of the vector class make a copy of the original object and store inside them. We know auto_ptr copies are not equivalent due to the strict ownership semantics. You might be safe in just having those auto_ptrs stored inside a vector but you get a few examples around this in the Gotw link below that show how the problem comes out in the open, for example, with say, a sort algorithm that holds a copy of an element as a pivot? There is nothing to guarantee or prevent that.


• Another disastrous situation would be when you are initializing (setting/assigning) another auto_ptr object with an object held by the vector. You would think that the object inside the vector is okay and retains the same state but it will not. The ownership by this time would get transferred to the external object just initialized/assigned.


• What if there is a member function that returns the contained vectors element is returned by value?


• What if you make another copy of the vector? Or for that matter, create some other container out of its elements?


• Use of member functions of vector like - constructor, assign, insert, resize - will they succeed? Will they definitely succeed?

It just gets too messy to handle. Sure some operations will work out fine but quite a few will not and you would need to keep worrying about 'em.

To help us out of this disaster, the C++ standards did some tweaking. The copy constructor and the copy assignment operator of auto_ptr take a reference to a non-const right hand side objects and the standard containers' insert() takes a reference to a const and hence auto_ptr cannot be used. It will just not allow you to create a copy of auto_ptr inside the vector (standard containers) members. That will cause you compilation errors on a compliant compiler. Nice solution..isnt it? But still, as a strict guideline..we should not use the auto_ptr wherever a copy of theirs is expected to be made.

For more information, visit these : Using auto_ptr effectively by Herb Sutter, Smart Pointers - What, Why and Which, Smart pointers

Cya!