From: Roy Smith on
We're looking for places in our code where memory is mis-used. One
tool we've used is a custom malloc library which writes DEADBEEF or
whatever into newly allocated and freed memory. I'm wondering if
there are similar things you can do with the stack.

I'm envisioning something that whenever a function returns, as the
stack is popped, the memory which is newly uncovered by the stack
pointer automatically gets zeroed out (or filled with DEADBEEF). I
don't see any way to do this is user code, so presumably it would need
to be a debugging option supported by the compiler. Does such a thing
exist?

We're currently using Visual Studio 2008, g++ 3.2.3, and SunStudio 12.

--
[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

From: Lailoken on
On Jun 17, 11:57 am, Roy Smith <r...(a)panix.com> wrote:
> We're looking for places in our code where memory is mis-used. One
> tool we've used is a custom malloc library which writes DEADBEEF or
> whatever into newly allocated and freed memory. I'm wondering if
> there are similar things you can do with the stack.
>
> I'm envisioning something that whenever a function returns, as the
> stack is popped, the memory which is newly uncovered by the stack
> pointer automatically gets zeroed out (or filled with DEADBEEF). I
> don't see any way to do this is user code, so presumably it would need
> to be a debugging option supported by the compiler. Does such a thing
> exist?
>
> We're currently using Visual Studio 2008, g++ 3.2.3, and SunStudio 12.

As far as I know Microsoft already does some of this, even for stack
if you can believe some of the comments here:

http://en.wikipedia.org/wiki/Magic_number_(programming)#Magic_debug_values

"CCCCCCCC Used by Microsoft's C++ debugging runtime library to mark
uninitialised stack memory"

You can also read up more here:
http://msdn.microsoft.com/en-us/library/aa260966(VS.60).aspx#debugging_failure

and here:
http://www.microsoft.com/msj/0298/hood0298.aspx

Hope this is useful to you.
Marius.


--
[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

From: Martin B. on
On 17.06.2010 20:57, Roy Smith wrote:
> We're looking for places in our code where memory is mis-used. One
> tool we've used is a custom malloc library which writes DEADBEEF or
> whatever into newly allocated and freed memory. I'm wondering if
> there are similar things you can do with the stack.
>
> I'm envisioning something that whenever a function returns, as the
> stack is popped, the memory which is newly uncovered by the stack
> pointer automatically gets zeroed out (or filled with DEADBEEF). I
> don't see any way to do this is user code, so presumably it would need
> to be a debugging option supported by the compiler. Does such a thing
> exist?
>
> We're currently using Visual Studio 2008, g++ 3.2.3, and SunStudio 12.
>

Have a look at the /RTC switch of Visual C++
MSDN:
/RTCs - Enables stack frame run-time error checking
* Initialization of local variables to a nonzero value.
* Detection of overruns .. of local variables such as arrays.
* Stack pointer verification, which detects stack pointer corruption.


is that what you're looking for?

br,
Martin

--
[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

From: Mathias Gaunard on
On 17 juin, 19:57, Roy Smith <r...(a)panix.com> wrote:
> We're looking for places in our code where memory is mis-used. One
> tool we've used is a custom malloc library which writes DEADBEEF or
> whatever into newly allocated and freed memory. I'm wondering if
> there are similar things you can do with the stack.

See valgrind, purify, or insure++.
The latter is probably the only one that can give you useful info
about stack buffer overflows.


--
[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

From: Piyush on
On Jun 18, 6:13 am, Mathias Gaunard <loufo...(a)gmail.com> wrote:
> On 17 juin, 19:57, Roy Smith <r...(a)panix.com> wrote:
>
> > We're looking for places in our code where memory is mis-used. One
> > tool we've used is a custom malloc library which writes DEADBEEF or
> > whatever into newly allocated and freed memory. I'm wondering if
> > there are similar things you can do with the stack.
>
> See valgrind, purify, or insure++.
> The latter is probably the only one that can give you useful info
> about stack buffer overflows.

{ quoted banner removed; please do it yourself. really. -mod }

Sun studio 12 can also be enabled to check for the stack overflow/
corruption. check xcheck=%all

this will enable the run time checking for solaris\linux build. you
should also check GS flags on VS 2008 if you are using the make file
based build setup.

--Piyush


--
[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]