Prev: ml.exe
Next: beginner asm, where next?
From: swishhh on 20 Jan 2007 09:19

japheth wrote:
> > I would not really like to use exception handeling, because it slows
> > the application down. I CAN NOT use a different DLL.
>
> Exception handling in Win32 is not slow, and with ASM it is even
> trivial. Here's an example:
>
> testmem proc pMem:ptr
>
> xor edx, edx
> push offset exception_occured
> push fs:[edx]
> mov fs:[edx], esp
>
> mov eax,1 ;1 means error (IsBadReadPtr() emulation)!
> mov ecx, pMem
> mov dl,[ecx]
> dec eax ;this code runs only if no exception occured
> done:
> xor edx, edx
> pop fs:[edx]
> pop ecx ;adjust stack (offset exception)
> ret
> exception_occured:
> mov eax, [esp+12] ;get context pointer
> mov [eax].CONTEXT.rEip, offset done
> xor eax, eax ;== _XCPT_CONTINUE_EXECUTION
> retn
> testmem endp

Thank you very much for the solutions. I tried the fs:[0] solution and
it works fine. It doesn't slow the application noticable down.
The dll I spoke about calculates in a own thread; and when I call the
function it gives me the pointer to the data, (or sometimes not :-( )

But now it runs.

Thx,

Sasha G.

From: "//o//annabee Free" " on 20 Jan 2007 09:32
> One of major problems with the IsBadXxxPtr() functions is that they can
> cause the access exception for a stack guard page to be eaten if you
> try validating a bad pointer that happens to point to a guard page.
> Then when the stack does grow to the point where it hits the guard
> page, the guard page does not trigger the correct sort of exception and
> instead of growing the stack (the normal result of hitting the guard
> page), the OS gets an access violation and the application abends.

Thanks.
Here is another:
Could any acceess to a yet not committed page, in the stack, or elsewhere,
that is silenced by an exception, possibly lead to the same result?
(An access by a program, but not with a call to IsBadXxxPtr() ?

> To the OP: The IsBadXxxPtr() functions provide, at best, an illusion
> of doing what you want. It is very, very, common that a bad pointer is
> pointing at memory that exists, in which case the IsBad functions will
> do precisely nothing.

> This memory may (and very commonly *does*) exist
> even if it's not a currently malloc()'d piece of memory (or whatever
> your heap management primitive is called), since the heap will
> typically retain memory allocated from the OS for a long time even if
> the area is freed.

> You can't even reliably walk the heap (assuming
> you're using a heap which exposes an API or something that allows you
> to do that), since a bad pointer can easily be pointing into a
> malloc()'d bit of memory, and still be completely wrong.

In one of my memorymangers, I can determine 100% if any pointer is valid.
I can even determine what kind of data it contain.
Exception are for memory allocated outside the mm. But I can determine if
it is inside, or outside.

> In short, validating a pointer based on nothing more than OS page
> allocation and the heap manager is essentially impossible. While you
> may make some protection faults go away, you're probably replacing them
> with some storage corruptions, which is usually a pretty bad trade off.



From: David Jones on 20 Jan 2007 12:39
//\\o//\\annabee <Free" <Wannabee(a)wannabee.org> wrote...
> > One of major problems with the IsBadXxxPtr() functions is that they can
> > cause the access exception for a stack guard page to be eaten if you
> > try validating a bad pointer that happens to point to a guard page.
> > Then when the stack does grow to the point where it hits the guard
> > page, the guard page does not trigger the correct sort of exception and
> > instead of growing the stack (the normal result of hitting the guard
> > page), the OS gets an access violation and the application abends.
>
> Thanks.
> Here is another:
> Could any acceess to a yet not committed page, in the stack, or elsewhere,
> that is silenced by an exception, possibly lead to the same result?
> (An access by a program, but not with a call to IsBadXxxPtr() ?

Now you've got it -- exception handling is good for many things, but as
a way to validate pointers, the cure is worse than the disease. Of
course you are entirely correct that calling a function that does
something has the same consequence as doing the same thing inline
directly.

This is why, generally, you want to use exceptions for truly exceptional
failure modes. Bad pointers, on the other hand, are bugs and should be
treated as such, not wrapped in a handler that swallows all exceptions
and pretends nothing went wrong.


> In one of my memorymangers, I can determine 100% if any pointer is valid.
> I can even determine what kind of data it contain.
> Exception are for memory allocated outside the mm. But I can determine if
> it is inside, or outside.

The two statements "I can determine _100%_ if any pointer is _VALID_"
(emphasis mine) and "I can determine if it is inside or outside" do not
mean the same thing. The fact that memory is allocated by your memory
manager or outside your memory manager has little to do with whether a
pointer points to valid data, unless your memory manager keeps track of
pushes on the stack, for example.

I don't doubt that you can determine the status wrt the mm in your code
(in fact, if you couldn't, it would be a pretty crappy memory manager),
but you should be careful using absolutes like "100%" since it's not
true.

Even the operating system can't determine whether a pointer is truly
valid, as Robert was trying to explain. All that has to happen is an
application writing their own memory manager <grin> where a pointer
could point to OS-allocated memory (the custom-mm heap) but not custom-
mm-allocated memory, thus making the pointer invalid.

Moreover, if the pointer points to valid memory but not of the type
described, is it really a valid pointer? (In this case, the OP was
talking about a pointer to a structure, so what if it pointed to a
string instead? Or a DWORD?)

The problem is a lot harder than you make it out to be. It's a fool's
errand, IMHO.

David
From: "//o//annabee Free" " on 20 Jan 2007 17:37
P� Sat, 20 Jan 2007 18:39:27 +0100, skrev David Jones <ncic(a)tadmas.com>:

> //\\o//\\annabee <Free" <Wannabee(a)wannabee.org> wrote...
>> > One of major problems with the IsBadXxxPtr() functions is that they
>> can
>> > cause the access exception for a stack guard page to be eaten if you
>> > try validating a bad pointer that happens to point to a guard page.
>> > Then when the stack does grow to the point where it hits the guard
>> > page, the guard page does not trigger the correct sort of exception
>> and
>> > instead of growing the stack (the normal result of hitting the guard
>> > page), the OS gets an access violation and the application abends.
>>
>> Thanks.
>> Here is another:
>> Could any acceess to a yet not committed page, in the stack, or
>> elsewhere,
>> that is silenced by an exception, possibly lead to the same result?
>> (An access by a program, but not with a call to IsBadXxxPtr() ?
>
> Now you've got it -- exception handling is good for many things, but as
> a way to validate pointers, the cure is worse than the disease. Of
> course you are entirely correct that calling a function that does
> something has the same consequence as doing the same thing inline
> directly.
>
> This is why, generally, you want to use exceptions for truly exceptional
> failure modes. Bad pointers, on the other hand, are bugs and should be
> treated as such, not wrapped in a handler that swallows all exceptions
> and pretends nothing went wrong.

Ok. Then I guess lots of bugs are brought to life this way.

>
>
>> In one of my memorymangers, I can determine 100% if any pointer is
>> valid.
>> I can even determine what kind of data it contain.
>> Exception are for memory allocated outside the mm. But I can determine
>> if
>> it is inside, or outside.
>
> The two statements "I can determine _100%_ if any pointer is _VALID_"
> (emphasis mine) and "I can determine if it is inside or outside" do not
> mean the same thing.



> The fact that memory is allocated by your memory
> manager or outside your memory manager has little to do with whether a
> pointer points to valid data, unless your memory manager keeps track of
> pushes on the stack, for example.

> I don't doubt that you can determine the status wrt the mm in your code
> (in fact, if you couldn't, it would be a pretty crappy memory manager),
> but you should be careful using absolutes like "100%" since it's not
> true.

What I mean is, that I can determine, by looking at a pointer, if that
pointer points to memory allocated by me, in all cases, and in the case it
is allocated by me, I can determine, with 100% accuracy, if this memory is
now pointing to "valid" memory or not. In the case that the pointer is
"inside" (allocated via mm) I can also determine with 100& accuracy what
type of memory it points to. E.g I can call to a routine to format it for
visual verification. And vs I can determine if
1) It is not in mm
2) it is not in memory allocated by me, outside mm (e.g stanalone
virtalloc, etc)

Thus, I can determine if the pointer is in the WILD.
E.g. a IsBadXxxPtr() by me, would work, 100% and would not cause any
unexpected exceptions. (Unless the mm structures wore allready corrupted).


> Even the operating system can't determine whether a pointer is truly
> valid, as Robert was trying to explain.

:)) The mm of windows, has several servere bugs. (Talking about the mm
accessed via VirtualAlloc) It has no verification system. It is slow
O(n**). It also cannot manage objects above a certain size correctly
(cannot free them) (win2000). Even if only present in win2000, theese bugs
affects code for other plattforms. Another thing is that the virtual
addresspace changes between OS versions (it gets smaller), and so forces
developer to study the bugs and write around them. Also destroys many of
the nice features of having virtual address, and in addition fully
destroys the philosophy of the OS. The addressspace even differ inside
versions. In XP, turing themes on or off, will affect the configuation of
the virtual memoryspace of each application. Because the themes are hacked
into the userspace. What has the OS todo in the userspace of an
application in the first place. It has allready 2 giga of system space.

> All that has to happen is an
> application writing their own memory manager <grin> where a pointer
> could point to OS-allocated memory (the custom-mm heap) but not custom-
> mm-allocated memory, thus making the pointer invalid.

Sorry. I dont comprehend this paragraph.

> Moreover, if the pointer points to valid memory but not of the type
> described, is it really a valid pointer? (In this case, the OP was
> talking about a pointer to a structure, so what if it pointed to a
> string instead? Or a DWORD?)

This is my point. I could determine this. If this poiner was returned from
a DLL i had written, using the mm I am talking about, I would know what it
pointed to. If it was pointing to memory that was recorded in use, or not,
and if in use I could determine what kind of memory, and call to a routine
for printing its content. I could also determine if the pointer was
pointing inside a string, or inside a record, and thus I could subtract
the offset, print the content in a messagebox, with a note saying : this
pointer is pointing inside the middle of a bitmap, bitmapname = "Such and
Such". In the worst cases, I could a least print out a message saying this
pointer is pointing at offset xxxx, inside an untyped binary chunck of
this size. And then I could give an dump of the memory. And if the memory
was uncounted for, I would know this by not finding any referance to it,
in any of my allocations records, and thus I would print "This pointer
points in the wild, or to memory allocated by windows, inside an API, not
recorded by mm".

> The problem is a lot harder than you make it out to be. It's a fool's
> errand, IMHO.

:)) Sure. And why is that. Its because the windows MM is a true disaster.

>
> David

From: David Jones on 23 Jan 2007 12:16
//\\o//\\annabee <Free" <Wannabee(a)wannabee.org> wrote...
> På Sat, 20 Jan 2007 18:39:27 +0100, skrev David Jones <ncic(a)tadmas.com>:
>
> > The fact that memory is allocated by your memory
> > manager or outside your memory manager has little to do with whether a
> > pointer points to valid data, unless your memory manager keeps track of
> > pushes on the stack, for example.
>
> > I don't doubt that you can determine the status wrt the mm in your code
> > (in fact, if you couldn't, it would be a pretty crappy memory manager),
> > but you should be careful using absolutes like "100%" since it's not
> > true.
>
> What I mean is, that I can determine, by looking at a pointer, if that
> pointer points to memory allocated by me, in all cases, and in the case it
> is allocated by me, I can determine, with 100% accuracy, if this memory is
> now pointing to "valid" memory or not. In the case that the pointer is
> "inside" (allocated via mm) I can also determine with 100& accuracy what
> type of memory it points to. E.g I can call to a routine to format it for
> visual verification. And vs I can determine if
> 1) It is not in mm
> 2) it is not in memory allocated by me, outside mm (e.g stanalone
> virtalloc, etc)
>
> Thus, I can determine if the pointer is in the WILD.
> E.g. a IsBadXxxPtr() by me, would work, 100% and would not cause any
> unexpected exceptions. (Unless the mm structures wore allready corrupted)..

Let me restate what I stated earlier, since you obviously missed it:
Yes, your memory manager can determine 100% about memory it allocates;
otherwise, it's a crappy memory manager. Explaining intricate details
of your mm to further illustrate you can determine what goes on inside
your memory manager isn't really making a point with respect to the
validity of pointers.

Let's say I push a DWORD on the stack and then pass esp as a pointer to
a function that you wrote. Is the memory valid? Your function must
determine this. The real answer is yes -- if not, you have no stack, so
you're in big trouble. *Your* test (allocated inside your memory
manager) fails, so you answer no. Ergo, being able to determine the
status with respect to your memory manager does not answer the question
of whether a pointer is truly valid.

It's a red herring.


> > Even the operating system can't determine whether a pointer is truly
> > valid, as Robert was trying to explain.
>
> :)) The mm of windows, has several servere bugs. (Talking about the mm
> accessed via VirtualAlloc) It has no verification system. It is slow
> O(n**). It also cannot manage objects above a certain size correctly
> (cannot free them) (win2000). Even if only present in win2000, theese bugs
> affects code for other plattforms. Another thing is that the virtual
> addresspace changes between OS versions (it gets smaller), and so forces
> developer to study the bugs and write around them. Also destroys many of
> the nice features of having virtual address, and in addition fully
> destroys the philosophy of the OS. The addressspace even differ inside
> versions. In XP, turing themes on or off, will affect the configuation of
> the virtual memoryspace of each application. Because the themes are hacked
> into the userspace. What has the OS todo in the userspace of an
> application in the first place. It has allready 2 giga of system space.

What does any of this have to do with ascertaining pointer validity?
The inability to ascertain pointer validity with 100% correctness has
nothing to do with bugs in the implementation -- it's inherent to the
system itself.

You might think you could do better, but if Windows presented you a
strongly-typed mm with strict control so it _could_ ascertain this,
you'd complain about a lack of flexibility.


> > All that has to happen is an
> > application writing their own memory manager <grin> where a pointer
> > could point to OS-allocated memory (the custom-mm heap) but not custom-
> > mm-allocated memory, thus making the pointer invalid.
>
> Sorry. I dont comprehend this paragraph.

And you wrote your own memory manager...? Where did you get the memory
to manage? The OS gives you a chunk (either through you asking
explicitly for a chunk through a XxxxAlloc-type function or implicitly
by reserving space in your executable), and you partition it as needed.
So what happens when a pointer falls in a block allocated by one mm and
not the other? Who can determine if it's *VALID*? Only the mm that
allocated it, so any mm-centric view is doomed to fail (just switch the
roles).

David
First  |  Prev  |  Next  |  Last
Pages: 1 2 3 4
Prev: ml.exe
Next: beginner asm, where next?