Vista Magnification API: MagSetImageScalingCallback() doesn'twork?
in [Win32 API]
|
From: Grzegorz Wróbel on 15 Jun 2008 07:09 Grzegorz Wr�bel wrote: > Hi, > > I am able to succesfully call MagSetImageScalingCallback() to set new > MagImageScalingCallback function, however my callback function gets > called only 2-3 times and it provides 4x4 pixel source data (with some > garbage instead source pixels). My source rectangle is 256x256 pixels. > Well, calling MagSetWindowSource() *before* MagSetImageScalingCallback() makes my own MagImageScalingCallback function to be called with correct pixel source data, however it still gets called only once. -- Grzegorz Wr�bel 677265676F727940346E6575726F6E732E636F6D
From: Grzegorz Wróbel on 17 Jun 2008 08:45 Ivan Brugiolo [MSFT] wrote: > The MagXXX APIs in Vista are meaningful only when > milcore.dll is being used to produce a magnified view of the desktop, > that may or may not be the case in your specific configuration. > For most of practical usages, the image filtering callback is not > that interesting. > In the "debug version" I do have the Vista magnification control visible on the screen. The successfully registered callback is called only once (or on some machines not at all). Since it was hard to believe that some Vista APIs might be broken, I thought that maybe it is that my graphic drivers that are not 100% WDDM compatible. But I asked few persons with different hardware configuration to run my code and they confirmed it that the registered magnification callback is called only once (or not at all). The most interesting fact is that this happens only when the DWM is on, when it's off then magnification callback works! Only that I don't need it to work with DWM off (especially that other magnification functions like MagSetWindowFilterList() will not work with DWM off), I need it to work with DWM on. So if something doesn't work in current Vista release maybe it would be worth to mention it in the documentation. ... Well the word "documentation" sounds too proud. If you look at MSDN entry about MagImageScalingCallback function: http://msdn.microsoft.com/en-us/library/ms692392(VS.85).aspx A child after few C++ classes will tell you a function declared like that has no right to compile. Even if you fix obvious typos it is still inconsistent with what I find in "magnification.h" from Windows Vista SDK. So all I have is the function declaration from magnification.h But that's apparently the declaration of a function that doesn't work. -- Grzegorz Wr�bel 677265676F727940346E6575726F6E732E636F6D
From: Grzegorz Wróbel on 19 Jun 2008 17:25 Ivan Brugiolo [MSFT] wrote: > With desktop composition ON, window filtering will work, > but not image filtering callback. > With desktop composion OFF, window filtering will NOT work, > and image filtering will work. > > `window-filtering` can work with composition-OFF using > private APIs that relay on layered windows, and only on layered windows. > This limitation cannot be fixed, and, will stay there. > Lack of window filtering with DWM composition off is not that much of a problem, since with DWM composition off the display is working the same as on all pre-Vista Windows systems (where there was no Magnification API). And for now I hardly imagine any developers releasing software that has Windows Vista as minimum OS requirement. Not working filtering callback while DWM composition is on is a problem. For me it meant the decision of dropping support for Windows Vista (the code that works on pre-DWM displays doesn't work with DWM on). > > Most of these limitations will be addressed in the next version of the > operative system. > Does it mean Windows Vista SP2 or Windows NT 7.0? -- Grzegorz Wr�bel 677265676F727940346E6575726F6E732E636F6D
From: Grzegorz Wróbel on 21 Jun 2008 11:14 Ivan Brugiolo [MSFT] wrote: > What is the scenario why the Image Filtering Callback is interesting ? > The most trivial scenario for which it is interesting is the one described in Remarks section for MagSetImageScalingCallback Function. Ie ability to provide more advanced filtering algorithms than what offers the default MagSetWindowTransform(), which is just the simplest nearest neighbour algorithm. This remarks section is just a science fiction for DWM enabled desktops as long as the callback doesn't work. But there is second use - the working magnification callback seems to be the only way to get to the desktop bits filtered out of the given window(s) with DWM composition turned on. And by stating "get to the bits" I mean being able to read them programmatically. Because, although you can use magnification control to filter out any windows, you *can't* get to its bits, it's a black-box control. The working callback for external filtering and scaling would be the only way to get them (they come in the srcdata buffer). You notice that this problem of getting desktop bits obscured by given window is unsolved in any earlier Windows editions, and you could find many posts in these newsgroups asking about that over the years with no really good answer. However on pre-DWM displays it was possible to filter out WS_EX_LAYERED windows. On Vista with DWM composition it is not possible anymore. Actually we had a conversation about this few months ago in different set of newsgroups. And you suggested Magnification API for that in general and magnification image scaling callback in particular. Now when it turns out the callback doesn't work you are saying that "image filtering callback is not that interesting". Which honestly I'm not sure how to interpret. Does it mean that there is a new generation of MS programmers who implement only this part of specification that they find "interesting"? The documentation for both MagSetImageScalingCallback() and MagImageScalingCallback() is there, there is no mention they don't work with WDDM, on the contrary. At which point they stopped being interesting? Are there any performance issues with having such callback working under WDDM? The documentation says that these functions should work with WDDM (and not work with XDM), which is completely opposite to reality. "This function requires Windows Display Driver Model (WDDM)-capable video cards." from http://msdn.microsoft.com/en-us/library/ms692389(VS.85).aspx "This function requires LDDM capable video cards. It returns FALSE on XPDM." from http://msdn.microsoft.com/en-us/library/aa468587.aspx Is there a bug in documentation or in implementation? Normally I would always assume the former, but in case where the new API is added to the old, dying technology and not to the next-generation one I can't help but thinking that someone mistakenly implemented the API for the wrong driver model. -- Grzegorz Wr�bel 677265676F727940346E6575726F6E732E636F6D
From: Grzegorz Wróbel on 23 Jun 2008 14:16
That was my impression that performance issues were the reason of "dropping" the idea of magnification callback somewhere on the way. Just by seeing how having magnification control affects overall performance or the mere fact that creating magnification control restarts entire DWM. The point in which magnification callback became so interesting for us was the moment when we realized that (according to documentation) this callback was the only way (on Vista with DWM on) to achieve what we wanted. I agree that PrintWindow() would be simpler solution (and that was what we tried), but there is nothing in documentation that would tell us PrintWindow() should work with magnification control. You notice that even with magnification callback working it probably wouldn't be possible to use magnification control directly, due to some of its other limitations. But once I am able to get access the bits that wouldn't be a problem anymore, I could process them and display in my own control/window. I would only have to send the original magnification control out of the desktop or make it invisible in any other way that wouldn't stop callback from working. (Hoping that callback won't clip according to the visibility of magnification control itself, only according to the changes in source rectangle.) The main theoretical advantage of using the callback comparing to the PrintWindow() is that it was supposed to provide that "clipping magic" telling which part of underlaying screen has changed ie which part exactly you need to update, which would be critical for performance reasons. But even without this magic I can do it. It comes with a price, it means necessity of performing sophisticated filtering algorithm on entire client area, which among other things forces to put additional constraints on the application, like limiting maximal window size, limiting maximal refreshing frequency, etc. not to eat too much CPU power. But what does matter is I can do it and leave it to the end user to decide whether the result is worth the cost (which will vary depending on the user's hardware, personal preferences and so on). On Windows Vista with DWM on I can't do it. I can only run it in "compatibility mode", by disabling DWM. But such application is not attractive to the Vista users, disabling DWM without prior notice that explains what will happen and why is going to result in the users' feedback like "There is a bug in your program that disables aero on my computer whenever I run it", etc. And I can hardly say that the application "supports" Windows Vista. Anyway, thanks for detailed explanation. My understanding is that the next version of Windows OS won't support any other display driver model than WDDM+DWM, so this problem will have to be addressed there and the callback will be working without significant performance impact on entire system. Ivan Brugiolo [MSFT] wrote: > First of all, let me clarify the context where the image filtering > callback was thought to be interesting, and, why that very same > rationale made that un-interesting. > The magnification APIs were created and exposed to allow > simple magnification application (like the Intellimouse Magnifier, > or the in-box magnifier) to provide a magnified view of the desktop > in Vista with desktop composition enabled. > All of the ISV developing solution for people with disabilities > (mild visual impairment, blinds who needs screen readers, etc) > have always used Mirror Drivers, or DDI filtering techniques > in Win32k.sys to get the raw bits, and, to imlement image filtering > and other sophisticated image enahancing post-filtering, > OCR of text, etc, etc. > Since with Vista the use of Mirror Drivers do disable Desktop compositon, > the image filtering callback was thought as a cheap replacement for > the need of installing a mirror driver, but, at the end of the day, > the image filtering callback was of no practical use, because it lacked > the other correlation and sepration that were availalble with WNDOBJs > in the mirror drivers (together with the per DDI filtering). > > That said, let's discuss the magnification.dll limitations of Vista. > The basic magnification functionality without desktop compositon > is really a GDI-based stretch-blt with the source set to be the HDC of the > screen (or of the desktop). > In Vista, if you do not have WDDM, that is pretty much what you will get. > With WDDM, and without desktop compositon, you can have some > image enhancement, such as the ability to apply an arbitrary > color-transform > using the DX9 hardware. And, since the mechanism used to produce > the source image for the transformation is in this scenario is always > GDI based, > the path to screen can easily be extended to implement an image > filtering callback. > GetDC() -> BitBlt() -> Invoke-Image-Callback -> > UpdateTexture -> rasterize with d3d9. > With WDDM and with desktop compositon active, the path to screen is > different. The process hosting Magnification.dll has a worker thread > (called the compositor thread) that receives a stream of compostion > instructions > and, produces a scene that is more-or-less faithful replica of the > screen visual scene. > At this point, in order to implement a maningful image filtering callback, > you would need to render to a back-buffer, read the back-buffer to system > memory, invoke the image filtering callback, push the bits to video-memory > and then flip the content to the screen (I will not digress on how d3d9 > apps > get their windowed content in Vista, since it would be a very long story). > > I'm not sure if the full explanation is going to help solve your problem, > but, if you feel that the image filtering callback is a product deficiency, > feel free to open an incident case with CSS/PSS of your country. > > The problem of getting the desktop bits without any window in front of it > may or may not have a meaningful solution. > For example, assuming you have image-filtering and window-exclusion. > Would you define the desktop as the Explorer List-View > that hosts the backgroung image ? > Would you define the desktop as the bottom-most window and all > of the Sidebar gadgets ? > In the first hypotesis, what would prevent calling PrintWindow on the > explorer list-view ? > > >> However on pre-DWM displays it was possible to filter out >> WS_EX_LAYERED windows. On Vista with DWM composition it is not >> possible anymore. > > With desktop compostion ON, > all of the top level window are forcefully layered > (we call this step `redirection`). > It would certainly be possible to implement a compatile behavior where > only Layered-for-DWM window are captured, and the other are filtered > unless the CAPTUREBITS flag is specified to bitblt (and in that case, > even in WinXp, you would have gotten the layered windows as well). > > I'm still interested to the scenario: > With Desktop Compostion ON, you should have pixel-perfect window-frames, > glass and blur and shadows, and you should have linear filtering for the > GDI content > inside the windows. Maybe, if you had the image-filtering-callback, > you could do bilinear or bicubic on the GDI content, but, that would > require > some non trivial clipping magic to know which portion is already magnified, > which one is not magnified, and, which part are going to affect the blur > of the glass > in the sorrounding pixels. > > The other scenario that you seem to have exposed, is the simultaneous use > of 1x magnification factor and window-filtering in order to get content > for obscrured windows. While the idea is indeed intersting, > the amount of resources consumed by the Vista magnifier for that scenario > would not be worth the while, and, the PrintWindow API (or any API > that references the window DC) would be a chaper solution, because you > would force Dwm.exe to run a render-pass on an off-screen texture for > the window > (to get both the GDI content and the frame done rught) and return that > content > in a DibSection. Still not perfect, but, you would not need to duplicate > the entire visual tree in the magnification.dll host process. > -- Grzegorz Wr�bel 677265676F727940346E6575726F6E732E636F6D |