input: mt: Document the MT event slot protocol (rev2)
in [Kernel]
|
Prev: [GIT PULL v2 0/5] perf tools fixes and improvements
Next: [RESEND -v2][PATCH 3/3] mem-hotplug: fix potential race while building zonelist for new populated zone
From: Rafi Rubin on 19 May 2010 21:10 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 05/19/10 20:51, Ping Cheng wrote: > On Wed, May 19, 2010 at 5:26 PM, Rafi Rubin <rafi(a)seas.upenn.edu> wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> On 05/19/10 20:13, Ping Cheng wrote: >>> On Wed, May 19, 2010 at 4:34 PM, Rafi Rubin <rafi(a)seas.upenn.edu> wrote: >>>> My understanding is that it would be more like >>>> + SYN_MT_SLOT 0 >>>> + ABS_MT_POSITION_X x >>>> + ABS_MT_POSITION_Y y >>>> + SYN_REPORT >>>> + SYN_MT_SLOT 0 >>>> + ABS_MT_POSITION_X x >>>> + ABS_MT_POSITION_Y y >>>> + SYN_REPORT >>>> + SYN_MT_SLOT 0 >>>> + ABS_MT_POSITION_X x >>>> + ABS_MT_POSITION_Y y >>>> + SYN_MT_SLOT 1 >>>> + ABS_MT_POSITION_X x >>>> + ABS_MT_POSITION_Y y >>>> + SYN_REPORT >>> >>> You are right if one slot only has or is only allowed to have one >>> point. My scenario is that one slot can have more than one point. >>> Basically, my intention is to utilize the MT_SLOT and MT_TRACKING_ID >>> in such a way that it avoids as much overlap as possible. >>> >>> And hopefully it makes sesne in the reality too. >> >> Please clarify by what you mean by more than one point. > > I might have been confused myself by ABS_MT_BLOB_ID and SYN_MT_SLOT > here. What I meant by "more than one point" is a contact (or touch, I > am not sure which one is the right term :) is represented by a few > (x,y) coordinates. Maybe we should use SYN_MT_SLOT for my case? > >> I may be misunderstanding, but I thought that these slots are basically a >> superior replacement to tracking id. >> >> one finger -> one slot > > This is what I needed to understand. Is slot for one (x,y) only or can > it also be used for more than one set of (x,y)? > >> But with slots we can use the filtering that input provides, which we've been >> by-passing with the existing MT protocol (at least that's what I think Henrik's >> goal is). > > Good to know that filtering has already been considered. I know I must > be out of sync with Henrik's goal. That's why I wanted to show my > ignorance :). > > Ping > Hey, sometimes Mark Twain's old advice is good, and sometimes its not. It's better to keep your mouth shut and be thought a fool than to open it and leave no doubt. --Mark Twain I think this is definitely a case where he's wrong. We need to sync up so that we're all implementing the same protocol, and can move on to the next interesting problem. Rafi -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkv0ik4ACgkQwuRiAT9o608XmQCdFiTOymC+OEVyQ+atbEdpCiDd RRYAoIsNpOZpI0Yxr4BG1uEj7Fja2Fyo =lK01 -----END PGP SIGNATURE----- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Ping Cheng on 20 May 2010 00:20 On Wed, May 19, 2010 at 6:03 PM, Rafi Rubin <rafi(a)seas.upenn.edu> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 05/19/10 20:51, Ping Cheng wrote: >> On Wed, May 19, 2010 at 5:26 PM, Rafi Rubin <rafi(a)seas.upenn.edu> wrote: >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA1 >>> >>> On 05/19/10 20:13, Ping Cheng wrote: >>>> On Wed, May 19, 2010 at 4:34 PM, Rafi Rubin <rafi(a)seas.upenn.edu> wrote: >>>>> My understanding is that it would be more like >>>>> + � SYN_MT_SLOT 0 >>>>> + � ABS_MT_POSITION_X x >>>>> + � ABS_MT_POSITION_Y y >>>>> + � SYN_REPORT >>>>> + � SYN_MT_SLOT 0 >>>>> + � ABS_MT_POSITION_X x >>>>> + � ABS_MT_POSITION_Y y >>>>> + � SYN_REPORT >>>>> + � SYN_MT_SLOT 0 >>>>> + � ABS_MT_POSITION_X x >>>>> + � ABS_MT_POSITION_Y y >>>>> + � SYN_MT_SLOT 1 >>>>> + � ABS_MT_POSITION_X x >>>>> + � ABS_MT_POSITION_Y y >>>>> + � SYN_REPORT >>>> >>>> You are right if one slot only has or is only allowed to have one >>>> point. My scenario is that one slot can have more than one point. >>>> Basically, my intention is to utilize the MT_SLOT and MT_TRACKING_ID >>>> in such a way that it avoids as much overlap as possible. >>>> >>>> And hopefully it makes sesne in the reality too. >>> >>> Please clarify by what you mean by more than one point. >> >> I might have been confused myself by ABS_MT_BLOB_ID and SYN_MT_SLOT >> here. �What I meant by "more than one point" is a contact (or touch, I >> am not sure which one is the right term :) is represented by a few >> (x,y) coordinates. Maybe we should use SYN_MT_SLOT for my case? The last sentence above should be "Maybe we should NOT use SYN_MT_SLOT for my case?" or "Maybe I should use MT_BLOB_ID for that case?". A typical typo for those whose hands are slower than their brains :). >>> I may be misunderstanding, but I thought that these slots are basically a >>> superior replacement to tracking id. >>> >>> one finger -> one slot >> >> This is what I needed to understand. Is slot for one (x,y) only or can >> it also be used for more than one set of (x,y)? >> >>> But with slots we can use the filtering that input provides, which we've been >>> by-passing with the existing MT protocol (at least that's what I think Henrik's >>> goal is). >> >> Good to know that filtering has already been considered. I know I must >> be out of sync with Henrik's goal. �That's why I wanted to show my >> ignorance :). >> >> Ping >> > > Hey, sometimes Mark Twain's old advice is good, and sometimes its not. > > It's better to keep your mouth shut and be thought a fool than to open it and > leave no doubt. --Mark Twain > > > I think this is definitely a case where he's wrong. �We need to sync up so that > we're all implementing the same protocol, and can move on to the next > interesting problem. I hope this is the common goal for everyone who has been involved in the _MT_ support. Ping -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Henrik Rydberg on 20 May 2010 03:10 Ping Cheng wrote: > Hi Henrik, > > I am trying to link the protocol to the actual multi-touch devices in > my "mind". Hope it helps you to point out the mismatch between my > imagination and the protocol. Please see details in line. > > Ping Hi Ping, first out, thank you for your detailed analysis, it aids in removing ambiguities and defining the borders of the protocol. > Am I right in thinking that SYN_MT_SLOT represents to the actual touch > area/finger on the surface? There could be more than one (x,y) (a few > points that form an irregular shape) that represents one finger. The > following example shows that slot 0 (finger 1) touched three points on > the surface while slot 1 (finger 2) only has one point reported: > > + SYN_MT_SLOT 0 > + ABS_MT_TRACKING_ID 45 > + ABS_MT_POSITION_X x[0] > + ABS_MT_POSITION_Y y[0] > + ABS_MT_TRACKING_ID 46 > + ABS_MT_POSITION_X x[1] > + ABS_MT_POSITION_Y y[1] > + ABS_MT_TRACKING_ID 47 > + ABS_MT_POSITION_X x[2] > + ABS_MT_POSITION_Y y[2] > + SYN_MT_SLOT 1 > + ABS_MT_TRACKING_ID 30 > + ABS_MT_POSITION_X x[3] > + ABS_MT_POSITION_Y y[3] > + SYN_REPORT > It helps to think of both TRACKING_ID and BLOB_ID as labels of a single identified contact which occupies one slot. To represent a set of contacts as an entity, one needs to add a label to the slot, representing that entity. As pointed out in a later reply by Peter, the BLOB_ID serves this purpose well. The name is slightly unfortunate, being a bit too generic. Let us use this discussion to pin down a more exact definition: ABS_MT_BLOB_ID is a label which groups contacts in close relation to each other, such as a hand. With this in mind, the sequence becomes SYN_MT_SLOT 0 ABS_MT_BLOB_ID 11 ABS_MT_TRACKING_ID 45 ABS_MT_POSITION_X x[0] ABS_MT_POSITION_Y y[0] SYN_MT_SLOT 1 ABS_MT_BLOB_ID 11 ABS_MT_TRACKING_ID 46 ABS_MT_POSITION_X x[1] ABS_MT_POSITION_Y y[1] SYN_MT_SLOT 2 ABS_MT_BLOB_ID 11 ABS_MT_TRACKING_ID 47 ABS_MT_POSITION_X x[2] ABS_MT_POSITION_Y y[2] SYN_MT_SLOT 3 ABS_MT_BLOB_ID 89 ABS_MT_TRACKING_ID 30 ABS_MT_POSITION_X x[3] ABS_MT_POSITION_Y y[3] SYN_REPORT Here, we are looking at one blob (11) consisting of three contacts (45, 46, 47), and another blob (89) consisting of one contact (30). [...] > So, an EVIO for X driver to retrieve the number of SLOTs would be very > helpful. Something like the following would do the work: > > input_set_abs_params(input_dev, ABS_MT_SLOT, 0, 12, 0, 0); > > which tells the user land clients that they can expect up to 13 touch areas. The SYN_MT_SLOT is a synchronization control event (EV_SYN), so it would require a different way to report value ranges, but the idea is sound. I will think about how to achieve this. >> +The main difference between the raw type A protocol and the higher level >> +type B slot protocol lies in the usage of identifiable contacts. The slot >> +protocol requires the use of the ABS_MT_TRACKING_ID, > > With what I said above, I think ABS_MT_TRACKING_ID is not the unique > identifier for type B protocol. It is the fact that we can identify > individual touch areas and use ABS_MT_SLOT to report them that makes > it a type B event. This is correct, but the TRACKING_ID is strong evidence that the device is capable of identifying contacts. >> ABS_MT_TRACKING_ID, either provided by the >> +hardware of computed from the raw data [5]. > ^^ or (is it?) > > I agree with this ABS_MT_TRACKING_ID definition. I would think something like: > > input_set_abs_params(input_dev, ABS_MT_TRACKING_ID, 0, 47, 0, 0); > > which tells the clients that total of 48 points are tracked, would be helpful. Agreed. > Another topic that may be irrelevant to this patch is the filter. With > the use of ABS_MT_TRACKING_ID, a filter can be applied to discard the > useless repeated points or less than a certain number of points > movement. As pointed out by Rafi in a later post, this is indeed one of the major points of the slot protocol. The filtering details can be found in the patch accompanying this documentation. Cheers, Henrik -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Dmitry Torokhov on 20 May 2010 03:20 On Thu, May 20, 2010 at 10:13:24AM +1000, Peter Hutterer wrote: > On Wed, May 19, 2010 at 02:12:14PM +0200, Henrik Rydberg wrote: > > Peter Hutterer wrote: > > > On Tue, May 18, 2010 at 10:10:29PM +0200, Henrik Rydberg wrote: > > >> This patch adds documentation for the SYN_MT_SLOT event and gives > > >> examples of how to use the event slot protocol. > > > > > > thanks, this is really nice documentation! the approach seems good, though I > > > do have a few questions inline. > > > > > [...] > > > > > > Is there a limit on the number of slots? > > > > The slots are dynamically allocated by the driver, so there is no practical > > limit. Each slot currently takes 44 bytes, and allocating a few kilobytes of > > kernel memory is not a problem. > > > > > Will all drivers with ABS_MT_TRACKING_ID use slots? if not, how can I know > > > in advance if a device may use slots or not? > > > > Eventually, this might become true, but you are pointing at one of the weaker > > points of the current setup. There is no bit field for the EV_SYN events, so > > there is no way to know in advance if SYN_MT_SYNC or SYN_MT_SLOT is used. This > > could quite possibly be added to the EVIO interface. Meanwhile, the method I use > > is to detect the first SYN_MT_SLOT and select parser based on that information. > > I'd really prefer if there was some way to detect this. While I'm not quite > sure how the matching X drivers would look like it's likely that the setups > will be different for drivers that support slots and those that don't. > e.g. those that don't have slots simply send events as valuators, those that > do may split into multiple devices. > > Doing this at runtime - after the device has been set up is...tricky. > Right, I think we should try our hardest to allow userspace base protocol decoding decisions on device capabilities and not the event datastream. What about moving from SYN_MT_SLOT to ABS_MT_SLOT? I think it makes more sense since it is not a simple marker but carries data (slot number). -- Dmitry -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Henrik Rydberg on 20 May 2010 06:50
Peter Hutterer wrote: [...] >>> Will all drivers with ABS_MT_TRACKING_ID use slots? if not, how can I know >>> in advance if a device may use slots or not? >> Eventually, this might become true, but you are pointing at one of the weaker >> points of the current setup. There is no bit field for the EV_SYN events, so >> there is no way to know in advance if SYN_MT_SYNC or SYN_MT_SLOT is used. This >> could quite possibly be added to the EVIO interface. Meanwhile, the method I use >> is to detect the first SYN_MT_SLOT and select parser based on that information. > > I'd really prefer if there was some way to detect this. While I'm not quite > sure how the matching X drivers would look like it's likely that the setups > will be different for drivers that support slots and those that don't. > e.g. those that don't have slots simply send events as valuators, those that > do may split into multiple devices. > > Doing this at runtime - after the device has been set up is...tricky. Changing SYN_MT_SLOT to ABS_MT_SLOT will take care of this. >> If you are thinking of a setup where one program is already hooked up to the >> device, and a new one comes in just as the message in question appears on the >> wire, it just means the new program will have to spend some time catching up. If >> this should ever become a problem, one could possibly add a send-full-state >> method to the input layer, to be issued when the new program opens the device. I >> doubt this will be needed in practice though, since contacts change all the time. > > This was the case I was wondering about. While I agree with the last part > (contacts change all the time) the side-effect that you'll get from this is > that devices can seemingly be non-responsive for random periods of time. > > If you have a finger down when the X server starts or after a VT switch (or > even a fast user switch), the driver will have to drop events. So you can > wriggle your finger around and nothing happens, but once you lift it goes > "well, now it works again". Since this happens only once in a while, it can > make the whole lot appear flaky. Especially if one finger works and the > other one doesn't. > > So I guess it comes down to whether sending SYN_MT_SLOT with every event > costs too much compared to these admittedly rare use-cases. They're not much > different to the current button events either, if you weren't there when a > button was pressed you won't know. So I'm somewhat indifferent about this > bit. Yes, it is a inherent property of the input protocol between the kernel and user space. A different problem, in other words. > And yes, you could add it once we find it's an issue, but by then someone > has already spent time to work around this. And when you then start sending > slot events all the time, you admit that writing the workaround was just a > time waster :) Work around what, exactly? >> As a side note, the notion of a used slot depends on how the attributes of the >> slot are interpreted. The method described in the document treats an >> ABS_MT_TRACKING_ID value outside of the driver-specified value range as an >> unused slot. > > It's easier for a latecomer to guess a tracking ID since you can just > assign any random value to it, provided the kernel guarantees to only send > updates on the tracking ID for changes. This doesn't quite work for slots. > > But I'm not sure what your last sentence means. Does this mean the kernel > will open a new slot for out-of-range tracking IDs? I'm missing something > here. I am referring to the notion of create-move-destroy from earlier discussions, and the question of how it is implemented using slots. The document is a bit unclear on this point, so I will add a note stating something like this: * Every slot with a tracking id within the value range represents a contact. * Tracking ids not previously present are considered new. * Tracking ids no longer present are considered removed. Cheers, Henrik -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |