xstat: Add a pair of system calls to make extended file stats available [ver #6]
in [Kernel]
|
Prev: perf annotate segfaults when source code has goto label that looks like hex number
Next: [PATCH 4/5] staging: wlags49_h2: remove custom hex_to_bin() method
From: Jeff Layton on 6 Aug 2010 07:20 On Thu, 5 Aug 2010 22:55:06 -0500 Steve French <smfrench(a)gmail.com> wrote: > On Thu, Aug 5, 2010 at 10:38 PM, Neil Brown <neilb(a)suse.de> wrote: > > On Thu, 5 Aug 2010 16:52:18 -0700 > > Jeremy Allison <jra(a)samba.org> wrote: > > >> Don't add it as an EA. It's *not* an EA, it's a timestamp. > > > > I'm curious. �Why do you particularly care what interface the kernel uses to > > provide you with access to this attribute? > > > > And given that it is an attribute that is not part of 'POSIX' or "UNIX", it > > would seem to be an extension - an extended attribute. > > As the Linux kernel does virtually nothing with this attribute except provide > > access, it seems to be a very different class of thing to other timestamps. > > Surely it is simply some storage associated with a file which is capable of > > storing a timestamp, which can be set or retrieved by an application, and > > which happens to be initialised to the current time when a file is created. > > > > Yes, to you it is a timestamp. �But to Linux it is a few bytes of > > user-settable metadata. �Sounds like an EA to me. > > > > Or do you really want something like BSD's 'btime' which as I understand it > > cannot be set. �Would that be really useful to you? > > Obviously the cifs and SMB2 protocols which Samba server support can > ask the server to set the create time of a file (this is handled > through xattrs today along with the "dos attribute" flags such as > archive/hidden/system), but certainly it is much more common (and > important) to read the creation time of an existing file. > > > > Is there something important that I am missing? > > It is another syscall that Samba server would have to make - and xattr > performance is extremely slow on some file systems (although > presumably this one would be more likely to be stored in inode and > perhaps not as bad on ext4, cifs and a few others such as ntfs). > > Right. One has to consider that samba has to satisfy READDIRPLUS-like calls, and on a large directory all of those extra syscalls are likely to impact performance. In my view, the ideal thing would be to add this field as an EA and continue work on implementing xstat(). Adding it as an EA gives userland a way to set this value, without needing to add a new utimes() variant. If/when xstat becomes available, samba could use that instead of the EA for reading this value. -- Jeff Layton <jlayton(a)redhat.com> -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Neil Brown on 6 Aug 2010 19:40 On Thu, 5 Aug 2010 22:55:06 -0500 Steve French <smfrench(a)gmail.com> wrote: > On Thu, Aug 5, 2010 at 10:38 PM, Neil Brown <neilb(a)suse.de> wrote: > > On Thu, 5 Aug 2010 16:52:18 -0700 > > Jeremy Allison <jra(a)samba.org> wrote: > > >> Don't add it as an EA. It's *not* an EA, it's a timestamp. > > > > I'm curious. Why do you particularly care what interface the kernel uses to > > provide you with access to this attribute? > > > > And given that it is an attribute that is not part of 'POSIX' or "UNIX", it > > would seem to be an extension - an extended attribute. > > As the Linux kernel does virtually nothing with this attribute except provide > > access, it seems to be a very different class of thing to other timestamps. > > Surely it is simply some storage associated with a file which is capable of > > storing a timestamp, which can be set or retrieved by an application, and > > which happens to be initialised to the current time when a file is created. > > > > Yes, to you it is a timestamp. But to Linux it is a few bytes of > > user-settable metadata. Sounds like an EA to me. > > > > Or do you really want something like BSD's 'btime' which as I understand it > > cannot be set. Would that be really useful to you? > > Obviously the cifs and SMB2 protocols which Samba server support can > ask the server to set the create time of a file (this is handled > through xattrs today along with the "dos attribute" flags such as > archive/hidden/system), but certainly it is much more common (and > important) to read the creation time of an existing file. > Just a point of clarification - when you say it is common and important to be able to read the creation time on an existing file, and you still talking in the context of cifs/smb windows compatibility, or are you talking in the broader context? If you are referring to a broader context could be please give more details because I have not heard any mention of any real value of creation-time out side of window interoperability - have such a use clearly documented would assist the conversation I think. If on the other hand you are just referring the the windows interoperability context ... given that you have to read an EA if the create-time has been changed, you will always have to read and EA so having something else is pointless ... or I'm missing something. > > > Is there something important that I am missing? > > It is another syscall that Samba server would have to make - and xattr > performance is extremely slow on some file systems (although > presumably this one would be more likely to be stored in inode and > perhaps not as bad on ext4, cifs and a few others such as ntfs). > > Obviously if we were to make xattrs the preferred way to get create time out of the filesystem we would want to make sure it is efficient. It would seem to make perfect sense to add a 'getxattrat' syscall and allow an AT_NONBLOCK flag (which would probably be useful for statat too). The AT_NONBLOCK flag would only get attributes if they were available immediately without going to storage/network/whatever. And if it is simply a case of too many syscalls per file, then getxattrat_multi would seem to be the most general way to go. NeilBrown -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Neil Brown on 6 Aug 2010 20:30 On Fri, 6 Aug 2010 18:58:42 -0500 Steve French <smfrench(a)gmail.com> wrote: > On Fri, Aug 6, 2010 at 6:30 PM, Neil Brown <neilb(a)suse.de> wrote: > > On Thu, 5 Aug 2010 22:55:06 -0500 > > Steve French <smfrench(a)gmail.com> wrote: > > > >> On Thu, Aug 5, 2010 at 10:38 PM, Neil Brown <neilb(a)suse.de> wrote: > >> > On Thu, 5 Aug 2010 16:52:18 -0700 > >> > Jeremy Allison <jra(a)samba.org> wrote: > >> > >> >> Don't add it as an EA. It's *not* an EA, it's a timestamp. > >> > > >> > I'm curious. Why do you particularly care what interface the kernel uses to > >> > provide you with access to this attribute? > >> > > >> > And given that it is an attribute that is not part of 'POSIX' or "UNIX", it > >> > would seem to be an extension - an extended attribute. > >> > As the Linux kernel does virtually nothing with this attribute except provide > >> > access, it seems to be a very different class of thing to other timestamps. > >> > Surely it is simply some storage associated with a file which is capable of > >> > storing a timestamp, which can be set or retrieved by an application, and > >> > which happens to be initialised to the current time when a file is created. > >> > > >> > Yes, to you it is a timestamp. But to Linux it is a few bytes of > >> > user-settable metadata. Sounds like an EA to me. > >> > > >> > Or do you really want something like BSD's 'btime' which as I understand it > >> > cannot be set. Would that be really useful to you? > >> > >> Obviously the cifs and SMB2 protocols which Samba server support can > >> ask the server to set the create time of a file (this is handled > >> through xattrs today along with the "dos attribute" flags such as > >> archive/hidden/system), but certainly it is much more common (and > >> important) to read the creation time of an existing file. > >> > > > > Just a point of clarification - when you say it is common and important to be > > able to read the creation time on an existing file, and you still talking in > > the context of cifs/smb windows compatibility, or are you talking in the > > broader context? > > If you are referring to a broader context could be please give more details > > because I have not heard any mention of any real value of creation-time out > > side of window interoperability - have such a use clearly documented would > > assist the conversation I think. > > > > If on the other hand you are just referring the the windows interoperability > > context ... given that you have to read an EA if the create-time has been > > changed, you will always have to read and EA so having something else is > > pointless ... or I'm missing something. > > There are other cases, less common than cifs and smb2. One > that comes to mind is NFS version 4, but there are a few other > cases that I have heard of (backup/archive applications). > The RFC recommends that servers return attribute 50 (creation > time). See below text: > > time_create 50 nfstime4 R/W The time of creation > of the object. This > attribute does not > have any relation to > the traditional UNIX > file attribute > "ctime" or "change > time". I really don't think NFSv4 is a separate justification. I'm fairly sure that attribute was only including in NFSv4 for enhanced Windows compatibility (windows interoperation was a big issue during the protocol development). That leaves hypothetical "backup/archive applications". Do you have a concrete example? Or we are left with just various flavours of Windows compatibility (not that I have a problem with Windows compatibility, but if that is the only reason that we have creation-time then I think it is important to be clear and open about that). NeilBrown -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Neil Brown on 6 Aug 2010 23:40 On Fri, 6 Aug 2010 21:54:49 -0500 Steve French <smfrench(a)gmail.com> wrote: > On Fri, Aug 6, 2010 at 9:42 PM, Steve French <smfrench(a)gmail.com> wrote: > > On Fri, Aug 6, 2010 at 7:29 PM, Neil Brown <neilb(a)suse.de> wrote: > >> On Fri, 6 Aug 2010 18:58:42 -0500 > >> Steve French <smfrench(a)gmail.com> wrote: > >> > >>> On Fri, Aug 6, 2010 at 6:30 PM, Neil Brown <neilb(a)suse.de> wrote: > >>> > On Thu, 5 Aug 2010 22:55:06 -0500 > >>> > Steve French <smfrench(a)gmail.com> wrote: > >>> > > >>> >> On Thu, Aug 5, 2010 at 10:38 PM, Neil Brown <neilb(a)suse.de> wrote: > >>> >> > On Thu, 5 Aug 2010 16:52:18 -0700 > >>> >> > Jeremy Allison <jra(a)samba.org> wrote: > >>> >> > >>> >> >> Don't add it as an EA. It's *not* an EA, it's a timestamp. > >>> >> > > >>> >> > I'm curious. Why do you particularly care what interface the kernel uses to > >>> >> > provide you with access to this attribute? > >>> >> > > >>> >> > And given that it is an attribute that is not part of 'POSIX' or "UNIX", it > >>> >> > would seem to be an extension - an extended attribute. > >>> >> > As the Linux kernel does virtually nothing with this attribute except provide > >>> >> > access, it seems to be a very different class of thing to other timestamps. > >>> >> > Surely it is simply some storage associated with a file which is capable of > >>> >> > storing a timestamp, which can be set or retrieved by an application, and > >>> >> > which happens to be initialised to the current time when a file is created. > >>> >> > > >>> >> > Yes, to you it is a timestamp. But to Linux it is a few bytes of > >>> >> > user-settable metadata. Sounds like an EA to me. > >>> >> > > >>> >> > Or do you really want something like BSD's 'btime' which as I understand it > >>> >> > cannot be set. Would that be really useful to you? > >>> >> > >>> >> Obviously the cifs and SMB2 protocols which Samba server support can > >>> >> ask the server to set the create time of a file (this is handled > >>> >> through xattrs today along with the "dos attribute" flags such as > >>> >> archive/hidden/system), but certainly it is much more common (and > >>> >> important) to read the creation time of an existing file. > >>> >> > >>> > > >>> > Just a point of clarification - when you say it is common and important to be > >>> > able to read the creation time on an existing file, and you still talking in > >>> > the context of cifs/smb windows compatibility, or are you talking in the > >>> > broader context? > >>> > If you are referring to a broader context could be please give more details > >>> > because I have not heard any mention of any real value of creation-time out > >>> > side of window interoperability - have such a use clearly documented would > >>> > assist the conversation I think. > >>> > > >>> > If on the other hand you are just referring the the windows interoperability > >>> > context ... given that you have to read an EA if the create-time has been > >>> > changed, you will always have to read and EA so having something else is > >>> > pointless ... or I'm missing something. > >>> > >>> There are other cases, less common than cifs and smb2. One > >>> that comes to mind is NFS version 4, but there are a few other > >>> cases that I have heard of (backup/archive applications). > >>> The RFC recommends that servers return attribute 50 (creation > >>> time). See below text: > >>> > >>> time_create 50 nfstime4 R/W The time of creation > >>> of the object. This > >>> attribute does not > >>> have any relation to > >>> the traditional UNIX > >>> file attribute > >>> "ctime" or "change > >>> time". > >> > >> I really don't think NFSv4 is a separate justification. I'm fairly sure > >> that attribute was only including in NFSv4 for enhanced Windows > >> compatibility (windows interoperation was a big issue during the protocol > >> development). > > > > Perhaps also useful for MacOS (and other BSD), not just Windows, > > although MacOS may use cifs more often than nfs. > > >> That leaves hypothetical "backup/archive applications". > >> Do you have a concrete example? > > A quick search for backup applications in Wikipedia came up with a > reference fairly easily (to backup app which uses creation > time) for Linux: > > http://www.aqualab.cs.northwestern.edu/publications/Cornell04VFS.html That publication seems to mention 'creation time' only as an abstract concept. The backup architecture keeps a history of the file all that way back to its "creation time". It doesn't appear to need or use a 'creation time' attribute stored with any file. > > Presumably Windows compat. is a stronger motivation, than BSD/MacOS > NFSv4 (returning birth time) compat, and backup applications > are a lesser motivations. There may also be some value in using creation > time as a generation number where no generation number is > available. > > Intuitively seems like creation time would be as "useful" as ctime (and probably > more so) to app developers ... but that is hard to prove. > I agree, it does seem like an intuitively valuable number - after all we each have a birthday which we are very aware of and often make use of. It is often treated as part of our identity - just like you were mentioning that the CIFS client uses creation-time to help identify files which lack the 'inode number' identifier that is the common tool in Unix and derivatives. But I'm not convinced that it is *practically* useful. The only practical use beyond windows-compatibility that has been mentioned is a stronger 'identity' tag. However inode+generation number, or "file-handle-fragment" are better things to use for identifying a file than "creation time", especially when the latter is settable. So if we were to add something for native applications to use, I doubt that it would be 'creation time' (but I'm still open to hearing a convincing use-case). So we are left with an attribute that is needed for windows compatibility, and so just needs to be understood by samba and wine. Some filesystems might support it efficiently, others might require the use of generic extended-attributes, still others might not support it at all (I guess you store it in some 'tdb' and hope it works well enough). Core-linux doesn't really need to know about this - there just needs to be a channel to pass it between samba/wine and the filesystem. xattr still seems the best mechanism to pass this stuff around. Team-samba can negotiate with fs developers to optimise/accelerate certain attributes, and linux-VFS doesn't need to know or care (except maybe to provide generic non-blocking or multiple-access interfaces). What is 'creation time' used for in the windows world??? Maybe there really is something valuable here that we are missing.... NeilBrown -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Jeff Layton on 7 Aug 2010 06:40
On Sat, 7 Aug 2010 13:32:40 +1000 Neil Brown <neilb(a)suse.de> wrote: > So we are left with an attribute that is needed for windows compatibility, > and so just needs to be understood by samba and wine. Some filesystems might > support it efficiently, others might require the use of generic > extended-attributes, still others might not support it at all (I guess you > store it in some 'tdb' and hope it works well enough). > > Core-linux doesn't really need to know about this - there just needs to be a > channel to pass it between samba/wine and the filesystem. xattr still seems > the best mechanism to pass this stuff around. Team-samba can negotiate with > fs developers to optimise/accelerate certain attributes, and linux-VFS > doesn't need to know or care (except maybe to provide generic non-blocking or > multiple-access interfaces). > IIUC, you're saying that we should basically just have samba stuff the current time into an xattr when it creates the file and leave the filesystems alone. If so, I disagree here. The problem with treating this as *just* an xattr is that it doesn't account for files that are created outside of samba but are then shared out by it. To handle this correctly, I believe it needs to be initialized by the kernel to the current time whenever an inode is created, even if samba doesn't create it. After that, it can be treated as just another xattr. -- Jeff Layton <jlayton(a)samba.org> -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |