From: Andreas Dilger on
On 2010-07-02, at 01:05, hch(a)infradead.org wrote:
> On Thu, Jul 01, 2010 at 10:02:29PM -0600, Andreas Dilger wrote:
>> I'd like to be able to use this interface to implement the distributed open call proposed by the POSIX HECWG. This allows one client to do the path traversal, broadcast the file handle to the (maybe) 1M processes in the job via MPI, and then the other clients can open the file by handle without doing 1M times the full path traversal (which might be 10's of RPCs per process).
>
> The proposal is doomed anyway. If we allow any sort of open by handle
> system call for unprivilegued users we need to do reconnect the dentry
> to the dcache path anyway (reconnect_path), which is more expensive than
> a normal path lookup.

I haven't looked at this part of the VFS in a while, but it looks like an implementation issue specific to knfsd, and shouldn't be needed for regular files. i.e. if exportfs_encode_fh() is never used on a disconnected file, then this overhead is not incurred.

The above use of open_by_handle() is not for userspace NFS/Samba re-export, but to allow applications to open regular files for IO.

Cheers, Andreas
--
Andreas Dilger
Lustre Technical Lead
Oracle Corporation Canada Inc.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Andreas Dilger on
On 2010-07-02, at 16:09, Neil Brown wrote:
> On Fri, 2 Jul 2010 10:12:47 -0600
> Andreas Dilger <andreas.dilger(a)oracle.com> wrote:
>>
>> I haven't looked at this part of the VFS in a while, but it looks like reconnect_path() is an implementation issue specific to knfsd, and shouldn't be needed for regular files. i.e. if exportfs_encode_fh() is never used on a disconnected file, then this overhead is not incurred.
>>
>> The above use of open_by_handle() is not for userspace NFS/Samba re-export, but to allow applications to open regular files for IO.
>
> Firstly it is needed for directories so that the VFS can effectively lock
> against directory rename races which could otherwise create disconnected
> subtrees (where the first parent is a member only of one of its
> descendants). So if you get a filehandle for a directory it *must* be
> properly connected to the root for rename to be safe. This operation is
> faster than a full path lookup if the dentry is already is cache, and slower
> if it and any of the path is not in cache.

OK, so this requirement is specific for directories, and not at all needed for regular files.

> Secondly it is needed if you want to enforce the rule that the contents of a
> directory are only accessible if the 'x' bit on the directory is set.
> kNFSd does not enforce this (unless subtree_check is specified), partly
> because it is hard to do correctly and partly because we have to trust the
> client any, so trusting it to check the 'x' bit is very little extra trust.

If the application that called name_to_handle() already had to traverse the whole pathname to get the file handle, then there shouldn't necessarily be a requirement to do this when calling open_by_handle(). The only possible permission checking in open_by_handle() is the permission on the inode itself.

> Note that it is not possible to reliably perform filehandle lookup for
> non-directories if you need a fully reconnected dentry, as
> cross-directory-renames can confuse the situation beyond recovery.

For normal file IO, a fully connected dentry is not needed, and in fact the handle_to_path->exportfs_decode_fh() code will accept any inode alias for reguar file use.

> Maybe open-by-handle should require DAC_OVERRIDE, or maybe a new
> DAC_X_OVERRIDE. And if those aren't provided it only works for directories.

That's the big question. If the file handle has some "non-public" information in it (i.e. a capability that cannot be (easily) guessed or forged), then there should not be any need for DAC_OVERRIDE. This could easily be enforced if there was a provision for "short term" file handles that only had to live a few minutes or less, so the kernel could just store a random cookie in each file handle and require applications to get a new handle if the cookie expires or the server crashes.

However, even a "plain" file handle containing only the inode/generation is relatively secure in this respect, since the only way to get the inode number of a particular file is "ls -li" (which either assumes path "x" traversal permission, OR guessing the inode number), and ioctl(FS_IOC_GETVERSION) which requires being able to open the inode already.

Guessing the inode number by itself is fairly weak, at most 2^32 inodes in most filesystems, usually far fewer. Guessing the generation number is much harder (though not impossible).

Cheers, Andreas
--
Andreas Dilger
Lustre Technical Lead
Oracle Corporation Canada Inc.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Aneesh Kumar K. V on
On Sat, 3 Jul 2010 08:09:04 +1000, Neil Brown <neilb(a)suse.de> wrote:
> On Fri, 2 Jul 2010 10:12:47 -0600
> Andreas Dilger <andreas.dilger(a)oracle.com> wrote:
>
> > On 2010-07-02, at 01:05, hch(a)infradead.org wrote:
> > > On Thu, Jul 01, 2010 at 10:02:29PM -0600, Andreas Dilger wrote:
> > >> I'd like to be able to use this interface to implement the distributed open call proposed by the POSIX HECWG. This allows one client to do the path traversal, broadcast the file handle to the (maybe) 1M processes in the job via MPI, and then the other clients can open the file by handle without doing 1M times the full path traversal (which might be 10's of RPCs per process).
> > >
> > > The proposal is doomed anyway. If we allow any sort of open by handle
> > > system call for unprivilegued users we need to do reconnect the dentry
> > > to the dcache path anyway (reconnect_path), which is more expensive than
> > > a normal path lookup.
> >
> > I haven't looked at this part of the VFS in a while, but it looks like an implementation issue specific to knfsd, and shouldn't be needed for regular files. i.e. if exportfs_encode_fh() is never used on a disconnected file, then this overhead is not incurred.
> >
> > The above use of open_by_handle() is not for userspace NFS/Samba re-export, but to allow applications to open regular files for IO.
> >
>
> From my recollection of implementing dentry reconnection there are two
> needs for it.
>
> Firstly it is needed for directories so that the VFS can effectively lock
> against directory rename races which could otherwise create disconnected
> subtrees (where the first parent is a member only of one of its
> descendants). So if you get a filehandle for a directory it *must* be
> properly connected to the root for rename to be safe. This operation is
> faster than a full path lookup if the dentry is already is cache, and slower
> if it and any of the path is not in cache.
> You could possibly delay the full-connection of the dentry until the first
> attempt to rename beneath it. I'm not sure how much VFS surgery that would
> require.
>
> Secondly it is needed if you want to enforce the rule that the contents of a
> directory are only accessible if the 'x' bit on the directory is set.
> kNFSd does not enforce this (unless subtree_check is specified), partly
> because it is hard to do correctly and partly because we have to trust the
> client any, so trusting it to check the 'x' bit is very little extra trust.
>
> Note that it is not possible to reliably perform filehandle lookup for
> non-directories if you need a fully reconnected dentry, as
> cross-directory-renames can confuse the situation beyond recovery.
>
> Maybe open-by-handle should require DAC_OVERRIDE, or maybe a new
> DAC_X_OVERRIDE. And if those aren't provided it only works for directories.
> ???
>

Currently I have the below in open_by_handle

/*
* With handle we don't look at the execute bit on the
* the directory. Ideally we would like CAP_DAC_SEARCH.
* But we don't have that
*/
if (!capable(CAP_DAC_READ_SEARCH)) {
retval = -EPERM;
goto out_err;
}

-aneesh
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Aneesh Kumar K. V on
On Tue, 6 Jul 2010 12:10:02 -0400, "J. Bruce Fields" <bfields(a)fieldses.org> wrote:
> On Fri, Jul 02, 2010 at 02:45:45AM +0530, Aneesh Kumar K. V wrote:
> > One use case i had was that if the userspace file server can directly
> > work with the returned file system UUID,
>
> I agree that the uuid should be split out from the rest of the
> filehandle, but ...
>
> > the it can build the file
> > handle for client in a single call.
>
> ... I don't understand why both need to come in the same system call.
> Is it purely an efficiency question? If so, why do you expect this to
> be significant?

Since we know that system wide file handle should include a file system
identifier and a file identifier my plan was to retrieve both in the
same syscall.


>
> (I would have thought that the system call overhead is so small, and so
> many calls will already be required to perform the typical rpc, that
> this would be insignificant.)
>
> A filesystem uuid seems like a generally useful thing (maybe more so
> than a filehandle), so it'd seem worth figuring out how to export that
> separately.
>

I can add a new syscall that returns

struct fs_uuid {
u8 fs_uuid[16];
};

long sys_get_fs_uuid(int dfd, char *name, struct fs_uuid *fsid, int flag);

-aneesh
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Andreas Dilger on
On 2010-07-06, at 11:09, Aneesh Kumar K. V wrote:
> On Tue, 6 Jul 2010 12:10:02 -0400, "J. Bruce Fields" <bfields(a)fieldses.org> wrote:
>> ... I don't understand why both need to come in the same system call.
>> Is it purely an efficiency question? If so, why do you expect this to
>> be significant?
>
> Since we know that system wide file handle should include a file system
> identifier and a file identifier my plan was to retrieve both in the
> same syscall.

Won't having it be in a separate system call be racy w.r.t. doing the pathname lookup twice?

>> A filesystem uuid seems like a generally useful thing (maybe more so
>> than a filehandle), so it'd seem worth figuring out how to export that
>> separately.
>>
>
> I can add a new syscall that returns
>
> struct fs_uuid {
> u8 fs_uuid[16];
> };
>
> long sys_get_fs_uuid(int dfd, char *name, struct fs_uuid *fsid, int flag);

While this might be useful, I think the file handle should identify the filesystem itself.

Cheers, Andreas
--
Andreas Dilger
Lustre Technical Lead
Oracle Corporation Canada Inc.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/