On Sep 6, 2007, at 6:50 PM, Herbert Poetzl wrote:
> On Thu, Sep 06, 2007 at 06:11:27PM -0700, Dallas Kashuba wrote:
>> We have several vserver guests NFS mounting volumes being served from
>> Network Appliance filers. The NFS mounting works as expected, other
>> than access to the snapshots provided by the NetApp filesystem.
>
> NFS tasgging is disabled, I presume?
Yes, it is.
>> Under normal circumstances (non-vserver), you access the snapshot
>> from anywhere along the filesystem by simply doing 'cd .snapshot'.
>> Within there you will see a directory for each snapshot named
>> something like hourly.0, nightly.0, or weekly.1 (etc), which contain
>> copies of the files from those points in time. They work like any
>> other directory aside from being read-only and being hidden from 'ls'
>> output. Any user can access the snapshots for any part of the
>> filesystem that user ordinarily has access to. No root access is
>> required.
>>
>> Inside the vserver guests it does not work as expected, though.
>> Attempting to 'cd .snapshot' never works at all unless BINARY_MOUNT
>> is added to ccapabilities. This seems odd to me as there is no
>> mounting taking place from what I understand, but the kernel seems to
>> think there is.
>
> there is a mount happening in this case,
> because the filer provides the snapshot
> as a different (subtree) export
> (but it happens inside the kernel)
>> With BINARY_MOUNT added, the root user is then able to access
>> .snapshot in all cases. non-root users cannot, though. A non-root
>> user
>> CAN access a .snapshot portion of the filesystem after the root user
>> has accessed it once previously, though. Here's an example...
>
> and this works fine on the host system?
> (I mean as non-root user on the host?)
I had not tried it on the host system itself, but after doing so I
have found that it does not work on the host system as non-root user
either. It acts the same as inside a guest and says 'operation not
permitted'. Is that a clue?
>> vserver03:/etc/vservers/patrick# vserver patrick enter
>> patrick:/# su - testuser
>> [patrick]$ whoami
>> testuser
>> [patrick]$ cd .snapshot
>> -su: cd: .snapshot: Operation not permitted
>> # (Note the "Operation not permitted" error)
>>
>> [patrick]$ logout
>> # (back to root now)
>>
>> patrick:/# cd /home/testuser/.snapshot
>> patrick:/# su - testuser
>> [patrick]$ cd .snapshot
>> # (after accessing it once as root, it's now accessible as the non-
>> root user)
>>
>> [patrick]$ ls
>> hourly.0 hourly.1 nightly.0 nightly.1 weekly.0 weekly.1
>> [patrick]$ ls hourly.0
>> ls: hourly.0: Operation not permitted
>> # (access one more level down the filesystem is still not permitted)
>>
>> [patrick]$ logout
>> patrick:/# cd /home/testuser/.snapshot/
>> patrick:/home/testuser/.snapshot# ls -l
>> total 32
>> drwxr-x--x 7 testuser users 4096 Jul 3 10:53 hourly.0
>> drwxr-x--x 7 testuser users 4096 Jul 3 10:53 hourly.1
>> drwxr-x--x 7 testuser users 4096 Jul 3 10:53 nightly.0
>> drwxr-x--x 7 testuser users 4096 Jul 3 10:53 nightly.1
>> drwxr-x--x 7 testuser users 4096 Jul 3 10:53 weekly.0
>> drwxr-x--x 7 testuser users 4096 Jul 3 10:53 weekly.1
>>
>> # (doing an ls -la as root accesses every directory below)
>>
>> patrick:/home/testuser/.snapshot# su - testuser
>> [patrick]$ cd .snapshot/hourly.0
>> [patrick]$ ls
>> Maildir logs testdomain.com svn svntest
>> [patrick]$ cd ../weekly.1
>> [patrick]$ ls
>> Maildir logs testdomain.com svn svntest
>> [patrick]$ logout
>>
>> After the root user ran 'ls -la' inside .snapshot, the non-root user
>> could then access all of the hourly.0, weekly.1 etc snapshot
>> directories below.
>
> this is most likely because the entries are
> already in the dentry cache on the 'client'
>
>> Does anyone have any clues about how to fix this?
>
>> Ideally BINARY_MOUNT would not be needed at all, but
>
> well, let's postpone this question ...
>
>> it's ok as long as it works normally for non-root users.
>>
>> Any ideas are appreciated!
>
> yeah, please provide the following debug info:
>
> - complete rpc and nfs(d) debug output for all
> cases, i.e. root/user on host/guest
> - Linux-VServer debug dmesg output
Is there a preferred format for this debug information? We are
looking into getting debug information from the NetApps themselves
but have not had luck so far. Would tcpdump output for the nfs
requests be useful? We're generating some of that now.
Thanks!
Dallas
> we know that the subtree checks (depending on
> the nfs version) cause some issues, and I think
> we can fix them with sufficient data ...
>
> TIA,
> Herbert
>
>> Thanks!
>> Dallas
>>
>>
>>
>
Received on Sat Sep 8 00:48:11 2007