Copyright © 2010 Jurjen Bokma
Abstract
Notes concerning the installation of NFSv4 in various configurations
Table of Contents
Table of Contents
This experiment with NFS and Kerberos uses at least three machines: a Kerberos server, an NFS server, and an NFS client. Both the NFS server and the NFS clients are kerberos clients. For the servers, we use Debian Lenny ('stable' as of this writing). The clients are supposed to become desktop machines later, so we put Ubuntu Lucid on them, the latest LTS version.
The Kerberos server is going to run the KDC
as well as the kadmind
.
The NFS server is going to use nfs-kernel-server
.
Both the NFS server and NFS client are going to run gssd
and idmapd
.
We are going to show the installation and configuration of the Kerberos server, the NFS server, and the NFS client. We show the full configuration of each, although when following the steps you may wish to do the actual work in parallel, taking small steps and verifying that things work when first trying, and to speed things up when familiar with the procedure.
If you are fiddling with Kerberos and NFS, you are supposed to be able to install an OS on your machines. We use unattended installs for both Debian and Ubuntu, so in our case, there isn't even something to show.
Table of Contents
To use NFSv4 with Kerberos authentication, we need a Kerberos server. Because we may later switch to a Novell Kerberos server, and Novell seems to ship Heimdal, we use Heimdal, on Debian Lenny.
If you wish to use a different Linux distribution, or a different UNIX, there is excellent documentation on the MIT.edu site. You probably want to read the part on installation.
But since the Heimdal and MIT implementations differ in details, I fall back on the more elaborate MIT docs if necessary, sticking first to the Heimdal-specific documentation on the Heimdal site, in particular on building and installing.
Procedure 2.1. Installation of a Kerberos server on Ubuntu Lucid
Note | |
---|---|
On Ubuntu, you may have to loosen security a bit and do sudo ufw disable before you proceed. We are using Debian Lenny, which comes with no iptables rules enabled by default. |
Preconfiguring the Heimdal packages
Create a file debconf-kerberos-settings
containing this:
# Kerberos servers for your realm: krb5-config krb5-config/kerberos_servers string krbserver.mydomain.com # Default Kerberos version 5 realm: krb5-config krb5-config/default_realm string MYDOMAIN.COM # Local realm name: heimdal-kdc heimdal/realm string MYDOMAIN.COM # Administrative server for your Kerberos realm: krb5-config krb5-config/admin_server string krbserver.mydomain.com # Does DNS contain pointers to your realm's Kerberos Servers? krb5-config krb5-config/dns_for_default boolean false # Add locations of default Kerberos servers to /etc/krb5.conf? krb5-config krb5-config/add_servers boolean true
This is the hostname of the Kerberos server, the machine we are configuring. We 've got only one at this stage, which is to be KDC as well as Administration server. | |
By convention, the Kerberos domain is identical to the uppercased domain name. |
Now tell debconf about these settings:
apprentice@krbserver:~$ cat debconf-kerberos-settings |sudo debconf-set-selections
apprentice@krbserver:~$
Installing the packages
Warning | |
---|---|
On Ubuntu Lucid, bug #579127 causes a hanging |
apprentice@krbserver:~$ sudo apt-get install -y heimdal-kdc
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
heimdal-clients krb5-config libasn1-8-heimdal libdb4.2 libgssapi2-heimdal libhdb9-heimdal libheimntlm0-heimdal libhesiod0 libhx509-3-heimdal
libkadm5clnt7-heimdal libkadm5srv8-heimdal libkafs0-heimdal libkdc2-heimdal libkrb5-25-heimdal libotp0-heimdal libroken18-heimdal
libsl0-heimdal libwind0-heimdal
Suggested packages:
heimdal-docs heimdal-kcm
The following NEW packages will be installed:
heimdal-clients heimdal-kdc krb5-config libasn1-8-heimdal libdb4.2 libgssapi2-heimdal libhdb9-heimdal libheimntlm0-heimdal libhesiod0
libhx509-3-heimdal libkadm5clnt7-heimdal libkadm5srv8-heimdal libkafs0-heimdal libkdc2-heimdal libkrb5-25-heimdal libotp0-heimdal
libroken18-heimdal libsl0-heimdal libwind0-heimdal
0 upgraded, 19 newly installed, 0 to remove and 0 not upgraded.
Need to get 2306kB of archives.
<snip a lot of Get, Unpacking and Selecting>
Processing triggers for man-db ...
Setting up krb5-config (1.22) ...
Setting up libroken18-heimdal (1.2.dfsg.1-2.1) ...
Setting up libasn1-8-heimdal (1.2.dfsg.1-2.1) ...
Setting up libdb4.2 (4.2.52+dfsg-5) ...
Setting up libwind0-heimdal (1.2.dfsg.1-2.1) ...
Setting up libhx509-3-heimdal (1.2.dfsg.1-2.1) ...
Setting up libkrb5-25-heimdal (1.2.dfsg.1-2.1) ...
Setting up libheimntlm0-heimdal (1.2.dfsg.1-2.1) ...
Setting up libgssapi2-heimdal (1.2.dfsg.1-2.1) ...
Setting up libhdb9-heimdal (1.2.dfsg.1-2.1) ...
Setting up libkadm5clnt7-heimdal (1.2.dfsg.1-2.1) ...
Setting up libkadm5srv8-heimdal (1.2.dfsg.1-2.1) ...
Setting up libhesiod0 (3.0.2-18.3) ...
Setting up libkafs0-heimdal (1.2.dfsg.1-2.1) ...
Setting up libotp0-heimdal (1.2.dfsg.1-2.1) ...
Setting up libsl0-heimdal (1.2.dfsg.1-2.1) ...
Setting up heimdal-clients (1.2.dfsg.1-2.1) ...
Setting up libkdc2-heimdal (1.2.dfsg.1-2.1) ...
Setting up heimdal-kdc (1.2.dfsg.1-2.1) ...
kstash: writing key to `/var/lib/heimdal-kdc/m-key'
Realm max ticket life [unlimited]:Realm max renewable ticket life [unlimited]:Starting Heimdal KDC: heimdal-kdc.
Starting Heimdal password server: kpasswdd.
apprentice@krbserver:~$
Note | |
---|---|
Because we preseeded the package, it is not necessary to initialize the |
Adding an admin principal
apprentice@krbserver:~$ sudo kadmin -l
kadmin> add apprentice/admin@MYDOMAIN.COM
Max ticket life [1 day]:
Max renewable life [1 week]:
Principal expiration time [never]:
Password expiration time [never]:
Attributes []:
apprentice/admin@MYDOMAIN.COM's Password:
Verifying - apprentice/admin@MYDOMAIN.COM's Password:
kadmin> quit
apprentice@krbserver:~$
Granting admin access to the newly created principal
Fix a small mistake in the kdc.conf
There is a reference to “FILE:/etc/heimdal-kdc/kadmind.acl” in /etc/heimdal-kdc/kdc.conf
, which the kadmind tires to open literally.
We need to fix that (actually, there is another instance of “FILE:”, which we also fix:
apprentice@krbserver:~$ sudo sed -i.bak 's%FILE:%%' /etc/heimdal-kdc/kdc.conf
apprentice@krbserver:~$
Grant all permissions to apprentice/admin
apprentice@krbserver:~$ sudo sh -c "echo 'apprentice/admin@MYDOMAIN.COM all' >> /etc/heimdal-kdc/kadmind.acl"
apprentice@krbserver:~$
Symlink the ACL
The kadmin daemon looks for the ACL file in /var/lib/heimdal
, so we need it to show up there:
apprentice@krbserver:~$ sudo ln -s /etc/heimdal-kdc/kadmind.acl /var/lib/heimdal-kdc/
apprentice@krbserver:~$
(Re)start the services
apprentice@krbserver:~$ sudo /etc/init.d/openbsd-inetd start
apprentice@krbserver:~$ sudo /etc/init.d/heimdal-kdc restart
Stopping Heimdal password server: kpasswdd.
Stopping Heimdal KDC: heimdal-kdc.
Starting Heimdal KDC: heimdal-kdc.
Starting Heimdal password server: kpasswdd.
apprentice@krbserver:~$
Creating a Kerberos principal for a user
apprentice@krbserver:~$ /usr/sbin/kadmin
kadmin> add joeuser@MYDOMAIN.COM
apprentice/admin@MYDOMAIN.COM's Password:
Max ticket life [1 day]:
Max renewable life [1 week]:
Principal expiration time [never]:
Password expiration time [never]:
Attributes []:
joeuser@MYDOMAIN.COM's Password:
Verifying - joeuser@MYDOMAIN.COM's Password:
kadmin> list *
default
joeuser
apprentice/admin
kadmin/admin
kadmin/hprop
krbtgt/MYDOMAIN.COM
kadmin/changepw
changepw/kerberos
kadmin> quit
apprentice@krbserver:~$
Note | |
---|---|
This time, we ran |
Table of Contents
Procedure 3.1. Installing and configuring an NFSv4 server
Mapping userIDs
If the UIDs of the files on your NFS server are going to be shown correctly, you must configure the NFS server to map them using idmapd
, which is configured in one of the steps below to use libnss
.
If getent passwd joeuser shows some output on the NFS server, then you 're all set.
If not, make it work first.
How to do that is outside of the scope of this document.
Something to serve
Note | |
---|---|
In this step we create and mount a filesystem that we are going to serve over NFS. It isn't necessary to serve an entire FS, you can serve a directory just as well. If you already have something to serve, you can skip this step. |
We are going to serve what is now mounted under /srv
, but from a different mount point:
apprentice@nfs-server:~$df -h|grep srv
/dev/mapper/nfs-server-lvsrv
2.4G 68M 2.2G 3% /srv
This little script makes the changes in my case:
apt-get install xfsprogs
umount /srv/
mkfs.xfs -f /dev/mapper/nfs-server-lvsrv
mkdir /lwphome
sed -i 's%/srv%/lwphome%' /etc/fstab
sed -i '/lwphome/ s%ext3%xfs%' /etc/fstab
sed -i '/lwphome/ s%defaults%defaults,uquota,pquota%' /etc/fstab
mount -a
chmod 1777 /lwphome/
Install xfsprogs | |
Unmount the existing fs from the old mount point | |
Put an XFS filesystem on the underlying block device | |
Create a new mount point | |
In /dev/mapper/machine.domain.com-lvsrv /srv ext3 defaults 0 2 with this one: /dev/mapper/machine.domain.com-lvsrv /lwphome xfs defaults,uquota,pquota 0 2
| |
Mount all filesystems, including the newly created one | |
Put mode 1777 on the mounted FS, as for the first setup we want anyone to be able to write files there as far as the FS is ruling permissions... |
If it ran correctly, we now have /lwphome mounted, like this:
apprentice@nfs-server:~$ mount|grep lwphome
/dev/mapper/nfs-server-lvsrv on /lwphome type xfs (rw,uquota,pquota)
apprentice@nfs-server:~$
Preparing the NFS server as a Kerberos client
The NFS server will speak Kerberos to the Kerberos server. We use the same preseeding as on the Kerberos server, and install the Heimdal clients (but not the KDC of course):
apprentice@nfs-server:~$ cat <<EOF > debconf-kerberos-settings
# Kerberos servers for your realm:
krb5-config krb5-config/kerberos_servers string krbserver.mydomain.com
# Default Kerberos version 5 realm:
krb5-config krb5-config/default_realm string MYDOMAIN.COM
# Local realm name:
heimdal-kdc heimdal/realm string MYDOMAIN.COM
# Administrative server for your Kerberos realm:
krb5-config krb5-config/admin_server string krbserver.mydomain.com
# Does DNS contain pointers to your realm's Kerberos Servers?
krb5-config krb5-config/dns_for_default boolean false
# Add locations of default Kerberos servers to /etc/krb5.conf?
krb5-config krb5-config/add_servers boolean true
EOF
apprentice@nfs-server:~$ sudo debconf-set-selections < debconf-kerberos-settings
apprentice@nfs-server:~$ sudo apt-get install -y heimdal-clients
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
krb5-config libasn1-8-heimdal libdb4.7 libgssapi2-heimdal libhdb9-heimdal libheimntlm0-heimdal libhesiod0 libhx509-4-heimdal libkadm5clnt7-heimdal
libkadm5srv8-heimdal libkafs0-heimdal libkrb5-25-heimdal libotp0-heimdal libroken18-heimdal libsl0-heimdal libwind0-heimdal
Suggested packages:
heimdal-docs heimdal-kcm
The following NEW packages will be installed:
heimdal-clients krb5-config libasn1-8-heimdal libdb4.7 libgssapi2-heimdal libhdb9-heimdal libheimntlm0-heimdal libhesiod0 libhx509-4-heimdal libkadm5clnt7-heimdal
libkadm5srv8-heimdal libkafs0-heimdal libkrb5-25-heimdal libotp0-heimdal libroken18-heimdal libsl0-heimdal libwind0-heimdal
0 upgraded, 17 newly installed, 0 to remove and 0 not upgraded.
Need to get 2,168kB of archives.
After this operation, 6,246kB of additional disk space will be used.
Get:1 http://mirror.mydomain.com/ubuntu/ lucid/main krb5-config 2.2 [23.0kB]
<snip>
Setting up heimdal-clients (1.2.e1.dfsg.1-1ubuntu1) ...
update-alternatives: using /usr/bin/krsh to provide /usr/bin/rsh (rsh) in auto mode.
update-alternatives: using /usr/bin/krcp to provide /usr/bin/rcp (rcp) in auto mode.
update-alternatives: using /usr/bin/kpagsh to provide /usr/bin/pagsh (pagsh) in auto mode.
Processing triggers for libc-bin ...
ldconfig deferred processing now taking place
apprentice@nfs-server:~$
Installing the NFS packages
apprentice@nfs-server:~$ sudo apt-get install -y nfs-kernel-server
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
nfs-kernel-server
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 162kB of archives.
After this operation, 319kB of additional disk space will be used.
Get:1 http://mirror.mydomain.com lenny/main nfs-kernel-server 1:1.1.2-6lenny1 [162kB]
Fetched 162kB in 0s (13.9MB/s)
Selecting previously deselected package nfs-kernel-server.
(Reading database ... 23720 files and directories currently installed.)
Unpacking nfs-kernel-server (from .../nfs-kernel-server_1%3a1.1.2-6lenny1_amd64.deb) ...
Processing triggers for man-db ...
Setting up nfs-kernel-server (1:1.1.2-6lenny1) ...
Creating config file /etc/exports with new version
Creating config file /etc/default/nfs-kernel-server with new version
Starting NFS common utilities: statd.
Exporting directories for NFS kernel daemon....
Starting NFS kernel daemon: nfsd mountd.
apprentice@nfs-server:~$
Creating a Kerberos principal for the NFS service
Kerberos is a protocol for mutual authentication.
So the NFS user should authenticate herself to the NFS service, but the NFS service should also authenticate itself to the user.
The NFS service therefore needs to have a principal
, which is named “nfs/nfs-server
@REALM
” by convention and by NFS server code.
(Actually, there is a short list <ToDo: find url of docs > of principals the NFS server tries to get credentials for, any of which my be used.)
apprentice@nfs-server:~$ sudo kadmin -p apprentice/admin@MYDOMAIN.COM
kadmin> add -r nfs/nfs-server.mydomain.com@MYDOMAIN.COM
apprentice/admin@MYDOMAIN.COM's Password:
Max ticket life [1 day]:
Max renewable life [1 week]:
Principal expiration time [never]:
Password expiration time [never]:
Attributes []:
kadmin> ext_keytab -k /etc/krb5.keytab nfs/nfs-server.mydomain.com@MYDOMAIN.COM
kadmin> quit
apprentice@nfs-server:~$
Note | |
---|---|
If you didn't succeed in getting the |
Configuring the NFS service
In /etc/default/nfs-common
, we put:
#<snip> # Do you want to start the statd daemon? It is not needed for NFSv4. NEED_STATD=no STATDOPTS= # Do you want to start the idmapd daemon? It is only needed for NFSv4. NEED_IDMAPD=yes # Do you want to start the gssd daemon? It is required for Kerberos mounts. NEED_GSSD=yes
... in /etc/default/nfs-kernel-server
, we have:
# Number of servers to start up RPCNFSDCOUNT=8 # Runtime priority of server (see nice(1)) RPCNFSDPRIORITY=0 # Options for rpc.mountd. # If you have a port-based firewall, you might want to set up # a fixed port here using the --port option. For more information, # see rpc.mountd(8) or http://wiki.debian.org/?SecuringNFS RPCMOUNTDOPTS= # Do you want to start the svcgssd daemon? It is only required for Kerberos # exports. Valid alternatives are "yes" and "no"; the default is "no". NEED_SVCGSSD=yes # Options for rpc.svcgssd. RPCSVCGSSDOPTS=-vvv
... and in /etc/exports
, we share /lwphome
(you want to specify you own IP range here, if any):
/lwphome 192.168.0.0/16(rw,sync,root_squash,subtree_check,sec=krb5p,fsid=0)
... and we edit /etc/krb5.conf
to work around bugs 575895 and 512110, gssd
:
[libdefaults] <snip> # default_tgs_enctypes = des3-hmac-sha1 # default_tkt_enctypes = des3-hmac-sha1 # permitted_enctypes = des3-hmac-sha1 # allow_weak_crypto = true # The following libdefaults parameters are only for Heimdal Kerberos. <snip>
And in /etc/idmapd.conf
:
[General] Verbosity = 0 Pipefs-Directory = /var/lib/nfs/rpc_pipefs Domain = rug.nl Local-Realms = TEST.MYDOMAIN.COM [Mapping] Nobody-User = nobody Nobody-Group = nogroup [Translation] Method = nsswitch
Restarting the NFS services
user@nfs-server:~$ sudo /etc/init.d/nfs-kernel-server restart
Stopping NFS kernel daemon: mountd svcgssd nfsd.
Unexporting directories for NFS kernel daemon....
Exporting directories for NFS kernel daemon....
Starting NFS kernel daemon: nfsd svcgssd mountd.
user@nfs-server:~$ sudo /etc/init.d/nfs-common restart
Stopping NFS common utilities: gssd idmapd.
Starting NFS common utilities: idmapd gssd.
user@nfs-server:~$
Procedure 3.2. Installing the NFS client
Making the PC a Kerberos client
We install the heimdal-clients with the same preconfiguration as on the NFS server and the Kerberos server:
apprentice@nfs-client:~$ cat debconf-kerberos-settings |sudo debconf-set-selections
apprentice@nfs-client:~$
apprentice@nfs-client:~$ sudo apt-get install -y heimdal-clients
Reading package lists... Done
<snip the usual apt-get output>
A principal for the NFS client
apprentice@nfs-client:~$ sudo kadmin -p apprentice/admin@MYDOMAIN.COM
kadmin> add -r nfs/nfs-client.mydomain.com@MYDOMAIN.COM
apprentice/admin@MYDOMAIN.COM's Password:
Max ticket life [1 day]:
Max renewable life [1 week]:
Principal expiration time [never]:
Password expiration time [never]:
Attributes []:
kadmin> ext_keytab -k /etc/krb5.keytab nfs/nfs-client.mydomain.com@MYDOMAIN.COM
kadmin> q
apprentice@nfs-client:~$
Installing the NFS client
apprentice@nfs-client:~$ sudo apt-get install nfs-common
<snip>
Configuring the NFS client
Copy /etc/idmapd.conf
from the NFS server.
In /etc/default/nfs-common
, put:
# Do you want to start the statd daemon? It is not needed for NFSv4. NEED_STATD=no STATDOPTS= # Do you want to start the idmapd daemon? It is only needed for NFSv4. NEED_IDMAPD=yes # Do you want to start the gssd daemon? It is required for Kerberos mounts. NEED_GSSD=yes
Edit /etc/krb5.conf
to work around bugs 575895 and 512110, gssd
:
[libdefaults] <snip> # default_tgs_enctypes = des3-hmac-sha1 # default_tkt_enctypes = des3-hmac-sha1 # permitted_enctypes = des3-hmac-sha1 # allow_weak_crypto = true # The following libdefaults parameters are only for Heimdal Kerberos. <snip>
Make sure gssd
will run even after reboot (idmapd
appears to run by default on Lenny:
apprentice@nfs-client:~$ sudo update-rc.d gssd defaults
update-rc.d: warning: /etc/init.d/gssd missing LSB information
update-rc.d: see <http://wiki.debian.org/LSBInitScripts>
Adding system startup for /etc/init.d/gssd ...
/etc/rc0.d/K20gssd -> ../init.d/gssd
/etc/rc1.d/K20gssd -> ../init.d/gssd
/etc/rc6.d/K20gssd -> ../init.d/gssd
/etc/rc2.d/S20gssd -> ../init.d/gssd
/etc/rc3.d/S20gssd -> ../init.d/gssd
/etc/rc4.d/S20gssd -> ../init.d/gssd
/etc/rc5.d/S20gssd -> ../init.d/gssd
And (re)start the services:
apprentice@nfs-client:~$ sudo /etc/init.d/gssd start
<snip>
gssd start/running, process 7178
apprentice@nfs-client:~$ sudo /etc/init.d/idmapd start
<snip>
idmapd start/running, process 7193
apprentice@nfs-client:~$
Creating a mount point
apprentice@nfs-client:~$ sudo mkdir /lwphome
apprentice@nfs-client:~$
Mounting
apprentice@nfs-client:~$ sudo mount.nfs4 nfs-server.mydomain.com:/ /lwphome -vvv -o sec=krb5p
mount.nfs4: timeout set for Tue Jun 29 23:41:27 2010
mount.nfs4: text-based options: 'sec=krb5p,clientaddr=192.168.0.48,addr=192.168.0.11'
nfs-server.mydomain.com:/ on /lwphome type nfs4 (sec=krb5p)
apprentice@nfs-client:~$ mount|grep lwphome
nfs-server.mydomain.com:/ on /lwphome type nfs4 (rw,sec=krb5p,clientaddr=192.168.0.48,addr=192.168.0.11)
apprentice@nfs-client:~$ sudo umount /lwphome
apprentice@nfs-client:~$
Putting it in fstab
/etc/fstab
:
nfs-server.mydomain.com:/ /lwphome nfs4 sec=krb5p 0 0
apprentice@nfs-client:~$ sudo mount -a
apprentice@nfs-client:~$ mount|grep lwphome
nfs-server.mydomain.com:/ on /lwphome type nfs4 (rw,sec=krb5p,clientaddr=192.168.0.48,addr=192.168.0.11)
apprentice@nfs-client:~$
Reading and writing
Now root is allowed to mount /lwphome from the NFS server on the NFS client:
apprentice@nfs-client:~$ sudo touch /lwphome/by-root
apprentice@nfs-client:~$ sudo ls /lwphome/by-root
/lwphome/by-root
But an ordinary user isn't allowed to:
apprentice@nfs-client:~$ touch /lwphome/by-apprentice
touch: cannot touch `/lwphome/by-apprentice': Permission denied
apprentice@nfs-client:~$
And the user we created a principal for earlier, joeuser
is, even though it is known on both the NFS server and the NFS client, cannot either:
apprentice@rc-706:~$ ssh joeuser@nfs-client.mydomain.com
<snip>
joeuser@nfs-client.mydomain.com's password:
Linux nfs-client 2.6.32-22-generic #36-Ubuntu SMP Thu Jun 3 19:31:57 UTC 2010 x86_64 GNU/Linux
<snip>
joeuser@nfs-client:~$ touch /lwphome/by-joeuser
touch: cannot touch `/lwphome/by-joeuser': Permission denied
... but once the user authenticates to the Kerberos server, they can write on the NFS share:
joeuser@nfs-client:~$ kinit joeuser
joeuser@MYDOMAIN.COM's Password:
joeuser@nfs-client:~$ touch /lwphome/by-joeuser
joeuser@nfs-client:~$
ACLs
We can even use ACLs on NFS:
apprentice@nfs-client:~$ sudo apt-get install nfs4-acl-tools
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
libpulse-browse0
Use 'apt-get autoremove' to remove them.
The following NEW packages will be installed:
nfs4-acl-tools
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 29.1kB of archives.
After this operation, 123kB of additional disk space will be used.
Get:1 http://mirror.mydomain.com/ubuntu/ lucid/universe nfs4-acl-tools 0.3.3-0ubuntu1 [29.1kB]
Fetched 29.1kB in 0s (2,006kB/s)
Selecting previously deselected package nfs4-acl-tools.
(Reading database ... 281523 files and directories currently installed.)
Unpacking nfs4-acl-tools (from .../nfs4-acl-tools_0.3.3-0ubuntu1_amd64.deb) ...
Processing triggers for man-db ...
Setting up nfs4-acl-tools (0.3.3-0ubuntu1) ...
apprentice@nfs-client:~$
joeuser@nfs-client:~$ nfs4_getfacl /lwphome/also-by-joeuser
A::OWNER@:rwatTcCy
A::GROUP@:rtcy
A::EVERYONE@:rtcy
joeuser@nfs-client:~$
Warning | |
---|---|
As of currently, bug #562913 is unfixed, and ACLs do not work with the default Ubuntu Lucid kernel. |
On the NFS server, these iptables
settings are sufficient to allow contact with the Kerberos Server and the Kerberos Admin Server:
# KDM is the IP of the Kerberos Server # KADMIN is the IP of the Kerberos Admin Server #Allow Kerberos (both TCP and UDP): iptables -A INPUT -i ${IFACE} -p tcp -s ${KDM} --sport 88 -m state --state NEW,ESTABLISHED -j ACCEPT iptables -A OUTPUT -o ${IFACE} -p tcp -d ${KDM} --dport 88 -m state --state NEW,ESTABLISHED -j ACCEPT iptables -A INPUT -i ${IFACE} -p udp -s ${KDM} --sport 88 -j ACCEPT iptables -A OUTPUT -o ${IFACE} -p udp -d ${KDM} --dport 88 -j ACCEPT #Allow Kadmin: iptables -A INPUT -i ${IFACE} -p tcp -s ${KADMIN} --sport 749 -m state --state NEW,ESTABLISHED -j ACCEPT iptables -A OUTPUT -o ${IFACE} -p tcp -d ${KADMIN} --dport 749 -m state --state NEW,ESTABLISHED -j ACCEPT
Table of Contents
NFSv4 ACLs do not have a 1-on-1 mapping to POSIX ACLs. But there exist the NFSv4 ACL tools, for which SuSE has some online docs, and IBM as well.
Procedure 4.1. Setting ACLs through NFSv4
Before setting the ACL
apprentice@nfs-server:~$ sudo getfacl /lwphome/numath/
getfacl: Removing leading '/' from absolute path names
# file: lwphome/numath/
# owner: joeuser
# group: joeuser
user::rwx
group::r-x
other::r-x
On the client, add to the ACL
joeuser@nfs-client:~$ mount|grep nfs-server
nfs-server.mydomain.com:/lwphome on /home type nfs4 (rw,sec=krb5p,rsize=32768,wsize=32768,acl,clientaddr=192.168.0.50,addr=192.168.0.52)
joeuser@nfs-client:~$ nfs4_setfacl -a 'A::janeuser@mydomain.com:RWX' /home/numath
joeuser@nfs-client:~$
After setting the ACL
apprentice@nfs-server:~$ sudo getfacl /lwphome/numath/
getfacl: Removing leading '/' from absolute path names
# file: lwphome/numath/
# owner: joeuser
# group: joeuser
user::rwx
user:janeuser:rwx
group::r-x
mask::rwx
other::r-x
On the client, in /etc/profile
, put something like
# Request a home directory if ! [ -e ${HOME} ] ; then touch /home/homedir-request/${USER}.req sleep 30 ls ${HOME} sleep 30 #Is this too much? cd ${HOME} fi
On the server, run
sudo inoticoming --logfile /tmp/inoti.log --pid-file /var/run/inoticoming-homedirreq /lwphome/homedir-request --suffix .req --stderr-to-log /bin/sh /usr/local/sbin/create-lwp-homedir /lwphome/homedir-request/{} \;
Note that the directory being watched must be repeated as the path to the request file being handled |
... and have a script /usr/local/sbin/create-lwp-homedir
(run as root) that creates the home directory.
Note | |
---|---|
I 've created a package that will this, with and init script for the inoticoming instance and log rotation. You can request it by mail if your're interested. |
Note | |
---|---|
Maybe it is better to do this from a PAM module instead of from the .profile.
But PAM runs with root permission, and root doesn't have access to the NFSv4 share (rootsquash is on).
|
The reason for pam_krb5_migrate
outputting Unknown code krb5 156 creating principal "joeuser@DOMAIN>COM"
is a non-responding kadmind
.
The nscd
can somehow cause files to show up as owned by nobody.
In one case, this was resolved by restarting the nscd. (Thanks, Stefan.)
The clientaddr
parameter of mount.nfs is important.
For a while, I didn't specify it in /etc/fstab
, and most clients automatically detected it, while others used 0.0.0.0 with impunity.
Then, one of the NFS servers, with no Kerberos, suddenly saw its load jump to 50 when 50 clients fetched a file at the same time, prompted by a cron job.
The load would be caused by processes waiting for I/O, not shortage of CPU, and while the file to be fetched was only 3kB or so, they would keep waiting for minutes.
And on some clients, no files could be read from the share in case, although directory listing could be obtained, as could stat
info.
It appeared that all of the clients affected had clientaddr=0.0.0.0 specified, and when they started using the proper address, the problem was over.
Increasing the number of daemons is done like this:
.. it is made persistent in
apprentice@nfs-server:~$ sudo rpc.nfsd 16
/etc/default/nfs-kernel-server
:
# Number of servers to start up RPCNFSDCOUNT=8 <snip>
Whether this is necessary can be judged from
cat /proc/net/rpc/nfsd|grep ^th
th 32 340905478 475057.300 442289.504 184302.020 1.312 95448.776 60302.696 43222.208 42993.392 0.000 224481.668
The last ten numbers form a histogram showing number of seconds at percentage full over all threads. In this case, 224481.668 seconds were spent at 90-100% full, so increasing the number of threads would seem justified. (Even though twice as much time was spent at 0-10% and 10-20%, it is peak load per thread that we want to reduce.)
List these URLs for documentation https://help.ubuntu.com/community/NFSv4Howto http://mailman.mit.edu/pipermail/kerberos/2008-May/013698.html https://help.ubuntu.com/9.04/serverguide/C/kerberos.html http://www.opinsys.fi/setting-up-nfsv4kerberos-on-ubuntu-10-04-alpha-2-lucid-part-6 http://www.dice.inf.ed.ac.uk/groups/services/file_service/docs/newfs-choice.html http://www.troubleshooters.com/linux/nfs.htm http://wiki.archlinux.org/index.php/NFSv4 http://www.citi.umich.edu/projects/nfsv4/crossrealm/libnfsidmap_config.html http://wiki.epfl.ch/icit/kb/linux-nfsv4-client https://we.riseup.net/stefani/kerberos-and-nfs4 https://bugs.launchpad.net/ubuntu/+source/nfs-utils/+bug/368153
Tighten security
Failover Kerberos
Copy LDAP accounts to Kerberos apprentice@nfs-client:~$ sudo apt-get install libpam-krb5-migrate-heimdal Reading package lists... Done Building dependency tree Reading state information... Done The following packages were automatically installed and are no longer required: libpulse-browse0 Use 'apt-get autoremove' to remove them. The following NEW packages will be installed: libpam-krb5-migrate-heimdal 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded. Need to get 11.2kB of archives. After this operation, 98.3kB of additional disk space will be used. Get:1 http://mirror.mydomain.com/ubuntu/ lucid/universe libpam-krb5-migrate-heimdal 0.0.9-1 [11.2kB] Fetched 11.2kB in 0s (617kB/s) Selecting previously deselected package libpam-krb5-migrate-heimdal. (Reading database ... 281535 files and directories currently installed.) Unpacking libpam-krb5-migrate-heimdal (from .../libpam-krb5-migrate-heimdal_0.0.9-1_amd64.deb) ... Processing triggers for man-db ... Setting up libpam-krb5-migrate-heimdal (0.0.9-1) ... apprentice@nfs-client:~$