How to build *BSD CVSup mirror
This document describes how to build *BSD (FreeBSD, NetBSD and OpenBSD)
CVSup mirror site.
Comments and suggestions are always welcome.
Contents
1. Hardware and Software Requirements
- CVSup binaries (cvsup and cvsupd)
- FreeBSD (i386 and alpha)
- use Ports Collection
(ports/net/cvsup-without-gui)
or binary package. - NetBSD (i386)
- Binary packages
are available. If you want to build CVSup binaries by yourself, use
pkgsrc. - OpenBSD (i386)
- use Ports Collection
(ports/net/cvsup)
or binary package (statically-linked/no-x11 binary package for
OpenBSD/i386 is available from
http://cvsup.jp.OpenBSD.org/).
- High performance disk storage
- FreeBSD
- 1.7GB for CVS repository, 3,2GB for all (CVS repository,
GNATS DB, mailing list archives, etc.) - FreeBSD Japan
- 0.4GB
- NetBSD
- 2.2GB
- NetBSD Japan
- 0.3GB
- OpenBSD
- 1.8GB
- Large amounts of physical memory
2. Setup
2.1. Common Setup
2.1.1. cvsup and cvsupd user
Example of /etc/master.passwd entries:
cvsup:*:10001:10001::0:0:CVSup Owner:/d/cvsup:/sbin/nologin cvsupd:*:10002:10002::0:0:CVSupd Owner:/d/cvsupd:/sbin/nologin
Example of /etc/group entries:
cvsup:*:10001: cvsupd:*:10002:
2.1.2. create directories
# mkdir /d/cvsup # cvsup home dir # mkdir /d/cvsupd # cvsupd home dir # chown root:wheel /d/cvsup /d/cvsupd # chmod 755 /d/cvsup /d/cvsupd # cd /d/cvsupd # mkdir prefixes # cvsupd collection (data) dir # mkdir scan # scan file dir # chown root:wheel prefixes scan # chmod 755 prefixes scan # cd /d/cvsup # ln -s ../cvsupd/scan sup # mkdir log # cvsup log dir # mkdir lock # cvsupd lock file dir # chown cvsup:cvsup /d/cvsup/log /d/cvsup/lock # chmod 755 /d/cvsup/log /d/cvsup/lock
2.2. Getting FreeBSD CVS repository using CVSup
2.2.1. create directory
# cd /d/cvsupd # mkdir prefixes/FreeBSD.cvs # CVS repository # mkdir prefixes/FreeBSD-gnats.current # GNATS DB # mkdir prefixes/FreeBSD-mail.current # mailing list archive # mkdir prefixes/FreeBSD-www.current # www.FreeBSD.org data # mkdir prefixes/distrib.self # CVSup config file # mkdir prefixes/FreeBSD-jp.cvs # JP CVS repository (for JP ONLY) # mkdir prefixes/FreeBSD-jp-distrib.self # CVSup config file (for JP ONLY) # chown cvsup:cvsup /d/cvsupd/prefixes/* # chmod 755 /d/cvsupd/prefixes/* # mkdir scan/cvs-all # mkdir scan/gnats # mkdir scan/www # mkdir scan/mail-archive # mkdir scan/distrib # mkdir scan/jp-all # for JP ONLY # mkdir scan/jp-distrib # for JP ONLY # chown cvsup:cvsup /d/cvsupd/scan/* # chmod 755 /d/cvsupd/scan/* # ln -s prefixes/distrib.self/sup sup # ln -s prefixes/jp-distrib.self/sup-jp sup-jp # for JP ONLY
2.2.2. create cvsup refuse file
Create cvsup refuse files (/d/cvsupd/scan/distrib/refuse.self
and /d/cvsupd/scan/jp-distrib/refuse.self) containing
the following contents. Please make sure that refuse files are owned by root
and their file modes are 0444.
*.sh cvsupd.access cvsupd.passwd prefixes sup.client supfile*
2.2.3. create supfile-freebsd
Create /d/cvsup/supfile-freebsd. The following is an example.
Please specify the nearest CVSup mirror site instead of
cvsup.jp.FreeBSD.org.
If you want to become an official FreeBSD CVSup mirror, you can access
CVSup master server. Please contact hub@FreeBSD.org
(cvsupadm@jp.FreeBSD.org if your site is in Japan).
# Supfile for FreeBSD CVSup mirror *default host=cvsup.jp.FreeBSD.org *default base=/d/cvsup *default delete use-rel-suffix umask=002 cvs-all release=cvs prefix=/d/cvsupd/prefixes/FreeBSD.cvs gnats release=current prefix=/d/cvsupd/prefixes/FreeBSD-gnats.current www release=current prefix=/d/cvsupd/prefixes/FreeBSD-www.current mail-archive release=current prefix=/d/cvsupd/prefixes/FreeBSD-mail.current distrib release=self prefix=/d/cvsupd/prefixes/distrib.self # following 2 lines are for JP mirror sites ONLY. jp-all release=cvs prefix=/d/cvsupd/prefixes/FreeBSD-jp.cvs jp-distrib release=self prefix=/d/cvsupd/prefixes/FreeBSD-jp-distrib.self
2.2.4. create update script
Create a shell script update-freebsd.sh. Please specify
S_TIME
so that cvsup runs without -s flag once a day.
#!/bin/sh
export PATH=/bin:/usr/bin:/usr/local/bin
export LANG=C
SUPFILE=/d/cvsup/supfile-freebsd
LOCK=/d/cvsup/lock/freebsd.lock
LOG=/d/cvsup/log/freebsd.log
ADMIN="your@address"
S_TIME=0430
if [ x`date "+%H%M"` = x$S_TIME ]; then
SFLAG=
else
SFLAG=-s
fi
(
echo
echo "========== CVSup $SFLAG started at `date`"
if ! cvsup $SFLAG -g -L 2 -r 1 -l $LOCK $SUPFILE; then
tail $LOG |\
mail -s "CVSup FreeBSD failed" $ADMIN
fi
echo "========== CVSup $SFLAG ended at `date`"
) >> $LOG
2.2.5. Execute update script
Become cvsup user (using sudo or su) and execute the update script.
If everything goes well, add an entry of the update script to crontab.
Most of FreeBSD cvsup mirror sites in Japan update their mirror at least
once an hour.
2.3. Getting NetBSD CVS repository using CVSup
2.3.1. create directory
# cd /d/cvsupd # mkdir prefixes/NetBSD.cvs # mkdir prefixes/NetBSD-jp.cvs # for JP ONLY # mkdir prefixes/NetBSD-distrib.self # chown cvsup:cvsup /d/cvsupd/prefixes/* # chmod 755 /d/cvsupd/prefixes/* # mkdir scan/netbsd # mkdir scan/netbsd-distrib # mkdir scan/netbsd-jp-all # for JP ONLY # mkdir scan/netbsd-jp-distrib # for JP ONLY # chown cvsup:cvsup /d/cvsupd/scan/* # chmod 755 /d/cvsupd/scan/* # ln -s prefixes/NetBSD-distrib.self/sup-netbsd sup-netbsd
2.3.2. create cvsup refuse file
Create cvsup refuse files
(/d/cvsupd/scan/netbsd-distrib/refuse.self and
/d/cvsupd/scan/netbsd-jp-distrib/refuse.self) containing
the following contents. Please make sure that refuse files are owned by root
and their file modes are 0444.
*.sh cvsupd.access cvsupd.passwd prefixes sup.client supfile*
2.3.3. create supfile-netbsd
Create /d/cvsup/supfile-netbsd.
# Supfile for NetBSD CVSup mirror *default host=cvsup.jp.NetBSD.org *default base=/d/cvsup *default delete use-rel-suffix umask=002 netbsd release=cvs prefix=/d/cvsupd/prefixes/NetBSD.cvs netbsd-distrib release=self prefix=/d/cvsupd/prefixes/NetBSD-distrib.self # the following 2 lines are for JP mirror sites ONLY. netbsd-jp-all release=cvs prefix=/d/cvsupd/prefixes/NetBSD-jp.cvs netbsd-jp-distrib release=self prefix=/d/cvsupd/prefixes/NetBSD-distrib.self
2.3.4. create update script
Create a shell script update-netbsd.sh. Please specify S_TIME
so that cvsup runs without -s flag once a day.
#!/bin/sh
export PATH=/bin:/usr/bin:/usr/local/bin
export LANG=C
SUPFILE=/d/cvsup/supfile-netbsd
LOCK=/d/cvsup/lock/netbsd.lock
LOG=/d/cvsup/log/netbsd.log
ADMIN="your@address"
S_TIME=0430
if [ x`date "+%H%M"` = x$S_TIME ]; then
SFLAG=
else
SFLAG=-s
fi
(
echo
echo "========== CVSup $SFLAG started at `date`"
if ! cvsup $SFLAG -g -L 2 -r 1 -l $LOCK $SUPFILE; then
tail $LOG |\
mail -s "CVSup NetBSD failed" $ADMIN
fi
echo "========== CVSup $SFLAG ended at `date`"
) >> $LOG
2.3.5. Execute update script
Become cvsup user (using sudo or su) and execute the update script.
If everything goes well, add an entry for the update script to crontab.
cvsup.jp.NetBSD.org updates its contents every 8 hours.
2.4. Getting OpenBSD CVS repository using CVSup
2.4.1. create directory
# cd /d/cvsupd # mkdir prefixes/OpenBSD.cvs # mkdir prefixes/OpenBSD-distrib.self # chown cvsup:cvsup /d/cvsupd/prefixes/* # chmod 755 /d/cvsupd/prefixes/* # mkdir scan/OpenBSD-all # mkdir scan/OpenBSD-distrib # chown cvsup:cvsup /d/cvsupd/scan/* # chmod 755 /d/cvsupd/scan/* # ln -s prefixes/OpenBSD-distrib.self/sup-openbsd sup-openbsd
2.4.2. create cvsup refuse file
Create cvsup refuse files
(/d/cvsupd/scan/OpenBSD-distrib/refuse.self) containing
the following contents. Please make sure that refuse files are owned by root
and their file modes are 0444.
*.sh cvsupd.access cvsupd.passwd prefixes sup.client supfile*
2.4.3. create supfile-openbsd
Create /d/cvsup/supfile-openbsd.
If you want to become an official OpenBSD CVSup mirror, please contact
brian@openbsd.org or beck@openbsd.org. OpenBSD does not
have the central CVSup server, so you have to use sup to update
the repository from an official root server. If you want to use
cvsup.jp.OpenBSD.org as a master server instead,
please contact cvsupadm@openbsd.bsdlab.org.
# Supfile for OpenBSD CVSup mirror *default host=cvsup.jp.OpenBSD.org *default base=/d/cvsup *default delete use-rel-suffix umask=002 OpenBSD-all release=cvs prefix=/d/cvsupd/prefixes/OpenBSD.cvs # NOTE: -distrib collection is available from cvsup.jp.OpenBSD.org only OpenBSD-distrib release=self prefix=/d/cvsupd/prefixes/OpenBSD-distrib.self
2.4.4. create update script
Create shell script update-openbsd.sh. Please specify
S_TIME
so that cvsup runs without -s flag once a day.
#!/bin/sh
export PATH=/bin:/usr/bin:/usr/local/bin
export LANG=C
SUPFILE=/d/cvsup/supfile-openbsd
LOCK=/d/cvsup/lock/openbsd.lock
LOG=/d/cvsup/log/openbsd.log
ADMIN="your@address"
S_TIME=0430
if [ x`date "+%H%M"` = x$S_TIME ]; then
SFLAG=
else
SFLAG=-s
fi
(
echo
echo "========== CVSup $SFLAG started at `date`"
if ! cvsup $SFLAG -g -L 2 -r 1 -l $LOCK $SUPFILE; then
tail $LOG |\
mail -s "CVSup OpenBSD failed" $ADMIN
fi
echo "========== CVSup $SFLAG ended at `date`"
) >> $LOG
2.4.5. Execute update script
Become cvsup user (using sudo or su) and execute the update script.
If everything goes well, add an entry for the update script to crontab.
cvsup.jp.OpenBSD.org updates its CVS repository every 2 hours.
For more information, please refer to
http://cvsup.jp.OpenBSD.org/.
2.5. Setup CVSup daemon
2.5.1 Create cvsupd.access
Create a CVSup daemon access control file /d/cvsupd/cvsupd.access.
A sample is as follows:
# ============================================================ # Limit 1 connection at a time for each host # ============================================================ -0.0.0.0/0/32 1 # ============================================================ # Always allow access from localhost (or ssh port forwarding) # ============================================================ +localhost # ============================================================ # Limit 10 connections total at any time # ============================================================ -0.0.0.0/0 10 # ============================================================ # Allow anonymous access # ============================================================ +0.0.0.0/0
2.5.2 logging
When “-l” flag is specified in the form of
“-l @local0,”
the logging is done via syslog to the local0 facility.
This is a recommended way since it makes the maintenance easier.
Probably you may want to add the following lines into
/etc/syslog.conf in order to separate log messages
generated by cvsupd processes from others.
!cvsupd *.* /var/log/cvsupd.log
Also, the default syslog.conf files in *BSD
include “*” that matches all facilities, for example:
*.notice;kern.debug;lpr.info;mail.crit;news.err /var/log/messages
When such lines are included, “local0″ messages will also be
logged in /var/log/messages. To prevent that behavior,
you can add “local0.none” into the lines where the
“*” is used:
*.notice;kern.debug;lpr.info;mail.crit;news.err;local0.none /var/log/messages
For more information, see syslog.conf(5).
2.5.3 create cvsupd startup script
An example of a startup script (/d/cvsupd/cvsupd.sh)
is as follows:
#!/bin/sh
export PATH=/usr/bin:/usr/local/bin
export LANG=C
exec nice -10 cvsupd -C 128 -l @local0 -b /proj/cvsupd -s scan \
-c sup:sup-jp:sup-netbsd:sup-openbsd
Add the following lines into /etc/rc.local:
[ -x /proj/cvsupd/cvsupd.sh ] && \
su -m cvsupd -c /proj/cvsupd/cvsupd.sh && echo -n ' cvsupd'
If your server is run by FreeBSD, you may use another startup script
that follows the framework of FreeBSD’s local startup scripts.
Here is a sample of /usr/local/etc/rc.d/cvsupd.sh:
#!/bin/sh
case "$1" in
start)
[ -x /d/cvsupd/cvsupd.sh ] || exit 1
su -m cvsupd -c /d/cvsupd/cvsupd.sh && echo -n ' cvsupd'
;;
stop)
killall cvsupd && echo -n ' cvsupd'
;;
esac
3. Getting server stats
Not yet.
4. Tuning CVSup server
Not yet.
5. Security
5.1. cvsup
CVSup client (cvsup) needs write permission of the following directories.
Make sure that there is no file or directory (except the following
directories) that cvsup user can write.
- CVS repository directories (
/d/cvsupd/prefixes/*) - scan files directory (
/d/cvsupd/scan/*) - log files directory (
/d/cvsup/log) - lock files directory (
/d/cvsup/lock)
5.2. cvsupd
CVSup daemon (cvsupd) does NOT create and/or write any file. Make sure
that there is no file or directory that cvsupd user can write.