Installation of various Astro-WISE Servers¶
This part describes how to setup and configure the Astro-WISE Servers.
The Database Viewer¶
The database viewer accepts on the command line a host name and (after switch -p) a port number. At the moment it can only listen on one ethernet interface and on one port number. The default hostname is localhost and the default port number is 8879. To start it from the command line one could type:
awe $AWEPIPE/Services/dbview/dbview.py dbv.aoc.astro-wise.org -p 8879
You can stop the process by sending it the INT signal. It can be started automatically via the following script:
#!/bin/sh
AWEPIPE=/home/astro-wise/awehome/opipe/current
awe=/home/astro-wise/local/bin/awe
host=dbv.aoc.astro-wise.org
port=8879
#
case $1 in
start)
p=${AWEPIPE}/Services/dbview/dbview.py
if [ -f ${p} ] ; then
${awe} -u ${p} -p ${port} ${host} > ${HOME}/dbview.log 2>&1 &
fi
;;
status)
pid=`ps -def | grep awe | awk '{if{$1=="'dbview'"&&$3=="1")print $2}'`
if [ "X${pid}" = "X" ] ; then
echo "Down"
else
echo "Up (pid = ${pid})"
fi
;;
stop)
pid=`ps -def | grep awe | awk '{if($1=="'dbview'"&&$3=="1")print $2}'`
if [ ! "X${pid}" = "X" ] ; then
kill ${pid}
fi
;;
*)
echo "Usage: $0 (start|stop|status)"
;;
esac
See the last section in this chapter for a sample script to start the database viewer at system start.
The Dataservers¶
The dataservers usually listen on port 8000 and can listen to any ethernet device. The example startup script below searches for all available ethernet devices and passes them on to the dataserver. The dataservers react on signal HUP (stop) and USR1 (rescan disk). Usually the fully qualified domainnames of the dataservers are put in the Astro-WISE DNS and point to all the local dataservers (like for example ds.astro.rug.astro-wise.org).
The following script will start a dataserver listening on all possible ethernet devices. It is assumed that the data server directory is in a subdirectory data of a directory which starts with the first three letters of the hostname, and the last two letters of the hostname (which is a sequence number in Groningen).
#!/bin/sh -f
AWEPIPE=/home/astro-wise/AWEHOME/current
awe=/home/astro-wise/AWEHOME/x86_64/local/bin/awe
mp='kgb@astro-wise.org'
log='ds.log'
pid='ds.pid'
dir=`hostname | awk -F. '{printf("%s%s\n",substr($1,1,3),substr($1,length($1)-1,2))}'`
port=8000
if [ ! -d ${dir}/data ] ; then
echo "Directory ${dir}/data missing"
exit 1
fi
args=`/sbin/ifconfig | awk '
BEGIN{e="";}
{
if (substr($0,1,3)=="eth") {
e=$1
} else {
if (e!=""&&$1=="inet"&&substr($2,1,5)=="addr:") {
printf("%s:'${port}'\n",substr($2,6,length($2)-5));
e=""
}
}
}
'`
cd ${dir}/data
case "$1" in
start)
if [ -f ${log} ] ; then
if [ -f ${log}.save ] ; then
cat ${log} >> ${log}.save
rm -f ${log}
else
mv ${log} ${log}.save
fi
fi
${awe} -u ${AWEPIPE}/common/net/dataserver_server.py ${args} \
-p ${pid} -m ${mp} > ${log} 2>&1 &
sleep 5
while [ ! -e ${pid} ] ; do
sleep 1
done
;;
stop)
if [ -e ${pid} ] ; then
kill -HUP `cat ${pid}`
while [ -e ${pid} ] ; do
sleep 1
done
fi
;;
reload)
if [ -e ${pid} ] ; then
kill -USR1 `cat ${pid}`
fi
;;
status)
${awe} -u ${AWEPIPE}/common/net/dataserver_server.py ${args} -status
;;
*)
echo "Usage: $0 (start|stop|status|reload)"
;;
esac
exit 0
The Distributed Processing Server¶
The distributed processing server can run on an openpbs managed cluster or on a unmanaged cluster. A script to start the DPU server could look like this:
#!/bin/sh
#
# dpu-server This shell script starts and stops the dpu server
#
# dpu-server: 345 98 90
#
# description: Astro-WISE dpu server.
#
# probe: true
port=9000
host=hpcibm1.service.rug.nl
tunnel=astro-wise@omegadlt.astro.rug.nl
infofile=.`basename $0`.info
logfile=`basename $0`.log
AWEHOME=${HOME}/awehome
AWEPIPE=${AWEHOME}/opipe/current
cd ${HOME}/dpu
export AWEHOME
export AWEPIPE
case "$1" in
start)
if [ "X${tunnel}" != "X" -a \
-z "`ps -def | grep ${port}:${host}:${port} | grep ssh`" ] ; then
ssh ${tunnel} -fN -R ${port}:${host}:${port}
fi
if [ ! -e ${infofile} ] ; then
awe -u ${AWEPIPE}/common/net/dpu_server.py hpcibm1.cluster:${port} \
${host}:${port} -pbs short,200 -ppn 2 > ${logfile} 2>&1 &
fi
;;
stop)
if [ -e ${infofile} ] ; then
pid=`awk '{if (NR == 3){print $0}}' ${infofile}`
kill -HUP ${pid}
while [ -e ${infofile} ] ; do
sleep 1
done
fi
;;
status)
if [ -e ${infofile} ] ; then
cat ${infofile}
else
echo Down
fi
;;
restart)
if [ -e ${infofile} ] ; then
pid=`awk '{if (NR == 3){print $0}}' ${infofile}`
kill -INT ${pid}
while [ -e ${infofile} ] ; do
sleep 1
done
awe -u ${AWEPIPE}/common/net/dpu_server.py hpcibm1.cluster:${port} \
${host}:${port} -pbs short,200 -ppn 2 > ${logfile} 2>&1 &
fi
;;
kill)
if [ -e ${infofile} ] ; then
pid=`awk '{if (NR == 3){print $0}}' ${infofile}`
kill -KILL ${pid}
while [ -e ${infofile} ] ; do
sleep 1
done
fi
;;
*)
echo $"Usage: $0 (start|status|stop)"
;;
esac
exit 0
This DPU server is running on a 200 node 2 processors per node cluster. When the DPU server starts up it generates a unique key based on the hostname and current directory. This key is needed for the clients to be able to get a secure connection to the DPU server. With the script dbdpu.py in the Toolbox directory an entry in the DPU_SERVER_TABLE can be made.
At the moment the DPU server cannot run on an unmanaged cluster, work is still in progress.
Sample startup script¶
For some services it will be necessary that they are started at system startup and stopped at system shutdown. Below is a sample script which could for example be used on a RedHat Linux distribution to startup an Astro-WISE service.
#!/bin/sh
#
# purpose: This shell script takes care of starting and stopping
# an Astro-WISE service. It is assumed that there
# is a script available somewhere in the searchpath of the
# specified user account with same name as this script
# which is capable of stopping and starting the service.
#
# chkconfig: 345 98 90
# description: Astro-WISE server starter/stopper
#
# probe: true
#
usr=astro-wise
exe=`basename $0`
case "$1" in
start)
su -l ${usr} -c "${exe} start"
sleep 2
;;
stop)
su -l ${usr} -c "${exe} stop"
sleep 2
;;
status)
su -l ${usr} -c "${exe} status"
sleep 2
;;
*)
echo "Usage: $0 (start|status|stop)"
;;
esac
exit 0
Note that this is a secure script, it is NOT starting the service as root!
