Albert's Useful Computer Tips: February 2013

Monday, February 25, 2013

image convert in Linux

convert -colorspace RGB -interlace none -density 300x300 -quality 100 mycode_pbs-crop.pdf mycode_pbs-crop.png

To see the status of all nodes:
$ pbsnodes -a

To see down nodes only
$ pbsnodes -l

====================

pbsnodes(8B) PBS pbsnodes(8B)

NAME
pbsnodes - pbs node manipulation

SYNOPSIS
pbsnodes [-{a|x}] [-q] [-s server] [node|:property]
pbsnodes -l [-q] [-s server] [state] [nodename|:property ...]
pbsnodes [-{c|d|o|r}] [-q] [-s server] [-n] [-N "note"] [node|:property]

DESCRIPTION
The pbsnodes command is used to mark nodes down, free or offline. It can
also be used to list nodes and their state. Node information is obtained
by sending a request to the PBS job server. Sets of nodes can be operated
on at once by specifying a node property prefixed by a colon.
Nodes do not exist in a single state, but actually have a set of states.
For example, a node can be simultaneously "busy" and "offline". The
"free" state is the absence of all other states and so is never combined
with other states.

In order to execute pbsnodes with other than the -a or -l options, the
user must have PBS Manager or Operator privilege.

OPTIONS
-a All attributes of a node or all nodes are listed. This is
the default if no flag is given.

-x Same as -a, but the output has an XML-like format.

-c Clear OFFLINE from listed nodes.

-d Print MOM diagnosis on the listed nodes. Not yet imple-
mented. Use momctl instead.

-o Add the OFFLINE state. This is different from being marked
DOWN. OFFLINE prevents new jobs from running on the speci-
fied nodes. This gives the administrator a tool to hold a
node out of service without changing anything else. The
OFFLINE state will never be set or cleared automatically by
pbs_server; it is purely for the manager or operator.

-p Purge the node record from pbs_server. Not yet imple-
mented.

-r Reset the listed nodes by clearing OFFLINE and adding DOWN
state. pbs_server will ping the node and, if they communi-
cate correctly, free the node.

-l List node names and their state. If no state is specified,
only nodes in the DOWN, OFFLINE, or UNKNOWN states are
listed. Specifying a state string acts as an output fil-
ter. Valid state strings are "free", "offline", "down",
"reserve", "job-exclusive", "job-sharing", "busy", "time-
shared", or "state-unknown".

-N Specify a "note" attribute. This allows an administrator
to add an arbitrary annotation to the listed nodes. To
clear a note, use -N "" or -N n.

-n Show the "note" attribute for nodes that are DOWN, OFFLINE,
or UNKNOWN. This option requires -l.

-q Supress all error messages.

-s Specify the PBS server’s hostname or IP address.

SEE ALSO
pbs_server(8B) and the PBS External Reference Specification

Local pbsnodes(8B)

Sunday, February 24, 2013

Efficient yum installation

Make a directory such as /etc/yum/Group, which is shared to all the nodes using NFS.
Download package files:

yum groupinstall -y 'Development Tools' --downloadonly --downloaddir=/opt/yum/Group
yum update --downloadonly --downloaddir=/opt/yum/ openmpi openmpi-devel openmpi-devel.x86_64 qt-devel cc gcc-c++ bison ncurses-devel tix.x86_64 glibc-devel flex zlib-devel libXt-devel binutils-devel gcc44 gcc44-c++ zlib-devel

If "--downloadonly" option does not work, then

yum install -y yum-downloadonly

To install downloaded package at the master as well as compute nodes:

yum localinstall -y /opt/yum/Group/*.rpm

If you get errors regarding yum execution, try

yum clean dbcache

yum download

Type the following command to install plugin, enter:

# yum install yum-downloadonly

To download http only:

# yum update httpd -y --downloadonly

Saturday, February 23, 2013

How to install OpenFOAM on CentOS

Package Installation

sudo yum groupinstall -y 'Development Tools'
sudo yum install -y openmpi openmpi-devel openmpi-devel.x86_64 qt-devel
sudo yum install -y cc gcc-c++ bison ncurses-devel tix.x86_64 glibc-devel flex zlib-devel libXt-devel binutils-devel
sudo yum install -y gcc44 gcc44-c++ zlib-devel
yum upgrade -y

Check your OS architecture to forcefully load Open-MPI into the environment

uname -m

If 32 bit:

mpi-selector --system --set openmpi-1.4-gcc-i386

If 64 bit:

mpi-selector --system --set openmpi-1.4-gcc-x86_64

In my case, "openmpi-1.4-gcc-x86_64" is not listed by executing "mpi-selector --list". To register openmpi-1.4-gcc-x86_64:

mpi-selector --register openmpi-1.4-gcc-x86_64 --source-dir /usr/lib64/openmpi/1.4-gcc/etc

Log out and log in again. If you did step 1 and 2 as root, at this time log in as a regular user.
Download and unpack

cd ~
mkdir OpenFOAM
cd OpenFOAM
wget "http://downloads.sourceforge.net/foam/OpenFOAM-2.1.1.tgz?use_mirror=mesh" -O OpenFOAM-2.1.1.tgz
wget "http://downloads.sourceforge.net/foam/ThirdParty-2.1.1.tgz?use_mirror=mesh" -O ThirdParty-2.1.1.tgz
tar xzf OpenFOAM-2.1.1.tgz
tar xzf ThirdParty-2.1.1.tgz

For X86 #here you can change 4 to the number of cores you've got

source $HOME/OpenFOAM/OpenFOAM-2.1.1/etc/bashrc WM_NCOMPPROCS=4 WM_COMPILER=Gcc44 WM_MPLIB=SYSTEMOPENMPI
export WM_CC='gcc44'
export WM_CXX='g++44'
echo "source $HOME/OpenFOAM/OpenFOAM-2.1.1/etc/bashrc $FOAM_SETTINGS" >> $HOME/.bashrc
echo "export WM_CC='gcc44'; export WM_CXX='g++44'" >> $HOME/.bashrc
sed -i -e 's=gcc=$(WM_CC)=' OpenFOAM-2.1.1/wmake/rules/linux64Gcc44/c
sed -i -e 's=g++=$(WM_CXX)=' OpenFOAM-2.1.1/wmake/rules/linux64Gcc44/c++

Now let's build OpenFOAM. It usually takes 30 minutes to 6 hours.

#Go into OpenFOAM's main source folder
- cd OpenFOAM-2.1.1
# This next command will take a while... somewhere between 30 minutes to 3-6 hours.

./Allwmake > make.log 2>&1

#Run it a second time for getting a summary of the installation

./Allwmake > make.log 2>&1

Preliminary
OpenFOAM uses libmpi.so.1. If this is not found in /usr/lib64/openmpi/1.4-gcc/lib, then do followings

Log in as root
cd /usr/lib64/openmpi/1.4-gcc/lib
ln -s libmpi.so libmpi.so.1

This is to make a symbolic link from libmpi.so to libmpi.so.1, which should be done all the nodes if OpenFoam will be run in parallel.

Wednesday, February 20, 2013

UseDNS no in sshd_config

"It turns out there is a setting in OpenSSH that controls whether SSHd should not only resolve remote host names but also check whether the resolved host names map back to remote IPs. Apparently, that setting is enabled by default in OpenSSH. The directive UseDNS controls this particular behaviour of OpenSSH, and while it is commented in sshd_config (which is the default configuration file for the OpenSSH daemon in most enviornments), as per the man page for sshd_config, the default for UseDNS is set to enabled. Uncommenting the line carrying the UseDNS directive and setting it to “no” disables the feature."

http://ayaz.wordpress.com/2008/02/03/usedns-resolution-and-openssh/

So to speed up connection speed of OpenSSH, in /etc/ssh/sshd_config file, put

UseDNS no

Tuesday, February 19, 2013

PBS scripts for mpirun

#!/bin/bash
#PBS -l walltime=00:30:00
#PBS -l nodes=3:ppn=2
#PBS -N mypi
#PBS -V
cd $PBS_O_WORKDIR
/opt/openmpi-1.6.3-intel-pbs/bin/mpirun -mca btl_tcp_if_include eth0 -mca btl tcp,self -mca plm tm -np 6 fpi.x

bash: /usr/bin/ssh: Permission denied

This message has been a headache, happening a couple of times when I upgrade or update packages.

albertsk@stokes:~$ ssh
bash: /usr/bin/ssh: Permission denied

root@stokes:ssh$ ll /usr/bin/wget
-rwxr-xr-x 1 root root 369848 Feb 10 2012 /usr/bin/wget*

root@stokes:ssh$ chmod og+x /usr/bin/ssh
chmod: changing permissions of `/usr/bin/ssh': Operation not permitted

root@stokes:ssh$ ll /usr/bin/ssh
-rwx------ 1 root root 650492 Feb 12 01:46 /usr/bin/ssh*

root@stokes:ssh$ chmod og+x /usr/bin/ssh
chmod: changing permissions of `/usr/bin/ssh': Operation not permitted

root@stokes:ssh$ lsattr /usr/bin/ssh
-u---a-------e- /usr/bin/ssh

root@stokes:ssh$ lsattr /usr/bin/wget
-------------e- /usr/bin/wget

root@stokes:ssh$ chattr -uai /usr/bin/ssh

root@stokes:ssh$ lsattr /usr/bin/ssh
-------------e- /usr/bin/ssh

root@stokes:ssh$ chmod og+xr /usr/bin/ssh
root@stokes:ssh$ ls /usr/bin/ssh
/usr/bin/ssh

albertsk@stokes:~$ ssh localhost

Last login: Tue Feb 19 15:10:25 2013 from localhost

Welcome to Ubuntu 12.04.2 LTS (GNU/Linux 3.2.0-37-generic x86_64)

* Documentation: https://help.ubuntu.com/

albertsk@stokes:~$

Monday, February 18, 2013

customized .bst file

/usr/share/texmf-texlive/bibtex/bst/natbib/

there are many .bst files, which one can modify.

After type/enter

$ texhash

Thursday, February 14, 2013

How to install Openmpi using ifort and icc

$ ./configure --with-tm=/usr/include/torque/ --prefix=/opt/openmpi-1.6.3-intel-pbs CC=icc CXX=icpc F77=ifort FC=ifort

$ make all install

openssh reinstallation

sudo apt-get remove --purge openssh-server

sudo apt-get install openssh-server

resolve memtest86+ (4.20-1.1ubuntu1) hangs

Quick workaround:

Append the following to /etc/default/grub:

GRUB_DISABLE_OS_PROBER=true

Run

$ sudo update-grub2

This will disable os-prober so grub doesn't see the other disks (this way it won't fail with an error).
Run sudo dpkg --configure -a to complete the configuration (should be successful this time).
The side-effect of this workaround is that you will loose all other boot entries in GRUB, so if you have a Windows dual boot or other distro's, you should remove or comment the line in /etc/default/grub and then run sudo update-grub2 once more to fully populate the GRUB menu.

How to resolve “Could not get lock /var/lib/dpkg/lock” problem.

Delete two "lock" files.

sudo rm /var/lib/dpkg/lock

sudo rm /var/cache/apt/archives/lock

Albert's Useful Computer Tips

Search This Blog