System Departmant Internship - İTÜ · System Departmant Internship 1st Week Report August 8, 2010...

System Departmant Internship

1st Week Report

www.cscrs.itu.edu.tr

August 8, 2010

Zemre Arslan [email protected]

MPICH2 Implementation on Virtual Machines by Zemre Arslan is licensed under Creative Commons

Attribution-NonCommercial-NoDerivs 3.0 Unported License.

www.cscrs.itu.edu.tr

[email protected]

http://www.students.itu.edu.tr/~arslanz/Staj%20Rapor%201.pdf

http://creativecommons.org/licenses/by-nc-nd/3.0/



MPICH2 Implementation on Virtual Machines

Contents

1. Creating Virtual Machine with Oracle VM VirtualBox

1.1 What is virtual machine?

1.2 Setting up Oracle VM VirtualBox

1.3 Configuring Ubuntu 10.04 on VirtualBox

2. MPICH2 Implementation

2.1 What is MPICH2?

2.2 Prerequisites

2.3 Setting up MPICH2

3. References

1. Creating Virtual Machine with Oracle VM VirtualBox

1.1 What is a virtual machine?

Virtual machine is a software that emulates the operating systems. It can be set to

any place in your current system without requiring any partition on your disk. Virtual

machine applications supports many OS, and the chosen OS can be installed via a CD

or from an image file. Through virtual machine platforms people can execute

programs just like a physical machine. There are many virtual machine applications

have been released so far, such as Vmware Server, VirtualPC, Oracle VM

VirtualBox...etc. We are going to use Oracle VirtualBox while implementing MPICH2

on a linux system.


Attribution-NonCommercial-NoDerivs 3.0 Unported License





1.2 Setting up Oracle VM VirtualBox

VirtualBox is easy to use and set up. It can be downloaded from the link below:

http://www.virtualbox.org/wiki/Downloads

You can choose the version of the VirtualBox depending on your current Operating System.

With regard to our Operating system, we’ve downloaded VirtualBox 3.2.8 for Windows

hosts release.

Since we are going to set up two Virtual Linux machines, we need to check our system if it

supplies the basic prerequisites:

OS: Windows 7 Professional

Installed memory (RAM) : 4GB

Processor: Intel(R) Core(TM)2 Duo CPU p8700 @2.53GHz

System type: 32-bit Operating System

That is substantial to create virtual machines, nevertheless you may set up virtual machines

even you have a nominal-featured system.



http://www.virtualbox.org/wiki/Downloads





1.3 Configuring Ubuntu 10.04 on VirtualBox

In order to create a new virtual machine, we press the red-highligted rectangle “Yeni” or

for english releases “New” button at the top-left of the window.

We are going to have two Virtual Linux machines to implement MPICH2. Let’s say Node0 and Node1

since one of them is the master node –Node 0-







Now here, we are going to choose the appropriate OS from the window shown below which is Linux

Ubuntu.

The next step is about the RAM that you are going to need for a Linux OS. We reserve 512 MB of our

RAM which is fairly enough for one virtual machine.







The other important point here choosing the image type of virtual disk. We face two alternatives ;

one of them is Dynamically expanding image and the other one is Fixed size image.

What does all that mean?

If Dynamically Expanding Image is chosen, the disk image file is created at a minumum size and then

according to the requirements it grows dynamically. This may be advantageous since disk space is

not used if not needed.

If Fixed Size Image is chosen, the disk space is allocated previously and there is no dynamic structure

here. This is more efficient which results good performance.

So here we choose to use fixed size image, both alternatives can be taken into consideration

depending on the requirements.







The next step is about deciding the disk size, we here reccomend about 8GB HDD for the optimum

performance. If you have a partiton in your HDD, it is highly reccomended to choose the other

partition (which there is no OS installed on it e.g. “D: disk” )

Now it is required to show an OS image or a CD/DVD to install Ubuntu 10.04-desktop-i386. It can be

downloaded from the link http://www.ubuntu.com/desktop/get-ubuntu/download.

Now we have an iso image to install Ubuntu. We need to show this to the virtual machine. When we

press the button “Start” , a window like below will be showed up:



http://www.ubuntu.com/desktop/get-ubuntu/download





After all these operations, we are going to pass to the next step: The installation. Desired language

and the appropriate time zone should be chosen. After that the partioning part is coming. Here we

have some suggestions:







We choose “Specify partitions manually (advanced). As seen below, we have approximately 8.6 GB

free space to partition. Red- highlighted below, press the Add button.

Here we need three partitions.

First one is about 100 MB primary partition which uses ext2 file format and the mount point is /boot;







Why do we need this boot partition?

This is important because system uses it to process the recovery in case of an unwanted situation. It

is not technically a must, but required.

Second partition is the swap partition. Either a primary or logical partition can be used for swap area.

Here we choose primary. Swap space is used when the RAM is full and the system needs more

memory. The size of your swap space should be equal to twice of your computer's RAM, but no more

than 2GB. We have 1024 MB swap area.

And thirdly, we need approximately 7.5 GB logical partition, ext3 file format.







After completing the installation we need to set the boot priority. That is pretty simple. We should

set the Boot order to Hard Disk –here “Sabit Disk”- as shown below:

Now completed the easy part and passing the MPICH2 installation. But before that make sure you

have at least two nodes to communicate each other. (Meaning that create two virtual machines

following the steps explained above!)







2. MPICH2 Implementation

2.1 What is MPICH2?

MPICH2 is an application to use th MPI programs easily designed by Argonne National Laboratory.

2.1 Prerequisites (what do we need to install mpich2)

1. MPICH2-1.2.1p1.tar.gz This can be downloaded from the link :

http://www.mcs.anl.gov/research/projects/mpich2/downloads/index.php?s=downlo

ads

2. A C compiler //It will be mentioned later

3. Unix based operating system //We choose Ubuntu 10.04

Changing the ip adress from DHCP to static

We are going to make the nodes’ ips static. We have two nodes with the hostnames UCG-15

and UCG-16. UCG-15 will get the ip 10.0.0.101 and UCG-16 will get the ip 10.0.0.102

Firstly we run #sudo pico /etc/network/interfaces command. Since we have a virtual

machine we see the following lines:

auto lo

iface lo inet loopback

We need to change it like this:

auto eth0

iface eth0 inet static

address 10.0.0.101

netmask 255.255.255.0

gateway 10.0.0.1

Of course you should write your own default gateway and netmask.



http://www.mcs.anl.gov/research/projects/mpich2/downloads/index.php?s=downloads






After all these we restart the networking in order to make your changes stable with

#sudo /etc/init.d/networking restart command.

Now we need to modify the hosts file, the commands are like below:

#sudo pico /etc/hosts

And edited the file like this:

127.0.0.1 localhost

10.0.0.101 UCG-15

10.0.0.102 UCG-16

This is an important step since we are creating a cluster, we could be able to ping or

ssh each other with their hostnames or ip’s.

Now we need NFS (Network File System) on the master node. This will allow us to

share a folder or file on the master node with all the other nodes.

#sudo apt-get install nfs-kernel-server command should be run on the master node.

In Node 0 we create a folder named mpich2 under /home/mpiuser to share the files.

Then we mount the slave node to the master node with the command #sudo mount UCG-15:/home/mpiuser/mpich2 /home/mpiuser/mpich2 //Here UCG-15 is the master node (Node0).

In order to make the nodes communicate each other we should be able to test the

ssh.







So we install ssh server to all nodes with #sudo apt-get install openssh-server

command. This will generate a new ssh key. We will be asked to enter a paraphrase.

We leave it blank since the aim is to create a passwordless ssh.

Next, it will inform you that the key created.

Under .ssh folder there is a file id_dsa.pub, this is the file that contains the dsa key.

We open it with an editor running the command

# sudo pico /etc/mpiuser/.ssh/id_dsa.pub

And then we copy the key into the authorized_keys file in the same directory. Now

we have a passwordless ssh.







At the beginning we have said that we need a compiler. With #sudo apt-get install build-

essential command gcc compiler and the other essentials are coming.

Now we are installing the MPICH2 from the link

http://www.mcs.anl.gov/research/projects/mpich2/downloads/index.php?s=downlo

ads

MPICH2-1.2.1p1.tar.gz file is placed in /home/mpiuser/Downloads by default. You can choose your place before the download. Now we should untar this file, you can do this with # tar xvf mpich2-1.2.1p1.tar.gz command. We did it with a right click and chose to extract. After that the configuration should be done.

# ./configure --prefix=/home/mpiuser/mpich2 # make # sudo make install Here, it is important that MPICH2 must be installed on all the nodes; in order not to confuse we reccommend to do the tasks simultaneously. Under the file /home/mpiuser/ mpich2 .bashrc the following lines must be written: export PATH=/home/mpiuser/ mpich2/bin:$PATH export PATH LD_LIBRARY_PATH="//home/mpiuser/ mpich2/lib:$LD_LIBRARY_PATH" export LD_LIBRARY_PATH Next we run this: #sudo echo /mirror/mpich2/bin >> /etc/environment To test if the mpi is running follow the commands: # which mpd # which mpiexec # which mpirun If these all work correctly that means mpi is running correctly. Now it is possible to make operations with mpd.









4. References

https://help.ubuntu.com/community/MpichCluster

http://www.redhat.com/docs/manuals/linux/RHL-9-Manual/custom-guide/

http://www.mcs.anl.gov/research/projects/mpich2/documentation/files/mpich2-1.2.1-

installguide.pdf

http://tldp.org/HOWTO/Partition/requirements.html#number

http://www.ubuntugeek.com/change-ubuntu-system-from-dhcp-to-a-static-ip-address.html

http://www.cyberciti.biz/faq/ubuntu-linux-configure-gateway-ip/

http://www.ipogren.com/k-terminal-komut-sistemi/linux_ip-network_komutlar%E4%B1/







System Departmant Internship - İTÜ · System Departmant Internship 1st Week Report August 8, 2010...

Documents

Transcript of System Departmant Internship - İTÜ · System Departmant Internship 1st Week Report August 8, 2010...