Toaster at 20:30, 11 September 2008

2008-09-11T20:30:58Z

← Older revision		Revision as of 20:30, 11 September 2008
Line 1:		Line 1:

	==Disk Config==		==Disk Config==

Line 156:		Line 155:
	Condor will be installed in /export/condor/condor-6.6.10 with a symlink created to /export/condor/condor. All scripts and such will use the symlink to aid in an easier upgrade. To make upgrades a truly copy binaries, the condor_config sits in /export/condor/condor_config. The environment CONDOR_CONFIG should be set so that condor can find this file.		Condor will be installed in /export/condor/condor-6.6.10 with a symlink created to /export/condor/condor. All scripts and such will use the symlink to aid in an easier upgrade. To make upgrades a truly copy binaries, the condor_config sits in /export/condor/condor_config. The environment CONDOR_CONFIG should be set so that condor can find this file.

	* [[Media:~~condor~~-install.log\|Installation Log]]		* [[Media:Condor-install.log\|Installation Log]]

	In ~condor/condor-6.6.10		In ~condor/condor-6.6.10

Toaster at 20:30, 11 September 2008

2008-09-11T20:30:08Z

New page

==Disk Config==

Disk space on terpcondor was snagged by umounting /tmp and setting it under the management of lvm.

Create physical volume and volume group.

<pre>
[root@terpcondor root]# vgscan
vgscan -- reading all physical volumes (this may take a while...)
vgscan -- "/etc/lvmtab" and "/etc/lvmtab.d" successfully created
vgscan -- WARNING: This program does not do a VGDA backup of your volume group

[root@terpcondor root]# pvcreate /dev/sda5
pvcreate -- physical volume "/dev/sda5" successfully created

[root@terpcondor root]# vgcreate internal-disk /dev/sda5
vgcreate -- INFO: using default physical extent size 32 MB
vgcreate -- INFO: maximum logical volume size is 2 Terabyte
vgcreate -- doing automatic backup of volume group "internal-disk"
vgcreate -- volume group "internal-disk" successfully created and activated
</pre>

Re-create tmp space.

<pre>
[root@terpcondor root]# lvcreate -L2G -ntmp internal-disk
lvcreate -- doing automatic backup of "internal-disk"
lvcreate -- logical volume "/dev/internal-disk/tmp" successfully created

[root@terpcondor root]# mke2fs -j /dev/internal-disk/tmp
mke2fs 1.32 (09-Nov-2002)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
262144 inodes, 524288 blocks
26214 blocks (5.00%) reserved for the super user
First data block=0
16 block groups
32768 blocks per group, 32768 fragments per group
16384 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912

Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 29 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
</pre>

Create condor volume, this will store software and configs

<pre>
[root@terpcondor root]# lvcreate -L2G -ncondor internal-disk
lvcreate -- doing automatic backup of "internal-disk"
lvcreate -- logical volume "/dev/internal-disk/condor" successfully created

[root@terpcondor root]# mke2fs -j /dev/internal-disk/condor
mke2fs 1.32 (09-Nov-2002)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
262144 inodes, 524288 blocks
26214 blocks (5.00%) reserved for the super user
First data block=0
16 block groups
32768 blocks per group, 32768 fragments per group
16384 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912

Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 31 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
</pre>

Now config home space in /export

<pre>
[root@terpcondor root]# lvcreate -L2G -nhomes internal-disk
lvcreate -- doing automatic backup of "internal-disk"
lvcreate -- logical volume "/dev/internal-disk/homes" successfully created

[root@terpcondor root]# mke2fs -j /dev/internal-disk/homes
mke2fs 1.32 (09-Nov-2002)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
262144 inodes, 524288 blocks
26214 blocks (5.00%) reserved for the super user
First data block=0
16 block groups
32768 blocks per group, 32768 fragments per group
16384 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912

Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 35 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
</pre>

Condor work directory (normally this just goes in /var/condor, but since this install is 100% relocatable, it's going in /export

<pre>

[root@terpcondor root]# lvcreate -ncondor-var -L2G internal-disk
lvcreate -- doing automatic backup of "internal-disk"
lvcreate -- logical volume "/dev/internal-disk/condor-var" successfully created

[root@terpcondor root]# mke2fs -j /dev/internal-disk/condor-var
mke2fs 1.32 (09-Nov-2002)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
262144 inodes, 524288 blocks
26214 blocks (5.00%) reserved for the super user
First data block=0
16 block groups
32768 blocks per group, 32768 fragments per group
16384 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912

Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 36 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
</pre>

Add to /etc/fstab

<pre>
/dev/internal-disk/tmp /tmp ext3 defaults 1 2
/dev/internal-disk/condor /export/condor ext3 defaults 1 2
/dev/internal-disk/condor-var /export/condor-var ext3 defaults 1 2
/dev/internal-disk/homes /export/homes ext3 defaults 1 2
</pre>

==Condor Base Install==

Condor will be installed in /export/condor/condor-6.6.10 with a symlink created to /export/condor/condor. All scripts and such will use the symlink to aid in an easier upgrade. To make upgrades a truly copy binaries, the condor_config sits in /export/condor/condor_config. The environment CONDOR_CONFIG should be set so that condor can find this file.

* [[Media:condor-install.log|Installation Log]]

In ~condor/condor-6.6.10
<pre>
./condor_install
Would you like to do a full installation of Condor? [yes]
...
...
Are you planning to setup Condor on multiple machines? [yes]
Will all the machines share files via a file server? [yes] no
...
...
Have you installed a release directory already? [no]

Where would you like to install the Condor release directory?
[/usr/local/condor] /export/condor/condor-6.6.10
That directory doesn't exist, should I create it now? [yes]
Installing a release directory into /export/condor/condor-6.6.10 ...
...
...
[root@terpcondor.umiacs.umd.edu] root@umiacs.umd.edu

What is the full path to a mail program that understands "-s" means
you want to specify a subject? [/bin/mail]
...
...
Do all of the machines in your pool from your domain ("umiacs.umd.edu")
share a common filesystem? [no]

Configuring each machine to be in its own filesystem domain.

Do all of the users across all the machines in your domain have a unique
UID (in other words, do they all share a common passwd file)? [no]

Configuring each machine to be in its own uid domain.
...
...
Enable Java Universe support? [yes]

I wasn't able to find a valid JVM.

Please enter the full path to the JVM, or "none" to leave unconfigured:
/opt/j2sdk1.4.2/bin/java

You entered: /opt/j2sdk1.4.2/bin/java
Is that right? [no] yes

Checking to see if you have a Sun JVM...yes.

Using JVM /opt/j2sdk1.4.2/bin/java for Java universe support.
...
...
Shall I create links in some other directory? [yes]

Where should I install these files?
[/usr/local/bin] /usr/bin
...
...
What is the full hostname of the central manager?
[terpcondor.umiacs.umd.edu]
...
...
You have a "condor" user on this machine. Do you want to put all the
Condor directories in /export/homes/condor? [yes] no

Do you want to put all the Condor directories in
/export/condor/condor-6.6.10/home? [yes] no

Where do you want the Condor directories?
/export/condor-var

Creating all necessary Condor directories ... done.
...
...
Should I put a "condor_config.local" file in /export/condor-var?
[yes]
Creating config files in "/export/condor-var" ... done.

Configuring global condor config file ... done.
Created /export/condor/condor-6.6.10/etc/condor_config.
...
...
Setting up terpcondor.umiacs.umd.edu as your central manager

What name would you like to use for this pool? This should be a
short description (20 characters or so) that describes your site.
For example, the name for the UW-Madison Computer Science Condor
Pool is: "UW-Madison CS". This value is stored in your central
manager's local config file as "COLLECTOR_NAME", if you decide to
change it later. (This shouldn't include any " marks).
UM College Park
...
...
Should I put in a soft link from /export/homes/condor/condor_config to
/export/condor/condor-6.6.10/etc/condor_config [yes]

Installing links for public binaries into /usr/bin ... done.

</pre>

==Post-Install cleanup==

To make condor a little more machine and install independant, we moved the condor_config to /export/condor/condor_config
<pre>
[root@terpcondor condor]# cd ~condor
[root@terpcondor condor]# ln -s /export/condor/condor_config
[root@terpcondor condor]# cd /export/condor
[root@terpcondor condor]# ln -s condor-6.6.10 condor
[root@terpcondor condor]# mv condor-6.6.10/etc/condor_config .
</pre>

==First Start==

Now, we should have a working condor install. This should be able to startup using the UMcondor init script. You'll notice that a few too many daemons startup. We'll fix this next.

(in ~condor)
<pre>
[root@terpcondor condor]# ./UMcondor start
[root@terpcondor condor]# ps -ef|grep cond
squid 12859 1 0 12:48 ? 00:00:00 /opt/stow/condor/sbin/condor_master
squid 12860 12859 0 12:48 ? 00:00:00 condor_collector -f
squid 12861 12859 0 12:48 ? 00:00:00 condor_negotiator -f
squid 12862 12859 22 12:48 ? 00:00:05 condor_startd -f
squid 12864 12859 0 12:48 ? 00:00:00 condor_schedd -f
root 12890 6380 0 12:48 pts/0 00:00:00 grep cond
[root@terpcondor condor]# ./UMcondor stop
Shutting down Condor (fast-shutdown mode)
</pre>

==Local Configuration==

In /export/condor/condor_config you'll need to make the following changes

<pre>
# version independant release dir
RELEASE_DIR = /export/condor/condor

# allow umiacs jobs to flow to this box
FLOCK_FROM = *.umiacs.umd.edu

# allow us to admin the box remotely
HOSTALLOW_ADMINISTRATOR = $(CONDOR_HOST), loach.umiacs.umd.edu

# only allow umd hosts to join this pool
HOSTALLOW_READ = *.umd.edu
HOSTALLOW_WRITE = *.umd.edu

</pre>

now, for the local config in /export/condor-var/condor_config.local

<pre>
DAEMON_LIST = MASTER, COLLECTOR, NEGOTIATOR, SCHEDD
</pre>

==Set Condor to start on boot==

Copy init script and start service.

<pre>
[root@terpcondor condor]# cp UMcondor /etc/init.d
[root@terpcondor condor]# /sbin/chkconfig UMcondor on
[root@terpcondor condor]# /sbin/service UMcondor start
[root@terpcondor condor]# ps -ef|grep cond
squid 13393 1 0 13:04 ? 00:00:00 /export/condor/condor/sbin/condor_master
squid 13394 13393 0 13:04 ? 00:00:00 condor_collector -f
squid 13396 13393 0 13:04 ? 00:00:00 condor_negotiator -f
squid 13397 13393 0 13:04 ? 00:00:00 condor_schedd -f
</pre>

==Test Condor==

On a standard umiacs host, add in /var/condor/condor_config.local

CONDOR_HOST = terpcondor.umiacs.umd.edu
FLOCK_FROM = rogueleader.umiacs.umd.edu

In the global umiacs config, we added a 'FLOCK_TO = termcondor...' line to allow jobs to flow to the new condor pool

-- Main.MikeSmorul - 17 Aug 2005

Toaster:UMD Condor Configuration - Revision history

Toaster at 20:30, 11 September 2008

Toaster at 20:30, 11 September 2008