Dpuiu Linux: Difference between revisions

From Cbcb
Jump to navigation Jump to search
No edit summary
 
(53 intermediate revisions by the same user not shown)
Line 1: Line 1:
= Sourceforge =
= Version Control =
== Sourceforge ==


== AMOS ==
  shell server:  shell.sourceforge.net
  CVS server:         PROJECTNAME.cvs.sourceforge.net
  Subversion server: PROJECTNAME.svn.sourceforge.net
Authentication:
* [http://alexandria.wiki.sourceforge.net/SSH+Key+Generation KeyGen]
* Generate the enw key:
  sh-keygen -t dsa -C dxpuiu@shell.sf.net
  cat /nfshomes/dpuiu/.ssh/id_dsa.pub >> /nfshomes/dpuiu/.ssh/authorized_keys
* Upload key to the Sourceforge server ("Account Maintenance" site)


* CO
== Patching ==
  diff  file.old file.new > file.diff
  patch file.old file.patch
 
== CVS ==
 
* Local CVS
  echo $CVSROOT
  /fs/szdevel/src/cvsroot
 
  /fs/szdevel/src/cvsroot/users/dpuiu/
 
* Checkout
   cvs -z3 -d:ext:dxpuiu@amos.cvs.sourceforge.net:/cvsroot/amos co -P AMOS
   cvs -z3 -d:ext:dxpuiu@amos.cvs.sourceforge.net:/cvsroot/amos co -P AMOS


* Add a new file
* Add a new file
   cd src/Utils  
   cd src/Utils  
   cp ~/bin/util .
   cp ~/bin/file .
    
    
   cvs add util
   cvs add file
   cvs ci
   cvs ci


* Add a new file
* Add a new dir
   cd src/Utils  
   cd src/Utils  
   cp -R ~/bin/dir .
   cp -R ~/bin/dir .
Line 23: Line 45:
   cvs add *
   cvs add *
   cvs ci
   cvs ci
* Update files
  cd src/Utils
 
  cvs update
* Check status
  cvs status -v | grep Status:
* Tagging  a file
  cvs tag file.tag file
* Downloading a tagged file
  cvs co -r file.tag file
* Options
  -l : Local; run only in current working directory, rather than recursing through subdirectories.
  -R: Recursive (default)
  -m message
* Keep track of the version in the code :
  Add following line: my $VERSION = '$Revision: 1.0 $ ';
  Revision gets automatically incremented when "cvs ci"; no need to "cvs update"
* View file
* View file
   http://amos.cvs.sourceforge.net/viewvc/*checkout*/amos/AMOS/src/Utils/seq2amos.pl
   http://amos.cvs.sourceforge.net/viewvc/*checkout*/amos/AMOS/src/Utils/seq2amos.pl
* View all files
* View all files
   http://amos.cvs.sourceforge.net/viewvc/amos/
   http://amos.cvs.sourceforge.net/viewvc/amos/
== Yum ==
  yum command [options]
* Examples:
  yum -y list installed                                # list installed packages
  yum -y list installed  | grep flash                  # check if flash is installed
  sudo yum remove flash-plugin.i386                    # remove flash
  sudo yum -y install flashplayer-square-plugin.x86_64  # install flashplayer-square
== Git ==
* yum install git-core
=  Build =
* [http://www.gnu.org/software/autoconf/ autoconf] extensible package of M4 macros that produce shell scripts to automatically configure software source code packages
* [http://www.gnu.org/software/m4/m4.html m4] [http://www.gnu.org/software/m4/manual/m4.pdf man] implementation of the traditional Unix macro processor
* [http://www.gnu.org/software/libtool/libtool.html libtool] generic library support script. Libtool hides the complexity of using shared libraries behind a consistent, portable interface.
* [http://developer.gnome.org/doc/GGAD/build-app.html Application Build]
* Example: copy executables to /fs/szdevel/dpuiu/; libs to ...
  ./configure --prefix=/fs/szdevel/dpuiu/ --libdir=/fs/szdevel/dpuiu/lib > configure.log
  make                                                                  > make.log
  make install                                                          > make.install.log
* Example: explicit compiler location
  ./configure CC=/usr/bin/gcc CXX=/usr/bin/g++
* Example: build for debug
  ./configure CFLAGS="-g -pg" CXXFLAGS="-g -pg"
  ...
= Redirect =
* tcsh : STDOUT & STDERR separately
  (command >stdout_file ) >&stderr_file
= Config =
* [http://en.wikipedia.org/wiki/Gconf-editor gconf-editor]
= Commands =
* ftp
  ftp - i  : turns interactive mode off
  > prompt : turns interactive mode off
* diff
  diff -rq dir1 dir2 : recursively compares 2 directories
* sort by multiple columns
  # column 2:alpha, 3:nucmeric
  cat prefix.posmap | msort -k2 -kn3
  cat prefix.sam| msort -k3 -kn4
= Utils =
* [http://en.wikipedia.org/wiki/Coreutils Coreutils]
* [http://en.wikipedia.org/wiki/Binutils Binutils]
  strip executable # removes debugging info => executable_new
 
  gcc -pg prefix.cc -o prefix  # -pg: profiling flag
  ./prefix #=> gmon.out
  gprof prefix > prefix.out
  /nfshomes/dpuiu/szdevel/bin/gprof2dot.py prefix.out > prefix.dot
  dot -Tpng -o prefix.png prefix.dot
== Convert profiling to dot format ==
* http://linux.softpedia.com/progDownload/gprof2dot-py-Download-27166.html
* http://jrfonseca.googlecode.com/svn/trunk/gprof2dot/gprof2dot.py
== Syncronize directories ==
* Example: syncronize FASTA files
cd dir1
rsync *fasta ../dir2
= System info =
* [http://pagesperso-orange.fr/sebastien.godard/ Sysstat package]
* Processors
  /usr/sbin/./x86info
  cat /proc/cpuinfo
* sycamore (8 processor):
  cat /proc/cpuinfo | more
  vendor_id      : AuthenticAMD
  cpu family      : 15
  model          : 33
  model name      : Dual Core AMD Opteron(tm) Processor 875
  stepping        : 0
  cpu MHz        : 1793.260
  cache size      : 1024 KB
  physical id    : 0
  siblings        : 2
  core id        : 0
  cpu cores      : 2
* walnut (16 processor):
  vendor_id      : AuthenticAMD
  cpu family      : 15
  model          : 65
  model name      : Dual-Core AMD Opteron(tm) Processor 8220
  stepping        : 3
  cpu MHz        : 2792.923
  cache size      : 1024 KB
  physical id    : 2
  siblings        : 2
  core id        : 0
  cpu cores      : 2
* ginkgo (32 processor)
  vendor_id      : AuthenticAMD
  cpu family      : 16
  model          : 2
  model name      : Quad-Core AMD Opteron(tm) Processor 8356
  stepping        : 3
  cpu MHz        : 2293.905
  cache size      : 512 KB
  physical id    : 1
  siblings        : 4
  core id        : 0
  cpu cores      : 4
* Memory
  cat /proc/meminfo
  free -mt
= Gnuplot =
Scripts for automatic generation:
* ~/bin/len-draw.pl prefix.len : draws a histogram of chromosome lengths
* ~/bin/map-draw.pl prefix.map : draws a synteny map
Example:
  perl ~/bin/len-draw.pl turkey.len | gnuplot ; display turkey.png
= Image manipulation =
* [http://www.imagemagick.org/script/command-line-tools.php ImageMagick]
* Merge:
  montage *a.jpg montage.jpg
* Get attributes
  identify prefix.jpg
= Audio/Video manipulation =
* [http://www.catswhocode.com/blog/19-ffmpeg-commands-for-all-needs]
* [http://www.ffmpeg.org/index.html FFMpeg]
  ffmpeg -i movieSample.avi -ab 256k -vn autioSample.mp3
* Youtube download:
  youtube-dl "url"
= Pretty printing =
* enscript
  enscript -G -r -B -f "Courier8" -p file.ps file.txt
  ps2pdf file.ps file.pdf
  lpr file.pdf
= Quick calculations =
* bc
echo 99 \* 8 | bc
echo 99 \< 8 | bc
= Commands =
== alias ==
* With arguments:
  alias backup cp -i  \!:1  \!:1.`date`
== mkfifo ==
  mkfifo fifo
  cat  fifo &
  ls >> fifo
==chgrp ==
* Make all new files craeted under /fs/www-umiacs-users belong to dpuiu
  chgrp cbcb /fs/www-umiacs-users/dpuiu
  chmod 775 /fs/www-umiacs-users/dpuiu
  chmod g+s /fs/www-umiacs-users/dpuiu
== time ==
* Format
  Time                                Memory                    IO
  %Uuser %Ssystem %Eelapsed  %PCPU    (%Xtext+%Ddata %Mmax)k    %Iinputs+%Ooutputs (%Fmajor+%Rminor)pagefaults %Wswaps
  %U    Total number of CPU-seconds that the process spent in user mode.
  %S    Total number of CPU-seconds that the process spent in kernel mode.
  %E    Elapsed real time (in [hours:]minutes:seconds).
  %P    Percentage of the CPU that this job got, computed as (%U + %S) / %E.
= Articles =
* [http://www.cyberciti.biz/tips/top-linux-monitoring-tools.html Most useful sysadmin tools]

Latest revision as of 13:08, 12 August 2011

Version Control

Sourceforge

 shell server:  	shell.sourceforge.net
 CVS server: 	        PROJECTNAME.cvs.sourceforge.net
 Subversion server: 	PROJECTNAME.svn.sourceforge.net

Authentication:

  sh-keygen -t dsa -C dxpuiu@shell.sf.net
  cat /nfshomes/dpuiu/.ssh/id_dsa.pub >> /nfshomes/dpuiu/.ssh/authorized_keys
  • Upload key to the Sourceforge server ("Account Maintenance" site)

Patching

 diff  file.old file.new > file.diff
 patch file.old file.patch

CVS

  • Local CVS
 echo $CVSROOT
 /fs/szdevel/src/cvsroot
 /fs/szdevel/src/cvsroot/users/dpuiu/
  • Checkout
 cvs -z3 -d:ext:dxpuiu@amos.cvs.sourceforge.net:/cvsroot/amos co -P AMOS
  • Add a new file
 cd src/Utils 
 cp ~/bin/file .
 
 cvs add file
 cvs ci
  • Add a new dir
 cd src/Utils 
 cp -R ~/bin/dir .
 
 cvs add dir
 cvs ci
 
 cd dir
 cvs add *
 cvs ci
  • Update files
 cd src/Utils 
 
 cvs update
  • Check status
 cvs status -v | grep Status:
  • Tagging a file
 cvs tag file.tag file
  • Downloading a tagged file
 cvs co -r file.tag file
  • Options
 -l : Local; run only in current working directory, rather than recursing through subdirectories.
 -R: Recursive (default)
 -m message
  • Keep track of the version in the code :
 Add following line: my $VERSION = '$Revision: 1.0 $ ';
 Revision gets automatically incremented when "cvs ci"; no need to "cvs update"
  • View file
 http://amos.cvs.sourceforge.net/viewvc/*checkout*/amos/AMOS/src/Utils/seq2amos.pl
  • View all files
 http://amos.cvs.sourceforge.net/viewvc/amos/

Yum

 yum command [options]
  • Examples:
 yum -y list installed                                 # list installed packages
 yum -y list installed  | grep flash                   # check if flash is installed
 sudo yum remove flash-plugin.i386                     # remove flash
 sudo yum -y install flashplayer-square-plugin.x86_64  # install flashplayer-square

Git

  • yum install git-core

Build

  • autoconf extensible package of M4 macros that produce shell scripts to automatically configure software source code packages
  • m4 man implementation of the traditional Unix macro processor
  • libtool generic library support script. Libtool hides the complexity of using shared libraries behind a consistent, portable interface.
  • Application Build
  • Example: copy executables to /fs/szdevel/dpuiu/; libs to ...
 ./configure --prefix=/fs/szdevel/dpuiu/ --libdir=/fs/szdevel/dpuiu/lib > configure.log 
 make                                                                   > make.log
 make install                                                           > make.install.log
  • Example: explicit compiler location
 ./configure CC=/usr/bin/gcc CXX=/usr/bin/g++
  • Example: build for debug
 ./configure CFLAGS="-g -pg" CXXFLAGS="-g -pg"
 ...

Redirect

  • tcsh : STDOUT & STDERR separately
 (command >stdout_file ) >&stderr_file

Config

Commands

  • ftp
 ftp - i  : turns interactive mode off
 > prompt : turns interactive mode off
  • diff
 diff -rq dir1 dir2 : recursively compares 2 directories
  • sort by multiple columns
 # column 2:alpha, 3:nucmeric
 cat prefix.posmap | msort -k2 -kn3
 cat prefix.sam| msort -k3 -kn4

Utils

 strip executable # removes debugging info => executable_new
 
 gcc -pg prefix.cc -o prefix   # -pg: profiling flag
 ./prefix #=> gmon.out
 gprof prefix > prefix.out
 /nfshomes/dpuiu/szdevel/bin/gprof2dot.py prefix.out > prefix.dot 
 dot -Tpng -o prefix.png prefix.dot

Convert profiling to dot format

Syncronize directories

  • Example: syncronize FASTA files
cd dir1
rsync *fasta ../dir2

System info

 /usr/sbin/./x86info
 cat /proc/cpuinfo
  • sycamore (8 processor):
 cat /proc/cpuinfo | more
 vendor_id       : AuthenticAMD
 cpu family      : 15
 model           : 33
 model name      : Dual Core AMD Opteron(tm) Processor 875
 stepping        : 0
 cpu MHz         : 1793.260
 cache size      : 1024 KB
 physical id     : 0
 siblings        : 2
 core id         : 0
 cpu cores       : 2
  • walnut (16 processor):
 vendor_id       : AuthenticAMD
 cpu family      : 15
 model           : 65
 model name      : Dual-Core AMD Opteron(tm) Processor 8220
 stepping        : 3
 cpu MHz         : 2792.923
 cache size      : 1024 KB
 physical id     : 2
 siblings        : 2
 core id         : 0
 cpu cores       : 2
  • ginkgo (32 processor)
 vendor_id       : AuthenticAMD
 cpu family      : 16
 model           : 2
 model name      : Quad-Core AMD Opteron(tm) Processor 8356
 stepping        : 3
 cpu MHz         : 2293.905
 cache size      : 512 KB
 physical id     : 1
 siblings        : 4
 core id         : 0
 cpu cores       : 4
  • Memory
 cat /proc/meminfo
 free -mt

Gnuplot

Scripts for automatic generation:

  • ~/bin/len-draw.pl prefix.len : draws a histogram of chromosome lengths
  • ~/bin/map-draw.pl prefix.map : draws a synteny map

Example:

 perl ~/bin/len-draw.pl turkey.len | gnuplot ; display turkey.png

Image manipulation

 montage *a.jpg montage.jpg
  • Get attributes
 identify prefix.jpg

Audio/Video manipulation

 ffmpeg -i movieSample.avi -ab 256k -vn autioSample.mp3
  • Youtube download:
 youtube-dl "url"

Pretty printing

  • enscript
 enscript -G -r -B -f "Courier8" -p file.ps file.txt
 ps2pdf file.ps file.pdf
 lpr file.pdf

Quick calculations

  • bc
echo 99 \* 8 | bc
echo 99 \< 8 | bc

Commands

alias

  • With arguments:
 alias backup cp -i  \!:1  \!:1.`date`

mkfifo

 mkfifo fifo
 cat   fifo &
 ls >> fifo

chgrp

  • Make all new files craeted under /fs/www-umiacs-users belong to dpuiu
 chgrp cbcb /fs/www-umiacs-users/dpuiu
 chmod 775 /fs/www-umiacs-users/dpuiu
 chmod g+s /fs/www-umiacs-users/dpuiu

time

  • Format
 Time                                Memory                    IO
 %Uuser %Ssystem %Eelapsed  %PCPU    (%Xtext+%Ddata %Mmax)k    %Iinputs+%Ooutputs (%Fmajor+%Rminor)pagefaults %Wswaps

 %U     Total number of CPU-seconds that the process spent in user mode.
 %S     Total number of CPU-seconds that the process spent in kernel mode.
 %E     Elapsed real time (in [hours:]minutes:seconds).

 %P     Percentage of the CPU that this job got, computed as (%U + %S) / %E.

Articles