Cbcb - User contributions [en]

Getting Started in CBCB

2013-04-30T15:13:14Z

Dsommer: /* Getting Building Access and Room Keys */

Last update 4/29/13 

[[Media:CBCB_orientation.pdf]]

==Getting Building Access and Room Keys==

CBCB is located on the 3rd floor of The Biomolecular Sciences Building identified on campus maps as Building #296. The building is secure and access is gained by either using your UM ID card, guest card or entering the 3-digit code of the person you to visit at the call box on the right side of the front door.

Contact the Center Coordinator, Denise Cross, about gaining card access to the building. She will need the following information:
*An notification email from your sponsor/adviser
*Your Name
*Your 9 Digit University ID number
*Your Contact email

Along with your assigned space and phone numbers, the coordinator will send your information to UMIACS Coordinator Edna Walker who will contact campus security to add you into their system. ''Note: clearance usually takes a number of days, so contact the coordinator as soon as possible.''

If you prefer not to send your information through email feel free to contact the coordinator in person.

You must get your key from the coordinator in person.

Denise Cross 
Room 3121 
Biomolecular Sciences Bldg #296 
301.405.5936 
dcross[at]umd.edu 

==Research In Progress Seminars==

These seminars are held through the year. For information go to [http://cbcb.umd.edu/node/18625 CBCB Research in Progress]
==Understanding the Layout of Available Resources==
When you first log into a server (eg. flicker01@umiacs.umd.edu), you will probably be placed in one of the following personalized directories:
*/fs/wrenhome/yourUserName/
*/nfshomes/yourUserName/
nfshomes has a limit on available disk space (in the double digit MBs) while wrenhomes allows you the more freedom. Therefore you should use wrenhomes for your personal work files, but should be very mindful of how much space you are using and how much free space remains. For large amounts of data, you should use one of the following (after first checking with your sponsor).
 

For a list of our Disk Storage and amount of available space left on each one, see [http://wiki.umiacs.umd.edu/cbcb-private/index.php/Storage wiki.umiacs.umd.edu/cbcb-private/index.php/Storage] 

== Configuring Your Home Directory and Shell ==

We import common settings files that set our path variables to include commonly shared software repositories.

As a start, add the following line to the top of the file called ".bashrc" located in your home directory (/nfshomes/username/):

. /fs/sz-user-supported/share/dotfiles/bashrc.cbcb

This will import the common bashrc.cbcb file into your own bashrc file every time you log in.

Now add this line to your ".bash_profile" file, also located in your home directory:

. ~/.bashrc

This will import your personal bashrc file every time you log in. Now you should have access to most of the locally installed software like "blastall" and "AMOS."

If you want to add any addition commands to your bashrc file, such as setting your default text editor to "vim" or formatting the output of bash commands (eg. "ls"), add the appropriate commands after the imported common files as shown in this example:

. /fs/sz-user-supported/share/dotfiles/bashrc.cbcb

alias vi='vim'

alias ls='ls --color'

If you decide you want to change the settings in the common bashrc.cbcb to better suit your personal needs, then please copy and paste its contents into your personal bashrc file. '''Do not modify the common bashrc.cbcb file as it will affect everyone's environment.''' Also check back periodically as people may add common paths for new software.

If you have any other problems, contact staff [at] umiacs.umd.edu or your PI. 

More resources can be found at [https://wiki.umiacs.umd.edu/umiacs/index.php/GettingStarted umiacs wiki]
== Printing ==
Go to the umiacs wiki to find [https://wiki.umiacs.umd.edu/umiacs/index.php/Printing system-specific guides for printing], and be sure to [https://wiki.umiacs.umd.edu/umiacs/index.php/PrinterQueueNaming add 'nb' to the end of your print queue] to avoid wasting paper printing banners.

== Using the Wiki ==
* [http://en.wikipedia.org/wiki/Wikipedia:Cheatsheet Wiki Formatting Cheatsheet]
* [http://www.mediawiki.org/wiki/Help:Configuration_settings Configuration settings list]
* [http://www.mediawiki.org/wiki/Help:FAQ MediaWiki FAQ]
* [http://mail.wikimedia.org/mailman/listinfo/mediawiki-announce MediaWiki release mailing list]

Getting Started in CBCB

2013-04-30T15:13:02Z

Dsommer: /* Understanding the Layout of Available Resources */

Last update 4/29/13 

==Getting Building Access and Room Keys==

CBCB is located on the 3rd floor of The Biomolecular Sciences Building identified on campus maps as Building #296. The building is secure and access is gained by either using your UM ID card, guest card or entering the 3-digit code of the person you to visit at the call box on the right side of the front door.

Contact the Center Coordinator, Denise Cross, about gaining card access to the building. She will need the following information:
*An notification email from your sponsor/adviser
*Your Name
*Your 9 Digit University ID number
*Your Contact email

Along with your assigned space and phone numbers, the coordinator will send your information to UMIACS Coordinator Edna Walker who will contact campus security to add you into their system. ''Note: clearance usually takes a number of days, so contact the coordinator as soon as possible.''

If you prefer not to send your information through email feel free to contact the coordinator in person.

You must get your key from the coordinator in person.

Denise Cross 
Room 3121 
Biomolecular Sciences Bldg #296 
301.405.5936 
dcross[at]umd.edu 

==Research In Progress Seminars==

These seminars are held through the year. For information go to [http://cbcb.umd.edu/node/18625 CBCB Research in Progress]
==Understanding the Layout of Available Resources==
When you first log into a server (eg. flicker01@umiacs.umd.edu), you will probably be placed in one of the following personalized directories:
*/fs/wrenhome/yourUserName/
*/nfshomes/yourUserName/
nfshomes has a limit on available disk space (in the double digit MBs) while wrenhomes allows you the more freedom. Therefore you should use wrenhomes for your personal work files, but should be very mindful of how much space you are using and how much free space remains. For large amounts of data, you should use one of the following (after first checking with your sponsor).
 

For a list of our Disk Storage and amount of available space left on each one, see [http://wiki.umiacs.umd.edu/cbcb-private/index.php/Storage wiki.umiacs.umd.edu/cbcb-private/index.php/Storage] 

== Configuring Your Home Directory and Shell ==

We import common settings files that set our path variables to include commonly shared software repositories.

As a start, add the following line to the top of the file called ".bashrc" located in your home directory (/nfshomes/username/):

. /fs/sz-user-supported/share/dotfiles/bashrc.cbcb

This will import the common bashrc.cbcb file into your own bashrc file every time you log in.

Now add this line to your ".bash_profile" file, also located in your home directory:

. ~/.bashrc

This will import your personal bashrc file every time you log in. Now you should have access to most of the locally installed software like "blastall" and "AMOS."

If you want to add any addition commands to your bashrc file, such as setting your default text editor to "vim" or formatting the output of bash commands (eg. "ls"), add the appropriate commands after the imported common files as shown in this example:

. /fs/sz-user-supported/share/dotfiles/bashrc.cbcb

alias vi='vim'

alias ls='ls --color'

If you decide you want to change the settings in the common bashrc.cbcb to better suit your personal needs, then please copy and paste its contents into your personal bashrc file. '''Do not modify the common bashrc.cbcb file as it will affect everyone's environment.''' Also check back periodically as people may add common paths for new software.

If you have any other problems, contact staff [at] umiacs.umd.edu or your PI. 

More resources can be found at [https://wiki.umiacs.umd.edu/umiacs/index.php/GettingStarted umiacs wiki]
== Printing ==
Go to the umiacs wiki to find [https://wiki.umiacs.umd.edu/umiacs/index.php/Printing system-specific guides for printing], and be sure to [https://wiki.umiacs.umd.edu/umiacs/index.php/PrinterQueueNaming add 'nb' to the end of your print queue] to avoid wasting paper printing banners.

== Using the Wiki ==
* [http://en.wikipedia.org/wiki/Wikipedia:Cheatsheet Wiki Formatting Cheatsheet]
* [http://www.mediawiki.org/wiki/Help:Configuration_settings Configuration settings list]
* [http://www.mediawiki.org/wiki/Help:FAQ MediaWiki FAQ]
* [http://mail.wikimedia.org/mailman/listinfo/mediawiki-announce MediaWiki release mailing list]

Getting Started in CBCB

2013-04-30T15:12:34Z

Dsommer: /* Understanding the Layout of Available Resources */

Last update 4/29/13 

==Getting Building Access and Room Keys==

CBCB is located on the 3rd floor of The Biomolecular Sciences Building identified on campus maps as Building #296. The building is secure and access is gained by either using your UM ID card, guest card or entering the 3-digit code of the person you to visit at the call box on the right side of the front door.

Contact the Center Coordinator, Denise Cross, about gaining card access to the building. She will need the following information:
*An notification email from your sponsor/adviser
*Your Name
*Your 9 Digit University ID number
*Your Contact email

Along with your assigned space and phone numbers, the coordinator will send your information to UMIACS Coordinator Edna Walker who will contact campus security to add you into their system. ''Note: clearance usually takes a number of days, so contact the coordinator as soon as possible.''

If you prefer not to send your information through email feel free to contact the coordinator in person.

You must get your key from the coordinator in person.

Denise Cross 
Room 3121 
Biomolecular Sciences Bldg #296 
301.405.5936 
dcross[at]umd.edu 

==Research In Progress Seminars==

These seminars are held through the year. For information go to [http://cbcb.umd.edu/node/18625 CBCB Research in Progress]
==Understanding the Layout of Available Resources==
When you first log into a server (eg. flicker01@umiacs.umd.edu), you will probably be placed in one of the following personalized directories:
*/fs/wrenhome/yourUserName/
*/nfshomes/yourUserName/
nfshomes has a limit on available disk space (in the double digit MBs) while wrenhomes allows you the more freedom. Therefore you should use wrenhomes for your personal work files, but should be very mindful of how much space you are using and how much free space remains. For large amounts of data, you should use one of the following (after first checking with your sponsor).
 

For a list of our Disk Storage and amount of available space left on each one, see [http://wiki.umiacs.umd.edu/cbcb-private/index.php/Storage wiki.umiacs.umd.edu/cbcb-private/index.php/Storage] 

[[Media:CBCB_orientation.pdf]]

== Configuring Your Home Directory and Shell ==

We import common settings files that set our path variables to include commonly shared software repositories.

As a start, add the following line to the top of the file called ".bashrc" located in your home directory (/nfshomes/username/):

. /fs/sz-user-supported/share/dotfiles/bashrc.cbcb

This will import the common bashrc.cbcb file into your own bashrc file every time you log in.

Now add this line to your ".bash_profile" file, also located in your home directory:

. ~/.bashrc

This will import your personal bashrc file every time you log in. Now you should have access to most of the locally installed software like "blastall" and "AMOS."

If you want to add any addition commands to your bashrc file, such as setting your default text editor to "vim" or formatting the output of bash commands (eg. "ls"), add the appropriate commands after the imported common files as shown in this example:

. /fs/sz-user-supported/share/dotfiles/bashrc.cbcb

alias vi='vim'

alias ls='ls --color'

If you decide you want to change the settings in the common bashrc.cbcb to better suit your personal needs, then please copy and paste its contents into your personal bashrc file. '''Do not modify the common bashrc.cbcb file as it will affect everyone's environment.''' Also check back periodically as people may add common paths for new software.

If you have any other problems, contact staff [at] umiacs.umd.edu or your PI. 

More resources can be found at [https://wiki.umiacs.umd.edu/umiacs/index.php/GettingStarted umiacs wiki]
== Printing ==
Go to the umiacs wiki to find [https://wiki.umiacs.umd.edu/umiacs/index.php/Printing system-specific guides for printing], and be sure to [https://wiki.umiacs.umd.edu/umiacs/index.php/PrinterQueueNaming add 'nb' to the end of your print queue] to avoid wasting paper printing banners.

== Using the Wiki ==
* [http://en.wikipedia.org/wiki/Wikipedia:Cheatsheet Wiki Formatting Cheatsheet]
* [http://www.mediawiki.org/wiki/Help:Configuration_settings Configuration settings list]
* [http://www.mediawiki.org/wiki/Help:FAQ MediaWiki FAQ]
* [http://mail.wikimedia.org/mailman/listinfo/mediawiki-announce MediaWiki release mailing list]

File:CBCB orientation.pdf

2013-04-30T15:11:05Z

Dsommer:

Communal Software

2012-08-03T13:04:08Z

Dsommer: /* Planned Maintenance (Summer 2010) */

== Introduction ==
CBCB users do not have root access on their machines or the communal servers. Communal CBCB software is therefore typically installed in
/fs/sz-user-supported/
There are two primary subdirectories which differ by architecture. For the most part, 32bit software is installed in
/fs/sz-user-supported/Linux-i686/
while 64bit software is installed in
/fs/sz-user-supported/Linux-x86_64/
The appropriate subdirectory is typically dynamically chosen by embedding
/fs/sz-user-supported/`uname`-`uname -m`
in a user's path variables in their bashrc/tcshrc file.

Installing software to these directories and updating a user's path can be tricky. For this reason, many users choose to maintain specialty software in personal directories.

== Core Software ==

The latest versions of the most frequently used CBCB software is installed under
/fs/szdevel/core-cbcb-software/
and follows a directory structure similar to
/fs/sz-user-supported/
The software is installed and maintained by the cbcb-staff group.

Software list:
'''Name version'''
Bio::Perl
Biopython
Bowtie 0.12.7
Celera Assembler 6.1
gcc 4.5.1
GMap,GSnap 2010-07-27
Java (JRE only) 1.6.0_22
NCBI blast
Newbler 2.5
R
Samtools 0.1.9
SOAPdeNovo 1.04
Velvet 1.0.13

== Planned Maintenance (Summer 2012) ==
The Pop lab is currently planning to update much of the communal software installed in sz-user-supported over the Summer of 2012. So far this list includes:
* [http://sourceforge.net/apps/mediawiki/amos/index.php?title=AMOS AMOS] -- Already installed
* [http://www.ncbi.nlm.nih.gov/staff/tao/URLAPI/blastall/index.html blastall 2.2.23] -- May not be worth upgrading from 2.2.18. Will look at release notes.
* [http://hmmer.janelia.org/ HMMer3] -- Installed.
** [http://pfam.sanger.ac.uk/ Pfam 24.0]
* PERL -- May attempt to standardize default version across servers.
** [http://www.bioperl.org/wiki/Main_Page BioPERL]
* Python -- May attempt to standardize default version across servers.
** [http://biopython.org/wiki/Main_Page BioPython]
*** [http://numpy.scipy.org/ NumPy]
*** [http://www.reportlab.com/software/opensource/ ReportLab]
Please feel free to contact Ted Gibbons at tgibbons@umd.edu to make suggestions or express concerns about tampering with the communal software installations and the corresponding paths in the communal CBCB bashrc file.

Communal Software

2012-08-03T13:03:45Z

Dsommer: /* Carl's World */

== Introduction ==
CBCB users do not have root access on their machines or the communal servers. Communal CBCB software is therefore typically installed in
/fs/sz-user-supported/
There are two primary subdirectories which differ by architecture. For the most part, 32bit software is installed in
/fs/sz-user-supported/Linux-i686/
while 64bit software is installed in
/fs/sz-user-supported/Linux-x86_64/
The appropriate subdirectory is typically dynamically chosen by embedding
/fs/sz-user-supported/`uname`-`uname -m`
in a user's path variables in their bashrc/tcshrc file.

Installing software to these directories and updating a user's path can be tricky. For this reason, many users choose to maintain specialty software in personal directories.

== Core Software ==

The latest versions of the most frequently used CBCB software is installed under
/fs/szdevel/core-cbcb-software/
and follows a directory structure similar to
/fs/sz-user-supported/
The software is installed and maintained by the cbcb-staff group.

Software list:
'''Name version'''
Bio::Perl
Biopython
Bowtie 0.12.7
Celera Assembler 6.1
gcc 4.5.1
GMap,GSnap 2010-07-27
Java (JRE only) 1.6.0_22
NCBI blast
Newbler 2.5
R
Samtools 0.1.9
SOAPdeNovo 1.04
Velvet 1.0.13

== Planned Maintenance (Summer 2010) ==
The Pop lab is currently planning to update much of the communal software installed in sz-user-supported over the Summer of 2010. So far this list includes:
* [http://sourceforge.net/apps/mediawiki/amos/index.php?title=AMOS AMOS] -- Already installed
* [http://www.ncbi.nlm.nih.gov/staff/tao/URLAPI/blastall/index.html blastall 2.2.23] -- May not be worth upgrading from 2.2.18. Will look at release notes.
* [http://hmmer.janelia.org/ HMMer3] -- Installed.
** [http://pfam.sanger.ac.uk/ Pfam 24.0]
* PERL -- May attempt to standardize default version across servers.
** [http://www.bioperl.org/wiki/Main_Page BioPERL]
* Python -- May attempt to standardize default version across servers.
** [http://biopython.org/wiki/Main_Page BioPython]
*** [http://numpy.scipy.org/ NumPy]
*** [http://www.reportlab.com/software/opensource/ ReportLab]
Please feel free to contact Ted Gibbons at tgibbons@umd.edu to make suggestions or express concerns about tampering with the communal software installations and the corresponding paths in the communal CBCB bashrc file.

User:Dsommer

2009-03-09T20:17:44Z

Dsommer:

{| width="100%"
| height="100%" |
{|
| height="0%" |

<center>[#top [[Image:Amostop.gif]]] [http://www.cbcb.umd.edu [[Image:umd-logo.gif]]] [[Image:middle1.gif]] [http://www.tigr.org [[Image:tigr-logo.gif]]] [[Image:middle2.gif]] [http://cruzi.cgb.ki.se [[Image:ki-logo.gif]]] [[Image:middle1.gif]] [http://jbpc.mbl.edu/mcarthur [[Image:mbl-logo.gif]]] [[Image:middle2.gif]] [http://sourceforge.net [[Image:sflogo.php|125px|SourceForge.net Logo]]]</center>
|-
| class="border" height="100%" | 
|}

 
| style="padding: 15px" width="100%" |
<div class="centered">

=AMOS: A Modular Open-Source Assembler=

</div> 

==1. AMOS overview==

The AMOS consortium is committed to the development of open-source whole genome assembly software. The project acronym (AMOS) represents our primary goal -- to produce '''A''' '''M'''odular, '''O'''pen-'''S'''ource whole genome assembler. ''Open-source'' so that everyone is welcome to contribute and help build outstanding assembly tools, and ''modular'' in nature so that new contributions can be easily inserted into an existing assembly pipeline. This modular design will foster the development of new assembly algorithms and allow the AMOS project to continually grow and improve in hopes of eventually becoming a widely accepted and deployed assembly infrastructure. In this sense, AMOS is both a design philosophy and a software system. 

[docs/tutorials/readme-first.html AMOS Getting Started]

[docs/tutorials/programmers-guide.html Programmer's guide]

[http://www.cbcb.umd.edu/research/AMOS-doc.html AMOS Documentation Project]

Quick links:

* [#download <big>'''Download'''</big>]
* [http://sourceforge.net/projects/amos SourceForge project page]
* [#libamos API documentation]
* [#specifications File format specs]
* [recentcvs.html Recent CVS Activity]

Module documentation:

* [docs/converters File conversion utilities]
* [hawkeye Hawkeye] - assembly viewer
* [forensics amosvalidate] - assembly validation
* [docs/pipeline/minimus.html minimus] - basic genome assembler for small datasets
* [docs/pipeline/minimus2.html minimus2] - basic genome assembler for two datasets; can also be used as an assembly merge pipeline
* [docs/pipeline/AMOScmp.html AMOScmp] - comparative assembler
* [docs/pipeline/AMOScmp-shortReads.html AMOScmp-shortReads] - comparative assembler for short reads (Solexa,454)
* [docs/pipeline/AMOScmp-shortReads-alignmentTrimmed.html AMOScmp-shortReads-alignmentTrimmed] - comparative assembler for short reads that uses alignment based trimming
* Figaro - statistical vector trimmer
* [#libamos libAMOS ]- the API
* [#libalign libAlign]
* [#libslice libSlice]
* [#bambus Bambus]
* [#autoeditor AutoEditor]

==2. Table of contents==

# [#overview AMOS overview]
# [#toc Table of contents]
# [#collaborators Collaborators]
# [#acknowledgements Acknowledgements]
# [#infrastructure Infrastructure]
## [#libamos libAMOS API]
## [#specifications Specifications]
# [#modules Modules and projects]
## [#pipeline Assembly pipeline]
## [#overlap Overlap detection]
## [#contig Contig construction]
## [#consensus Consensus]
## [#scaffold Scaffolding]
## [#error Error correction]
## [#validation Validation]
## [#utilities Utilities]
# [#download Download]
# [#join Join the consortium]
# [#bugs Bug reports]

 

==3. Consortium members==

There have been numerous positive responses regarding the AMOS initiative, and we expect the list of involved organizations to grow significantly as the project matures. [#join Please contact us] if you want to join. The groups currently involved with the development of AMOS are listed below, along with their responsibilities and areas of expertise.

* '''[http://www.cbcb.umd.edu University of Maryland, Center for Bioinformatics and Computational Biology]'''
** project organization and direction
** infrastructure
** consensus
** automated sequence editing
** scaffolding
** overlap detection
** contig construction
* '''[http://www.tigr.org The Institute for Genomic Research]'''
** production pipelines
** automated finishing tools
** error correction
* '''[http://cruzi.cgb.ki.se Karolinska Institutet]'''
** overlap detection
** error correction
* '''[http://jbpc.mbl.edu/mcarthur Marine Biological Laboratory - Woods Hole]'''
** graphical interface
** integration of assembly data with analysis (gene, polymorphism, etc.) information

 

==4. Acknowledgements==

The AMOS consortium would like to thank the following organizations for their funding and/or support:

[http://www.nih.gov/ The National Institutes of Health] - grants R01-LM06845, N01-AI-15447

[http://www.nsf.gov/ The National Science Foundation] - grants IIS-9902923, IIS-9820497

[http://www.dhs.gov Department of Homeland Security] - cooperative agreement W81XWH-05-2-0051

[http://sourceforge.net SourceForge.net]

 

==5. Infrastructure==

The principal benefit of the AMOS project is its modular design, but in order to facilitate many, isolated components, a robust infrastructure is desirable. In response to this need, TIGR has developed numerous C++ classes for the efficient storage of assembly data types. These assembly objects can be written to and read from a central data repository, allowing for separate modules to build on and improve existing assemblies in discrete steps. This allows an assembly pipeline to run its steps in any order, and for data snapshots to be preserved at any time. In order to convey the assembly data outside of the C++ classes, we have implemented an ASCII message format modeled on that used by Celera Assembler*. This message format will be the unifying standard for all external module communication, and allow for the data snapshots to be output in a concise, text format. The API (application programming interface) for the AMOS foundation classes and the specification for the AMOS message format can be found in the sections below.

* "A Whole-Genome Assembly of ''Drosophila''." Myers E, Sutton G, et. al., ''Science'', 2000. 287(5461):2196-204.

 

===5.1. Application programming interface===

The AMOS API describes the programming interface for all of the AMOS foundation classes. Currently these classes are implemented in C++, but could ported to other languages as long as the API was preserved. The implementation can be found in the latest distribution under the <code>src/AMOS</code> project directory. These classes comprise the <code>libAMOS.a</code> library. This library contains the tools necessary to handle and manipulate AMOS messages, data-banks and internal assembly data structures such as sequencing reads, contigs, scaffolds, etc. The C++ source code for libAMOS is freely available for download [#download here].

[docs/api AMOS infrastructure API]

 

===5.2. Specifications===

The AMOS file types and message formats are defined in various specification documents, which can be found by following the below link. These documents also provide information on how to use messages for module communication and general development procedure recommendations.

[docs/specs AMOS specification documents]

 

==6. Modules and projects==

The following sections list all modules currently in development and those modules that are already in production. Because AMOS is in a constant state of development, there is an ever expanding list of ongoing projects, and this section attempts to outline the basic function of each project along with its status and parent organization. Status descriptions are (in order of occurrence): '''planning''', '''development''', '''testing''', '''production''' and '''antiquated'''. These status descriptions appear to the right of the module name. Clicking on a module name will redirect you to the project homepage (if applicable). 

===6.1. Assembly pipeline===

{| width="100%"
| width="50%" |
====[docs/pipeline/runAmos.html runAmos]====
| class="right" | <code>STATUS: production</code>
|}

<code>runAmos</code> is the command executor for all of the AMOS pipelines.

 

{| width="100%"
| width="50%" |
====[docs/pipeline/minimus.html minimus]====
| class="right" | <code>STATUS: production</code>
|}

<code>minimus</code> is a lightweight assembly tool for performing small assembly tasks for which a the complexity of a full assembler is unnecessary. Some such tasks, commonly needed during genome finishing, include joining together two overlapping contigs, adding reads to an existing contig, and refining the multiple-alignment of the reads within a contig. We use a standard 3-step assembly process known as overlap-layout-consensus which is explained further on the minimus website. <code>minimus</code> is freely available as part of the AMOS distribution which can be downloaded [#download here].

Examples of a flu assembly and a Zebrafish gene can be found in the test/minimus directory created when the AMOS distribution is untarred. Documentation on the examples is included with the distribution in /docs subdirectory

Supported in part by [http://www.dhs.gov DHS] cooperative agreement W81XWH-05-2-0051.

 

{| width="100%"
| width="50%" |
====[docs/pipeline/minimus2.html minimus2]====
| class="right" | <code>STATUS: production</code>
|}

<code>minimus2</code> is modified version of the minimus pipeline designed for merging one or two sequence sets. 

{| width="100%"
| width="50%" |
====[docs/pipeline/AMOScmp.html AMOScmp]====
| class="right" | <code>STATUS: production</code>
|}

<code>AMOScmp</code> is a comparative assembly pipeline. With the rapid growth in the number of sequenced genomes has come an increase in the number of organisms for which two or more closely-related species have been sequenced. This has created the possibility of building a comparative genome assembly algorithm, which can assemble a newly sequenced genome by mapping it onto a reference genome. Methods are described in our paper (below) and on the AMOScmp website. The MUMmer whole genome alignment package is required for the mapping step of this pipeline, and is freely available from the [#mummer MUMmer homepage]. <code>AMOScmp</code> is freely available as part of the AMOS distribution, which can be downloaded [#download here].

=====Related publications=====

* "Comparative Genome Assembly." Pop M, Phillippy A, Delcher AL, Salzberg SL, ''Briefings in Bioinformatics'', 2004. 5(3):237-48.

 

{| width="100%"
| width="50%" |
====[docs/pipeline/AMOScmp-shortReads.html AMOScmp-shortReads]====
| class="right" | <code>STATUS: production</code>
|}

<code>AMOScmp-shortReads</code> is a modified version of AMOScmp designed for assembling short reads. 

{| width="100%"
| width="50%" |
====[docs/pipeline/AMOScmp-shortReads-alignmentTrimmed.html AMOScmp-shortReads-alignmentTrimmed]====
| class="right" | <code>STATUS: production</code>
|}

<code>AMOScmp-shortReads-alignmentTrimmed</code> is a modified version of AMOScmp designed for alignment based trimming and assembling of short reads. 

===6.2. Overlap detection===

{| width="100%"
| width="50%" |
====UMD overlapper====
| class="right" | <code>STATUS: testing</code>
|}

The <code>UMD overlapper</code> is designed to reduce the number of overlaps produced by the assembler by reducing the number of repeat-induced overlaps. Furthermore the algorithm is greatly enhanced through the use of minimizers - a technique for reducing the number of k-mers considered in the initial phase of overlapping by an order of magnitude. Most assemblers use exact k-mer matches in order to identify reads that potentially overlap.

=====Related publications=====

* "A preprocessor for shotgun assembly of large genomes." Roberts M, Hunt BR, Yorke JA, Bolanos R, Delcher A, ''Journal of Computational Biology'', 2004. 11(4):734-752
* "Reducing storage requirements for biological sequence comparison." Roberts M, Hayes W, Hunt BR, Mount SM, Yorke JA. ''Bioinformatics'', 2004, 20(18):3363-3369. 

 

{| width="100%"
| width="50%" |
====KI overlapper====
| class="right" | <code>STATUS: testing</code>
|}

The <code>Karolinska Institutet overlapper</code> is designed to handle the problems created by sequencing errors. Instead of exact k-mer matches - the approach used by most existing assemblers - the KI overlapper uses a q-gram based method to identify "near hits" - k-mers that differ at a small number of positions. This approach allows this overlapper to identify overlaps otherwise missed by other overlappers.

=====Related publications=====

* "Correcting errors in shotgun sequences." Tammi MT, Arner E, Kindlund E, Andersson B, ''Nucleic Acids Research'', 2003. 31(15):4663-72.
* "TRAP: Tandem Repeat Assembly Program produces improved shotgun assemblies of repetitive sequences." Tammi MT, Arner E, Andersson B, ''Computational Methods Programs Biomed'', 2003. 70(1):47-59.
* "Separation of nearly identical repeats in shotgun assemblies using defined nucleotide positions, DNPs." Tammi MT, Arner E, Britton T, Andersson B, ''Bioinformatics'', 2002. 18(3):379-88.

 

===6.3. Contig construction===

{| width="100%"
| width="50%" |
====UMD Contigger====
| class="right" | <code>STATUS: development</code>
|}

The <code>UMD contigger</code> uses the set of read overlaps generated during the overlap stage in order to identify unambiguous contigs - maximally consistent tilings of reads. These contigs represent stretches of the genome that can be unambiguously assembled and form a convenient backbone for further processing.

 

===6.4. Consensus===

{| width="100%"
| width="50%" |
====libSlice====
| class="right" | <code>STATUS: production</code>
|}

<code>libSlice</code> is a C++ library that provides the user with a parametric implementation of the Churchill-Waterman algorithm for computing the consensus base from a column in a multiple alignment of reads. This task is an essential part of any consensus module. The implementation can be found in the latest distribution under the <code>src/Slice</code> project directory. These C structs comprise the <code>libSlice.a</code> library.

 

{| width="100%"
| width="50%" |
====libAlign====
| class="right" | <code>STATUS: production</code>
|}

<code>libAlign</code> is a robust multi-alignment library for consensus generation. It can efficiently handle large inputs and is able to identify and correctly align slightly misplaced and/or low-similarity reads in the input. The implementation can be found in the latest distribution under the <code>src/Align</code> project directory. These classes comprise the <code>libAlign.a</code> library and depend on the [#libslice libSlice] library.

 

===6.5. Scaffolding===

{| width="100%"
| width="50%" |
====[docs/bambus/ Bambus]====
| class="right" | <code>STATUS: production</code>
|}

<code>Bambus</code> is the first general purpose scaffolder that is publicly available as an open source package. While most other scaffolders are closely tied to a specific assembly program, Bambus accepts the output from most current assemblers and provides the user with great flexibility in choosing the scaffolding parameters. In particular, Bambus is able to accept contig linking data other than specified by mate-pairs. Such sources of information include alignment to a reference genome (Bambus can directly use the output of MUMmer), physical mapping data, or information about gene synteny.

=====Related publications=====

* "Hierarchical scaffolding with Bambus." Pop M, Kosack DS, Salzberg SL, ''Genome Research'', 2004. 14(1):149-59.

 

===6.6. Error correction===

{| width="100%"
| width="50%" |
====[http://www.tigr.org/software/autoeditor/ AutoEditor]====
| class="right" | <code>STATUS: production</code>
|}

The <code>AutoEditor</code> is a tool developed at TIGR that combines the trace information with the tiling of reads within a contig in order to identify and correct sequencing errors. Note that unlike other methods for error correction, the Auto Editor will only modify a base if supporting evidence is found in the traces, thus greatly reducing the possibility of errors. In our tests the Auto Editor corrected up to 90% of the sequencing errors present in the data, leading to a corresponding reduction in the manual labor required during the finishing stages.

=====Related publications=====

* "Automated correction of genome sequence errors." Gajer P, Schatz M, Salzberg SL, ''Nucleic Acids Research'', 2004. 32(2):562-9.

 

{| width="100%"
| width="50%" |
====UMD error corrector====
| class="right" | <code>STATUS: testing</code>
|}

In conjunction with the UMD overlapper, the <code>UMD error corrector</code> identifies and corrects potential sequencing errors by detecting bases in a multiple alignment of reads that are supported by only one of the reads. The algorithm uses a heuristic rule called the 4-3 rule that examines overlapping sets of 4 reads at 3 positions in order to identify differences corresponding to distinct copies of a repeat.

=====Related publications=====

* "A preprocessor for shotgun assembly of large genomes." Roberts M, Hunt BR, Yorke JA, Bolanos R, Delcher A, ''Journal of Computational Biology'' (to appear)

 

{| width="100%"
| width="50%" |
====KI error corrector====
| class="right" | <code>STATUS: testing</code>
|}

Sequencing errors in combination with repeated regions cause major problems in shotgun sequencing, mainly due to the failure of assembly programs to distinguish single base differences between repeat copies from erroneous base calls. The <code>Karolinska Institutet error corrector</code> implements a new strategy to correct errors in shotgun sequence data using defined nucleotide positions, DNPs. The method distinguishes single base differences from sequencing errors by analyzing multiple alignments consisting of a read and all its overlaps with other reads. The construction of multiple alignments is performed using a novel pattern matching algorithm.

=====Related publications=====

* "Correcting errors in shotgun sequences." Tammi MT, Arner E, Kindlund E, Andersson B, ''Nucleic Acids Research'', 2003. 31(15):4663-72.
* "TRAP: Tandem Repeat Assembly Program produces improved shotgun assemblies of repetitive sequences." Tammi MT, Arner E, Andersson B, ''Computational Methods Programs Biomed'', 2003. 70(1):47-59.
* "Separation of nearly identical repeats in shotgun assemblies using defined nucleotide positions, DNPs." Tammi MT, Arner E, Britton T, Andersson B, ''Bioinformatics'', 2002. 18(3):379-88.

 

===6.7. Validation===

{| width="100%"
| width="50%" |
====[/forensics/ amosvalidate]====
| class="right" | <code>STATUS: production</code>
|}

<code>amosvalidate</code> is a validation pipeline for genome assemblies. This pipeline includes a collection of methods for ascertaining the quality of an assembly, and examines multiple measures of assembly quality to pinpoint potential mis-assemblies. Validation techniques include mate-pair validation, repeat analysis, coverage analysis, identification of correlated read polymorphisms, and read alignment breakpoint analysis. Regions of the assembly exhibiting multiple signatures of mis-assembly are flagged as suspicious and output by amosvalidate for further examination.

 

{| width="100%"
| width="50%" |
====[http://www.tigr.org/tdb/benchmark/ Benchmark]====
| class="right" | <code>STATUS: production</code>
|}

TIGR has assembled a set of <code>benchmark</code> assembly genomes. Each benchmark set comes with the sequence of the finished genome, random shotgun reads, closure reads, and ancillary library and insert information. Each sequence is categorized as matching or non-matching, based on its mapping to the finished genome. Sequences that match the finished genome at 90% identity for over 80% of their trimmed length (as aligned by [#mummer MUMmer]) are included in the matching set, while all other reads are grouped into the non-matching set. Ancillary information is presented in Trace Archive XML format. Please refer the the benchmark website for a more lengthy description and the actual data.

 

===6.8. [docs/utilities/ Utilities]===

{| width="100%"
| width="50%" |
====[docs/converters ASM File Converters]====
| class="right" | <code>STATUS: development - production</code>
|}

The <code>ASM File converters</code> are a collection of utilities for converting sequence and assembly data between the most widely used data formats as well as to and from the AMOS message format. Examples of the data handled by these utilities are: Trace Archive data and ancillary information, .ACE assembly format, TIGR Assembler input and output formats, Celera Assembler message format, and Arachne input and output formats.

 

{| width="100%"
| width="50%" |
====[docs/utilities/amoslib.html amoslib]====
| class="right" | <code>STATUS: production</code>
|}

<code>amoslib</code> is a PERL module for the handling of AMOS message files.

 

{| width="100%"
| width="50%" |
====[http://www.tigr.org/software/mummer MUMmer]====
| class="right" | <code>STATUS: production</code>
|}

<code>MUMmer</code> is a whole-genome alignment software suite developed and maintained by TIGR. It is extremely useful to mapping sequencing reads and assembly contigs to finished sequence. This can provide valuable information for assembly quality assessment and for comparative genomics.

 

{| width="100%"
| width="50%" |
====[hawkeye Hawkeye]====
| class="right" | <code>STATUS: production</code>
|}

<code>Hawkeye</code> serves as a visualization tool for AMOS assembly data. It works off of an AMOS data bank and produces an interactive display of the assembly data for quality assessment and assembly investigation. It was developed with the Qt GUI toolkit. 

==8. [http://sourceforge.net/project/showfiles.php?group_id=134326 Download]==

The AMOS source if freely available for download from the [http://sourceforge.net/project/showfiles.php?group_id=134326 File Release Section] of our [http://sourceforge.net/projects/amos SourceForge project page]. Please refer to the COPYING license included in the package for a description of the [http://www.opensource.org/licenses/artistic-license.php Artistic License], the same [http://www.opensource.org/docs/definition.php OSI certified open source] license used by Perl and countless other packages. Not all of the above packages are included with the standard AMOS distribution, please see the homepage for the software you wish to download to verify that it is included with the AMOS source distribution. 

==9. [http://sourceforge.net/projects/amos Join the consortium]==

All interested parties are welcome to join or aid the AMOS consortium. Please address all correspondence via Email to:

[http://lists.sourceforge.net/lists/listinfo/amos-help [[Image:amos-help.gif|amos-help(at)lists(dot)sourceforge(dot)net]]]

To receive information regarding new releases and developments, please [http://lists.sourceforge.net/lists/listinfo/amos-users subscribe] to our moderated, low-traffic users' mailing list:

[http://lists.sourceforge.net/lists/listinfo/amos-users [[Image:amos-users.gif|271px|amos-users(at)lists(dot)sourceforge(dot)net]]] 

==10. [http://sourceforge.net/support/getsupport.php?group_id=134326 Bug reports and support]==

For AMOS bug reports or support requests, please browse our [http://sourceforge.net/projects/amos SourceForge project page] or Email us at:

[http://lists.sourceforge.net/lists/listinfo/amos-help [[Image:amos-help.gif|amos-help(at)lists(dot)sourceforge(dot)net]]] 
|}

----

Cbcb:Pop-Lab:Dan-Report

2009-02-23T16:57:54Z

Dsommer: /* Feb 23, 2009 */

== Feb 23, 2009 ==
*'''EST Read Annotation of the Coral Reef'''
Performed a read analysis of EST sequences from soft corals.
The analysis followed the methods from the paper [http://aem.asm.org/cgi/content/abstract/AEM.02181-07v1 Metagenomics: Read length matters].

'''Short description of the method'''
* Homology detection by blasting(blastx) EST reads against NR database
* Use CDD Search to find conserved domains in detected proteins
* Blast proteins against COG database to map proteins to COGs and functional groups

*'''Diva: Comparative analysis tool'''
Finishing the development of Google Gears version of the viewer. Developed client side/javascript support of GFF relational database

*'''Grid Upgrade'''
Transitioned RunCmd software packaged to Condor version 7.
Worked with IT to upgrade grid to latest Redhat 64-bit OS.

Cbcb:Pop-Lab:Dan-Report

2009-02-23T16:57:27Z

Dsommer: /* Short description of the method */

== Feb 23, 2009 ==
*'''EST Read Annotation of the Coral Reef'''
Did a read analysis on EST sequences from soft corals.
The analysis followed the methods from the paper [http://aem.asm.org/cgi/content/abstract/AEM.02181-07v1 Metagenomics: Read length matters].

'''Short description of the method'''
* Homology detection by blasting(blastx) EST reads against NR database
* Use CDD Search to find conserved domains in detected proteins
* Blast proteins against COG database to map proteins to COGs and functional groups

*'''Diva: Comparative analysis tool'''
Finishing the development of Google Gears version of the viewer. Developed client side/javascript support of GFF relational database

*'''Grid Upgrade'''
Transitioned RunCmd software packaged to Condor version 7.
Worked with IT to upgrade grid to latest Redhat 64-bit OS.

Cbcb:Pop-Lab:Dan-Report

2009-02-23T16:56:45Z

Dsommer: /* Feb 23, 2009 */

== Feb 23, 2009 ==
*'''EST Read Annotation of the Coral Reef'''
Did a read analysis on EST sequences from soft corals.
The analysis followed the methods from the paper [http://aem.asm.org/cgi/content/abstract/AEM.02181-07v1 Metagenomics: Read length matters].

==Short description of the method==
* Homology detection by blasting(blastx) EST reads against NR database
* Use CDD Search to find conserved domains in detected proteins
* Blast proteins against COG database to map proteins to COGs and functional groups

*'''Diva: Comparative analysis tool'''
Finishing the development of Google Gears version of the viewer. Developed client side/javascript support of GFF relational database

*'''Grid Upgrade'''
Transitioned RunCmd software packaged to Condor version 7.
Worked with IT to upgrade grid to latest Redhat 64-bit OS.

Cbcb:Pop-Lab:Dan-Report

2009-02-23T16:30:42Z

Dsommer: /* Feb 23, 2009 */

== Feb 23, 2009 ==
*'''EST Read Annotation of the Coral Reef''' [http://aem.asm.org/cgi/content/abstract/AEM.02181-07v1 Metagenomics: Read length matters]
*'''Diva: Comparative analysis tool''' (more will come)
*'''Short Read Assembly: Minimus-SR''' (more will come)

Cbcb:Pop-Lab:Dan-Report

2009-02-23T16:21:09Z

Dsommer:

== Feb 23, 2009 ==
*'''EST Read Annotation of the Coral Reef''' (more will come)
*'''Antibiotic Resistance Genes in Metagenome.''' (more will come)
*'''Short Read Assembly: Minimus-SR''' (more will come)

Cbcb:Pop-Lab

2009-02-23T16:16:57Z

Dsommer: /* Progress Reports */

= Important pages =
* Group members [[Cbcb:Pop-Lab:Members]]
* Blog: [http://genomes.blogspot.com Genomes.blogspot.com]
* How-tos: [[Cbcb:Pop-Lab:How-to]] - List of documentation on how to perform certain analyses, use different software, etc.
* Software developed by us: [[Cbcb:Pop-Lab:Software]]
* Metagenomics papers: [[Cbcb:Pop-Lab:Papers]]

= Meetings/seminars =
* [[Pop_group_meeting|Group meeting]]: every Monday from 2:30pm in corner conference room (3120 C)
* [[seminars|CBCB seminar]]: Thursdays 2-3pm during the semester in main conference room

= Progress Reports =
* [[Cbcb:Pop-Lab:Ben-Report|Ben]]
* [[Cbcb:Pop-Lab:Niranjan-Report|Niranjan]]
* [[Cbcb:Pop-Lab:Chris-Report|Chris]]
* [[Cbcb:Pop-Lab:James-Report|James]]
* [[Cbcb:Pop-Lab:Serge-Report|Serge]]
* [[Cbcb:Pop-Lab:Bo-Report|Bo]]
* [[Cbcb:Pop-Lab:Dan-Report|Dan]]

= Pop Lab Presentations =
* [[Media:Xoo_Cripsr.odg|Xanthomonas Crispr Slides (openoffice format)]]
* Listeria Monocytogenes Slides (powerpoint)
* Metagenome pipelines overview [[Media:Pipeline_outline.pdf|Presentation(pdf)]]

Pop group meeting

2008-10-20T16:54:45Z

Dsommer: /* 10/13/2008 */

= 09/29/2008 =

# Benjamin Langmead [http://www.plosone.org/article/info:doi%2F10.1371%2Fjournal.pone.0003263 Metagenomic Analysis of Lysogeny in Tampa Bay: Implications for Prophage Gene Expression] (PLoS one)
# Bo Liu [http://genomebiology.com/2008/9/9/R139 Systematic bioinformatic analysis of expression levels of 17,330 human genes across 9,783 samples from 175 types of healthy and pathological tissues] (Genome Biology)
# Daniela Puiu [http://genome.cshlp.org/cgi/reprint/gr.080200.108v1 Sequencing of natural strains of Arabidopsis thaliana with short reads] (Genome Research)
# Dan Sommer [http://www.nature.com/nature/journal/v455/n7212/edsumm/e080925-04.html Metagenomics: questions to answer] (Nature)
# James Robert White [http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6V73-4SHVST0-1&_user=961305&_rdoc=1&_fmt=&_orig=search&_sort=d&view=c&_version=1&_urlVersion=0&_userid=961305&md5=4115e7b8186a36db60f6b11a217a8798 Microbial population dynamics during aerobic sludge granulation at different organic loading rates] (ScienceDirect)
# Mohammad Reza Ghodsi [http://nar.oxfordjournals.org/cgi/content/short/36/16/5180 A comparison of random sequence reads versus 16S rDNA sequences for estimating the biodiversity of a metagenomic library] (NAR)
# Niranjan Nagarajan [http://www.nature.com/nature/journal/v455/n7212/edsumm/e080925-04.html Metagenomics: questions to answer] (Nature)
# Sergey Koren [http://www.biomedcentral.com/1471-2105/9/390 R/parallel - speeding up bioinformatics analysis with R] (BMC Bioinformatics)
# Theodore Gibbons [http://www.ncbi.nlm.nih.gov/pubmed/18711340 High-resolution metagenomics targets specific functional types in complex microbial communities] (Nature Biotechnology) [http://www.nature.com/nrmicro/journal/v6/n9/abs/nrmicro1935.html Molecular eco-systems biology: towards an understanding of community function] (Systems Microbiology)

= 10/06/2008 =
# Bo Liu [http://www.pnas.org/content/105/39/15076.short The convergence of carbohydrate active gene repertoires in human gut microbes.] PNAS
# Daniela Puiu [http://bioinformatics.oxfordjournals.org/cgi/content/full/24/20/2395 SeqMap: mapping massive amount of oligonucleotides to the genome] Bioinformatics
# James Robert White [http://www.pnas.org/content/104/21/8918.full Artificial selection of simulated microbial ecosystems.] PNAS
# MohammadReza Ghodsi [http://aem.asm.org/cgi/content/abstract/74/5/1453 Metagenomics: Read Length Matters] Applied Environmental Microbiology

= 10/13/2008 =
# Daniela Puiu (Daniel Sommer, same paper) [http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0003373 MetaSim—A Sequencing Simulator for Genomics and Metagenomics] PLoS ONE
MetaSim software:
Location: /fs/sz-user-supported/common/packages/MetaSim/
Executable(Java): /fs/sz-user-supported/common/packages/MetaSim/metasim/MetaSim
Database: /fs/sz-user-supported/common/packages/MetaSim/database # create a symlink to it from your working directory, otherwise you have to import it
NCBI databases: [ftp://ftp.ncbi.nlm.nih.gov/genomes/Bacteria/all.fna.tar.gz Complete bacteria]; [ftp://ftp.ncbi.nlm.nih.gov/pub/taxonomy/gi_taxid_nucl.dmp.gz Taxonomy]

# Bo Liu & James Robert White [http://www.sciencemag.org/cgi/content/full/322/5899/275 Environmental Genomics Reveals a Single-Species Ecosystem Deep Within Earth.] Science
# MohammadReza Ghodsi [http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2557142 Probing Metagenomics by Rapid Cluster Analysis of Very Large Datasets] (PLoS one)

= 10/20/2008 =
# Daniel Sommer [http://www.ncbi.nlm.nih.gov/pubmed/18927115?dopt=Abstract] NCBI Reference Sequences: current status, policy and new initiatives

Pop group meeting

2008-10-20T16:52:11Z

Dsommer: /* 10/13/2008 */

Cbcb:Works-In-Progress

2008-07-29T18:28:41Z