https://wiki.umiacs.umd.edu/cbcb/api.php?action=feedcontributions&feedformat=atom&user=GhodsiCbcb - User contributions [en]2026-04-12T19:12:46ZUser contributionsMediaWiki 1.43.7https://wiki.umiacs.umd.edu/cbcb/index.php?title=Cbcb:Pop-Lab:16S-pipeline_16S_analysis_pipeline_(for_Gates_project)&diff=7584Cbcb:Pop-Lab:16S-pipeline 16S analysis pipeline (for Gates project)2010-09-27T19:16:29Z<p>Ghodsi: /* Step 6: Run clustering tool */</p>
<hr />
<div>= 16S analysis pipeline =<br />
<br />
== Overview ==<br />
=== Assumptions ===<br />
* 16S rRNA sequences are generated with barcoded 454 and are received as either: (i) .sff file; (ii) fasta and quality file; (iii) just fasta file<br />
* each batch is a single 96-well plate and is accompanied by a tab-delimited file containing information about the sample, including the "Sample ID", well on the plate, and additional information regarding the sample quality and DNA concentration<br />
* we have a file that specifies the barcode and sequencing adapter used for each well (these entities do not change).<br />
<br />
=== Directory structure ===<br />
<pre><br />
Gates_SOM/<br />
Main/<br />
samples.csv - information about all the samples available to us<br />
454.csv - information about all 454 runs (essentially concatenation of .csvs from 454 dir)<br />
phylochip.csv - information about all Phylochip runs<br />
IGS_Barcodes.csv - information about barcodes used to multiplex 454 runs<br />
scripts/ - scripts used to process the data (referred to as ${SCRIPTS} below)<br />
DB/ - scripts used to access the database (referred to as ${DB_SCRIPTS} below)<br />
454/ - here's where all 454 sequences live<br />
[batch1]/ - ... each batch in a separate directory<br />
[batch1].csv - meta-information about the batch <br />
[fasta1] - fasta files containing the batch<br />
... <br />
[fastan]<br />
[batch1].part - partition file describing how the sequences get split by barcode/sample<br />
part/ - directory where all the partitioned files live <br />
... <br />
[batchn]<br />
Phylochip/ - all the CEL files and auxiliary information on the Phylochip runs<br />
</pre><br />
<br />
== File-based approach == <br />
<br />
=== Step 0: Get the sequence information ===<br />
* From .SFF files (assuming these are 454 sequences)<br />
<br />
This step uses the sff_extract program from the Staden package (if I'm not mistaken)<br />
<pre><br />
for i in *.sff ;do<br />
name=`expr $i : '\(.*\)\.sff'`<br />
sff_extract -c -s $name.seq -q $name.qual $i<br />
done<br />
</pre><br />
<br />
=== Step 1: Cleanup meta-information ===<br />
* Convert the Excel sheet containing the batch information into tab-delimited file (and run dos2unix) making sure the quotes added by Excel/OOffice are removed, adding the date (if not already in the file), and sorting the file by Sample ID. At this stage also check that the header row information is in canonical format.<br />
* Add barcode information using add_barcode.pl<br />
<br />
<pre>${SCRIPTDIR}/add_barcode.pl [batch].csv ${MAINDIR}IGS_Barcodes.csv > [batch]_barcode.csv</pre><br />
<br />
=== Step 2: Create partition file ===<br />
<br />
First concatenate all the sequence files (if multiple) from a batch into a single file [batch].all.seq.<br />
<br />
<pre>${SCRIPTDIR}/code2part.pl [batch].csv [batch].all.seq [batch] > [batch].part</pre><br />
<br />
The files [batch].BAD.list and [batch].NONE.list contain the names of all sequences that have failed quality checks either because they are too short, or contain Ns (BAD), or because the barcode is not recognized (NONE). The files contain additional information that can be used to troubleshoot the pipeline: sequences that are too short (< 75 454 cycles) are followed by the number of cycles, sequences that either contain Ns or have an unknown barcode are followed by the first 8 characters in the sequence.<br />
<br />
=== Step 3: Break up the fasta file into separate batches by partition ===<br />
<br />
* Create partition directory<br />
<br />
<pre><br />
mkdir part<br />
cd part<br />
</pre><br />
<br />
* Partition main file into sub-parts<br />
<pre><br />
${SCRIPTS}/partitionFasta.pl ../[batch].all.seq ../[batch].part<br />
</pre><br />
<br />
The result is one fasta file per sub-partition (i.e. individual subject).<br />
<br />
* Remove barcodes <br />
(still in the part/ subdirectory)<br />
<pre><br />
for i in *.seq; do<br />
${SCRIPTS}/unbarcode.pl $i<br />
done<br />
</pre><br />
<br />
The output files will have the same name as the original file but with the addition of the .nbc suffix. You should remove the .nbc files from the BAD/NONE files in order to prevent their addition to the pipeline downstream.<br />
<br />
<pre> rm *.BAD.nbc.fa *.NONE.nbc.fa </pre><br />
<br />
=== Step 4: Add file names to sample tables ===<br />
<br />
The following needs to be run from the root of the 454 directory.<br />
<br />
<pre> ${SCRIPTS}/add_file.pl [batch]/[batch]_barcode.csv [batch]/part > [batch]/[batch]_names.csv </pre><br />
<br />
As a result, the file [batch]/[batch]_names.csv will associate each Sample ID to a file name and also record the number of sequences in that file. Note that only files ending in .nbc.fa are procesed.<br />
<br />
=== Step 5: Merge all the batch meta-info files into a same file at the top ===<br />
<br />
Note: the addition of file names must be done on a batch by batch basis as multiple files might refer to a same Sample ID - within each batch it can be assumed that the Sample ID -> Filename mapping is unique. In the 454.csv file in the top directory the unique key is the file name. <br />
<br />
<pre> <br />
mv ../454.csv ../454.csv.bak<br />
${SCRIPTS}/merge_csv.pl ../454.csv.bak [batch]/[batch]_names.csv "Sample ID" non-unique > ../454.csv<br />
</pre><br />
<br />
This should probably be run with the filename as the key but then the tables need to be sorted by filename. Ultimately, this will all work better once the data are in a relational database.<br />
<br />
=== Step 6: Update sample file ===<br />
From the top directory:<br />
<br />
* First add all new samples to the samples.csv file<br />
<pre> <br />
mv samples.csv samples.csv.bak<br />
${SCRIPTS}/merge_csv.pl samples.csv.bak 454.csv "Sample ID" unique merge > samples.csv<br />
</pre><br />
<br />
Note: merge means that if record keys conflict, the empty fields will be updated with the new data.<br />
<br />
* Update the tag indicating 454 sequences available for this sample<br />
<br />
<pre><br />
mv samples.csv samples.csv.bak<br />
${SCRIPTS}/update_field_csv.pl samples.csv.bak 454.csv "Sample ID" 454 Y > samples.csv<br />
</pre><br />
<br />
=== Step 7: Assign numbers to all filenames ===<br />
<br />
Each file in the 454.csv file will be assigned an integer (if one is not already available). This number will be used to prefix the sequences in the combined file for the project.<br />
<br />
<pre><br />
mv 454.csv 454.csv.bak<br />
${SCRIPTS}/add_filenum.pl 454.csv.bak > 454.csv<br />
</pre><br />
<br />
=== Step 8: Combine all fasta files into a single one ===<br />
<br />
'''Note: this assumes we're running the pipeline for the first time - a different protocol is necessary for adding new sequences to an already existing analysis'''<br />
<br />
In the new file all sequences will be named <n>_<nn> where <n> is the value in the "File #"<br />
field in 454.csv and <nn> is the index of the sequence in the file.<br />
<br />
<pre><br />
mkdir Analysis/Run[date]<br />
${SCRIPTS}/combinefa.pl -c Analysis/Run[date]/Run[date] -i 454.csv 454<br />
</pre><br />
<br />
The output will be in Analysis/Run[date]/Run[date].fna<br />
<br />
=== Step 9: Run clustering tool ===<br />
<br />
* First generate clusters<br />
'''Note: This part assumes we're running the whole set of sequences as one batch.'''<br />
<pre><br />
cd Analysis/Run[date]/Run[date].fna<br />
/fs/szasmg2/ghodsi/Src/dnaclust/dnaclust -i Run[date].fna > Run[date].fna.cluster<br />
/fs/szasmg2/ghodsi/Src/dnaclust/dnaclust -m -i Run[date].fna > Run[date].fna.align<br />
</pre><br />
<br />
Output will be in Run[date].fna.cluster, one cluster per line, cluster center listed as the first identifier.<br />
<br />
The .align file contains aligned FASTA records for all the sequences in each cluster. Clusters are separated by #<number> where <number> is the number of sequences in the cluster.<br />
<br />
<br />
* Then extract the cluster centers<br />
'''From here on the code runs the same in both full-run and batch modes'''<br />
<pre><br />
/fs/szasmg2/ghodsi/Src/dnaclust/fastaselect Run[date].fna < Run[date].fna.cluster > Run[date].centers.fna<br />
</pre><br />
<br />
=== Step 10: Assign putative taxonomic labels to clusters ===<br />
<br />
<pre><br />
/fs/szasmg2/ghodsi/rdp/findtax/findtaxid.sh Run[date].centers.fna > Run[date].centers.taxid<br />
</pre><br />
<br />
Output is tab-delimited: sequence name <TAB> taxid<br />
Note: using "findtax" instead of "findtaxid" will retrieve actual taxonomy names.<br />
<br />
=== Step 11: Build summary tables ===<br />
<br />
Using the output from steps 9 and 10 we construct a collection of tables linking OTUs, taxIDs, taxnames at various taxonomic levels to individual samples. The colums are the samples and the rows are the respective units. The cells are numbers of sequences assigned to the specific group. If looking at taxonomic levels, the sequences without an assignment at that level are assigned to a generic "No Assignment" bin.<br />
<br />
* First create a partition file that contains all the clusters and associated taxids (if they exist)<br />
<br />
<pre><br />
${SCRIPTS}/clusters2part.pl [batch].fna.cluster [batch] [batch].centers.taxid > [batch].part<br />
</pre><br />
<br />
* Using this partition, construct summary tables at various taxonomic levels<br />
<br />
<pre><br />
${SCRIPTS}/taxpart2summary.pl [batch].part ${MAIN}/454.csv [batch]<br />
</pre><br />
<br />
The following outputs will be created:<br />
<br />
<pre><br />
[batch].stats.txt - overall statistics for the data-set<br />
[batch].otus.count.csv - table containing OTUs as rows, samples as columns, and entries represent<br />
# of sequences in each OTU/Sample pair<br />
[batch].otus.percent.csv - same as "count" except that entries are percentages wrt total sequences<br />
in each sample<br />
[batch].[tax].[otu|count|percent] - same as the "otus" file except at varying taxonomic levels.<br />
[tax] is one of "strain", "species", "genus", "family", "order", "class", "phylum"<br />
the "count" and "percent" entries are the same as for the "otus" files<br />
the "otu" entries contain number of OTUs assigned to the taxonomic group/Sample pair.<br />
<br />
</pre><br />
<br />
== Database-based approach ==<br />
<br />
=== Database information === <br />
* To access the database through a command line use the command shown below with password "access":<br />
<pre><br />
mysql -u access -p -h cbcbmysql00 gems<br />
</pre><br />
* A stub Perl script for playing with the database is provided at /fs/szasmg2/Gates_SOM/Main/454/DB/stub.pl<br />
* All users can read the data from the database using users/password combo "access"/"access".<br />
* Database schema: [[Media:Schema.pdf]]<br />
* The commands listed below assume you have write access to the database.<br />
<br />
=== Step 0: Get the sequence information ===<br />
* From .SFF files (assuming these are 454 sequences)<br />
<br />
This step uses the sff_extract program from the Staden package (if I'm not mistaken)<br />
<pre><br />
for i in *.sff ;do<br />
name=`expr $i : '\(.*\)\.sff'`<br />
sff_extract -c -s $name.seq -q $name.qual $i<br />
done<br />
</pre><br />
<br />
=== Step 1: Cleanup meta-information ===<br />
* Convert the Excel sheet containing the batch information into tab-delimited file (and run dos2unix) making sure the quotes added by Excel/OOffice are removed, adding the date (if not already in the file), and sorting the file by Sample ID. At this stage also check that the header row information is in canonical format.<br />
* Add barcode information using add_barcode.pl<br />
<br />
<pre>${SCRIPTDIR}/add_barcode.pl [batch].csv ${MAINDIR}IGS_Barcodes.csv > [batch]_barcode.csv</pre><br />
<br />
=== Step 2: Create partition file ===<br />
<br />
First concatenate all the sequence files (if multiple) from a batch into a single file [batch].all.seq.<br />
<br />
<pre>${SCRIPTDIR}/code2part.pl [batch].csv [batch].all.seq [batch] > [batch].part</pre><br />
<br />
The files [batch].BAD.list and [batch].NONE.list contain the names of all sequences that have failed quality checks either because they are too short, or contain Ns (BAD), or because the barcode is not recognized (NONE). The files contain additional information that can be used to troubleshoot the pipeline: sequences that are too short (< 75 454 cycles) are followed by the number of cycles, sequences that either contain Ns or have an unknown barcode are followed by the first 8 characters in the sequence.<br />
<br />
=== Step 3: Break up the fasta file into separate batches by partition ===<br />
<br />
* Create partition directory<br />
<br />
<pre><br />
mkdir part<br />
cd part<br />
</pre><br />
<br />
* Partition main file into sub-parts<br />
<pre><br />
${SCRIPTS}/partitionFasta.pl ../[batch].all.seq ../[batch].part<br />
</pre><br />
<br />
The result is one fasta file per sub-partition (i.e. individual subject).<br />
<br />
* Remove barcodes <br />
(still in the part/ subdirectory)<br />
<pre><br />
for i in *.seq; do<br />
${SCRIPTS}/unbarcode.pl $i<br />
done<br />
</pre><br />
<br />
The output files will have the same name as the original file but with the addition of the .nbc suffix. You should remove the .nbc files from the BAD/NONE files in order to prevent their addition to the pipeline downstream.<br />
<br />
<pre> rm *.BAD.nbc.fa *.NONE.nbc.fa </pre><br />
<br />
=== Step 4: Upload file names ===<br />
Run from 454 directory<br />
<pre><br />
${DB_SCRIPTS}/add_file_db.pl [batch]/[batch]_barcode.csv [batch]/part<br />
</pre><br />
<br />
=== Step 5: Combine all fasta files into a single one ===<br />
<br />
'''Note: this assumes we're running the pipeline for the first time - a different protocol is necessary for adding new sequences to an already existing analysis'''<br />
<br />
In the new file all sequences will be named <n>_<nn> where <n> is the value in the "File #"<br />
field in 454.csv and <nn> is the index of the sequence in the file.<br />
<br />
<pre><br />
mkdir Analysis/Run[date]<br />
${DB_SCRIPTS}/combinefa_db.pl -c Analysis/Run[date]/Run[date] 454<br />
</pre><br />
<br />
The output will be in Analysis/Run[date]/Run[date].fna<br />
<br />
=== Step 6: Run clustering tool ===<br />
<br />
* First generate clusters<br />
'''Note: This part assumes we're running the whole set of sequences as one batch.'''<br />
<pre><br />
cd Analysis/Run[date]/Run[date].fna<br />
/fs/szasmg2/ghodsi/Src/dnaclust/dnaclust -i Run[date].fna > Run[date].fna.cluster<br />
/fs/szasmg2/ghodsi/Src/dnaclust/dnaclust -m -i Run[date].fna > Run[date].fna.align<br />
</pre><br />
<br />
Output will be in Run[date].fna.cluster, one cluster per line, cluster center listed as the first identifier.<br />
<br />
The .align file contains aligned FASTA records for all the sequences in each cluster. Clusters are separated by #<number> where <number> is the number of sequences in the cluster.<br />
<br />
<br />
* Then extract the cluster centers<br />
'''From here on the code runs the same in both full-run and batch modes'''<br />
<pre><br />
/fs/szasmg2/ghodsi/Src/dnaclust/fastaselect Run[date].fna < Run[date].fna.cluster > Run[date].centers.fna<br />
</pre><br />
<br />
=== Step 7: Assign putative taxonomic labels to clusters ===<br />
<br />
<pre><br />
/fs/szasmg2/ghodsi/rdp/findtax/findtaxid.sh Run[date].centers.fna > Run[date].centers.taxid<br />
</pre><br />
<br />
Output is tab-delimited: sequence name <TAB> taxid<br />
Note: using "findtax" instead of "findtaxid" will retrieve actual taxonomy names.<br />
<br />
=== Step 8: Upload OTU information into the database ===<br />
* First create a partition file that contains all the clusters and associated taxids (if they exist)<br />
<br />
<pre><br />
${SCRIPTS}/cluster2part.pl [batch].fna.cluster [batch] [batch].centers.taxid > [batch].part<br />
</pre><br />
<br />
* Then upload the resulting partition to the database<br />
<pre><br />
${DB_SCRIPTS}/upload_otus.pl [batch].part<br />
</pre><br />
<br />
=== Step 9: Generate summary tables ===</div>Ghodsihttps://wiki.umiacs.umd.edu/cbcb/index.php?title=Cbcb:Pop-Lab:16S-pipeline_16S_analysis_pipeline_(for_Gates_project)&diff=7583Cbcb:Pop-Lab:16S-pipeline 16S analysis pipeline (for Gates project)2010-09-27T19:14:21Z<p>Ghodsi: /* Step 9: Run clustering tool */</p>
<hr />
<div>= 16S analysis pipeline =<br />
<br />
== Overview ==<br />
=== Assumptions ===<br />
* 16S rRNA sequences are generated with barcoded 454 and are received as either: (i) .sff file; (ii) fasta and quality file; (iii) just fasta file<br />
* each batch is a single 96-well plate and is accompanied by a tab-delimited file containing information about the sample, including the "Sample ID", well on the plate, and additional information regarding the sample quality and DNA concentration<br />
* we have a file that specifies the barcode and sequencing adapter used for each well (these entities do not change).<br />
<br />
=== Directory structure ===<br />
<pre><br />
Gates_SOM/<br />
Main/<br />
samples.csv - information about all the samples available to us<br />
454.csv - information about all 454 runs (essentially concatenation of .csvs from 454 dir)<br />
phylochip.csv - information about all Phylochip runs<br />
IGS_Barcodes.csv - information about barcodes used to multiplex 454 runs<br />
scripts/ - scripts used to process the data (referred to as ${SCRIPTS} below)<br />
DB/ - scripts used to access the database (referred to as ${DB_SCRIPTS} below)<br />
454/ - here's where all 454 sequences live<br />
[batch1]/ - ... each batch in a separate directory<br />
[batch1].csv - meta-information about the batch <br />
[fasta1] - fasta files containing the batch<br />
... <br />
[fastan]<br />
[batch1].part - partition file describing how the sequences get split by barcode/sample<br />
part/ - directory where all the partitioned files live <br />
... <br />
[batchn]<br />
Phylochip/ - all the CEL files and auxiliary information on the Phylochip runs<br />
</pre><br />
<br />
== File-based approach == <br />
<br />
=== Step 0: Get the sequence information ===<br />
* From .SFF files (assuming these are 454 sequences)<br />
<br />
This step uses the sff_extract program from the Staden package (if I'm not mistaken)<br />
<pre><br />
for i in *.sff ;do<br />
name=`expr $i : '\(.*\)\.sff'`<br />
sff_extract -c -s $name.seq -q $name.qual $i<br />
done<br />
</pre><br />
<br />
=== Step 1: Cleanup meta-information ===<br />
* Convert the Excel sheet containing the batch information into tab-delimited file (and run dos2unix) making sure the quotes added by Excel/OOffice are removed, adding the date (if not already in the file), and sorting the file by Sample ID. At this stage also check that the header row information is in canonical format.<br />
* Add barcode information using add_barcode.pl<br />
<br />
<pre>${SCRIPTDIR}/add_barcode.pl [batch].csv ${MAINDIR}IGS_Barcodes.csv > [batch]_barcode.csv</pre><br />
<br />
=== Step 2: Create partition file ===<br />
<br />
First concatenate all the sequence files (if multiple) from a batch into a single file [batch].all.seq.<br />
<br />
<pre>${SCRIPTDIR}/code2part.pl [batch].csv [batch].all.seq [batch] > [batch].part</pre><br />
<br />
The files [batch].BAD.list and [batch].NONE.list contain the names of all sequences that have failed quality checks either because they are too short, or contain Ns (BAD), or because the barcode is not recognized (NONE). The files contain additional information that can be used to troubleshoot the pipeline: sequences that are too short (< 75 454 cycles) are followed by the number of cycles, sequences that either contain Ns or have an unknown barcode are followed by the first 8 characters in the sequence.<br />
<br />
=== Step 3: Break up the fasta file into separate batches by partition ===<br />
<br />
* Create partition directory<br />
<br />
<pre><br />
mkdir part<br />
cd part<br />
</pre><br />
<br />
* Partition main file into sub-parts<br />
<pre><br />
${SCRIPTS}/partitionFasta.pl ../[batch].all.seq ../[batch].part<br />
</pre><br />
<br />
The result is one fasta file per sub-partition (i.e. individual subject).<br />
<br />
* Remove barcodes <br />
(still in the part/ subdirectory)<br />
<pre><br />
for i in *.seq; do<br />
${SCRIPTS}/unbarcode.pl $i<br />
done<br />
</pre><br />
<br />
The output files will have the same name as the original file but with the addition of the .nbc suffix. You should remove the .nbc files from the BAD/NONE files in order to prevent their addition to the pipeline downstream.<br />
<br />
<pre> rm *.BAD.nbc.fa *.NONE.nbc.fa </pre><br />
<br />
=== Step 4: Add file names to sample tables ===<br />
<br />
The following needs to be run from the root of the 454 directory.<br />
<br />
<pre> ${SCRIPTS}/add_file.pl [batch]/[batch]_barcode.csv [batch]/part > [batch]/[batch]_names.csv </pre><br />
<br />
As a result, the file [batch]/[batch]_names.csv will associate each Sample ID to a file name and also record the number of sequences in that file. Note that only files ending in .nbc.fa are procesed.<br />
<br />
=== Step 5: Merge all the batch meta-info files into a same file at the top ===<br />
<br />
Note: the addition of file names must be done on a batch by batch basis as multiple files might refer to a same Sample ID - within each batch it can be assumed that the Sample ID -> Filename mapping is unique. In the 454.csv file in the top directory the unique key is the file name. <br />
<br />
<pre> <br />
mv ../454.csv ../454.csv.bak<br />
${SCRIPTS}/merge_csv.pl ../454.csv.bak [batch]/[batch]_names.csv "Sample ID" non-unique > ../454.csv<br />
</pre><br />
<br />
This should probably be run with the filename as the key but then the tables need to be sorted by filename. Ultimately, this will all work better once the data are in a relational database.<br />
<br />
=== Step 6: Update sample file ===<br />
From the top directory:<br />
<br />
* First add all new samples to the samples.csv file<br />
<pre> <br />
mv samples.csv samples.csv.bak<br />
${SCRIPTS}/merge_csv.pl samples.csv.bak 454.csv "Sample ID" unique merge > samples.csv<br />
</pre><br />
<br />
Note: merge means that if record keys conflict, the empty fields will be updated with the new data.<br />
<br />
* Update the tag indicating 454 sequences available for this sample<br />
<br />
<pre><br />
mv samples.csv samples.csv.bak<br />
${SCRIPTS}/update_field_csv.pl samples.csv.bak 454.csv "Sample ID" 454 Y > samples.csv<br />
</pre><br />
<br />
=== Step 7: Assign numbers to all filenames ===<br />
<br />
Each file in the 454.csv file will be assigned an integer (if one is not already available). This number will be used to prefix the sequences in the combined file for the project.<br />
<br />
<pre><br />
mv 454.csv 454.csv.bak<br />
${SCRIPTS}/add_filenum.pl 454.csv.bak > 454.csv<br />
</pre><br />
<br />
=== Step 8: Combine all fasta files into a single one ===<br />
<br />
'''Note: this assumes we're running the pipeline for the first time - a different protocol is necessary for adding new sequences to an already existing analysis'''<br />
<br />
In the new file all sequences will be named <n>_<nn> where <n> is the value in the "File #"<br />
field in 454.csv and <nn> is the index of the sequence in the file.<br />
<br />
<pre><br />
mkdir Analysis/Run[date]<br />
${SCRIPTS}/combinefa.pl -c Analysis/Run[date]/Run[date] -i 454.csv 454<br />
</pre><br />
<br />
The output will be in Analysis/Run[date]/Run[date].fna<br />
<br />
=== Step 9: Run clustering tool ===<br />
<br />
* First generate clusters<br />
'''Note: This part assumes we're running the whole set of sequences as one batch.'''<br />
<pre><br />
cd Analysis/Run[date]/Run[date].fna<br />
/fs/szasmg2/ghodsi/Src/dnaclust/dnaclust -i Run[date].fna > Run[date].fna.cluster<br />
/fs/szasmg2/ghodsi/Src/dnaclust/dnaclust -m -i Run[date].fna > Run[date].fna.align<br />
</pre><br />
<br />
Output will be in Run[date].fna.cluster, one cluster per line, cluster center listed as the first identifier.<br />
<br />
The .align file contains aligned FASTA records for all the sequences in each cluster. Clusters are separated by #<number> where <number> is the number of sequences in the cluster.<br />
<br />
<br />
* Then extract the cluster centers<br />
'''From here on the code runs the same in both full-run and batch modes'''<br />
<pre><br />
/fs/szasmg2/ghodsi/Src/dnaclust/fastaselect Run[date].fna < Run[date].fna.cluster > Run[date].centers.fna<br />
</pre><br />
<br />
=== Step 10: Assign putative taxonomic labels to clusters ===<br />
<br />
<pre><br />
/fs/szasmg2/ghodsi/rdp/findtax/findtaxid.sh Run[date].centers.fna > Run[date].centers.taxid<br />
</pre><br />
<br />
Output is tab-delimited: sequence name <TAB> taxid<br />
Note: using "findtax" instead of "findtaxid" will retrieve actual taxonomy names.<br />
<br />
=== Step 11: Build summary tables ===<br />
<br />
Using the output from steps 9 and 10 we construct a collection of tables linking OTUs, taxIDs, taxnames at various taxonomic levels to individual samples. The colums are the samples and the rows are the respective units. The cells are numbers of sequences assigned to the specific group. If looking at taxonomic levels, the sequences without an assignment at that level are assigned to a generic "No Assignment" bin.<br />
<br />
* First create a partition file that contains all the clusters and associated taxids (if they exist)<br />
<br />
<pre><br />
${SCRIPTS}/clusters2part.pl [batch].fna.cluster [batch] [batch].centers.taxid > [batch].part<br />
</pre><br />
<br />
* Using this partition, construct summary tables at various taxonomic levels<br />
<br />
<pre><br />
${SCRIPTS}/taxpart2summary.pl [batch].part ${MAIN}/454.csv [batch]<br />
</pre><br />
<br />
The following outputs will be created:<br />
<br />
<pre><br />
[batch].stats.txt - overall statistics for the data-set<br />
[batch].otus.count.csv - table containing OTUs as rows, samples as columns, and entries represent<br />
# of sequences in each OTU/Sample pair<br />
[batch].otus.percent.csv - same as "count" except that entries are percentages wrt total sequences<br />
in each sample<br />
[batch].[tax].[otu|count|percent] - same as the "otus" file except at varying taxonomic levels.<br />
[tax] is one of "strain", "species", "genus", "family", "order", "class", "phylum"<br />
the "count" and "percent" entries are the same as for the "otus" files<br />
the "otu" entries contain number of OTUs assigned to the taxonomic group/Sample pair.<br />
<br />
</pre><br />
<br />
== Database-based approach ==<br />
<br />
=== Database information === <br />
* To access the database through a command line use the command shown below with password "access":<br />
<pre><br />
mysql -u access -p -h cbcbmysql00 gems<br />
</pre><br />
* A stub Perl script for playing with the database is provided at /fs/szasmg2/Gates_SOM/Main/454/DB/stub.pl<br />
* All users can read the data from the database using users/password combo "access"/"access".<br />
* Database schema: [[Media:Schema.pdf]]<br />
* The commands listed below assume you have write access to the database.<br />
<br />
=== Step 0: Get the sequence information ===<br />
* From .SFF files (assuming these are 454 sequences)<br />
<br />
This step uses the sff_extract program from the Staden package (if I'm not mistaken)<br />
<pre><br />
for i in *.sff ;do<br />
name=`expr $i : '\(.*\)\.sff'`<br />
sff_extract -c -s $name.seq -q $name.qual $i<br />
done<br />
</pre><br />
<br />
=== Step 1: Cleanup meta-information ===<br />
* Convert the Excel sheet containing the batch information into tab-delimited file (and run dos2unix) making sure the quotes added by Excel/OOffice are removed, adding the date (if not already in the file), and sorting the file by Sample ID. At this stage also check that the header row information is in canonical format.<br />
* Add barcode information using add_barcode.pl<br />
<br />
<pre>${SCRIPTDIR}/add_barcode.pl [batch].csv ${MAINDIR}IGS_Barcodes.csv > [batch]_barcode.csv</pre><br />
<br />
=== Step 2: Create partition file ===<br />
<br />
First concatenate all the sequence files (if multiple) from a batch into a single file [batch].all.seq.<br />
<br />
<pre>${SCRIPTDIR}/code2part.pl [batch].csv [batch].all.seq [batch] > [batch].part</pre><br />
<br />
The files [batch].BAD.list and [batch].NONE.list contain the names of all sequences that have failed quality checks either because they are too short, or contain Ns (BAD), or because the barcode is not recognized (NONE). The files contain additional information that can be used to troubleshoot the pipeline: sequences that are too short (< 75 454 cycles) are followed by the number of cycles, sequences that either contain Ns or have an unknown barcode are followed by the first 8 characters in the sequence.<br />
<br />
=== Step 3: Break up the fasta file into separate batches by partition ===<br />
<br />
* Create partition directory<br />
<br />
<pre><br />
mkdir part<br />
cd part<br />
</pre><br />
<br />
* Partition main file into sub-parts<br />
<pre><br />
${SCRIPTS}/partitionFasta.pl ../[batch].all.seq ../[batch].part<br />
</pre><br />
<br />
The result is one fasta file per sub-partition (i.e. individual subject).<br />
<br />
* Remove barcodes <br />
(still in the part/ subdirectory)<br />
<pre><br />
for i in *.seq; do<br />
${SCRIPTS}/unbarcode.pl $i<br />
done<br />
</pre><br />
<br />
The output files will have the same name as the original file but with the addition of the .nbc suffix. You should remove the .nbc files from the BAD/NONE files in order to prevent their addition to the pipeline downstream.<br />
<br />
<pre> rm *.BAD.nbc.fa *.NONE.nbc.fa </pre><br />
<br />
=== Step 4: Upload file names ===<br />
Run from 454 directory<br />
<pre><br />
${DB_SCRIPTS}/add_file_db.pl [batch]/[batch]_barcode.csv [batch]/part<br />
</pre><br />
<br />
=== Step 5: Combine all fasta files into a single one ===<br />
<br />
'''Note: this assumes we're running the pipeline for the first time - a different protocol is necessary for adding new sequences to an already existing analysis'''<br />
<br />
In the new file all sequences will be named <n>_<nn> where <n> is the value in the "File #"<br />
field in 454.csv and <nn> is the index of the sequence in the file.<br />
<br />
<pre><br />
mkdir Analysis/Run[date]<br />
${DB_SCRIPTS}/combinefa_db.pl -c Analysis/Run[date]/Run[date] 454<br />
</pre><br />
<br />
The output will be in Analysis/Run[date]/Run[date].fna<br />
<br />
=== Step 6: Run clustering tool ===<br />
<br />
* First generate clusters<br />
'''Note: This part assumes we're running the whole set of sequences as one batch.'''<br />
<pre><br />
cd Analysis/Run[date]/Run[date].fna<br />
/fs/szasmg2/ghodsi/Src/clusterk/clusterk7 -r 2 -i Run[date].fna > Run[date].fna.cluster<br />
/fs/szasmg2/ghodsi/Src/clusterk/clusterk7 -r 2 -m -i Run[date].fna > Run[date].fna.align<br />
</pre><br />
<br />
Output will be in Run[date].fna.cluster, one cluster per line, cluster center listed as the first identifier.<br />
<br />
The .align file contains aligned FASTA records for all the sequences in each cluster. Clusters are separated by #<number> where <number> is the number of sequences in the cluster.<br />
<br />
<br />
* Then extract the cluster centers<br />
'''From here on the code runs the same in both full-run and batch modes'''<br />
<pre><br />
/fs/szasmg2/ghodsi/Src/clusterk/fastaselect Run[date].fna < Run[date].fna.cluster > Run[date].centers.fna<br />
</pre><br />
<br />
=== Step 7: Assign putative taxonomic labels to clusters ===<br />
<br />
<pre><br />
/fs/szasmg2/ghodsi/rdp/findtax/findtaxid.sh Run[date].centers.fna > Run[date].centers.taxid<br />
</pre><br />
<br />
Output is tab-delimited: sequence name <TAB> taxid<br />
Note: using "findtax" instead of "findtaxid" will retrieve actual taxonomy names.<br />
<br />
=== Step 8: Upload OTU information into the database ===<br />
* First create a partition file that contains all the clusters and associated taxids (if they exist)<br />
<br />
<pre><br />
${SCRIPTS}/cluster2part.pl [batch].fna.cluster [batch] [batch].centers.taxid > [batch].part<br />
</pre><br />
<br />
* Then upload the resulting partition to the database<br />
<pre><br />
${DB_SCRIPTS}/upload_otus.pl [batch].part<br />
</pre><br />
<br />
=== Step 9: Generate summary tables ===</div>Ghodsihttps://wiki.umiacs.umd.edu/cbcb/index.php?title=Cbcb_talk:Pop-Lab:16S-pipeline_16S_analysis_pipeline_(for_Gates_project)&diff=7582Cbcb talk:Pop-Lab:16S-pipeline 16S analysis pipeline (for Gates project)2010-09-27T19:11:28Z<p>Ghodsi: </p>
<hr />
<div>=== Step 9: Run OLD clustering tool ===<br />
<br />
* First generate clusters<br />
'''Note: This part assumes we're running the whole set of sequences as one batch.'''<br />
<pre><br />
cd Analysis/Run[date]/Run[date].fna<br />
/fs/szasmg2/ghodsi/Src/clusterk/clusterk7 -r 2 -i Run[date].fna > Run[date].fna.cluster<br />
/fs/szasmg2/ghodsi/Src/clusterk/clusterk7 -r 2 -m -i Run[date].fna > Run[date].fna.align<br />
</pre><br />
<br />
Output will be in Run[date].fna.cluster, one cluster per line, cluster center listed as the first identifier.<br />
<br />
The .align file contains aligned FASTA records for all the sequences in each cluster. Clusters are separated by #<number> where <number> is the number of sequences in the cluster.<br />
<br />
<br />
* Then extract the cluster centers<br />
'''From here on the code runs the same in both full-run and batch modes'''<br />
<pre><br />
/fs/szasmg2/ghodsi/Src/clusterk/fastaselect Run[date].fna < Run[date].fna.cluster > Run[date].centers.fna<br />
</pre></div>Ghodsihttps://wiki.umiacs.umd.edu/cbcb/index.php?title=Cbcb:Pop-Lab:16S-pipeline_16S_analysis_pipeline_(for_Gates_project)&diff=6109Cbcb:Pop-Lab:16S-pipeline 16S analysis pipeline (for Gates project)2009-11-25T18:10:58Z<p>Ghodsi: /* Step 9: Run clustering tool */</p>
<hr />
<div>== 16S analysis pipeline ==<br />
<br />
Assumptions:<br><br />
* 16S rRNA sequences are generated with barcoded 454 and are received as either: (i) .sff file; (ii) fasta and quality file; (iii) just fasta file<br />
* each batch is a single 96-well plate and is accompanied by a tab-delimited file containing information about the sample, including the "Sample ID", well on the plate, and additional information regarding the sample quality and DNA concentration<br />
* we have a file that specifies the barcode and sequencing adapter used for each well (these entities do not change).<br />
<br />
=== Directory structure ===<br />
<pre><br />
Gates_SOM/<br />
Main/<br />
samples.csv - information about all the samples available to us<br />
454.csv - information about all 454 runs (essentially concatenation of .csvs from 454 dir)<br />
phylochip.csv - information about all Phylochip runs<br />
IGS_Barcodes.csv - information about barcodes used to multiplex 454 runs<br />
scripts/ - scripts used to process the data<br />
454/ - here's where all 454 sequences live<br />
[batch1]/ - ... each batch in a separate directory<br />
[batch1].csv - meta-information about the batch <br />
[fasta1]- fasta files containing the batch<br />
... <br />
[fastan]<br />
[batch1].part - partition file describing how the sequences get split by barcode/sample<br />
part/ - directory where all the partitioned files live <br />
... <br />
[batchn]<br />
Phylochip/ - all the CEL files and auxiliary information on the Phylochip runs<br />
</pre><br />
<br />
=== Step 0: Get the sequence information ===<br />
* From .SFF files (assuming these are 454 sequences)<br />
<br />
This step uses the sff_extract program from the Staden package (if I'm not mistaken)<br />
<pre><br />
for i in *.sff ;do<br />
name=`expr $i : '\(.*\)\.sff'`<br />
sff_extract -c -s $name.seq -q $name.qual $i<br />
done<br />
</pre><br />
<br />
=== Step 1: Cleanup meta-information ===<br />
* Convert the Excel sheet containing the batch information into tab-delimited file (and run dos2unix) making sure the quotes added by Excel/OOffice are removed, adding the date (if not already in the file), and sorting the file by Sample ID. At this stage also check that the header row information is in canonical format.<br />
* Add barcode information using add_barcode.pl<br />
<br />
<pre>${SCRIPTDIR}/add_barcode.pl [batch].csv ${MAINDIR}IGS_Barcodes.csv > [batch]_barcode.csv</pre><br />
<br />
=== Step 2: Create partition file ===<br />
<br />
First concatenate all the sequence files (if multiple) from a batch into a single file [batch].all.seq.<br />
<br />
<pre>${SCRIPTDIR}/code2part.pl [batch].csv [batch].all.seq [batch] > [batch].part</pre><br />
<br />
The files [batch].BAD.list and [batch].NONE.list contain the names of all sequences that have failed quality checks either because they are too short, or contain Ns (BAD), or because the barcode is not recognized (NONE). The files contain additional information that can be used to troubleshoot the pipeline: sequences that are too short (< 75 454 cycles) are followed by the number of cycles, sequences that either contain Ns or have an unknown barcode are followed by the first 8 characters in the sequence.<br />
<br />
=== Step 3: Break up the fasta file into separate batches by partition ===<br />
<br />
* Create partition directory<br />
<br />
<pre><br />
mkdir part<br />
cd part<br />
</pre><br />
<br />
* Partition main file into sub-parts<br />
<pre><br />
${SCRIPTS}/partitionFasta.pl ../[batch].all.seq ../[batch].part<br />
</pre><br />
<br />
The result is one fasta file per sub-partition (i.e. individual subject).<br />
<br />
* Remove barcodes <br />
(still in the part/ subdirectory)<br />
<pre><br />
for i in *.seq; do<br />
${SCRIPTS}/unbarcode.pl $i<br />
done<br />
</pre><br />
<br />
The output files will have the same name as the original file but with the addition of the .nbc suffix. You should remove the .nbc files from the BAD/NONE files in order to prevent their addition to the pipeline downstream.<br />
<br />
<pre> rm *.BAD.nbc.fa *.NONE.nbc.fa </pre><br />
<br />
=== Step 4: Add file names to sample tables ===<br />
<br />
The following needs to be run from the root of the 454 directory.<br />
<br />
<pre> ${SCRIPTS}/add_file.pl [batch]/[batch]_barcode.csv [batch]/part > [batch]/[batch]_names.csv </pre><br />
<br />
As a result, the file [batch]/[batch]_names.csv will associate each Sample ID to a file name and also record the number of sequences in that file. Note that only files ending in .nbc.fa are procesed.<br />
<br />
=== Step 5: Merge all the batch meta-info files into a same file at the top ===<br />
<br />
Note: the addition of file names must be done on a batch by batch basis as multiple files might refer to a same Sample ID - within each batch it can be assumed that the Sample ID -> Filename mapping is unique. In the 454.csv file in the top directory the unique key is the file name. <br />
<br />
<pre> <br />
mv ../454.csv ../454.csv.bak<br />
${SCRIPTS}/merge_csv.pl ../454.csv.bak [batch]/[batch]_names.csv "Sample ID" non-unique > ../454.csv<br />
</pre><br />
<br />
This should probably be run with the filename as the key but then the tables need to be sorted by filename. Ultimately, this will all work better once the data are in a relational database.<br />
<br />
=== Step 6: Update sample file ===<br />
From the top directory:<br />
<br />
* First add all new samples to the samples.csv file<br />
<pre> <br />
mv samples.csv samples.csv.bak<br />
${SCRIPTS}/merge_csv.pl samples.csv.bak 454.csv "Sample ID" unique merge > samples.csv<br />
</pre><br />
<br />
Note: merge means that if record keys conflict, the empty fields will be updated with the new data.<br />
<br />
* Update the tag indicating 454 sequences available for this sample<br />
<br />
<pre><br />
mv samples.csv samples.csv.bak<br />
${SCRIPTS}/update_field_csv.pl samples.csv.bak 454.csv "Sample ID" 454 Y > samples.csv<br />
</pre><br />
<br />
=== Step 7: Assign numbers to all filenames ===<br />
<br />
Each file in the 454.csv file will be assigned an integer (if one is not already available). This number will be used to prefix the sequences in the combined file for the project.<br />
<br />
<pre><br />
mv 454.csv 454.csv.bak<br />
${SCRIPTS}/add_filenum.pl 454.csv.bak > 454.csv<br />
</pre><br />
<br />
=== Step 8: Combine all fasta files into a single one ===<br />
<br />
'''Note: this assumes we're running the pipeline for the first time - a different protocol is necessary for adding new sequences to an already existing analysis'''<br />
<br />
In the new file all sequences will be named <n>_<nn> where <n> is the value in the "File #"<br />
field in 454.csv and <nn> is the index of the sequence in the file.<br />
<br />
<pre><br />
mkdir Analysis/Run[date]<br />
${SCRIPTS}/combinefa.pl -c Analysis/Run[date]/Run[date] -i 454.csv 454<br />
</pre><br />
<br />
The output will be in Analysis/Run[date]/Run[date].fna<br />
<br />
=== Step 9: Run clustering tool ===<br />
<br />
* First generate clusters<br />
<pre><br />
cd Analysis/Run[date]/Run[date].fna<br />
/fs/szasmg2/ghodsi/Src/clusterk/clusterk7 -r 2 -i Run[date].fna > Run[date].fna.clusters<br />
</pre><br />
<br />
Output will be in Run[date].fna.clusters, one cluster per line, cluster center listed as the first identifier.<br />
<br />
* Then extract the cluster centers<br />
<pre><br />
/fs/szasmg2/ghodsi/Src/clusterk/fastaselect Run[date].fna < Run[date].fna.clusters > Run[date].centers.fna<br />
</pre><br />
<br />
=== Step 10: Assign putative taxonomic labels to clusters ===<br />
<br />
<pre><br />
/fs/szasmg2/ghodsi/rdp/findtax/findtaxid.sh Run[date].centers.fna > Run[date].centers.taxid<br />
</pre><br />
<br />
Output is tab-delimited: sequence name <TAB> taxid<br />
Note: using "findtax" instead of "findtaxid" will retrieve actual taxonomy names.<br />
<br />
=== Step 11: Build summary tables ===<br />
<br />
Using the output from steps 9 and 10 we construct a collection of tables linking OTUs, taxIDs, taxnames at various taxonomic levels to individual samples. The colums are the samples and the rows are the respective units. The cells are numbers of sequences assigned to the specific group. If looking at taxonomic levels, the sequences without an assignment at that level are assigned to a generic "No Assignment" bin.</div>Ghodsihttps://wiki.umiacs.umd.edu/cbcb/index.php?title=Cbcb:Pop-Lab:Mohammad-Report&diff=4858Cbcb:Pop-Lab:Mohammad-Report2009-03-11T17:40:36Z<p>Ghodsi: </p>
<hr />
<div>Known bounds for embedding Levenstein distance into metric spaces<br />
*[http://portal.acm.org/citation.cfm?id=1109557.1109669 Lower bound of <math>\Omega(\log n)</math> where n is number of points]<br />
*[http://portal.acm.org/citation.cfm?id=1284322 Upper bound of <math>2^{O(\sqrt(\log d \log \log d))}</math> where d is dimension (i.e. length)]<br />
<br />
FFT and random projections<br />
*[https://wiki.umiacs.umd.edu/cbcb/images/f/f6/Projection.pdf Presentation]<br />
*[https://wiki.umiacs.umd.edu/cbcb/images/1/13/Fft2.pdf Writeup]</div>Ghodsihttps://wiki.umiacs.umd.edu/cbcb/index.php?title=Cbcb:Pop-Lab:Mohammad-Report&diff=4857Cbcb:Pop-Lab:Mohammad-Report2009-03-11T17:39:48Z<p>Ghodsi: </p>
<hr />
<div>Bounds for embedding Levenstein distance into metric spaces<br />
*[http://portal.acm.org/citation.cfm?id=1109557.1109669 Lower bound of <math>\Omega(\log n)</math> where n is number of points]<br />
*[http://portal.acm.org/citation.cfm?id=1284322 Upper bound of <math>2^{O(\sqrt(log d log log d))}</math> where d is dimension (i.e. length)]<br />
<br />
FFT and random projections<br />
*[https://wiki.umiacs.umd.edu/cbcb/images/f/f6/Projection.pdf Presentation]<br />
*[https://wiki.umiacs.umd.edu/cbcb/images/1/13/Fft2.pdf Writeup]</div>Ghodsihttps://wiki.umiacs.umd.edu/cbcb/index.php?title=Cbcb:Pop-Lab:Mohammad-Report&diff=4856Cbcb:Pop-Lab:Mohammad-Report2009-03-11T17:39:31Z<p>Ghodsi: </p>
<hr />
<div>Bounds for embedding Levenstein distance into metric spaces<br />
*[http://portal.acm.org/citation.cfm?id=1109557.1109669 Lower bound of <math>\Omega(\log n)</math> where n is number of points<br />
*[http://portal.acm.org/citation.cfm?id=1284322 Upper bound of <math>2^{O(\sqrt(log d log log d))}</math> where d is dimension (i.e. length)<br />
<br />
FFT and random projections<br />
*[https://wiki.umiacs.umd.edu/cbcb/images/f/f6/Projection.pdf Presentation]<br />
*[https://wiki.umiacs.umd.edu/cbcb/images/1/13/Fft2.pdf Writeup]</div>Ghodsihttps://wiki.umiacs.umd.edu/cbcb/index.php?title=Cbcb:Pop-Lab:Mohammad-Report&diff=4855Cbcb:Pop-Lab:Mohammad-Report2009-03-11T17:31:47Z<p>Ghodsi: </p>
<hr />
<div>Bounds for embedding Levenstein distance into l1<br />
* <math>\Omega(n)</math><br />
<br />
FFT and random projections<br />
*[https://wiki.umiacs.umd.edu/cbcb/images/f/f6/Projection.pdf Presentation]<br />
*[https://wiki.umiacs.umd.edu/cbcb/images/1/13/Fft2.pdf Writeup]</div>Ghodsihttps://wiki.umiacs.umd.edu/cbcb/index.php?title=Cbcb:Pop-Lab:Mohammad-Report&diff=4729Cbcb:Pop-Lab:Mohammad-Report2009-02-26T22:34:00Z<p>Ghodsi: </p>
<hr />
<div>FFT and random projections<br />
*[https://wiki.umiacs.umd.edu/cbcb/images/f/f6/Projection.pdf Presentation]<br />
*[https://wiki.umiacs.umd.edu/cbcb/images/1/13/Fft2.pdf Writeup]</div>Ghodsihttps://wiki.umiacs.umd.edu/cbcb/index.php?title=File:Fft2.pdf&diff=4728File:Fft2.pdf2009-02-26T22:32:13Z<p>Ghodsi: FFT for overlapping</p>
<hr />
<div>FFT for overlapping</div>Ghodsihttps://wiki.umiacs.umd.edu/cbcb/index.php?title=Cbcb:Pop-Lab&diff=4727Cbcb:Pop-Lab2009-02-26T17:47:07Z<p>Ghodsi: /* Progress Reports */</p>
<hr />
<div>= Important pages =<br />
* Group members [[Cbcb:Pop-Lab:Members]] <br />
* Blog: [http://genomes.blogspot.com Genomes.blogspot.com]<br />
* How-tos: [[Cbcb:Pop-Lab:How-to]] - List of documentation on how to perform certain analyses, use different software, etc.<br />
* Software developed by us: [[Cbcb:Pop-Lab:Software]]<br />
* Metagenomics papers: [[Cbcb:Pop-Lab:Papers]]<br />
<br />
= Meetings/seminars =<br />
* [[Pop_group_meeting|Group meeting]]: every Monday from 2:30pm in corner conference room (3120 C)<br />
* [[seminars|CBCB seminar]]: Thursdays 2-3pm during the semester in main conference room<br />
<br />
= Progress Reports =<br />
* [[Cbcb:Pop-Lab:Ben-Report|Ben]]<br />
* [[Cbcb:Pop-Lab:Niranjan-Report|Niranjan]]<br />
* [[Cbcb:Pop-Lab:Chris-Report|Chris]]<br />
* [[Cbcb:Pop-Lab:James-Report|James]]<br />
* [[Cbcb:Pop-Lab:Serge-Report|Serge]]<br />
* [[Cbcb:Pop-Lab:Bo-Report|Bo]]<br />
* [[Cbcb:Pop-Lab:Dan-Report|Dan]]<br />
* [[Cbcb:Pop-Lab:Mohammad-Report|Mohammad]]<br />
<br />
= Pop Lab Presentations =<br />
* [[Media:Xoo_Cripsr.odg|Xanthomonas Crispr Slides (openoffice format)]] <br />
* Listeria Monocytogenes Slides (powerpoint)<br />
* Metagenome pipelines overview [[Media:Pipeline_outline.pdf|Presentation(pdf)]]</div>Ghodsihttps://wiki.umiacs.umd.edu/cbcb/index.php?title=Cbcb:Pop-Lab:Mohammad-Report&diff=4726Cbcb:Pop-Lab:Mohammad-Report2009-02-26T17:36:50Z<p>Ghodsi: </p>
<hr />
<div>FFT and random projections<br />
*[https://wiki.umiacs.umd.edu/cbcb/images/f/f6/Projection.pdf Presentation]<br />
*[https://wiki.umiacs.umd.edu/cbcb/images/f/f6/Projection.pdf Writeup]</div>Ghodsihttps://wiki.umiacs.umd.edu/cbcb/index.php?title=Cbcb:Pop-Lab:Mohammad&diff=4725Cbcb:Pop-Lab:Mohammad2009-02-26T17:30:04Z<p>Ghodsi: </p>
<hr />
<div></div>Ghodsihttps://wiki.umiacs.umd.edu/cbcb/index.php?title=Cbcb:Pop-Lab:Mohammad&diff=4724Cbcb:Pop-Lab:Mohammad2009-02-26T17:29:15Z<p>Ghodsi: </p>
<hr />
<div>Presentation</div>Ghodsihttps://wiki.umiacs.umd.edu/cbcb/index.php?title=File:Projection.pdf&diff=4723File:Projection.pdf2009-02-26T17:27:53Z<p>Ghodsi: Presentation to meeting on FFT and random projections</p>
<hr />
<div>Presentation to meeting on FFT and random projections</div>Ghodsihttps://wiki.umiacs.umd.edu/cbcb/index.php?title=Cbcb:Pop-Lab:Mohammad-Report&diff=4722Cbcb:Pop-Lab:Mohammad-Report2009-02-26T17:23:26Z<p>Ghodsi: </p>
<hr />
<div>FFT and random projections<br />
presentation<br />
writeup</div>Ghodsihttps://wiki.umiacs.umd.edu/cbcb/index.php?title=Pop_group_meeting&diff=3998Pop group meeting2008-12-02T20:18:42Z<p>Ghodsi: </p>
<hr />
<div>= 09/29/2008 =<br />
<br />
# Benjamin Langmead [http://www.plosone.org/article/info:doi%2F10.1371%2Fjournal.pone.0003263 Metagenomic Analysis of Lysogeny in Tampa Bay: Implications for Prophage Gene Expression] (PLoS one)<br />
# Bo Liu [http://genomebiology.com/2008/9/9/R139 Systematic bioinformatic analysis of expression levels of 17,330 human genes across 9,783 samples from 175 types of healthy and pathological tissues] (Genome Biology)<br />
# Daniela Puiu [http://genome.cshlp.org/cgi/reprint/gr.080200.108v1 Sequencing of natural strains of Arabidopsis thaliana with short reads] (Genome Research)<br />
# Dan Sommer, Niranjan Nagarajan [http://www.nature.com/nature/journal/v455/n7212/edsumm/e080925-04.html Metagenomics: questions to answer] (Nature)<br />
# James Robert White [http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6V73-4SHVST0-1&_user=961305&_rdoc=1&_fmt=&_orig=search&_sort=d&view=c&_version=1&_urlVersion=0&_userid=961305&md5=4115e7b8186a36db60f6b11a217a8798 Microbial population dynamics during aerobic sludge granulation at different organic loading rates] (ScienceDirect)<br />
# Mohammad Reza Ghodsi [http://nar.oxfordjournals.org/cgi/content/short/36/16/5180 A comparison of random sequence reads versus 16S rDNA sequences for estimating the biodiversity of a metagenomic library] (NAR)<br />
# Sergey Koren [http://www.biomedcentral.com/1471-2105/9/390 R/parallel - speeding up bioinformatics analysis with R] (BMC Bioinformatics)<br />
# Ted Gibbons [http://www.ncbi.nlm.nih.gov/pubmed/18711340 High-resolution metagenomics targets specific functional types in complex microbial communities] (Nature Biotechnology) <BR> [http://www.nature.com/nrmicro/journal/v6/n9/abs/nrmicro1935.html Molecular eco-systems biology: towards an understanding of community function] (Systems Microbiology)<br />
<br />
= 10/06/2008 =<br />
# Bo Liu [http://www.pnas.org/content/105/39/15076.short The convergence of carbohydrate active gene repertoires in human gut microbes.] PNAS<br />
# Daniela Puiu [http://bioinformatics.oxfordjournals.org/cgi/content/full/24/20/2395 SeqMap: mapping massive amount of oligonucleotides to the genome] Bioinformatics<br />
# James Robert White [http://www.pnas.org/content/104/21/8918.full Artificial selection of simulated microbial ecosystems.] PNAS<br />
# Mohammad Reza Ghodsi [http://aem.asm.org/cgi/content/abstract/74/5/1453 Metagenomics: Read Length Matters] Applied Environmental Microbiology<br />
# Sergey Koren [http://www.ploscompbiol.org/article/info:doi%2F10.1371%2Fjournal.pcbi.1000186 Gene-Boosted Assembly of a Novel Bacterial Genome from Very Short Reads] PLoS Computational Biology<br />
<br />
= 10/13/2008 =<br />
# Daniela Puiu, Daniel Sommer [http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0003373 MetaSim—A Sequencing Simulator for Genomics and Metagenomics] PLoS ONE<br />
# Bo Liu, James Robert White [http://www.sciencemag.org/cgi/content/full/322/5899/275 Environmental Genomics Reveals a Single-Species Ecosystem Deep Within Earth.] Science<br />
# MohammadReza Ghodsi [http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2557142 Probing Metagenomics by Rapid Cluster Analysis of Very Large Datasets] (PLoS one)<br />
<br />
MetaSim software:<br />
Location: /fs/sz-user-supported/common/packages/MetaSim/ <br />
Executable(Java): /fs/sz-user-supported/common/packages/MetaSim/metasim/MetaSim <br />
Database: /fs/sz-user-supported/common/packages/MetaSim/database <br />
create a symlink to it from your working directory, otherwise you have to import it <br />
NCBI databases: [ftp://ftp.ncbi.nlm.nih.gov/genomes/Bacteria/all.fna.tar.gz Complete bacteria]; [ftp://ftp.ncbi.nlm.nih.gov/pub/taxonomy/gi_taxid_nucl.dmp.gz Taxonomy]<br />
<br />
= 10/20/2008 =<br />
# Bo Liu [http://www.pnas.org/content/105/37/13977.short?rss=1 Accelerated evolution of resistance in multidrug environments] PNAS <br> [http://www.pnas.org/content/105/39/14918.short?rss=1 Drug interactions modulate the potential for evolution of resistance] PNAS<br />
# Daniel Sommer [http://www.ncbi.nlm.nih.gov/pubmed/18927115?dopt=Abstract NCBI Reference Sequences: current status, policy and new initiatives] NAR<br />
# Daniela Puiu [http://genomebiology.com/content/pdf/gb-2008-9-10-r151.pdf A simple, fast, and accurate method of phylogenomic inference] Genome Biology<br />
# Niranjan Nagarajan [http://www.springerlink.com/content/cn3551154w232l73/fulltext.pdf The Stastical Power of Phylogenetic Motif Models] RECOMB 2008<br />
# Sergey Koren [http://www.nature.com/nmeth/journal/v5/n10/full/nmeth1008-903.html Scientific software: seeing the SNPs between us] Nature Methods<br />
# Mohammadreza Ghodsi [http://www.biomedcentral.com/1471-2105/9/386 The metagenomics RAST server – a public resource for the automatic phylogenetic and functional analysis of metagenomes] BMC Bioinformatics<br />
<br />
= 10/27/2008 =<br />
# Bo Liu [http://www.pnas.org/content/105/37/14130.short?rss=1 Frequent emergence and limited geographic dispersal of methicillin-resistant Staphylococcus aureus] PNAS<br />
# Sergey Koren [http://bioinformatics.oxfordjournals.org/cgi/content/abstract/btn548 Aggressive Assembly of Pyrosequencing Reads with Mates] Bioinformatics<br />
# Daniela Puiu [http://bioinformatics.oxfordjournals.org/cgi/content/full/24/21/2431 ZOOM! Zillions of oligos mapped] Bioinformatics<br />
# Mohammadreza Ghodsi [http://www.ncbi.nlm.nih.gov/pubmed/18692931 Biodiversity of the microbial community in a Spanish farmhouse cheese as revealed by culture-dependent and culture-independent methods]<br />
<br />
= 11/03/2008 = <br />
# Daniela Puiu [http://genome.cshlp.org/cgi/content/abstract/gr.7088808v1 Short read fragment assembly of bacterial genomes] (Genome Research)<br />
<br />
= 11/10/2008 = <br />
# Bo Liu [http://www.pnas.org/content/early/2008/11/04/0802782105.abstract Metagenome analysis of an extreme microbial symbiosis reveals eurythermal adaptation and metabolic flexibility] PNAS<br />
<br />
= 11/17/2008 =<br />
# Bo Liu [http://bioinformatics.oxfordjournals.org/cgi/content/short/24/22/2579?rss=1 Phylogenetic distances are encoded in networks of interacting pathways] Bioinformatics<br />
# Daniela Puiu [http://bioinformatics.oxfordjournals.org/cgi/content/abstract/btn582 Human genomes as email attachments] Bioinformatics<br />
# Mohammad Ghodsi [http://nar.oxfordjournals.org/cgi/content/short/36/18/e120 Accurate taxonomy assignments from 16S rRNA sequences produced by highly parallel pyrosequencers] Nucleic Acids Research<br />
# Sergey Koren [http://www.biomedcentral.com/1471-2105/9/420/abstract XplorSeq: A software environment for integrated management and phylogenetic analysis of metagenomic sequence data] (BMC Bioinformatics)<br />
# James Robert White [http://www.pnas.org/content/early/2008/08/08/0801925105.abstract?etoc Resistance, resilience, and redundancy in microbial communities] PNAS<br />
<br />
= 11/24/2008 =<br />
# Ted Gibbons [http://biology.plosjournals.org/perlserv/?request=get-document&doi=10.1371%2Fjournal.pbio.0060295 Gut Reaction: Pyrosequencing Provides the Poop on Distal Gut Bacteria (summary)] PLoS <br> [http://biology.plosjournals.org/perlserv/?request=get-document&doi=10.1371%2Fjournal.pbio.0060280 The Pervasive Effects of an Antibiotic on the Human Gut Microbiota, as Revealed by Deep 16S rRNA Sequencing] PLoS <br> [http://www.plosgenetics.org/article/info%3Adoi%2F10.1371%2Fjournal.pgen.1000255 Exploring Microbial Diversity and Taxonomy Using SSU rRNA Hypervariable Tag Sequencing] PLoS Genetics<br />
<br />
= 12/1/2008 =<br />
# Mohammad Ghodsi [http://www.plosone.org/article/info:doi%2F10.1371%2Fjournal.pone.0002836 Comparative Analysis of Human Gut Microbiota by Barcoded Pyrosequencing] PLoS ONE</div>Ghodsihttps://wiki.umiacs.umd.edu/cbcb/index.php?title=Pop_group_meeting&diff=3883Pop group meeting2008-11-17T19:15:46Z<p>Ghodsi: /* 11/10/2008 */</p>
<hr />
<div>= 09/29/2008 =<br />
<br />
# Benjamin Langmead [http://www.plosone.org/article/info:doi%2F10.1371%2Fjournal.pone.0003263 Metagenomic Analysis of Lysogeny in Tampa Bay: Implications for Prophage Gene Expression] (PLoS one)<br />
# Bo Liu [http://genomebiology.com/2008/9/9/R139 Systematic bioinformatic analysis of expression levels of 17,330 human genes across 9,783 samples from 175 types of healthy and pathological tissues] (Genome Biology)<br />
# Daniela Puiu [http://genome.cshlp.org/cgi/reprint/gr.080200.108v1 Sequencing of natural strains of Arabidopsis thaliana with short reads] (Genome Research)<br />
# Dan Sommer, Niranjan Nagarajan [http://www.nature.com/nature/journal/v455/n7212/edsumm/e080925-04.html Metagenomics: questions to answer] (Nature)<br />
# James Robert White [http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6V73-4SHVST0-1&_user=961305&_rdoc=1&_fmt=&_orig=search&_sort=d&view=c&_version=1&_urlVersion=0&_userid=961305&md5=4115e7b8186a36db60f6b11a217a8798 Microbial population dynamics during aerobic sludge granulation at different organic loading rates] (ScienceDirect)<br />
# Mohammad Reza Ghodsi [http://nar.oxfordjournals.org/cgi/content/short/36/16/5180 A comparison of random sequence reads versus 16S rDNA sequences for estimating the biodiversity of a metagenomic library] (NAR)<br />
# Sergey Koren [http://www.biomedcentral.com/1471-2105/9/390 R/parallel - speeding up bioinformatics analysis with R] (BMC Bioinformatics)<br />
# Theodore Gibbons [http://www.ncbi.nlm.nih.gov/pubmed/18711340 High-resolution metagenomics targets specific functional types in complex microbial communities] (Nature Biotechnology) <BR> [http://www.nature.com/nrmicro/journal/v6/n9/abs/nrmicro1935.html Molecular eco-systems biology: towards an understanding of community function] (Systems Microbiology)<br />
<br />
= 10/06/2008 =<br />
# Bo Liu [http://www.pnas.org/content/105/39/15076.short The convergence of carbohydrate active gene repertoires in human gut microbes.] PNAS<br />
# Daniela Puiu [http://bioinformatics.oxfordjournals.org/cgi/content/full/24/20/2395 SeqMap: mapping massive amount of oligonucleotides to the genome] Bioinformatics<br />
# James Robert White [http://www.pnas.org/content/104/21/8918.full Artificial selection of simulated microbial ecosystems.] PNAS<br />
# MohammadReza Ghodsi [http://aem.asm.org/cgi/content/abstract/74/5/1453 Metagenomics: Read Length Matters] Applied Environmental Microbiology<br />
# Sergey Koren [http://www.ploscompbiol.org/article/info:doi%2F10.1371%2Fjournal.pcbi.1000186 Gene-Boosted Assembly of a Novel Bacterial Genome from Very Short Reads] PLoS Computational Biology<br />
<br />
= 10/13/2008 =<br />
# Daniela Puiu, Daniel Sommer [http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0003373 MetaSim—A Sequencing Simulator for Genomics and Metagenomics] PLoS ONE<br />
# Bo Liu, James Robert White [http://www.sciencemag.org/cgi/content/full/322/5899/275 Environmental Genomics Reveals a Single-Species Ecosystem Deep Within Earth.] Science<br />
# MohammadReza Ghodsi [http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2557142 Probing Metagenomics by Rapid Cluster Analysis of Very Large Datasets] (PLoS one)<br />
<br />
MetaSim software:<br />
Location: /fs/sz-user-supported/common/packages/MetaSim/ <br />
Executable(Java): /fs/sz-user-supported/common/packages/MetaSim/metasim/MetaSim <br />
Database: /fs/sz-user-supported/common/packages/MetaSim/database <br />
create a symlink to it from your working directory, otherwise you have to import it <br />
NCBI databases: [ftp://ftp.ncbi.nlm.nih.gov/genomes/Bacteria/all.fna.tar.gz Complete bacteria]; [ftp://ftp.ncbi.nlm.nih.gov/pub/taxonomy/gi_taxid_nucl.dmp.gz Taxonomy]<br />
<br />
= 10/20/2008 =<br />
# Bo Liu [http://www.pnas.org/content/105/37/13977.short?rss=1 Accelerated evolution of resistance in multidrug environments] PNAS <br> [http://www.pnas.org/content/105/39/14918.short?rss=1 Drug interactions modulate the potential for evolution of resistance] PNAS<br />
# Daniel Sommer [http://www.ncbi.nlm.nih.gov/pubmed/18927115?dopt=Abstract NCBI Reference Sequences: current status, policy and new initiatives] NAR<br />
# Daniela Puiu [http://genomebiology.com/content/pdf/gb-2008-9-10-r151.pdf A simple, fast, and accurate method of phylogenomic inference] Genome Biology<br />
# Niranjan Nagarajan [http://www.springerlink.com/content/cn3551154w232l73/fulltext.pdf The Stastical Power of Phylogenetic Motif Models] RECOMB 2008<br />
# Sergey Koren [http://www.nature.com/nmeth/journal/v5/n10/full/nmeth1008-903.html Scientific software: seeing the SNPs between us] Nature Methods<br />
# Mohammadreza Ghodsi [http://www.biomedcentral.com/1471-2105/9/386 The metagenomics RAST server – a public resource for the automatic phylogenetic and functional analysis of metagenomes] BMC Bioinformatics<br />
<br />
= 10/27/2008 =<br />
# Bo Liu [http://www.pnas.org/content/105/37/14130.short?rss=1 Frequent emergence and limited geographic dispersal of methicillin-resistant Staphylococcus aureus] PNAS<br />
# Sergey Koren [http://bioinformatics.oxfordjournals.org/cgi/content/abstract/btn548 Aggressive Assembly of Pyrosequencing Reads with Mates] Bioinformatics<br />
# Daniela Puiu [http://bioinformatics.oxfordjournals.org/cgi/content/full/24/21/2431 ZOOM! Zillions of oligos mapped] Bioinformatics<br />
# Mohammadreza Ghodsi [http://www.ncbi.nlm.nih.gov/pubmed/18692931 Biodiversity of the microbial community in a Spanish farmhouse cheese as revealed by culture-dependent and culture-independent methods]<br />
<br />
= 11/03/2008 = <br />
# Daniela Puiu [http://genome.cshlp.org/cgi/content/abstract/gr.7088808v1 Short read fragment assembly of bacterial genomes] (Genome Research)<br />
<br />
= 11/10/2008 = <br />
# Bo Liu [http://www.pnas.org/content/early/2008/11/04/0802782105.abstract Metagenome analysis of an extreme microbial symbiosis reveals eurythermal adaptation and metabolic flexibility] PNAS<br />
# Sergey Koren [http://www.biomedcentral.com/1471-2105/9/420/abstract XplorSeq: A software environment for integrated management and phylogenetic analysis of metagenomic sequence data] (BMC Bioinformatics)<br />
<br />
= 11/17/2008 =<br />
# Daniela Puiu [http://bioinformatics.oxfordjournals.org/cgi/content/abstract/btn582 Human genomes as email attachments] Bioinformatics<br />
# Mohammad Ghodsi [http://nar.oxfordjournals.org/cgi/content/short/36/18/e120 Accurate taxonomy assignments from 16S rRNA sequences produced by highly parallel pyrosequencers] Nucleic Acids Research</div>Ghodsihttps://wiki.umiacs.umd.edu/cbcb/index.php?title=Pop_group_meeting&diff=3785Pop group meeting2008-10-27T22:43:36Z<p>Ghodsi: /* 10/27/2008 */</p>
<hr />
<div>= 09/29/2008 =<br />
<br />
# Benjamin Langmead [http://www.plosone.org/article/info:doi%2F10.1371%2Fjournal.pone.0003263 Metagenomic Analysis of Lysogeny in Tampa Bay: Implications for Prophage Gene Expression] (PLoS one)<br />
# Bo Liu [http://genomebiology.com/2008/9/9/R139 Systematic bioinformatic analysis of expression levels of 17,330 human genes across 9,783 samples from 175 types of healthy and pathological tissues] (Genome Biology)<br />
# Daniela Puiu [http://genome.cshlp.org/cgi/reprint/gr.080200.108v1 Sequencing of natural strains of Arabidopsis thaliana with short reads] (Genome Research)<br />
# Dan Sommer, Niranjan Nagarajan [http://www.nature.com/nature/journal/v455/n7212/edsumm/e080925-04.html Metagenomics: questions to answer] (Nature)<br />
# James Robert White [http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6V73-4SHVST0-1&_user=961305&_rdoc=1&_fmt=&_orig=search&_sort=d&view=c&_version=1&_urlVersion=0&_userid=961305&md5=4115e7b8186a36db60f6b11a217a8798 Microbial population dynamics during aerobic sludge granulation at different organic loading rates] (ScienceDirect)<br />
# Mohammad Reza Ghodsi [http://nar.oxfordjournals.org/cgi/content/short/36/16/5180 A comparison of random sequence reads versus 16S rDNA sequences for estimating the biodiversity of a metagenomic library] (NAR)<br />
# Sergey Koren [http://www.biomedcentral.com/1471-2105/9/390 R/parallel - speeding up bioinformatics analysis with R] (BMC Bioinformatics)<br />
# Theodore Gibbons [http://www.ncbi.nlm.nih.gov/pubmed/18711340 High-resolution metagenomics targets specific functional types in complex microbial communities] (Nature Biotechnology) <BR> [http://www.nature.com/nrmicro/journal/v6/n9/abs/nrmicro1935.html Molecular eco-systems biology: towards an understanding of community function] (Systems Microbiology)<br />
<br />
= 10/06/2008 =<br />
# Bo Liu [http://www.pnas.org/content/105/39/15076.short The convergence of carbohydrate active gene repertoires in human gut microbes.] PNAS<br />
# Daniela Puiu [http://bioinformatics.oxfordjournals.org/cgi/content/full/24/20/2395 SeqMap: mapping massive amount of oligonucleotides to the genome] Bioinformatics<br />
# James Robert White [http://www.pnas.org/content/104/21/8918.full Artificial selection of simulated microbial ecosystems.] PNAS<br />
# MohammadReza Ghodsi [http://aem.asm.org/cgi/content/abstract/74/5/1453 Metagenomics: Read Length Matters] Applied Environmental Microbiology<br />
# Sergey Koren [http://www.ploscompbiol.org/article/info:doi%2F10.1371%2Fjournal.pcbi.1000186 Gene-Boosted Assembly of a Novel Bacterial Genome from Very Short Reads] PLoS Computational Biology<br />
<br />
= 10/13/2008 =<br />
# Daniela Puiu, Daniel Sommer [http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0003373 MetaSim—A Sequencing Simulator for Genomics and Metagenomics] PLoS ONE<br />
# Bo Liu, James Robert White [http://www.sciencemag.org/cgi/content/full/322/5899/275 Environmental Genomics Reveals a Single-Species Ecosystem Deep Within Earth.] Science<br />
# MohammadReza Ghodsi [http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2557142 Probing Metagenomics by Rapid Cluster Analysis of Very Large Datasets] (PLoS one)<br />
<br />
MetaSim software:<br />
Location: /fs/sz-user-supported/common/packages/MetaSim/ <br />
Executable(Java): /fs/sz-user-supported/common/packages/MetaSim/metasim/MetaSim <br />
Database: /fs/sz-user-supported/common/packages/MetaSim/database <br />
create a symlink to it from your working directory, otherwise you have to import it <br />
NCBI databases: [ftp://ftp.ncbi.nlm.nih.gov/genomes/Bacteria/all.fna.tar.gz Complete bacteria]; [ftp://ftp.ncbi.nlm.nih.gov/pub/taxonomy/gi_taxid_nucl.dmp.gz Taxonomy]<br />
<br />
= 10/20/2008 =<br />
# Bo Liu [http://www.pnas.org/content/105/37/13977.short?rss=1 Accelerated evolution of resistance in multidrug environments] PNAS <br> [http://www.pnas.org/content/105/39/14918.short?rss=1 Drug interactions modulate the potential for evolution of resistance] PNAS<br />
# Daniel Sommer [http://www.ncbi.nlm.nih.gov/pubmed/18927115?dopt=Abstract NCBI Reference Sequences: current status, policy and new initiatives] NAR<br />
# Daniela Puiu [http://genomebiology.com/content/pdf/gb-2008-9-10-r151.pdf A simple, fast, and accurate method of phylogenomic inference] Genome Biology<br />
# Niranjan Nagarajan [http://www.springerlink.com/content/cn3551154w232l73/fulltext.pdf The Stastical Power of Phylogenetic Motif Models] RECOMB 2008<br />
# Sergey Koren [http://www.nature.com/nmeth/journal/v5/n10/full/nmeth1008-903.html Scientific software: seeing the SNPs between us] Nature Methods<br />
# Mohammadreza Ghodsi [http://www.biomedcentral.com/1471-2105/9/386 The metagenomics RAST server – a public resource for the automatic phylogenetic and functional analysis of metagenomes] BMC Bioinformatics<br />
<br />
= 10/27/2008 =<br />
<br />
# Bo Liu [http://www.pnas.org/content/105/37/14130.short?rss=1 Frequent emergence and limited geographic dispersal of methicillin-resistant Staphylococcus aureus] PNAS<br />
# Sergey Koren [http://bioinformatics.oxfordjournals.org/cgi/content/abstract/btn548 Aggressive Assembly of Pyrosequencing Reads with Mates] Bioinformatics<br />
# Daniela Puiu [http://bioinformatics.oxfordjournals.org/cgi/content/full/24/21/2431 ZOOM! Zillions of oligos mapped] Bioinformatics<br />
# Mohammadreza Ghodsi [http://www.ncbi.nlm.nih.gov/pubmed/18692931 Biodiversity of the microbial community in a Spanish farmhouse cheese as revealed by culture-dependent and culture-independent methods]</div>Ghodsihttps://wiki.umiacs.umd.edu/cbcb/index.php?title=Pop_group_meeting&diff=3781Pop group meeting2008-10-27T15:06:40Z<p>Ghodsi: /* 10/20/2008 */</p>
<hr />
<div>= 09/29/2008 =<br />
<br />
# Benjamin Langmead [http://www.plosone.org/article/info:doi%2F10.1371%2Fjournal.pone.0003263 Metagenomic Analysis of Lysogeny in Tampa Bay: Implications for Prophage Gene Expression] (PLoS one)<br />
# Bo Liu [http://genomebiology.com/2008/9/9/R139 Systematic bioinformatic analysis of expression levels of 17,330 human genes across 9,783 samples from 175 types of healthy and pathological tissues] (Genome Biology)<br />
# Daniela Puiu [http://genome.cshlp.org/cgi/reprint/gr.080200.108v1 Sequencing of natural strains of Arabidopsis thaliana with short reads] (Genome Research)<br />
# Dan Sommer, Niranjan Nagarajan [http://www.nature.com/nature/journal/v455/n7212/edsumm/e080925-04.html Metagenomics: questions to answer] (Nature)<br />
# James Robert White [http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6V73-4SHVST0-1&_user=961305&_rdoc=1&_fmt=&_orig=search&_sort=d&view=c&_version=1&_urlVersion=0&_userid=961305&md5=4115e7b8186a36db60f6b11a217a8798 Microbial population dynamics during aerobic sludge granulation at different organic loading rates] (ScienceDirect)<br />
# Mohammad Reza Ghodsi [http://nar.oxfordjournals.org/cgi/content/short/36/16/5180 A comparison of random sequence reads versus 16S rDNA sequences for estimating the biodiversity of a metagenomic library] (NAR)<br />
# Sergey Koren [http://www.biomedcentral.com/1471-2105/9/390 R/parallel - speeding up bioinformatics analysis with R] (BMC Bioinformatics)<br />
# Theodore Gibbons [http://www.ncbi.nlm.nih.gov/pubmed/18711340 High-resolution metagenomics targets specific functional types in complex microbial communities] (Nature Biotechnology) <BR> [http://www.nature.com/nrmicro/journal/v6/n9/abs/nrmicro1935.html Molecular eco-systems biology: towards an understanding of community function] (Systems Microbiology)<br />
<br />
= 10/06/2008 =<br />
# Bo Liu [http://www.pnas.org/content/105/39/15076.short The convergence of carbohydrate active gene repertoires in human gut microbes.] PNAS<br />
# Daniela Puiu [http://bioinformatics.oxfordjournals.org/cgi/content/full/24/20/2395 SeqMap: mapping massive amount of oligonucleotides to the genome] Bioinformatics<br />
# James Robert White [http://www.pnas.org/content/104/21/8918.full Artificial selection of simulated microbial ecosystems.] PNAS<br />
# MohammadReza Ghodsi [http://aem.asm.org/cgi/content/abstract/74/5/1453 Metagenomics: Read Length Matters] Applied Environmental Microbiology<br />
# Sergey Koren [http://www.ploscompbiol.org/article/info:doi%2F10.1371%2Fjournal.pcbi.1000186 Gene-Boosted Assembly of a Novel Bacterial Genome from Very Short Reads] PLoS Computational Biology<br />
<br />
= 10/13/2008 =<br />
# Daniela Puiu, Daniel Sommer [http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0003373 MetaSim—A Sequencing Simulator for Genomics and Metagenomics] PLoS ONE<br />
# Bo Liu, James Robert White [http://www.sciencemag.org/cgi/content/full/322/5899/275 Environmental Genomics Reveals a Single-Species Ecosystem Deep Within Earth.] Science<br />
# MohammadReza Ghodsi [http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2557142 Probing Metagenomics by Rapid Cluster Analysis of Very Large Datasets] (PLoS one)<br />
<br />
MetaSim software:<br />
Location: /fs/sz-user-supported/common/packages/MetaSim/ <br />
Executable(Java): /fs/sz-user-supported/common/packages/MetaSim/metasim/MetaSim <br />
Database: /fs/sz-user-supported/common/packages/MetaSim/database <br />
create a symlink to it from your working directory, otherwise you have to import it <br />
NCBI databases: [ftp://ftp.ncbi.nlm.nih.gov/genomes/Bacteria/all.fna.tar.gz Complete bacteria]; [ftp://ftp.ncbi.nlm.nih.gov/pub/taxonomy/gi_taxid_nucl.dmp.gz Taxonomy]<br />
<br />
= 10/20/2008 =<br />
# Bo Liu [http://www.pnas.org/content/105/37/13977.short?rss=1 Accelerated evolution of resistance in multidrug environments] PNAS <br> [http://www.pnas.org/content/105/39/14918.short?rss=1 Drug interactions modulate the potential for evolution of resistance] PNAS<br />
# Daniel Sommer [http://www.ncbi.nlm.nih.gov/pubmed/18927115?dopt=Abstract NCBI Reference Sequences: current status, policy and new initiatives] NAR<br />
# Daniela Puiu [http://genomebiology.com/content/pdf/gb-2008-9-10-r151.pdf A simple, fast, and accurate method of phylogenomic inference] Genome Biology<br />
# Niranjan Nagarajan [http://www.springerlink.com/content/cn3551154w232l73/fulltext.pdf The Stastical Power of Phylogenetic Motif Models] RECOMB 2008<br />
# Sergey Koren [http://www.nature.com/nmeth/journal/v5/n10/full/nmeth1008-903.html Scientific software: seeing the SNPs between us] Nature Methods<br />
# Mohammadreza Ghodsi [http://www.biomedcentral.com/1471-2105/9/386 The metagenomics RAST server – a public resource for the automatic phylogenetic and functional analysis of metagenomes] BMC Bioinformatics<br />
<br />
= 10/27/2008 =<br />
<br />
# Sergey Koren [http://bioinformatics.oxfordjournals.org/cgi/content/abstract/btn548 Aggressive Assembly of Pyrosequencing Reads with Mates] Bioinformatics<br />
# Daniela Puiu [http://bioinformatics.oxfordjournals.org/cgi/content/full/24/21/2431 ZOOM! Zillions of oligos mapped] Bioinformatics</div>Ghodsihttps://wiki.umiacs.umd.edu/cbcb/index.php?title=Pop_group_meeting&diff=3592Pop group meeting2008-10-13T20:05:38Z<p>Ghodsi: /* 10/13/2008 */</p>
<hr />
<div>= 09/29/2008 =<br />
<br />
# Benjamin Langmead [http://www.plosone.org/article/info:doi%2F10.1371%2Fjournal.pone.0003263 Metagenomic Analysis of Lysogeny in Tampa Bay: Implications for Prophage Gene Expression] (PLoS one)<br />
# Bo Liu [http://genomebiology.com/2008/9/9/R139 Systematic bioinformatic analysis of expression levels of 17,330 human genes across 9,783 samples from 175 types of healthy and pathological tissues] (Genome Biology)<br />
# Daniela Puiu [http://genome.cshlp.org/cgi/reprint/gr.080200.108v1 Sequencing of natural strains of Arabidopsis thaliana with short reads] (Genome Research)<br />
# Dan Sommer [http://www.nature.com/nature/journal/v455/n7212/edsumm/e080925-04.html Metagenomics: questions to answer] (Nature)<br />
# James Robert White [http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6V73-4SHVST0-1&_user=961305&_rdoc=1&_fmt=&_orig=search&_sort=d&view=c&_version=1&_urlVersion=0&_userid=961305&md5=4115e7b8186a36db60f6b11a217a8798 Microbial population dynamics during aerobic sludge granulation at different organic loading rates] (ScienceDirect)<br />
# Mohammad Reza Ghodsi [http://nar.oxfordjournals.org/cgi/content/short/36/16/5180 A comparison of random sequence reads versus 16S rDNA sequences for estimating the biodiversity of a metagenomic library] (NAR)<br />
# Niranjan Nagarajan [http://www.nature.com/nature/journal/v455/n7212/edsumm/e080925-04.html Metagenomics: questions to answer] (Nature)<br />
# Sergey Koren [http://www.biomedcentral.com/1471-2105/9/390 R/parallel - speeding up bioinformatics analysis with R] (BMC Bioinformatics)<br />
# Theodore Gibbons [http://www.ncbi.nlm.nih.gov/pubmed/18711340 High-resolution metagenomics targets specific functional types in complex microbial communities] (Nature Biotechnology) <BR> [http://www.nature.com/nrmicro/journal/v6/n9/abs/nrmicro1935.html Molecular eco-systems biology: towards an understanding of community function] (Systems Microbiology)<br />
<br />
= 10/06/2008 =<br />
# Bo Liu [http://www.pnas.org/content/105/39/15076.short The convergence of carbohydrate active gene repertoires in human gut microbes.] PNAS<br />
# Daniela Puiu [http://bioinformatics.oxfordjournals.org/cgi/content/full/24/20/2395 SeqMap: mapping massive amount of oligonucleotides to the genome] Bioinformatics<br />
# James Robert White [http://www.pnas.org/content/104/21/8918.full Artificial selection of simulated microbial ecosystems.] PNAS<br />
# MohammadReza Ghodsi [http://aem.asm.org/cgi/content/abstract/74/5/1453 Metagenomics: Read Length Matters] Applied Environmental Microbiology<br />
<br />
= 10/13/2008 =<br />
# Daniela Puiu<br />
# Bo Liu & James Robert White [http://www.sciencemag.org/cgi/content/full/322/5899/275 Environmental Genomics Reveals a Single-Species Ecosystem Deep Within Earth.] Science<br />
# MohammadReza Ghodsi [http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2557142 Probing Metagenomics by Rapid Cluster Analysis of Very Large Datasets] (PLoS one)</div>Ghodsihttps://wiki.umiacs.umd.edu/cbcb/index.php?title=Pop_group_meeting&diff=3560Pop group meeting2008-10-08T19:25:07Z<p>Ghodsi: /* 10/06/2008 */</p>
<hr />
<div>= 09/29/2008 =<br />
<br />
# Benjamin Langmead [http://www.plosone.org/article/info:doi%2F10.1371%2Fjournal.pone.0003263 Metagenomic Analysis of Lysogeny in Tampa Bay: Implications for Prophage Gene Expression] (PLoS one)<br />
# Bo Liu [http://genomebiology.com/2008/9/9/R139 Systematic bioinformatic analysis of expression levels of 17,330 human genes across 9,783 samples from 175 types of healthy and pathological tissues] (Genome Biology)<br />
# Daniela Puiu [http://genome.cshlp.org/cgi/reprint/gr.080200.108v1 Sequencing of natural strains of Arabidopsis thaliana with short reads] (Genome Research)<br />
# Dan Sommer [http://www.nature.com/nature/journal/v455/n7212/edsumm/e080925-04.html Metagenomics: questions to answer] (Nature)<br />
# James Robert White [http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6V73-4SHVST0-1&_user=961305&_rdoc=1&_fmt=&_orig=search&_sort=d&view=c&_version=1&_urlVersion=0&_userid=961305&md5=4115e7b8186a36db60f6b11a217a8798 Microbial population dynamics during aerobic sludge granulation at different organic loading rates] (ScienceDirect)<br />
# Mohammad Reza Ghodsi [http://nar.oxfordjournals.org/cgi/content/short/36/16/5180 A comparison of random sequence reads versus 16S rDNA sequences for estimating the biodiversity of a metagenomic library] (NAR)<br />
# Niranjan Nagarajan [http://www.nature.com/nature/journal/v455/n7212/edsumm/e080925-04.html Metagenomics: questions to answer] (Nature)<br />
# Sergey Koren [http://www.biomedcentral.com/1471-2105/9/390 R/parallel - speeding up bioinformatics analysis with R] (BMC Bioinformatics)<br />
# Theodore Gibbons [http://www.ncbi.nlm.nih.gov/pubmed/18711340 High-resolution metagenomics targets specific functional types in complex microbial communities] (Nature Biotechnology) <BR> [http://www.nature.com/nrmicro/journal/v6/n9/abs/nrmicro1935.html Molecular eco-systems biology: towards an understanding of community function] (Systems Microbiology)<br />
<br />
= 10/06/2008 =<br />
# Bo Liu [http://www.pnas.org/content/105/39/15076.short The convergence of carbohydrate active gene repertoires in human gut microbes.] PNAS<br />
# Daniela Puiu [http://bioinformatics.oxfordjournals.org/cgi/content/full/24/20/2395 SeqMap: mapping massive amount of oligonucleotides to the genome] Bioinformatics<br />
# James Robert White [http://www.pnas.org/content/104/21/8918.full Artificial selection of simulated microbial ecosystems.] PNAS<br />
# MohammadReza Ghodsi [http://aem.asm.org/cgi/content/abstract/74/5/1453 Metagenomics: Read Length Matters] Applied Environmental Microbiology</div>Ghodsihttps://wiki.umiacs.umd.edu/cbcb/index.php?title=Pop_group_meeting&diff=3559Pop group meeting2008-10-08T19:23:28Z<p>Ghodsi: /* 10/06/2008 */</p>
<hr />
<div>= 09/29/2008 =<br />
<br />
# Benjamin Langmead [http://www.plosone.org/article/info:doi%2F10.1371%2Fjournal.pone.0003263 Metagenomic Analysis of Lysogeny in Tampa Bay: Implications for Prophage Gene Expression] (PLoS one)<br />
# Bo Liu [http://genomebiology.com/2008/9/9/R139 Systematic bioinformatic analysis of expression levels of 17,330 human genes across 9,783 samples from 175 types of healthy and pathological tissues] (Genome Biology)<br />
# Daniela Puiu [http://genome.cshlp.org/cgi/reprint/gr.080200.108v1 Sequencing of natural strains of Arabidopsis thaliana with short reads] (Genome Research)<br />
# Dan Sommer [http://www.nature.com/nature/journal/v455/n7212/edsumm/e080925-04.html Metagenomics: questions to answer] (Nature)<br />
# James Robert White [http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6V73-4SHVST0-1&_user=961305&_rdoc=1&_fmt=&_orig=search&_sort=d&view=c&_version=1&_urlVersion=0&_userid=961305&md5=4115e7b8186a36db60f6b11a217a8798 Microbial population dynamics during aerobic sludge granulation at different organic loading rates] (ScienceDirect)<br />
# Mohammad Reza Ghodsi [http://nar.oxfordjournals.org/cgi/content/short/36/16/5180 A comparison of random sequence reads versus 16S rDNA sequences for estimating the biodiversity of a metagenomic library] (NAR)<br />
# Niranjan Nagarajan [http://www.nature.com/nature/journal/v455/n7212/edsumm/e080925-04.html Metagenomics: questions to answer] (Nature)<br />
# Sergey Koren [http://www.biomedcentral.com/1471-2105/9/390 R/parallel - speeding up bioinformatics analysis with R] (BMC Bioinformatics)<br />
# Theodore Gibbons [http://www.ncbi.nlm.nih.gov/pubmed/18711340 High-resolution metagenomics targets specific functional types in complex microbial communities] (Nature Biotechnology) <BR> [http://www.nature.com/nrmicro/journal/v6/n9/abs/nrmicro1935.html Molecular eco-systems biology: towards an understanding of community function] (Systems Microbiology)<br />
<br />
= 10/06/2008 =<br />
# Bo Liu [http://www.pnas.org/content/105/39/15076.short The convergence of carbohydrate active gene repertoires in human gut microbes.] PNAS<br />
# Daniela Puiu [http://bioinformatics.oxfordjournals.org/cgi/content/full/24/20/2395 SeqMap: mapping massive amount of oligonucleotides to the genome] Bioinformatics<br />
# James Robert White [http://www.pnas.org/content/104/21/8918.full Artificial selection of simulated microbial ecosystems.] PNAS<br />
# MohammadReza Ghodsi [http://aem.asm.org/cgi/content/abstract/74/5/1453 Metagenomics: Read Length Matters]</div>Ghodsi