Setting up visualization. Make output folder for visualization files Log into vieques $ ssh

Slides:



Advertisements
Similar presentations
Introduction to Linux command line for bioinformatics Wenjun Kang, MS Jorge Andrade, PhD 6/28/2013 Bioinformatics Core, Center.
Advertisements

CS1020 Week 3: 29th January 2015.
Learning Unix/Linux Bioinformatics Orientation 2008 Eric Bishop.
MCB Lecture #10 Sept 25/14 SRA, Illumina data QC.
ISG We build general capability Job Submission on the Olympus Cluster J. DePasse; S. Brown, PhD; T. Maiden Pittsburgh Supercomputing Center Public Health.
ATS Programming Short Course I INTRODUCTORY CONCEPTS Tuesday, Jan. 27 th, 2009 Essential Unix Commands.
Introducing the Command Line CMSC 121 Introduction to UNIX Much of the material in these slides was taken from Dan Hood’s CMSC 121 Lecture Notes.
Introduction to RCC for Intro to MRI 2014 July 25, 2014.
1 Some basic Unix commands u Understand the concept of loggin into and out of a Unix shell u Interact with the system in a basic way through keyboard and.
Understanding the Basics of Computational Informatics Summer School, Hungary, Szeged Methos L. Müller.
7/17/2009 rwjBROOKDALE COMMUNITY COLLEGE1 Unix Comp-145 C HAPTER 2.
CS 141 Labs are mandatory. Attendance will be taken in each lab. Make account on moodle. Projects will be submitted via moodle.
Using Macs and Unix Nancy Griffeth January 6, 2014 Funding for this workshop was provided by the program “Computational Modeling and Analysis of Complex.
Learning basic Unix command IT 325 operating system.
Hola Hadoop. 0. Clean-Up The Hard-disks Delete tmp/ folder from workspace/mdp-lab3 Delete unneeded downloads.
Unix Primer. Unix Shell The shell is a command programming language that provides an interface to the UNIX operating system. The shell is a “regular”
 Accessing the NCCS Systems  Setting your Initial System Environment  Moving Data onto the NCCS Systems  Storing Data on the NCCS Systems  Running.
Linux environment ● Graphical interface – X-window + window manager ● Text interface – terminal + shell.
CSN08101 Digital Forensics Lecture 1B: Essential Linux and Caine Module Leader: Dr Gordon Russell Lecturers: Robert Ludwiniak.
BIOSTAT LINUX CLUSTER By Helen Wang October 11, 2012.
AN INTRO TO UNIX/LINUX COMMANDS BY: JIAYANG WANG.
CSCB07 TUT3 TA: Andrey Dyblenko Website:
PROGRAMMING PROJECT POLICIES AND UNIX INTRO Sal LaMarca CSCI 1302, Fall 2009.
UNIX Commands. Why UNIX Commands Are Noninteractive Command may take input from the output of another command (filters). May be scheduled to run at specific.
HPC for Statistics Grad Students. A Cluster Not just a bunch of computers Linked CPUs managed by queuing software – Cluster – Node – CPU.
 Name: Santiago Bock   Telephone:  Office Number: 5106 Sennott Square  Office Hours: Tuesdays.
Linux and Java Basics. What is Linux? Operating system by Linus Torvalds that was a clone of Unix (thus Linux) Free and open source – this is the reason.
Linux Commands C151 Multi-User Operating Systems.
Cluster Computing Applications for Bioinformatics Thurs., Sept. 20, 2007 process management shell scripting Sun Grid Engine running parallel programs.
EGEE-III INFSO-RI Enabling Grids for E-sciencE Apr. 25, Grid Computing Hands On Training for Users Faculty of Sciences, University.
AN INTRO TO UNIX/LINUX COMMANDS BY: JIAYANG WANG.
Linux Stuff Last Update Copyright 2014 Kenneth M. Chipps Ph.D. 1.
A Brief Overview of Unix Brandon Bohrer. Topics What is Unix? – Quick introduction Documentation – Where to get it, how to use it Text Editors – Know.
 Last lesson, the Windows Operating System was discussed along with the Windows command shell  Unix is a computer operating system, that similarly manages.
Basic Unix Commands. Listing files and directories ● ls:command is used to list the files and ● directories in present working directory ● ls command.
+ Vieques and Your Computer Dan Malmer & Joey Azofeifa.
+ Introduction to Unix Joey Azofeifa Dowell Lab Short Read Class Day 2 (Slides inspired by David Knox)
1 Linux Commands. 2 Path You specify a file or directory by its path name:  the full, or absolute, path name or the one relative to a location. The full.
Access QA servers Install SSH/SFTP software –T:\QualityAssurance\Tools\SSH.
Editing, Transferring, and Running Files on Vieques Daniel Malmer Dowell Lab Short Reads Course 6/9/15.
Hackinars in Bioinformatics
GRID COMPUTING.
UNIX To do work for the class, you will be using the Unix operating system. Once connected to the system, you will be presented with a login screen. Once.
Welcome to Indiana University Clusters
Stubbs Lab Bioinformatics - 2 Retrieving sequence data files and Linux commands Nov 17, 2016 Joe Troy.
Linux 101 Training Module Linux Basics.
Welcome to Indiana University Clusters
How to use the HPCC to do stuff
Andy Wang Object Oriented Programming in C++ COP 3330
Short Read Sequencing Analysis Workshop
Some Linux Commands.
C151 Multi-User Operating Systems
Assignment Preliminaries
Practice #0: Introduction
Tutorial of Unix Command & shell scriptS 5027
Postdoctoral researcher Department of Environmental Sciences, LSU
Tutorial of Unix Command & shell scriptS 5027
Compiling and Job Submission
Tutorial of Unix Command & shell scriptS 5027
The Unix File System.
Lab 00 Discussion Linux Basics
Andy Wang Object Oriented Programming in C++ COP 3330
Tutorial Unix Command & Makefile CIS 5027
Module 6 Working with Files and Directories
Video Notes.
Lab 2: Terminal Basics.
Hola Hadoop.
DIBBs Brown Dog Tutorial Setup
Presentation transcript:

Setting up visualization

Make output folder for visualization files Log into vieques $ ssh Change directories $ cd /projects/sreadgroup/students $ pwd $ ls Make output folder $ mkdir $ ls

Follow directions for setting up your vnc session Check from Jamie Kershner for port and link to setup vnc session short-read-class/visualization-information

What do I do when I get data back from the sequencer?

Prep your home directory $ cd /Users/identikey/ $ mkdir Day4 $ cd Day4 $ mkdir fastq $ mkdir $ mkdir PBS $ mkdir PBS/eofiles/

Copy today’s files $ scp /projects/sreadgroup/Day4/fastq/ * /Users/identikey/Day4/fastq $ cp /projects/sreadgroup/Day4/PBS/* /Users/identikey/Day4/PBS

Unzip your files $ cd /Users/identikey/Day4/fastq $ ls $ gunzip * $ ls $ less Example_A_01.fastq

What is a Fastq file? Read Identifier Sequenc e Quality

For the remainder of the FastQC examples, the class will be split into 4 groups Group A – Work with Example_A files Group B – Work with Example_B files Group C – Work with Example_C files Group D – Work with Example_D files DO NOT DELETE the other files from your folder, they will be used in your homework

Concatenate fastq files $ wc –l Example_X_01.fastq $ wc –l Example_X_02.fastq $ cat Example_X_0*.fastq > Example_X.fastq $ ls -lh $ wc –l Example_X.fastq $ rm Example_X_0*.fastq

Evaluating Sequencing and Library Quality

Where to find FastQC options $ /opt/fastQC/fastqc --help -OR- $ module load fastqc_ $ fastqc --help Don’t know how to use a program? Look at the help page for options!

Running FastQC $ cd /Users/identikey/Day4/PBS/ $ ls $ nano run_fastqc.pbs

Specify Job name Update resources if necessary Specify Queue Update eo file path #PBS –e /Users/identikey/Day4/PBS/eofiles/ #PBS –o /Users/identikey/Day4/PBS/eofiles/ Update Specify output dir -o /Users/identikey/Day4/QC/ Specify input file /Users/identikey/Day4/Example_X.fastq Replace anything with " " with your desired value & "amso5880" with your identikey

Submit FastQC job $ module list $ module load fastqc_ $ qsub run_fastqc.pbs $ qstat –u identikey

Look at eofiles $ cd eofiles $ ls $ less jobname.e12345 $ less jobname.o12345

Locate output files $ cd /Users/identikey/Day4/QC $ ls Move html file to visualizer folder $ cp Example_X_fastqc.html / projects/sreadgroup/students/ identikey/ OR: Use FileZilla to transfer html files to your own computer

Running Trimmomatic: Quick Start Settings $ java –jar /opt/trimmomatic/ 0.32/trimmomatic-0.32.jar $ cd /Users/identikey/PBS $ nano QS_trim.pbs

Job name Queue eofiles Replace anything with " " with your desired value and "amso5880" with your identikey

Trimmomatic Settings java -jar /opt/trimmomatic/0.32/trimmomati c-0.32.jar SE -threads 4 - phred33 call program Single end setting Multithreading Phred33 quality scores

Trimmomatic Settings Trimlog -trimlog /Users/identikey/Day4/ fastq/input_fastq.trimlog Input file /Users/identikey/Day4/fastq/input.f astq Output file /Users/identikey/Day4/fastq/input_t rimmed.fastq

Trimmomatic Settings ILLUMINACLIP:/opt/trimmomatic/0. 32/adaptors/TruSeq3-PE- 2.fa:2:30:10 \ LEADING:3 \ TRAILING:3 \ SLIDINGWINDOW:4:15 \ MINLEN:36

Run trimmomatic & look at eofiles $ qsub QS_trim.pbs $ qstat –u identikey $ cd eofiles $ ls $ less job_name.e12345 $ less job_name.o12345

Check trimlog and fastq_trimmed files $ cd /Users/identikey/Day4/fastq $ ls $ head -12 input.fastq $ head -12 input_trimmed.fastq $ less input_fastq.trimlog

Copy trimmomatic PBS to a new file $ cd../PBS $ ls $ cp QS_trim.pbs new_trim.pbs $ nano new_trim.pbs

Update new_trim.pbs Job name Settings java -jar /opt/trimmomatic/0.32/trimmomatic jar SE -threads 4 -phred33 \ -trimlog /Users/amso5880/Day4/fastq/ input_fastq_cropped.trimlog \ input.fastq\ input_cropped.fastq \ CROP:40 \ HEADCROP:10

Run trimmomatic & look at eofiles $ qsub new_trim.pbs $ qstat –u identikey $ cd eofiles $ ls $ less new_job_name.e12345 $ less new_job_name.o12345

Check trimlog and fastq_trimmed files $ cd /Users/identikey/Day4/fastq $ ls $ head -12 input.fastq $ head -12 input_cropped.fastq $ less input_fastq_cropped.trimlog