Either (r)ead, (w)rite, or e(x)ecute on 'user', 'group', 'others'
Example: -rw-r--r-- User can read and write, group and others can only read
The first column can either be
- regular file
d directory
l symlink
Execute permissions are necessary to remove directories
The '*' can be used as a wildcard:
ls *.fasta List anything that ends with .fasta
But this also works: ls *[0-9].fasta List anything that ends with number.fasta ls *[3-5][5]0.fasta List anything that ends with 350, 450, 550
Ctrl-a Jump to the start of the line Ctrl-e Jump to the end of the line Ctrl-k Remove everything between cursor and end of line (K is for KILL) Ctrl-u Remove everything between cursor and start of line Ctrl-w Remove word backwards Alt-d Remove word forwards Ctrl-y Paste the last thing you removed with Ctrl-K or Ctrl-W or Ctrl-U (Y is for YANK) Ctrl-l Same as 'clear' Ctrl-d Same as 'exit' Ctrl-d If text in command, same as DELETE Ctrl-h If text in command, same as BACKSPACE Ctrl-x-e Open up in-terminal editor (emacs / vim) and write multi line command. Exiting the editor will then execute the command Ctrl-p Cycle through previous commands. Same as arrow up key Ctrl-N When cycling through previous commands, go to next command. Same arrow down key
!$ Bang-Dollar. Wildcard for the last argument of the previous command Esc-. Will return the last argument of the previous command Esc could be Alt in other systems Esc-Bksp Remove everything between special character (_./) and cursor Esc-F Move forward one word in the command line Esc-B Move backwards one word in the command line
See Corey Schafer video on YouTube The prompt is determined by the PS1 variable
PS1="-> " Sets the prompt literally to -> PS1="\u@\h \W -> " username@hostname currentdir ->
\h hostname up to the first . \n newline \s the name of the shell \t the current time in 24 hour format \u username of the current user \w current working directory - full absolute path \W the basename of the current working directory - only current directory \D timestamp, e.g. \D{%H:%M:%S} will display hour:minute:second
The custom PS1 variable typically gets very long, so you may want to break it up in several pieces, so its more human understandable Has added benefit that you can describe the different parts with comments.
PS1="$(tput setaf 166)\u" # orange user PS1+="$(tput setaf 228)@\h" # yellow host PS1+="$(tput setaf 71)\W -> " # green directory PS1+="$(tput sgr0)" # reset formatting so colors do not bleed into command line
You should also envelope the tput commands with [ and ], i.e. [$(tput setaf 166)] to prevent the cursor from acting up in the terminal export PS1
lscpu Reports the number of cpus, architecture, and all other relevant CPU information of the system cat /proc/cpuinfo Bunch of info per CPU on the computer cat /etc/os-release Name and version of current operating system
sh, bash, Perl, Python, Ruby etc are scripting languages in which the "program" is a regular file containing plain text that is interpreted into MACHINE CODE at the time you run it. Other languages like c++, java, haskell, rust have a separate COMPILATION step to turn their regular text source file into a BINARY EXECUTABLE.
A script from sh, bash, Perl, Python, Ruby etc become EXECUTABLE simply by turning on the executable permission.
#!/bin/bash Allows the script interpreter to recognize that the file should be executed with bash #!/usr/bin/env bash Same, but now the script is portable. I.e., different systems will recognize it as a bash script The env program will find the bash that is found in your environment #!/usr/bin/env python3 Use the python3 that is found by the environment
Everything in UNIX is case sensitive. So $NAME ne $name ne $Name Probably a good idea to use lower case variable names in scripts, so not to accidentally overwrite environment variables
Analogous to 'use warnings' in perl? Write 'set -u' on top of your shell script
program 2> file.e Redirect STDERR to file.e program 1> file.o Redirect STDOUT to file.o program 1> file.o 2> file.e Redirect STDOUT to file.o and STDERR to file.e program 2>/dev/null Redirect output to /dev/null , where data is gone forever
file Truncate a file to size 0 if it exists, create an empty file if it doesnt exist
IF LOOP SYNTAX if []; then ... else ... fi
NESTED FOR LOOPS for i in {}; do for j in {}; do cmd $i $j; done; done;
FOR LOOPS
for i in {,,}; do cmd1 $i; cmd2; done
sometimes
for i in *.txt; do cmd; done Loops over all files that match the *.txt pattern.
WHILE READ LOOP Read a file line by line and assign variables to the line's content.
Example: ls -la | while read PERM USR USRGRP SIZE; do echo $PERM $USR $USRGRP $SIZE; done
while read -r LINE; do echo "$LINE" done < file.txt
cat file.txt | while read LINE; do echo $LINE done
cat file.txt | while IFS=$'\t' read -r WORD; do echo $WORD done
Use the read '-r' flag, 'while read -r STRING' to prevent read from eating '' in $STRING
LOOP CONTROL break Breaks and leaves the loop continue Skip to the next iteration of the loop (definitely analogous to next of Perl)
LOOP OVER RANGE OF NUMBERS for i in $(seq 89 2 97); do echo $i done
loops from 89 to 97 with steps of 2
Will return 89 91 93 95 97
COMPARE TWO STRINGS if [ "$S" -ge "$Q" ] ...
SHELL BOOLEANS FOR INTEGERS -eq Equal -ne Not Equal -gt Greater Than -lt Lesser Than -ge Greater or Equal then -le Lesser or Equal then
STRING COMPARISON OPERATORS = Match != Does not match
Example: Checks if the string "BACL10" is part of $alphabin
if [[ "$alphabin" != "BACL10" ]]; then echo "blastp $alphabin ..." fi
OTHER COMPARISONS [[ -e FILE ]] True if FILE exists [[ -f FILE ]] True if FILE exists and is a regular file [[ -h FILE ]] True if FILE exists and is a symbolic link [[ -s FILE ]] True if FILE is non-empty [[ -d DIR ]] True if DIR exists [[ -z $STRING ]] True if $STRING is null (doesn't exist) [[ -L SYMLINK ]] True if SYMLINK exists
cmd1 && cmd2 && is the AND operator. execute cmd1 and only if it succeeds execute cmd2 cmd1 || cmd2 || is the OR operator. execute cmd2 only if cmd1 fails Note that these operators already have an 'if' functionality built in!
&& is used to chain commands together, such that the command after the && is only run when the command prior to the && is run without any errors && is also used to mean AND in comparisons that return True/False
Example [[ ! -d "$OUT_DIR" ]] && mkdir -p "$OUT_DIR" if $OUT_DIR does not exist is TRUE, create $OUT_DIR
Alternatively
if ls *.orig.tsv 1> /dev/null 2>&1; then ... ; fi
if ! grep -q $pattern file.list; then ... ; fi
POSITIONAL ARGUMENTS FOR BASH SCRIPT
$0 Name of the bash script
$1 First argument
$2 Second argument
If multiple words on the command line are flanked by quotes, whatever is within the quotes is one positional argument ./script.sh ONE TWO "THREE PIGS" $1 -> ONE $2 -> TWO $3 -> THREE PIGS
CASE Matches a variable against a range of options described in the case loop
Example: var=apple case $var in apple) echo "variable was apple";; pear) echo "variable was pear";; mango) echo "variable was mango";; esac Will print "variable was apple"
GETOPTS Stating options in a bash script. Only works with one letter flags.
while getopts ":s:m:z:n:t:g:" opt; do
case
The : after the single letter in the getopts line, states that it expects an argument after the flag. I.e. -s bladiebla.aln The first : in that line prevents 'verbose error handling' :) gets activated when an option/flag that needs an argument is called without an argument ?) gets activated when an option/flag that is not specified in the code is called.
USING DELIMITER OTHER THAN WHITESPACE: grep -c ">" foo.fasta | while IFS=':' do echo $FILE $SEQCOUNT; done
i=teststring
echo
OUTFMT="6 std stitle" If you pass $OUTFMT to some command line tool, it will evaluate to 6 std stitle as separate arguments If you pass "$OUTFMT" instead, it will evalute '6 std stitle' as a single argument
string=${var:-default} $string gets value $var if $var is set previously. Otherwise,
sum=$((1+1)) $((1+1)) returns to 2, prior to assigning sum the value 2
let sum=1+1 $sum is assigned the value 2 through simple arithmetic evaluation
declare -A arr Declare a new array arr arr=() Empty/reset arr arr=( ["moo"]="cow" ["woof"]="dog") Create a dictionary/hash. Supported since bash v4. Make sure to use #!/bin/bash and not #!/bin/sh while read -r one two; do arr[$one]="$two" done < input_file Fill up an array/dictionary/hash by reading in a file. It seems critical to have the file read in like this: done < input_file echo "${!arr[@]}" Return the KEYS of the dictionary echo "${arr[@]}" Return the VALUES of the dictionary
join <(sort FILE1) <(sort FILE2) > FILE3 <(sort FILE) obsoletes an intermediate file, it creates like a temporary file within the command. Otherwise you would have done: sort FILE1 > FILE1-sort sort FILE2 > FILE2-sort join FILE1-sort FILE2-sort > FILE3 rm FILE1-sort rm FILE2-sort So it saves you a lot of work! Amazing!!
cmd > file.txt Write STDOUT to file.txt cmd >> file.txt Append file.txt with STDOUT cmd &> file.txt Write STDOUT and STDERR to file.txt
Every command in Unix returns an exit status. A successfull exit status returns 0, an un-succesfull one returns a non-zero value. The last command executed determines the exit status. Within a script, when exiting, you can pass on a custom exit status through exit , where nnn is an integer between 0 and 255
The special variable
export VARIABLE="FOO" # set "FOO" as environment variable VARIABLE unset VARIABLE # remove variable VARIABLE from environment
All environment variables are uppercase !
$PATH Contains all directories that contain executables that can be directly invoked from the command line $HOME Contains the absolute home directory $PWD Contains current absolute directory $USER The username $SHELL The program running the current shell
alias Returns all active aliases
To add default aliases, write in your .bashrc or .profile files alias=='' Example: alias l='ls -la'
unalias Remove from alias list
Execute a bash script at a given time Example: at -f <commands.sh> 21.00
Trims paths and extensions froms strings (opposite of DIRNAME)
Example: FASTA returns panorthologs/famcl00140.faa basename $FASTA returns famcl00140.faa basename $FASTA .faa returns famcl00140
see also DIRNAME
Calculator echo "100/50" | bc Yields 2 echo "scale=2:100/60 | bc -l Yields 1.67
-e Show newline characters as '$'
-A Show tabs (^I), newline characters (
-A Shows identity of whitespace characters. Much nicer than cat -A --config-file Show location of your bat config file
Neat trick: cat > new_file Then type whatever texts you want to type End with Ctrl-D You now have a new file with the typed content!
column -t [file] Prints the file so that all columns are nicely aligned in view column -t [file] -s $'\t' Use TAB as separator instead of the default whitespace You can also type 'Ctrl-v TAB' while typing the comment. "
cd - Returns to the previous directory
Change rights of owner / user, group, others or all of a file to either read, write or execute. Example: chmod u+x Adds the e(x)ecutable right for (u)ser to this chmod ug+x Add executable rights to user and group chmod u-x Remove executable rights for user chmod 755 7=4+2+1=r+w+x for user. 7=4+2+1=r+w+x for group. 5=4+1=r+x for others chmod 600 6=4+2=r+w for user, 0=--- for group, 0=--- for others
Change ownership of a given directory Example: chown kasiazar kasia/ -R Recursive (any directory below will also be changed ownership)
Compares to files and lists the common lines and unique files Files have to be sorted first (in the same way ofcourse) They have to be sorted lexicographically apparently (-V sort on both files won't work with comm) Column1: unique to file1 '-1' to suppress this column Column2: unique to file2 '-2' to suppress this column Column3: appear in both files '-3' to suppress this column
cp -P Copy a symbolic link
Converts between image file formats Example: convert filename.pdf filename.png convert -resize 200x100 filename.pdf filename.png Resizes the image while converting convert -resize 200 filename.pdf filename.png Resizes the image to 200px wide, but keeps aspect ratio for length convert -append *.png output.png Merge images into a single file, stack images vertically convert +append *.png output.png Merge images into a single file, stack images horizontally
Convert xslx to csv Example: in2csv in.xlsx > out.csv Install with 'brew install csvkit'
-l List all current crontabs -e Edit crontab commands crontab command format: [minute] [hour] [day of the month] [week] [month] [day of the week] Example 0 12 * * * * Every day, at 12.00
cut Remove sections from lines -f Select column Example: 3. Or 1,3,4. Or 1-5. Or 1-3,5-7 Example: 3- to print 3rd column until the last -d Select delimiter to use to creat fields, Example: '_' or ' '. Default is '\t' ? -d' Set literal single tick as delimiter --complement Prints all fields except those specified in -f cmd | rev | cut -f 1 | rev Trick to print the last column. Be ware: you may need to put cut -f 2 instead.
datamash -s -g 1 collapse 2 A 1 A 2 B 3 C 4 D 5 D 6
into
A 1,2 B 3 C 4 D 5,6
Shows differences between two files diff
-y invoke side-by-side mode, really nice -w set width of visibles lines of differing lines in side by side mode --suppress-common-lines --color=always color persists to less -R
Example: diff -W 200 -y --color=always | less -R
Trims filenames from strings (opposite of BASENAME)
Example: FILE returns /path/to/panorthologs/famcl00140.faa dirname $FILE returns /path/to/panorthologs
DNSDOMAINNAME dnsdomainname Reveals the domain name of the current computer where you are logged in Example: icm.uu.se
-h <file|directory> Human readable numbers du -h . | sort -h -r | head -n 20 Nice quick command to look for the top biggest directories from current directory du -sh
Report total size of dir only. Will not report any nested directories or filesecho "$STRING" Prints string to STDOUT echo $STRING Same thing but converts tabs to spaces
Options: -e Enables interpretation of backslash escapes and certain characters are recognized, like \t and \n -n Removes the newline character
Takes a string as an argument and evaluates it as if you'd typed that string on a command line
exit Exit with an exit code. Any non-zero exit code will be interpreted as exited with an error.
find <PATH> <flags> \;
-name <EXPRESSION>
-maxdepth <number> The level of directories down find will try to find your attern
-size 0
-exec <cmd> {} {} refers to each file that was found
-xtype l Find broken symlinks
-newermt "2022-05-10" Find files that are newer (modified later) than May 10th 2022
-ls Print found files like an 'ls -l' print
Example:
find . -name "All.htm"
find . -size 0 -exec rm {} \; Remove empty files
find . -xtype l -exec rm {} \; Remove broken symlinks
# sort all files in directory and subdirectories by modification time
find . -type f -printf '%TY-%Tm-%Td %TH:%TM: %Tz %p\n' | sort -n
# list all directories that do not contain a file that has pattern
find . -type d ! -exec sh -c 'ls "{}" | grep -q "PATTERN"' \; -printFG Bring back jobs that are running in the background, into the "foreground". Jobs that are running in the background can be displayed with "jobs" Ctrl-Z to put current windown in the background. Then you can use 'fg' to put it back to the foreground
GREP grep "PATTERN" OUTPUTS THE ENTIRE LINE OF WHERE IT FINDS ITS HITS grep -v "^$" REMOVES EMPTY LINES (no special character, but clever use of regexp: ^ start of line, $ end of line) Flags: -i Ignores case -n REPORTS LINE NUMBER OF HITS -c COUNTS HITS -c "" Counts lines in the file -A PRINTS IN ADDITION TO HIT LINE ALSO int LINES AFTER HIT -v PRINTS ALL LINES EXCEPT LINES WITH HITS -P Allows for Perl regexp in PATTERN -f Prints all lines that have the PATTERNS listed in FILE, one per line -x The PATTERN must encompass the entire line. Handy for speeding up grep --no-group-separator Suppresses the '--' seperator when using -A, -B, or -C -o If you have multiple hits in a single line, it will report all the hits on that line, seperated by newlines. It will not report the whole line, but just the match. Hence the name '-only_matching' --color=always Keep coloring of match even if you pipe into a new grep command. Even less keeps color Can even do 2 different colors. Example echo "hello" | grep --color=always "he" | GREP_COLORS="mt=01;34" grep "lo" -e Multiple search patterns at the same time. grep -e "pattern1" -e "pattern2" -m Only reports the first lines with match -H Always reports the filename that it found its hit in -q Quiet mode
..AGREP Grep that allows mismatching. Very handy to search sequence data still allowing sequence errors --color -E Allowed of mismatches
..RIPGREP (rg) rg -f <pattern_file> Prints all lines that have the PATTERNS listed in FILE, one per line. MUCCCH FASTER than grep -f Other options: -N Suppress line numbers -c Count hits -z Search compressed files --color=always Ensures that color highlighting persists when for example piping to less -SR --no-filename Hides filenames when reporting lines that match expression
rg --files-without-match 'Best FunDi parameter' *log
HOSTNAME hostname Reveals the name of the host Example: molev209
IDENTIFY Reports properties of an image file. Part of the ImageMagick suite. Example: identify filename.png
Reports:
filename.png PNG 200x3510 200x3510+0+0 16-bit DirectClass 1.267MB 0.000u 0:00.000
JOBS Lists all 'jobs', that can either be commands, or scripts or anything like that in the background. Putting an '&' sign after the job command puts it in the background. With FG (foreground), you put the 'job' back in the foreground. Then you can kill it for example. JOBS only lists those jobs that are running in that specific terminal
Join two files based on a common field between two files to create a new, merged file NOTE: FILE1 and FILE2 need to be sorted (on their common field) prior to attempting to join!
# complicated example
join -a 1 -a 2 -e '0' -1 2 -2 2 -o '1.1,2.1,0' -t $'\t' FILE1 FILE2 > FILE3
# Switch to print unpairable fields of FILE1 and FILE2
-a 1 -a 2
# Replace missing fields with 0 (if a column value exists in FILE1 and not in FILE2 or vice versa)
-e '0'
# Join using the 2nd field in file 1 and 2nd field in file 2
-1 2 -2 2
# Print the field 1 of file 1, field 1 of file 2, and the field used to join them (0 special meaning)
-o '1.1,2.1,0'
# Set field delimiter. Tab is default, but by setting this explicitly, Tab becomes the delimiter in the output too
-t $'\t'
KILL Killing defunct processes. For example when Firefox crashes ps -e | grep firefox kill -9 <process_id_of_firefox>
LESS /PATTERN Search for regex pattern in file from this point on forward ?PATTERN Search for regex pattern in file from this point on backwards Press 'n' to go to the next hit Press 'N' to go the previous hit then g Go to line
Flags: -S Show file without line breaks -N Show line numbers -R Show font colors +F
LN
Symbolic links
ln -s
LS ls -S Sorts all files in directory by size ls -t Sorts all files in directory by last time modified ls -d */ List only directories ls -L List the file that the symbolic link is pointing to ls --ignore= List all files except those that match
ls --full-time
LSOF
lsof | grep '.nfs00000000107c4e6200248a94'
MAN man -l Will open the file with man man ./ Will open the file with man
# Create nested directories in a single command. 'p' stands for parent directory
# -p also prevents the 'does already exists' error if directory already exists
mkdir -p newdir1/newdir2
# Create directory with these permissions ??
mkdir -m 777tmp=$(mktemp) mktemp returns a random temporary file name and creates that file tmp_dir=$(mktemp -d) mktemp -d returns a random temporary directory name and creates that directory
more Opens the file for reading. Can for some files display colors, underlines and bold text
nohub Continue running the program even if the session ends
Basic usage
# feed in a glob referencing all the files you want to process
parallel -j[n_processors] "[command you want to parallelize] {}" ::: [files to process]
# also works with single quotes
parallel -j[n_processors] '[command you want to parallelize] {}' ::: [files to process]
# feed in a file that references all the files you want to process
parallel -j[n_processors] "[command you want to parallelize] {}" :::: [file_containing_patterns_to_process]
# storing variables inside a parallel command is possible
# it seems important to use single quotes for outer, and inner quotes for inner
parallel -j10 'LL=$(rg "FunDi log-likelihood" {}.log | cut -f3 -d" "); echo {} "$LL"' :::: ../leca_fams.list
# process all commands listed in $JOBS with parallel
parallel -j2 < $(JOBS)
# prevent new jobs from being invoked if one of the processed jobs returns an error
parallel -j2 --halt soon,fail=1 < $(JOBS)Example
parallel -j8 "seqret -sequence <(zcat {}) -outseq fasta/{/.}.fasta" ::: ../../data/wgs_*.dat.gzThis command will use 8 processors, so 8 seqret commands will be executed at the same time until all files have been processed.
{} holds the name of the file, including the path, for example in this case ../../data/wgs_cevg01_env.dat.gz {/.} holds the basname of the file, and the extension trimmed, for example in this case wgs_cevg01_env.dat If you don't provide a number of processors, parallel will take ALL available processors
Currently running jobs are still finished properly
paste Merge lines of files
# join all the lines in a file into a single line, using tab as delimiter
paste -s /path/to/file
# join all the lines into a single line, using a custom delimiter
paste -s -d ',' /path/to/file
# merge two files side by side
paste /path/to/file1 /path/to/file2
# example usecase
cat [list] | paste -sd+ | bc easy code to get the sum of column of numbersAs a replacement for sed. Perl -PE has a better regular expressions support -p assumes the code 'while () { ##code## } ' -e anything within '' will be seen as a perl script
Example: perl -pe 's:^.+$:TEST: if /^>/' aaa_pooled-hfix.fasta -i Changes input file. (Just like sed). But the -i has to be first flag for some reason -w use warnings
Concatenate PDFs pdfunite *.pdf out.pdf or pdftk *.pdf cat output <new.pdf>
Subset pages of a PDF pdftk <in.pdf> cat 1-4 6-7 output <out.pdf> Put pages 1-4 and 6-7 of <in.pdf> into <out.pdf> pdftk <in.pdf> cat 8-end output <out.pdf> Put pages 8 until the last page into <out.pdf>
Concatenate PNGs
# concatenate left to right
magick convert +append one.png two.png three.png out.png
# concatenate top to bottom
magick convert -append one.png two.png three.png out.pngps Shows a list of active processes with process IDs (PIDs) and the command used to start these processes ps -fu $USER Shows all actives processes invoked by $USER ps -o <field1,field2,etc> Show process key fields. Example: ps -o user,uid,comm,pid,pcpu,tty ps -r Sort by pcpu (%CPU)
PPID: Parent Process ID
readlink -f <file.txt> Returns the absolute path of the file If realpath is not available
realpath <file.txt> Returns the absolute path of the file
reboots the system
rsync
For updating chain backups, this command is really useful rsync -avP --inplace --append chain*.* chain_backup/
screen Open up a new screen screen -S Create a new screen with screen -ls List down the current open screens screen -r Reattach to the screen screen -r Reattach to screen with the set sessionname screen -r If there is only one screen, it automatically will reattach to that one. exit Exit the current screen you are in (this will kill any processses in this screen?) killall screen Kill all screens
WHILE IN SCREEN ctrl-a + ctrl-d Detach from the screen while not interuppting any processes in the screen ctrl-a + ':' + sessionname + Enter Rename a session to something easy to remember ctrl-a + Esc You can now scroll up and down using arrow keys and PgUp PgDwn
SEQ Creates a sequence of numbers seq -s seq -w 000 050 500 returns 000 050 100 150 etc. -w ensures that all numbers have the same digit length.
Example: END=5 seq 0 $END Returns 0 1 2 3 4 5, separated by
SIPS
Rotate png / jpg images a certain number of degrees
sips
-r <degrees_rotating_clockwise>
--padColor <color_of_resulting_deadspace> (FFFFFF = white)
--out <out_file>
If you want to rotate counter clockwise e.g. 10 degrees, simply rotate clockwise 360-10=350 degrees.
-t "_" Set field separator to underscore
-r Reverses the sort output (for example instead of large to small go from small to large)
-u -k 1,1 Omits lines that are copies in the first field after sorting
-g Sort numerical lists that include exponential values (e.g. ending with e-06)
-h Sort human readable numbers, incompatible with -n
-k 1 All fields from field 1 until the last field
-k 1,1 Just field 1
-k 1,1n Sort field 1 numerically
-o FILE FILE Sort the file in place
-V Sort based on the numbers in a field, also if it contains letters and dots etc. Version sortSHUTDOWN shutdown -r +5 "Rebooting soon, log out ASAP!" Gives people who are logged in some time to log out.
SPLIT Split a file by size, by lines or other things split -l 10000 Splits up a file into multiple files with 10000 lines each split -d Makes numerical suffixes instead of alphabetical split -n l/ Splits file into chunks while not splitting lines
SSH Login as root: Type either 'su' and then enter the root password or ssh into your own machine as root as follows: ssh -i .ssh/root_rsa root@molev209 and enter the passphrase for key '.ssh/root_rsa' (not the same as the root password) ssh -X enable X11-forwarding ssh -Y enable trusted X11-forwarding Once you are in the server, you can check if X11 server is active with echo $DISPLAY which should return something like 'localhost:15.0'
add ssh key so you only need to type passphrase once per session ssh-add ~/.ssh/perun_rsa
SVN cd desired/directory svn checkout svn://foo DOWNLOADS REPOSITORY, THIS IS ONLY NEEDED TO DO ONCE Before you can add a new file to the svn, you actually need to physically copy it to the svn folder. svn add [new dir] svn add [new file] svn commit -m "Message to the subversion system DATE" svn update UDPATES YOUR LOCAL REPOSITORY WITH LATEST CHANGES ON SUBVERSION SYSTEM
TAIL / HEAD tail Shows the last 10 lines of a file - Shows the last [number] lines of a file -n Shows the last [number] lines of a file -n + Shows the last part of a file starting at line [number] -f Keeps updating, if the file is being appended continuously
head Shows the first 10 lines of a file - Shows the first lines of a file -n Shows the first lines of a file -n - Shows the first part of a file until line
note that tail -n+ and head -n- only work with GNU head/tail ?
TAR tar is short for "tape archive" tar -tvf Lists the contents of the .tar tar -ztvf Lists the contents of the .tar.gz tar -jtvf Lists the contents of the .tar.bz2 tar -czvf .tar.gz /path/to/dir-or-file> (Create) an archive with g(Z)ip, with (V)erbose and specify (F)ilename tar xzf <tar_ball> -C <output_dir>
TR tr -s ' ' If you have multiple white space characters in a row, it will truncate it to just a single white space
tr '\t' '\n' Transform all tab into newlines tr '[abc]' '[def]' Transform all a's into d's, b's into e's etc. cat FILE | tr -dc '(' | wc -c Counts number of '(' in file tr -d ' >' Deletes space ánd > tr -d '\n' Removes all newline characters
TIME time [cmd] Will print out the time it takes to execute a given command
UNAME Prints system information -a Prints all information
Prints among other stuff, hostname, 32-bit or 64-bit
UNIQ -u show which lines are not repeated (?) -d show only duplicate lines
VI(M) :x Quit
W Shows you who is currently logged in
WGET wget <source_url> -P <out_dir>
WC wc -m Prints the number of characters in the entire file
WHATIS whatis Gives a oneline summary of what the is
WHEREIS whereis Searches standard unix/linux locations, like /usr/local/bin, /usr/bin, /bin/ etc for Shows ALL locations of where a tool is installed. For instance whereis python can show you all different python installations
WHICH which Searches your $PATH for the executable that is and returns the directori(es) in your $PATH that have said
perl -pe 'if (/^>/) {my @line = split /|/,$_; s/.+/>$line[3]/;}' 16S_Pooled_id83-otu-0.89-pruned.fasta
perl -pe 's/)([1-9]|[1-6][0-9]):/):/g' newick > newick-edit.out
perl -pe "s/)\d+:/):/g" newick > newick-edit.out
cut -f 5 | tr '\t' '+' | sed 's/+$/\n/' | bc
sed -n 'n;p;n;n' <.fastq> | tr -d "\n" | wc -c
sort -k1,1 -k12,12gr -k11,11g -k3,3gr contigs_vs_nt.blastn | sort -u -k1,1 --merge 1,1 query name 12,12 bit score 11,11 evalue 3,3 id
sed -i -r "s/)([0-9]{1,2}).[0-9]:/)\1:/g"