Je is a suite to handle barcoded fastq files with (or without) Unique Molecule Identifiers (UMIs) and filter read duplicates using these UMIs 

If you have barcodes and/or UMIs in your fastq files, you'll most likely enjoy Je. Je currently offers 4 tools :

1. demultiplex to demultiplex multi-samples fastq files which reads contain barcodes and UMIs (or not) 
2. demultiplex-illu to demultiplex fastq files according to associated index files (contain the sample encoding barcodes). Reads can additionally contain UMIs (inline)
3. clip to remove UMIs contained in reads of fastq files that do not need sample demultiplexing
4. markdupes to filter BAM files for read duplicates taking UMIs into account

In short, Je demultiplex, demultiplex-illu and clip add extracted barcodes and UMIs to the read headers and reformat read headers to fulfill read mappers requirements. Indeed most read mappers (bowtie, bwa...) expect headers for read_1 and read_2 to be strictly identical. After mapping, markdupes identifies PCR (and optical) read duplicates based on their mapping positions and UMIs found in read headers.

Source Code & Executables

Je source code (Java) is available in git at EMBL GitLab and executables in the repository dist directory. In addition, we provide the je-suite through bioconda and galaxy.

Installation

Using conda

"conda install -c bioconda je-suite"

Manual

1. Download Je latest tar.gz (e.g. je_1.2.tar.gz) from the code repository
   • ​simply go to EMBL GitLab and navigate to the dist directory (Files menu)
2. Unpack
3. Call "~/jesuite/je -h" to see the available options 

Note that java 7 is expected.

For Galaxy

As an admin of your local Galaxy installation, simply look for "suite_je" in the toolshed and click install!

Getting help

• Read this page
• Please subscribe to the Je's mailing list ( je -at- embl.de):  subscribe to the ML! Or read the archive.
• Consider the command line help i.e. -h

Citing Je

Selecting correct demultiplexing options (graphical decision trees)

Below are decision workflow to help you pick the right key options depending on your situation (click pictures to enlarge).

Je's demultiplex module (no Illumina index files)

Option Selection for Premultiplexed Libraries

Je's demultiplex-illu module (with Illumina index files)

Option Selection for libraries multiplexed with Illumina index(es)

Practical demultiplexing examples

Single End

Simply call (will use defaults) :

> je demultiplex F1=file1.txt.gz BF=barcodes.txt

Paired End without UMIs

Scenario 1: the same barcode is found in both reads

Maximize the number of reads : reads are attributed to samples even if only one of the two barcode resolves to the sample S=false, the default (reads are ignored if barcodes resolve to different samples)

> je demultiplex F1=file1.txt.gz F2=file2.txt.gz BF=barcodes.txt

Some as above but a bit more stringent i.e. keep reads only if both barcodes resolve to the same sample

> je demultiplex F1=file1.txt.gz F2=file2.txt.gz BF=barcodes.txt S=true

Note that in both situations:

• Input FASTQ file are expected to be encoded using phred scale + 33 (V option)
• the output files are automatically gzipped (GZ=true), named after a pattern like samplename_barcode.txt.gz (can be overridden by providing file names in extra columns of the barcode file) and placed in the current dir (output dir can be adapted using O=/path/to/dir).
• Unassigned reads are saved (UN=true) in unassigned_1.txt and unassigned_2.txt files (current dir) as well as a summary file jemultiplexer_out_stats.txt (all these file names and path can be adapted if needed).
• the barcode look up use a maximum mismatch (MM=1) of 1, a minimal mismatch delta (MMD=1) with second best barcode match of 1 and a minimum quality of 10 (Q=10)
• the barcodes are removed from reads (C=true) and added to the FASTQ header (ADD=true)

Keep reads only if both barcodes resolve to the same sample, adapt the output dir (all files will be now saved in this dir) and provide demultiplexed file names (unassigned read files will still be named automatically)

> je demultiplex F1=file1.txt.gz F2=file2.txt.gz BF=barcodes.txt S=true O=/tmp/jemultiplexer_out/

with a barcode file like:

sample1     ATATAT     sample1.txt.gz
sample2    GACGAC    sample2.txt.gz

Scenario 2: the barcode is found in only one read

The barcode is in READ_2

> je demultiplex F1=file1.txt.gz F2=file2.txt.gz BF=barcodes.txt O=/tmp/jemultiplexer_out/ BPOS=READ_2

N.B.: BM will be automatically set to READ_2, BRED and S will be ignored. 

N.B. 2: reads will have different length as only one read will have the barcode sequence removed , see below example to make sure reads have same length.

The barcode is in READ_2 => make sure read 1 is of same resulting size by using unbalanced trimming options

We need to compensate 6 bases i.e. assuming barcode length is 6, READ_2 will be trimmed of 6 bases 

• Example 1: also remove 6 bases at start of READ_1 (XT=6:1)

> je demultiplex F1=file1.txt.gz F2=file2.txt.gz BF=barcodes.txt O=/tmp/jemultiplexer_out/ BPOS=READ_2 XT=6:1

• Example 2: remove 7 bases at the end of READ_1 and none at the end of read 2 (ZT=6:0)

> je jemultiplex F1=file1.txt.gz F2=file2.txt.gz BF=barcodes.txt O=/tmp/jemultiplexer_out/ BPOS=READ_2 ZT=6:0

• Example 3: or use a mix ...

> je demultiplex F1=file1.txt.gz F2=file2.txt.gz BF=barcodes.txt O=/tmp/jemultiplexer_out/ BPOS=READ_2 ZT=6:0 XT=1:1

Scenario 3: a different barcode is found in both read with both barcodes being needed for sample lookup

The barcode file must be like:

sample1    ATATAT:CGCGCG
sample2   GACGAC:TACGTT

 i.e. sample 1 has barcode ATATAT in READ_1 and CGCGCG in READ_2 ; and

 sample 2 has barcode GACGAC in READ_1 and TACGTT in READ_2

Then :

> je demultiplex F1=file1.txt.gz F2=file2.txt.gz BF=barcodes.txt BPOS=BOTH BM=BOTH BRED=false O=/tmp/jemultiplexer_out/

Note that you could use different barcode lookup parameters (if this makes sense!) using the syntax MM=1:2 MMD=2:1 Q=30:10

Paired-end with UMIs or degenerate positions

Scenario 1: one barcode encodes the sample identity (e.g. READ_1) while the other barcode (in READ_2) is a Unique Molecular Identifier

In this example we'll consider that the sample encoding barcode is 6 bases long and the UMI is 10 bases long. 

The barcode file will look like:

sample1     ATATAT
sample2     GACGAC

To specify we hane 2 barcodes, we need BPOS=BOTH

To tell Je only the first read encodes sample identity, we need to explicitely mention BM=READ_1

We must also specify the individual barcode length ie 6 and 10 : LEN=6:10

Due to the barcode length difference, we'll also compensate length difference using ZT of 4 on READ_1 only (ie to end up with reads of identical length) so we'll add ZT=4:0

> je demultiplex F1=file1.txt.gz F2=file2.txt.gz BF=barcodes.txt BPOS=BOTH BM=READ_1 LEN=6:10 ZT=4:0 O=/tmp/jemultiplexer_out/

Scenario 2 : iCLIP barcodes i.e. barcodes mixed with random sequence

For the iCLIP experiment, barcodes contain random letters (A,T,G, or C). For example, a barcode may look like NNNNATTCNN, where N can be A, T, G, C or N. Only the barcode letters from 5th to 8th positions are used to resolve a sample i.e. Ns are ignored. .

The barcode file for an iCLIP may look like:

sample1     NNNNATATATNNN  
sample2     NNNNGACGACNNN

PS: All other command line options are like in the other scenarii. 

Important : when barcodes (defined in the barcode file) contain Ns, Je demulitplex will copy the extracted sequence at the end of header line (CLIP and ADD options set to true) instead of the barcode (as found in the barcode file, which the default behavior). Note that many tools (bowtie2, ...) will clip this information from the read header (due to the space separator). Because you might need to have this information available down your workflow (when dealing with duplicates), we suggest you to use the RCHAR option (e.g. add RCHAR=':' in the command line) so all spaces are replaced with a ':'. 

Scenario 3 : composite barcodes 

In case of composite barcodes i.e. sample-encoding barcode followed or prefixed with a UMI, two approaches could be considered.

The first approach to deal with such a scenario is to define a barcode with degenerate position, using Ns, as explained in the previous section, e.g. (6 bases UMI first then sample barcode :

sample1     NNNNNNATATAT  
sample2     NNNNNNGACGAC

sample1     ATATAT
sample2     GACGAC

Je USE_EMBASE run mode

 

In this running mode, simply call Je demultiplex on the lane file(s) and demultiplex will use information stored in emBASE to demultiplex and place the demultiplexed files directly in your group NGS library (an example of such a group library is shown below and decribed at here).

 

 

Structure of a Group NGS library managed by emBASE (located on the group file server)

 

 

1. No barcode file : barcodes are fetched from emBASE
2. File location is used to look up emBASE
3. User calling Je demultiplex is used for authentication and rights
4. Demultiplexed files are named according a pre-defined naming scheme and directly placed where they should => emBASE will automatically know about them
5. Demultiplexing options right at GCBridge form validation

 

 

Comparison with other demultiplexers

Single end mode : fastx barcode splitter

• Matched options for comparison
• Exactly the same results

Paired-end mode : fastq-multx barcode splitter

• Matched options for comparison
• Can only compare in the ‘standard situation’ (one sample == one barcode)
• Exactly the same results