ENCODE pilot phase

ENCODE began as a pilot project focusing on 1% of the human genome. Results from this phase of ENCODE were reported in Nature (2) and a special issue of Genome Biology in June 2007 (3).

Data from this phase are available at UCSC in designated ENCODE ‘track groups’ within the UCSC browsers for the hg16, hg17 and hg18 human genome assemblies (NCBI Builds 34–36) (4–6). The pilot section of the UCSC ENCODE web portal (http://genome.ucsc.edu/ENCODE/pilot.html) supplies information about this phase of ENCODE, and a ‘Regions’ link on this page (http://genome.ucsc.edu/ENCODE/encode.hg18.html) provides convenient access to the areas of the genome with ENCODE pilot phase annotations.

ENCODE production (scale-up) phase

In September 2007, the ENCODE project scaled up to production mode, with the goal of generating high-throughput annotations on the full human genome. In addition to the increased scale and data volume, other aspects of the project expanded in an effort to standardize results and facilitate integrative analysis. Significant differences from the pilot phase include:

• Common cell types (http://www.genome.gov/26524238) and approved cell culture protocols

• Specification of standards for experiment verification and reporting

• Capture of experiment metadata using controlled vocabularies

• New experimental technologies based on high-throughput sequencing

• A data release policy restricting use of data for nine months following release

To accommodate the increased scale and volume of ENCODE data submissions, the ENCODE project at UCSC was expanded to include a more formal data submission process with substantial automation. The browser and download sites were expanded to include new data types, the capture of additional metadata, and new track organization features (described below).

Visualizing data in the genome browser

Whole-genome ENCODE data generated during the ENCODE production phase are loaded into the standard browser track groups in the UCSC genome database (in contrast to pilot phase data, which were placed in ENCODE-specific groups). Nearly all of the ENCODE data can be found in the ‘expression’ and ‘regulation’ track groups; a few ENCODE tracks are located in the ‘mapping’, ‘genes’ and ‘variation’ groups. ENCODE tracks are highlighted in the browser track menus by an NHGRI helix logo (Figure 1). The ‘Release Log’ link at the UCSC ENCODE portal (http://genome.ucsc.edu/ENCODE/releaseLog.html) provides access to the list of released ENCODE tracks, along with links to the methods description and configuration for each track.

Open in a separate window

Figure 1.

A portion of the Genome Browser track group controls section on the hg18 human assembly, showing tracks in the ‘expression’ and ‘regulation’ track groups. The ENCODE tracks are distinguished by the NHGRI helix icon appearing in the label.

To make the hundreds of ENCODE tracks more manageable for users, we have enhanced the UCSC Genome Browser track configuration to provide more power, flexibility and interactivity. Subtracks can now be individually customized, organized into multiple ‘views’, and reordered by column sort or by drag-and-drop. We have incorporated a structured metadata display on Genome Browser track details pages and have added a link to facilitate bulk download of data files associated with a track.

Figure 2 provides a detailed look at these new features. The ‘Views’ section near the top of the track configuration page shows the potentially multiple data representations for a single experiment. Efforts have been made to standardize ‘views’ across similar datasets in ENCODE. Most tracks follow one of two patterns:

1. Regulatory elements: Peaks (discrete sites) and Signal (continuous graph of enrichment)

2. Gene expression: Plus and Minus Signal (coverage graph of reads on forward and reverse strand) and Alignments (short reads aligned to genome)

Below the ‘Views’ section, configuration pages for ENCODE tracks typically include a matrix of checkboxes that allow the selection of subtracks by experimental variables such as cell type or antibody. Subtracks can also be selected individually from the list of all subtracks displayed at the bottom of the configuration section. The column headers of this section (which include the experimental variables shown in the matrix) define the ordering of subtracks within the track display. The subtrack ordering can be changed by clicking the column headers to reorder by group, or by dragging and dropping individual subtracks in the list.

Open in a separate window

Figure 2.

Example configuration and details pages for an ENCODE track, showing important navigation and informational items.

The clickable (…) icons expand the display to show the metadata (experiment type and variables, data format and data freeze) for each subtrack. Clicking the ‘schema’ link for any subtrack listed on the track configuration page displays a full description of the data representation. The database representations and file formats for the peaks and alignments data were designed specifically for ENCODE. Signal views use one of the standard UCSC graphing formats: wiggle, bedGraph or bigWig.

Finally, note the ‘restricted until’ date for each subtrack, which shows the date when restricted use of the data expires. The data use policy for ENCODE is described in more detail below.

Bulk downloads of data

The DCC provides both raw data (sequence reads and quality scores) and processed data files (alignments, density graphs and peak calls). The raw data from high-throughput sequencing are provided in FASTQ format when feasible. SOLID colorspace sequences and quality are provided in CSFASTA and CSQUAL format.

ENCODE files can be retrieved by web access or anonymous FTP from the UCSC download server. Due to the large size of most ENCODE data sets, FTP retrieval is recommended.

The ENCODE portal includes a Downloads index page (http://genome.ucsc.edu/ENCODE/downloads.html) that provides convenient web access to data files by track. The top-level download area for ENCODE data is at http://hgdownload.cse.ucsc.edu/goldenPath/hg18/encodeDCC.

For FTP access, connect to the FTP server at ‘hgdownload.cse.ucsc.edu’, then move to the ‘goldenPath/hg18/encodeDCC’ directory. Each of the listed subdirectories contains the data files for an individual ENCODE track (one track for each data type per lab), along with an index.html page listing the data files, metadata describing the experiment, the type, experimental variables, the data format and a data restriction timestamp. An example is shown in Figure 2.

For convenient access to the ENCODE data in the Genome Browser, a Downloads link is included on the track configuration page below the subtrack selection list.

Data use policy

The following guidelines should be followed when using ENCODE data:

1. Data users may freely use ENCODE data, but may not, without prior consent, submit publications that use an unpublished ENCODE dataset until nine months following the release of the dataset (see time stamp for release date).

2. Data users should properly acknowledge the ENCODE Project and resource producer(s) as the source of the data in any publication.

3. See the full ENCODE Data Release Policy (2008–present) document (http://www.genome.gov/Pages/Research/ENCODE/ENCODEDataReleasePolicyFinal2008.pdf) for further details.

Mouse genome

The ENCODE project plans to expand to include the study of the Mus musculus genome beginning in late 2009.

Track search tool

The breadth of ENCODE data creates a challenge in terms of presentation—how to provide access to the full range of data without overwhelming the user? The extension of the existing track organization mechanisms to provide a hierarchy of data (i.e. multiview) improves on a linear listing of thousands of datasets and files. To further facilitate the dataset selection process, UCSC is planning to develop a more intuitive track search mechanism that supports the entry of keywords indicating the type of data desired.

We thank the members of the ENCODE Consortium for their collaborative spirit and stamina over the six years of data production, submission and analysis that the ENCODE project has required to date. We also acknowledge Hiram Clawson, a core Genome Browser engineer who has contributed greatly to its overall success by his work to keep the browser reliable, fast and annotation-rich. We thank the UCSC CCDS team, Mark Diekhans and Rachel Harte, for their contributions to the Gencode genes and their tireless advocacy for the best data representations and display for the challenging and high-value RNA-seq data. Nicole Washington and Lincoln Stein at the modENCODE DCC have graciously shared DCC processes and strategies. Melissa Cline provided technical review and editing for this paper, for which we thank her. And finally, we acknowledge our dedicated team of system administrators, Jorge Garcia, Erich Weiler, Victoria Lin and Alex Wolfe, for their relentless provision of more cycles and megabytes, valiant swat-team trouble-shooting and for generally providing an outstanding computing environment.