Abstract

Background

Metabolic network reconstructions are knowledge bases of key components in the metabolic reaction network of a particular organism or cell type [1,2]. Reconstructions vary in size from small-scale reconstructions where only a few pathways of interest are included, to genome-scale reconstructions where the aim is to include all known components in a network. The components of a metabolic network reconstruction are the metabolic reactions in the network, the metabolites that participate in those reactions, the enzymes that catalyze the reactions, and the genes that encode those enzymes. Individual components are linked with mathematical structures that enable computational analysis of the network as an integrated system. Computational analysis of genome-scale metabolic reconstructions has, for example, been used for biochemical [3,4], biomedical [5-7], and bioengineering [8,9] purposes.

An important step in the reconstruction of a metabolic network is the annotation of reconstruction components [2]. The identities and functions of components included in metabolic network reconstructions are generally known. Annotation here refers to the process of attaching various metadata for components to a reconstruction. These annotations serve to unambiguously identify components and enable efficient mapping of data to the reconstruction for analysis. Components are generally annotated with an appropriate type of identifier, e.g., an Entrez gene ID for genes and an Enzyme Classification (EC) number for enzymes. Metabolites are usually annotated with several types of identifiers. A comprehensive protocol for genome-scale metabolic reconstruction [2] recommended annotating metabolites with a primary identifier in at least one of the following three databases: ChEBI [10], KEGG Compound [11,12], or PubChem Compound [13]. The protocol also recommended annotating metabolites with structure-based identifiers, such as the IUPAC International Chemical Identifier (InChI) and the Simplified Molecular-Input Line-Entry System (SMILES). We add that further annotation with primary identifiers in databases that are specific to the reconstructed organism is also advisable, e.g., the Human Metabolome Database (HMDB) [14,15] for human metabolic reconstructions.

Database identifiers have the advantage of providing direct links to data that are stored in each database. Data types that can be mapped to the reconstruction via metabolite identifiers include physicochemical data, metabolomics data, metabolic pathways and metabolite structures. Different types of data are available in each database. Whereas KEGG has more data on metabolic genes, enzymes and reactions, HMDB provides more information on metabolites. Since not all chemical databases provide cross-references to all other databases, it is usually not enough to annotate metabolites with only one type of identifier. Instead they should be annotated with identifiers in as many relevant databases as possible. Multiple annotations also aid in identification since any one database may not contain all metabolites in a given reconstruction.

Advantages of structure-based identifiers are that they are unambiguous and database independent. InChIs and SMILES strings can also be converted to metabolite structures that can be used directly for various computational analyses [16-19]. Although SMILES strings have a simpler syntax and are more human readable, InChIs are preferable in many ways [20]. Firstly, they have a layered structure that makes them highly flexible and easy to manipulate. Secondly, they can account for tautomerism. Multiple tautomers of the same compound can be represented with the same standardized or “standard” InChI. Alternatively, a specific tautomer can be represented with a nonstandard InChI. A third advantage of InChIs is that the InChI algorithm in non-propriatory and is implemented in open source software. Version 1 of the InChI is currently in use.

A disadvantage of the InChI is that its length increases with molecular size and level of structural detail. Also, it includes non-alphabetical characters such as /, \, - and +. These features make the InChI inconvenient for internet and database searches [20]. A hashed version, the InChIKey, was therefore created [20]. The InChIKey has a fixed length of 27 characters, and only includes uppercase English letters and dashes. These features also make it a good choice as a database independent identifier for metabolites in metabolic reconstructions. Most chemical databases now include an InChIKey (as well as an InChI and SMILES string) in each database entry. The hashing algorithm that generates InChIKeys from InChIs is not reversible [20], meaning that there is no algorithm that can convert an InChIKey back to an InChI. InChIKeys are therefore not directly convertible to metabolite structure. To retrieve an InChI from an InChIKey it is necessary to use a lookup table or a chemical structure resolver such as the Chemical Identifier Resolver [21] or ChemSpider [22].

Genome-scale reconstructions usually include hundreds of metabolites. The latest human reconstruction, Recon 2, includes over 2500 metabolites [23]. Manual annotation of such a large number of metabolites, with multiple identifiers each, is extremely laborious. Metabolites in early reconstructions were generally annotated manually. Today, reconstruction tools such as the Model SEED [24], rBioNet [25] and the SuBliMinaL Toolbox [26] facilitate the process by populating new reconstructions with pre-annotated components from source databases. Metabolites in the source databases, however, may have incomplete or incorrect annotations, which will then be propagated to all new reconstructions. Metabolites in existing reconstructions may likewise have incomplete or incorrect annotations. Metabolite annotations in metabolic reconstructions may therefore need to be updated prior to their use. Software applications that enable automatic updating of annotations are desirable.

The SuBliMinaL Toolbox comes with an annotation module that can be used to retrieve ChEBI identifiers for metabolites by searching the ChEBI database with metabolite names. A recently published annotation tool called Metingear [27] also features a name search for database identifiers, but against multiple databases. In our experience, metabolite names are poor identifiers. Most metabolites have several synonyms and databases differ in the use of those synonyms. Searching a database with a metabolite name may therefore yield no results if the metabolite is registered under a different synonym. The same name is also often associated with multiple stereoisomers of the same compound, across different databases and within the same database. The name dextrose, for example, is associated with four entries in PubChem Compound: D-glucose (PubChem Compound ID (CID) 5793), α-D-glucose (PubChem CID 79025), β-D-glucose (PubChem CID 64689) and a generic hexopyranose (PubChem CID 206). As the last entry (PubChem CID 206) demonstrates, names are also sometimes associated with the incorrect structures. Numerous other examples of incorrect associations between names and structures are given in [28]. A name search can therefore yield a list of several candidate identifiers that must be sorted through manually to find the one that best matches the target compound. It is therefore not conducive to automatic updating of identifiers.

A name search is the best option available for updating identifiers for metabolites that are only annotated with a name. However, most metabolites in source databases for metabolic reconstruction tools are annotated with at least one identifier besides name. The same goes for metabolites in existing metabolic reconstructions, especially those that were built according to the aforementioned protocol [2]. The non-name identifiers are generally more specific than names as they refer to specific structures or database entries. Software applications that enable mapping between non-name identifiers could therefore facilitate automatic updating of metabolite identifiers in metabolic network reconstructions.

The problem of annotating large sets of metabolites is well known in metabolomics and chemical informatics. Applications that can be used to partially automate annotation have been developed for these fields. We searched among these for applications that were suitable for mapping between metabolite identifiers in metabolic reconstructions. We only considered open-source applications as these can readily be adapted to the needs of the metabolic reconstruction community and integrated into metabolic reconstruction tools. Three applications that met these criteria were MetMask [29], the Chemical Translation System (CTS) [30] and UniChem [31]. These applications implement annotation strategies that go beyond name search. They enable mapping between multiple types of identifiers, including chemical names. Different annotation strategies are implemented in each of the three applications. Here, we compare these applications, to determine which annotation strategy is best suited for annotation of metabolites in genome scale metabolic reconstructions. We then apply the top application to update annotations of metabolites in the latest human reconstruction Recon 2 [23].

Results

Identifier mapping tests

The tests described in Methods revealed a number of differences between the three mapping applications. The overall performance of each application is quantified in Table ​Table1.1. Performance on individual mapping tests is given in Additional file 1: Table S1. UniChem performed best on tests involving identifier types that it covered. UniChem generally only returned the preferred output identifier, for each input identifier that was associated with at least one output identifier in the UniChem database. Many input identifiers, however, were not associated with any output identifier. This was also the case for CTS, which indicates that it is a characteristic of InChI based mapping strategies. The fact that no output identifier of a particular type is returned for a given input identifier does not necessarily mean that the input identifier is not in the database. It only means that no identifier of the requested output type is associated with the exact same standard InChI. A single compound can have more than one valid structure, each represented with a distinct standard InChI. The distinct, equally valid structures are usually stereoisomers. If two databases associate different stereoisomers with their respective entries for the same compound, the two entries cannot be linked through InChIs. The KEGG Compound and HMDB entries for lactose are good examples of this (see Figure ​Figure1).1). Because the lactose stereoisomers in the two entries have different InChIs, neither UniChem nor CTS can map between them.

Table 1

Quantified overall performance of the three mapping applications

	Mean counts
	In	Hits	Out	Matches	Mean score
MetMask	99	98	146	93	0.63
CTS	99	80	105	75	0.57
UniChem	99	74	77	72	0.70

Open in a separate window

Scores were calculated as described in Section Scoring. Means were taken over all input-output pairs that were covered by each application. The means reflect well general characteristics of each application observed in individual tests (see Additional file 1: Table S1).

Open in a separate window

Figure 1

Lactose stereoisomers. Two epimers of lactose occur in nature, α-lactose and β-lactose. The epimers differ by the configuration of structural groups around a single stereogenic carbon atom (top right). (a) In KEGG Compound the synonyms lactose and milk sugar are assigned to a generic stereoisomer, where the configuration around this stereogenic carbon is not specified (C00243). Reactions, enzymes and pathways involving lactose are linked to this entry in KEGG. (b) The same synonyms and most lactose-related data are linked to the α-epimer in HMDB (HMDB00186). There is neither an entry for the generic stereoisomer in HMDB, nor an entry for the α-epimer in KEGG Compound.

On average, a slightly higher number of input identifiers were associated with at least one output identifier with CTS than with UniChem. CTS, however, also returned a higher number of non-preferred identifiers, and thus received a lower overall score (Table ​(Table1).1). These differences between CTS and UniChem are attributable to two factors; the greater number of identifier types covered by CTS, and the checks implemented by UniChem to prevent errors in their database (see Section Applications). The fact that CTS covers chemical names as an identifier type has a particularly large effect. Chemical names are ambiguous identifiers and the same name can be associated with a number of different structures [28]. Using names as input identifiers will therefore often result in a long list of candidate output identifiers (see Additional file 1: Table S1). Referring to the previous example, the name lactose is associated with different structures in KEGG Compound and HMDB. Inputting the name lactose into CTS will return all identifiers that are associated with either structure. The number of incorrect output identifiers returned for names is also generally higher than for other input types, because names are more frequently associated with an incorrect structure [28]. When chemical name is included as an input identifier type (Figure ​(Figure2a),2a), the number of incorrect identifiers returned by CTS is much greater. When only identifiers that are covered by UniChem are considered (Figure ​(Figure2b),2b), CTS results are more similar to those of UniChem. The remaining difference between the two applications is presumably due to the quality checks implemented by UniChem.

Open in a separate window

Figure 2

Identifiers output in identifier mapping tests. Annotations of unique identifiers returned by each application, (a) when all mapping tests are included, and (b) when only tests involving identifier types covered by UniChem are included. The output identifiers returned in all included tests were pooled and duplicates removed. If the same identifier was returned in more than one test it was only counted once. The annotations are explained in Section Scoring.

MetMask returned the greatest number of preferred identifiers on average, but it also returned the greatest number of non-preferred identifiers. It received a score in between those of UniChem and CTS (Table ​(Table1).1). MetMask returned the greatest number of output identifiers on individual mapping tests but when all tests were combined, MetMask returned fewer unique identifiers than CTS (Figure ​(Figure2).2). The reason for this is that MetMask generally returns the same set of output identifiers, when it is queried with different input identifiers for the same compound. MetMask will, for example, return the same set of ChEBI ID whether it is queried with the KEGG Compound ID (CID) for lactose or the HMDB ID. The ChEBI ID for both lactose stereoisomers in Figure ​Figure11 will be returned for either query. This is a consequence of the way metabolite masks are defined. All identifiers for lactose belong to the same mask. Querying MetMask with any identifier in a particular mask will always yield all identifiers of the requested output type that are in the same mask. Queries with different types of identifiers for the same compounds are therefore not independent. This also explains why the difference between the total number of unique identifiers returned by MetMask, when all tests are included (Figure ​(Figure2a)2a) and when only UniChem-compatiple tests are included (Figure ​(Figure2b),2b), is so much smaller than for CTS.

Taken together, these results demonstrate that regardless of the choice of mapping application, some manual effort is required for proper annotation of metabolites with database identifiers. With MetMask this effort is mainly directed at sorting through candidate output identifiers to locate the preferred ones. With UniChem it is directed at searching for missing identifiers. Some effort is required for both sorting and gap filling with CTS, although less for each than with the other two applications. In the next section we seek ways to minimize this manual effort.

Update of Recon 2 metabolite annotations

Recon 2 includes 2,626 unique metabolites. During the reconstruction of Recon 2, 1,690 metabolites were annotated with a standard InChIKey, 1,125 with a ChEBI ID, 1,040 with an HMDB ID, 396 with a KEGG CID, and 150 with a PubChem CID. All metabolites were annotated with a metabolite name. We updated metabolite identifiers in Recon 2 using additive mapping with CTS as described in the previous section. We used CTS both to review existing annotations and to add as many new ones as possible. In addition to updating identifier types that were already present in Recon 2, we added identifiers for the LIPID MAPS Structure Database (LMSD) [32] which were previously missing. To speed up the process of updating identifiers, we took advantage of the extensive metabolite annotations that are available in HMDB [14,15]. The step-by-step process is described in Additional file 1: Section 2. We added a total of 3,049 new identifiers to Recon 2, removed 124 incorrect identifiers, and replaced 569 identifiers. We therefore updated a total of 3,746 identifiers. All except 233 identifiers were updated automatically. These 233 identifiers were selected manually from a list of 2,790 candidates that were returned by CTS. Manually sorting through the list of candidates required approximately ten man-hours. A random sample of 100 automatically updated identifiers were also checked manually. All 100 were found to be correct. The full list of updated metabolite annotations in Recon 2 is included in the Additional file 2.

The majority (1,962/2,660) of added identifiers were KEGG and PubChem CID (Figure ​(Figure3a),3a), which were previously lacking in Recon 2. We also added LMSD ID for 389 metabolites. Around half (65/124) of all incorrect identifiers were PubChem CID. The remaining half was evenly distributed among ChEBI ID, HMDB ID and KEGG CID. Incorrect identifiers were identified and removed automatically (see Additional file 1: Section 2). The majority of replaced identifiers (464/569) were ChEBI ID. In most cases, we replaced a ChEBI ID for a charged metabolite, with a ChEBI ID for the same metabolite in its neutral state. ChEBI and PubChem Compound often include separate entries for metabolites in neutral and various charged states, but HMDB, KEGG Compound and LMSD usually only include metabolites in their neutral state. For the sake of mapping and other comparisons between databases, it is therefore preferable to include identifiers for metabolites in their neutral states in metabolic reconstructions. If metabolite charge is required, it can be predicted with software tools, such as ChemAxon’s Calculator Plugins (ChemAxon Kft., Budapest, Hungary).

Open in a separate window

Figure 3

Recon 2 identifiers. Identifier statistics for Recon 2 before and after metabolite annotations were updated with CTS. (a) Number of unique metabolites with each of the seven types of identifiers. n: names, i: InChIKeys, c: ChEBI ID, h: HMDB ID, k: KEGG CID, p: PubChem CID, l: LipidMAPS ID. (b) Number of unique metabolites with one, and up to seven, identifiers each.

Most of the added identifiers (2,991/3,049) were for metabolites that were already annotated with at least one identifier besides a chemical name in Recon 2 (Figure ​(Figure3b).3b). Non-name identifiers were added for 28 metabolites that were previously only annotated with a name. That leaves 594 Recon 2 metabolites with name as the only annotation. There are two possible reasons for this; either these metabolites are not included in the CTS database, or the synonyms used for them in Recon 2 is not included. The CTS name search is currently only capable of matching names exactly, so even slight differences between a metabolite name in Recon 2 and the synonyms listed for that metabolite in CTS would prevent finding a match. A majority (432/594) of the metabolites for which no identifier was found consisted of macromolecules and metabolites with variable structures (i.e., R groups), such as polysaccharides and proteins. Such metabolites are seldom included in the chemical databases considered here, so it is not surprising that no identifiers were found. In addition, metabolites with variable structures cannot be represented with an InChI. An InChI based application such as CTS therefore cannot cover them. The remaining metabolites (162/594) are more likely to be included in the databases considered here. If they are, they must be registered under different synonyms than the names included in Recon 2. Non-name identifiers for these metabolites will need to be searched for manually.

Discussion

The three applications compared in this work implement two different metabolite annotation strategies. MetMask implements what can be termed a “metabolite masking” strategy (see Section MetMask), whereas CTS and UniChem implement two different versions of an InChI based strategy (see Sections The Chemical Translation System and UniChem). We found the InChI based strategy to be better suited for annotation of metabolites in genome-scale metabolic reconstructions. The main advantage of this strategy over the metabolite masking strategy is that multiple types of information about a metabolite can be used to perform independent searches for missing annotations. The independent search results can then be combined to increase both coverage and specificity. Candidate annotations are thereby found for a greater number of metabolites (increased coverage), while candidate annotations for each metabolite are fewer and usually include the preferred annotations (increased specificity). Candidate annotations, which are returned by multiple independent searches, can additionally be assigned in an automatic manner with more confidence. Neither the CTS nor UniChem user interfaces currently offer the possibility of combining multiple types of information to search for missing annotations. Here we performed each search separately and combined results afterwards. This process was slow and required a considerable amount of programming. An InChI based application that allows simultaneous input of multiple types of metabolite information would greatly simplify and accelerate annotation.

When multiple independent search results were combined, the InChI based strategy implemented in CTS gave qualitatively better results than the version implemented in UniChem. Although UniChem gave more specific results, CTS covered a greater number of metabolites. The main advantage of CTS over UniChem is that it can map between a greater number of identifier types. We chose to map between a limited number of identifier types, that are relevant for the human metabolic reconstruction Recon 2, but the same strategy could be used to map between any of the 215 types of identifiers covered by CTS.

The greater metabolite coverage of CTS was mostly due to the fact that CTS allows chemical names as inputs. This was also the main reason for the lower specificity of CTS results. Chemical names are rather generic metabolite identifiers, or at least they are used rather generically in chemical databases. The same name is often associated with multiple different structures, sometimes incorrectly [28]. When names are input to an InChI based mapping application, identifiers for different but equally valid structures may be returned leading to increased coverage. However, identifiers for nonpreferred, invalid or even incorrect structures may also be returned leading to reduced specificity. Inputting the name lactose to CTS, for example, will return both a KEGG CID and an HMDB ID, for different but equally valid lactose stereoisomers (see Figure ​Figure1).1). However, it will also return a total of four PubChem CID, one of which is invalid as it refers to a generic disaccharide (PubChem CID 294). To retain the coverage obtained with chemical names as inputs, while minimizing the adverse effects it has on specificity, we introduced a confidence score that gave annotations returned for names a lower priority. A similar mechanism could be built into the metabolite annotation application suggested above, where multiple types of metabolite information could be input simultaneously as search criteria.

Although the fact that CTS allows names as inputs explains most of the difference between the specificity of CTS and UniChem, it does not explain all of it (see Figure ​Figure22 and Additional file 1: Table S1). Some of this difference is also due to the quality checks performed by UniChem before data from external databases is imported into the UniChem database (see Section UniChem). Any InChI based application would benefit from similar quality checks. A recent study [33] showed that different structural representations (Molfiles, InChIs, SMILES) within the same database entry often do not represent the same structure. Such mismatches are indicative of errors in database entries. Quality checks such as the ones implemented in UniChem hinder such errors from being propagated to local databases for annotation applications. Additional quality checks could also be performed, such as checking whether the two dimensional structure (e.g., in Molfile format) and the chemical formula in an external database entry match the standard InChI. Chemical formulas can also be used to check candidate annotations returned for metabolites and to weed out incorrect ones. All metabolites in metabolic network reconstructions should be annotated with their chemical formulas. We used metabolite formulas in Recon 2 to review database identifiers that were added to the reconstruction (see Additional file 1: Section 2). If the metabolite formula in the database entry associated with an identifier did not match the formula in Recon 2, we assumed the identifier was incorrect and rejected it. Differences in numbers of hydrogen atoms between formulas were ignored. The metabolite annotation application suggested above could include metabolite formulas as one type of input information about metabolites. Candidate identifiers associated with different formulas than the input formula would then be rejected before they were added to the reconstruction.

The coverage of any InChI based application is limited to metabolites with defined structures that can be represented with InChIs. Metabolic reconstructions often include generic metabolites with undefined structural elements such as R groups. These generic metabolites represent whole classes of structurally similar metabolites that undergo the same metabolic transformations in vivo. They are introduced into reconstructions for simplification. Such generic metabolites cannot be represented with an InChI and therefore cannot be covered by an InChI based metabolite annotation application. An annotation application based on a metabolite masking strategy would be better suited to mapping between identifiers for such metabolites. The fact that an InChI based application cannot cover generic metabolites of this type does not decrease its value much, since these metabolites are generally a minority of all metabolites in metabolic reconstructions and only a minority of those is expected to be included in chemical databases of interest.

Generic metabolites of a different type that are also found in metabolic reconstructions are generic stereoisomers, i.e., metabolites with undefined stereochemistry at one or more stereocenters. An example found in Recon 2 is lactose in Figure ​Figure1a.1a. As enzymes that catalyze metabolic reactions involving lactose generally don’t have known stereospecificity, both the α- and β-epimers are assumed to participate in the same reactions. Instead of needlessly complicating the reconstruction by writing the same reactions twice, the two epimers are collapsed into a single generic stereoisomer. The metabolic component lactose in Recon 2 therefore encompasses both α- and β-lactose. Similar examples are frequent in fatty acid metabolism where cis-trans isomerism is not specified unless necessary, i.e., for fatty acids that participate in reactions that are catalyzed by enzymes with known stereospecificity.

Generic stereoisomers can be represented with an InChI and can therefore be covered by an InChI based application. CTS often returned several candidate identifiers for such metabolites that needed to be sorted through manually to select the preferred ones. One reason for this is that the names of generic stereoisomers in Recon 2 are often associated with more specific or even more generic (and thus invalid) stereoisomers in the databases considered here. As discussed above, the name lactose is associated with α-lactose (more specific) in HMDB and a disaccharide with no specified stereochemistry (more generic) in PubChem among other database identifiers. Another reason for why CTS often returned several candidate identifiers for generic stereoisomers was that preexisting annotations of these metabolites in Recon 2 were sometimes rather ambiguous. Lactose in Recon 2, for example, was annotated with the KEGG CID for the generic stereoisomer (Figure ​(Figure1a)1a) and the HMDB ID for the α-epimer (Figure ​(Figure1b).1b). When these two identifiers were used in combination to find the PubChem CID for lactose, CTS naturally returned PubChem CID for both stereoisomers. This raises the question of how generic stereoisomers should be annotated in metabolic reconstructions. The general rule should be to annotate each metabolite with the most generic identifier of each type that is still valid. Lactose therefore should be annotated with identifiers for the generic stereoisomer in Figure ​Figure1a.1a. However, there is no entry for this lactose stereoisomer in HMDB. Instead, HMDB includes separate entries for the more specific α- and β-epimers of lactose. In such cases, the general rule in the past appears to have been to select the identifier for the most prevalent specific stereoisomer, e.g., the HMDB ID for α-lactose, as relevant biochemical data is more likely to be associated with that identifier. The advantage of annotating generic stereoisomers with identifiers for more specific ones is precisely that they provide links to such data. The disadvantage is that the identity of metabolic components becomes somewhat ambiguous. It may, for example, not be obvious to all users of Recon 2 whether the metabolic component lactose represents the generic stereoisomer or only α-lactose since it is annotated with identifiers for both.

Reconstruction of the metabolic network of an organism or cell type is an iterative process. Recon 2 is the latest iteration of the human metabolic network reconstruction. While Recon 2 is much more comprehensive than its predecessor Recon 1 [34], it probably does not capture the entire human metabolic network and further iterations are expected in the future [23]. Our results suggest some guidelines for researchers to keep in mind when annotating new metabolites that are added to metabolic reconstructions, human or otherwise. Firstly, each metabolite should be annotated with at least one identifier besides name if that is possible. Secondly, each metabolite should generally be annotated with identifiers for its neutral form. Exceptions exist for metabolites that only participate in metabolic reactions in a particular charged state, e.g. inorganic ions such as Cl ⁻ and Mg ²⁺. Thirdly, each metabolite should preferably be annotated with the most generic identifier of each type that is still valid. This also applies to metabolite names. The name D-glucose, for example, should not be used for a metabolic component that is meant to represent α-D-glucose or vice versa. If no generic identifier of a particular type is available for a metabolite, a more specific identifier may be used. Researchers should, however, be aware that doing so makes the identity of that metabolite somewhat ambiguous. Best practices would be to include a note that specifies the relationship of an identifier to a metabolic component. The ChEBI ontology could serve as a guideline for how relationships between metabolites should be specified. So, for example, it would be noted that the metabolite with HMDB ID HMDB00186 (Figure ​(Figure1b)1b) is a Lactose.

An InChI based metabolite annotation application has the potential to enable fully automatic mapping between identifiers for metabolites with defined structures. Fully automatic mapping, however, would require that both databases and reconstructions were free of errors and ambiguity. As several authors have demonstrated [28,33,35], errors and ambiguities are quite common in publicly available chemical databases. If a database identifier is associated with an incorrect InChI in a database it will not be mapped to the correct metabolite in a reconstruction. As we demonstrated here, erroneous metabolite annotations are also found in metabolic reconstructions. In particular, we removed 124 incorrect identifiers from Recon 2. As discussed above metabolite annotations in metabolic reconstructions can also be ambiguous, e.g., when a metabolite is annotated with the name of one stereoisomer but the KEGG CID of another. If a metabolite is annotated with an incorrect identifier in a metabolic reconstruction it may not be mapped to the correct database or structure based identifier. Fully automatic mapping between metabolite identifiers will not be possible until such errors and ambiguities are resolved. An InChI based application such as CTS, however, with the modifications suggested above, can significantly reduce the manual effort required for mapping between metabolite identifiers.

Experimental

CTS was accessed by calling the “Convert” web service as described at http://cts.fiehnlab.ucdavis.edu/moreServices/index. UniChem was accessed by calling the web service method "Get src_compound_ids from src_compound_id" as described at https://www.ebi.ac.uk/unichem/info/webservices. Web services were called from MATLAB (version R2009b, MathWorks, Natick, MA) using the built-in function urlread. MetMask (version 0.5.3) was installed on a desktop computer running Windows 7. MetMask databases were created by importing identifier groups from Recon 2, HMDB and ChEBI. Database queries were formulated as described at http://metmask.sourceforge.net/manual.htmlwith all options set to their default values. Output from all mapping applications was parsed in MATLAB using the built-in function regexp.

Conclusions

We found that an application implementing an InChI based strategy could facilitate automatic mapping between metabolite identifiers in metabolic network reconstructions. Of the two InChI based applications evaluated here we found CTS to be qualitatively better. The main advantage of CTS is the large number of identifier types it can map between. As CTS is open source it can be adapted to the task of mapping between metabolite identifiers in metabolic network reconstructions with relative ease. We suggest several features that could be added to CTS to optimize its performance on this task. In particular, we suggest combining multiple types of information about metabolites to find new identifiers. A confidence score can be used to account for the fact that some types of input information, in particular metabolite names, are less reliable than others. We further suggest implementing various quality checks, similar to those implemented in UniChem, to limit the number of incorrect identifiers returned for a metabolite. When simple versions of some of the suggested features were implemented, CTS allowed us to update more than 3,500 metabolite identifiers in Recon 2. Most were updated automatically. Based on this experience, we suggest some guidelines for future annotation of metabolites in metabolic network reconstructions. We hope that the updated Recon 2 identifiers will facilitate application of Recon 2 in the future. More generally, we hope that our results will guide developers of reconstruction tools in implementing strategies for automatically updating metabolite identifiers in metabolic network reconstructions.

Methods

Scoring

To quantify the relative performance of the three mapping applications, we devised a simple scoring system. For each input-output pair, each mapping application returned a list of candidate output identifiers associated with the set of input identifiers. The number of output identifiers associated with a single input identifier ranged from zero to several. We annotated each returned output identifier as preferred, valid, invalid, incorrect or nonexistent (see Figure ​Figure44 for an example). Preferred identifiers point to the preferred stereoisomer of each compound, which is generally the same as the input stereoisomer. There is exactly one preferred output identifier for each input identifier. Valid identifiers point to valid but not preferred stereoisomers, invalid identifiers point to invalid stereoisomers or mixtures, incorrect identifiers point to different compounds, and nonexistent identifiers do not point to anything. Once all output identifiers had been annotated in this manner, we calculated a score based on the number of input identifiers (In), the number of input identifiers for which at least one output identifier was returned (Hits), the total number of returned output identifiers (Out), and the number of preferred output identifiers (Matches). The score was calculated as

Open in a separate window

Figure 4

Annotation of output identifiers. An example demonstrating annotation of output PubChem Compound identifiers (b-e), when the KEGG Compound identifier for D-glucose (a) is input to a mapping application. The preferred output identifier is for D-glucose (b), but an identifier for alpha-D-glucose (c) is also valid since it is a D-glucose. An identifier for a generic hexose (d), however, is not valid. Finally, an identifier for phospholactic acid (e), which is a completely different compound, is incorrect.

$$\text{Score}=\frac{\text{Hits}\times \text{Matches}}{\text{In}\times \text{Out}}.$$

(1)

This score can range from 0 to 1. An application receives the maximum score if it returns the preferred output identifier, and no other, for each input identifier. It receives a lower score if it returns non-preferred identifiers, or none at all, for a subset of input identifiers, since some manual effort is then required to sort through results and fill gaps. Note that the number of input identifiers (In) varies between identifier types, because two of the 100 test metabolites were not found in ChEBI (In=98) and three were not found in KEGG Compound (In=97).

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

All authors conveived and designed performance tests. HSH performed the tests and analyzed results. HSH updated metabolite annotations in Recon 2. All authors wrote the manuscript. All authors read and approved the final manuscript.

Supplementary Material

References