Accessing component databases of VEuPathDB
Rohit Satyam, Alberto Maillo, David Gomez-Cabrero, Arnab Pain
King Abdullah University of Science & Technology, Saudi Arabia22 August, 2026
Source:vignettes/Gene_ID_Conversion.Rmd
Gene_ID_Conversion.RmdAbstract
This section covers the capability of plasmoRUtils
to access various VEuPathDb database and carry out ID conversion
tasks, fetch preconfigured data tables and few other tasks.
Introduction
The plasmoRUtils package streamlines access to VEuPathDB’s family of 12 specialized databases such as ToxoDB, PlasmoDB, and PiroplasmaDB (Amos et al. 2022). It provides direct data retrieval capabilities through VEuPathDB’s RESTful API, enabling seamless integration of biological data into R workflows. The package supports downloading both standard and customized data tables, making it particularly valuable for researchers needing to combine data from multiple sources for downstream analysis.
With release 71 of VEuPathDB, only users with subscription and API
key can access the database programmatically. Use
usethis::edit_r_environ(scope = "user") to add VEuPathDB
API using VEUPATHDB_API_KEY variable. By default, all the
functions look for VEUPATHDB_API_KEY variable for API key.
Should you store your key using any other variable, supply it using
api_key argument in VEuPathDB asscoiated functions.
# Load package and some other useful packages by using
suppressPackageStartupMessages(
suppressWarnings({
library(plasmoRUtils)
library(dplyr)
library(plyr)}))Gene ID Conversion
A common challenge in bioinformatics involves mapping between
different identifier systems across databases. For apicomplexan
research, this might include converting between UniProt IDs, legacy gene
identifiers, and current Ensembl gene IDs. The toGeneid()
function addresses this need by retrieving up-to-date annotations from
VEuPathDB databases, supporting bidirectional conversion between various
ID types through flexible parameter specification.
Retrieving Gene Annotations and Alternative IDs
The toGeneid() function enables annotation retrieval
when provided with Ensembl gene IDs. By default, it returns essential
information including gene names, symbols, outdated gene IDs, and
protein Uniprot IDs. The function’s versatility extends to supporting
custom field requests through its customFields parameter,
with available options documented in the getTable() help
section.
## Get annotations for list of geneIDs for PF3D7
toGeneid(c("PF3D7_0420300", "PF3D7_0621000"), from="ensembl")
#> # A tibble: 2 × 10
#> `Gene ID` `Product Description` `Gene Strand` `Gene Name or Symbol`
#> <chr> <chr> <chr> <chr>
#> 1 PF3D7_0420300 AP2 domain transcription fa… forward ApiAP2
#> 2 PF3D7_0621000 RNA polymerase subunit sigm… forward ApSigma
#> # ℹ 6 more variables: `Previous ID(s)` <chr>, `Entrez Gene ID` <chr>,
#> # `UniProt ID(s)` <chr>, `Protein Length` <chr>, `# TM Domains` <chr>,
#> # `SignalP 6.0 Signal Probability` <chr>
## Get annotations for list of geneIDs for organisms other than PF3D7
toGeneid(inputid = c("TGME49_304740","TGME49_208030"),from="ensembl",org="Toxoplasma gondii ME49", db="toxodb")
#> # A tibble: 2 × 10
#> `Gene ID` `Product Description` `Gene Strand` `Gene Name or Symbol`
#> <chr> <chr> <chr> <chr>
#> 1 TGME49_208030 microneme protein MIC4 forward MIC4
#> 2 TGME49_304740 rhoptry kinase family prote… reverse ROP35
#> # ℹ 6 more variables: `Previous ID(s)` <chr>, `Entrez Gene ID` <chr>,
#> # `UniProt ID(s)` <chr>, `Protein Length` <chr>, `# TM Domains` <chr>,
#> # `SignalP 6.0 Signal Probability` <chr>
## Convert uniprot IDs back to gene IDs. It will also provide Product description and Gene Symbol
toGeneid(inputid = c("Q8I1N6","C6KT48"),from="uniprot",to="ensembl" )
#> # A tibble: 2 × 2
#> `Gene ID` `UniProt ID(s)`
#> <chr> <chr>
#> 1 PF3D7_0420300 Q8I1N6
#> 2 PF3D7_0621000 C6KT48
## Using customFields to get only columns of interest
toGeneid(inputid = c("TGME49_304740","TGME49_208030"),
from="ensembl",org="Toxoplasma gondii ME49",
db="toxodb",
customFields=c("primary_key","predicted_go_component","annotated_go_function"))
#> # A tibble: 2 × 3
#> `Gene ID` `Computed GO Components` `Curated GO Functions`
#> <chr> <chr> <chr>
#> 1 TGME49_208030 extracellular region N/A
#> 2 TGME49_304740 N/A N/ANote: Successful ID conversion requires precise organism nomenclature matching VEuPathDB’s conventions. For example,
Toxoplasma gondii ME49must include proper spacing and special characters. Invalid query would beToxoplasma gondiiME49or short formsTgME49.
This functionality proves particularly valuable for enhancing differential expression analysis results with comprehensive annotations, enabling complete workflow automation without leaving command-line interfaces on HPC systems.
With toGeneid() function, one-to-many relationships are
retained in long format by default, allowing all possible mappings to
remain visible. The decision about how to handle such cases is therefore
left to the user, rather than being imposed by the function.
For instance, the two protein IDs Q8I0P6 and
A0A2P1JI60 correspond to
elongation factor 1-alpha. Although the genes
PF3D7_1357100 and PF3D7_1357000 encode the
same protein (thereby same uniprot ID Q8I0P6), they are
located on opposite strands: PF3D7_1357100 is on the
forward strand, whereas PF3D7_1357000 is on the reverse
strand. They also differ in transcript length. In this example if the
task is Uniprot to Gene ID mapping, users can group the results returned
by toGeneid() function by Uniprot IDs and collapse rows
thereby concatinating two gene IDs separated by semicolon or comma. An
alternative is to randomly choose one lexicographically. Because there
is no universally appropriate rule for resolving this type of ambiguity,
we avoid making this choice for the end user.
Accessing Preconfigured Data Tables from VEuPathDB’s component sites
Since VEuPathDB API documentation specifically encourages to use
specific organism database as quoted below, we developed
getTable() function to fetch fields of interests from each
database separately.
There are 12 component sites and one portal: VEuPathDB.org. The component sites are: AmoebaDB, CryptoDB, FungiDB, GiardiaDB, HostDB, MicrosporidiaDB, PiroplasmaDB, PlasmoDB, ToxoDB, TrichDB, TriTrypDB and VectorBase. For most record types (all but dataset and organism), when running a search, the portal reaches out to component sites to get the search results. That means it will be faster to use a component site directly when you can.
Some frequently required fields have been provided in the help
section of getTable().
## To fetch table for all the genes present in an organism
getTable(org="Plasmodium falciparum 3D7", db="plasmodb",
api_key = Sys.getenv("VEUPATHDB_API_KEY")) %>% head()
#> # A tibble: 6 × 10
#> `Gene ID` `Product Description` `Gene Strand` `Gene Name or Symbol`
#> <chr> <chr> <chr> <chr>
#> 1 PF3D7_0100100 erythrocyte membrane protei… forward VAR
#> 2 PF3D7_0100200 rifin reverse RIF
#> 3 PF3D7_0100300 erythrocyte membrane protei… reverse VAR
#> 4 PF3D7_0100400 rifin forward RIF
#> 5 PF3D7_0100500 erythrocyte membrane protei… reverse N/A
#> 6 PF3D7_0100600 rifin reverse RIF
#> # ℹ 6 more variables: `Previous ID(s)` <chr>, `Entrez Gene ID` <chr>,
#> # `UniProt ID(s)` <chr>, `Protein Length` <chr>, `# TM Domains` <chr>,
#> # `SignalP 6.0 Signal Probability` <chr>
## User can also provide custom fields. For example we wish to download the Transcript abundance in Transcripts per Million (TPM) [Dataset: Transcriptome in severe vs uncomplicated malaria]
getTable(org="Plasmodium falciparum 3D7", db="plasmodb", customFields = c("primary_key","pan_24905_ns_2917")) %>% head()
#> # A tibble: 6 × 2
#> `Gene ID` `unstranded - SFM-3_severe`
#> <chr> <dbl>
#> 1 PF3D7_0100100 0.16
#> 2 PF3D7_0100200 0
#> 3 PF3D7_0100300 0.13
#> 4 PF3D7_0100400 0
#> 5 PF3D7_0100500 0
#> 6 PF3D7_0100600 0
## The files to be supplied to customFields arguments can be found using
attrs <- getTableAttributes(
db = "plasmodb",
record_type = "transcript", api_key=Sys.getenv("VEUPATHDB_API_KEY")
)
head(attrs)
#> name displayName columnDataType isDisplayable isSortable
#> 1 primary_key Gene ID STRING TRUE TRUE
#> 2 overview Overview STRING TRUE TRUE
#> 3 gbrowseLink gbrowseLink STRING TRUE TRUE
#> 4 JBrowse JBrowse OTHER TRUE FALSE
#> 5 gene_source_id gene_source_id STRING FALSE TRUE
#> 6 source_id source_id STRING FALSE TRUE
#> isInReport help
#> 1 TRUE <NA>
#> 2 FALSE <NA>
#> 3 FALSE <NA>
#> 4 FALSE View this Gene in the Genome Browser
#> 5 TRUE <NA>
#> 6 TRUE <NA>For more information on fields that can be supplied to
getTable(), use the following steps:
Go to database of your interest (Say “PlasmoDB”)
Click on
Annotation, curation and identifierstab on your left and selectList of IDsScroll down and click on
Build a Web Services URL from this Search >>hyperlink.Under the section
Choose Columns:choose fields of your interest. Most of these fields are included in help section ofgetTable(). However, fields specific to a particular database such as dataset related fields (starts with “pan_”) are excluded.Once you select fields, they are updated in POST section of the webpage query builder.
The constituent databases of VEuPathDB also provide some
preconfigured tables which can not be fetched via
getTable() function. To enable users to fetch such tables,
we wrote another function called getPreconfiguredTable()
the usage of which has been shown below.
## To get list of preconfigured tables
preconf <- getPreconfiguredTable(listtables = TRUE)
head(preconf)
#> [1] "3dPreds"
#> [2] "afumA1163_TF_KO_Image_NAFeaturePhenotypeImage_RSRC_PhenotypeImages"
#> [3] "afumAf293_TF_KO_Image_NAFeaturePhenotypeImage_RSRC_PhenotypeImages"
#> [4] "Alias"
#> [5] "AllProducts"
#> [6] "AlphaFoldLinkouts"
## Fetch pathway table for all the genes from MPMP database
getPreconfiguredTable(org = "Plasmodium falciparum 3D7",db = "plasmodb",customField = "MetabolicPathwaysMPMP") %>% head()
#> # A tibble: 6 × 4
#> `Gene ID` pathway_id Pathway Activity
#> <chr> <chr> <chr> <chr>
#> 1 PF3D7_0100100 cytoadherencescheme Interactions between modified host… erythro…
#> 2 PF3D7_0100100 exportome Exportome - compiled from various … erythro…
#> 3 PF3D7_0100100 gene_lumef Gene expression affected by lumefa… erythro…
#> 4 PF3D7_0100100 lys_met Peptides with confirmed methylated… erythro…
#> 5 PF3D7_0100100 Par_RBC Protein-Protein Interactions betwe… erythro…
#> 6 PF3D7_0100100 PfEMP1 PfEMP1 domain architectures erythro…Please note that the MPMP pathway version provided by PlasmoDB is outdated
(03-2019). Some pathways have been revised or removed in
its entirety. If you wish to access the latest MPMP version, you can use
data("mpmp.28Aug2024") for you analysis which was scraped
by us. If you wish to use this geneset for MPMP pathway enrichment
analysis using pathfindR,
you can do so by using data("pathfindrMPMP").
Note: We urge the users to cite the original articles of the related datasets alongside plasmoRUtils.
Fetching genome metadata and strain names
In above examples, we saw the importance of passing exact name to the
org argument for toGeneid() to function
properly. A helper function is provided to achieve this called
listVeupathdb(). By default, 11 columns are returned
including organism name, the respective database present in “VEuPathDB
Project” column and some more additional information. However, you can
limit the search to columns of your interests as shown below. As stated
by VEuPathDB:
The best use of the VEuPathDB portal is to get a table with all organisms in our sites, and for each organism: the component site, and the urls to access their fasta and gff files.
listVeupathdb() %>% head()
#> # A tibble: 6 × 11
#> Organism Species Genome Fasta Downloa…¹ CDS Fasta Download L…²
#> <chr> <chr> <chr> <chr>
#> 1 Acanthamoeba astronyxis… Acanth… http://AmoebaDB.org/a… http://AmoebaDB.org/a…
#> 2 Acanthamoeba castellani… Acanth… http://AmoebaDB.org/a… http://AmoebaDB.org/a…
#> 3 Acanthamoeba castellani… Acanth… http://AmoebaDB.org/a… http://AmoebaDB.org/a…
#> 4 Acanthamoeba castellani… Acanth… http://AmoebaDB.org/a… http://AmoebaDB.org/a…
#> 5 Acanthamoeba castellani… Acanth… http://AmoebaDB.org/a… http://AmoebaDB.org/a…
#> 6 Acanthamoeba culbertson… Acanth… http://AmoebaDB.org/a… http://AmoebaDB.org/a…
#> # ℹ abbreviated names: ¹`Genome Fasta Download Link`,
#> # ²`CDS Fasta Download Link`
#> # ℹ 7 more variables: `Transcript Fasta Download Link` <chr>,
#> # `Protein Fasta Download Link` <chr>, project_id <chr>, Genes <chr>,
#> # `GFF Download Link` <chr>, `Genome Source` <chr>,
#> # `Structural Annotation Source` <chr>
listVeupathdb(customFields=c("species", "project_id")) %>% head()
#> # A tibble: 6 × 3
#> Organism Species project_id
#> <chr> <chr> <chr>
#> 1 Acanthamoeba astronyxis Unknown Acanthamoeba astronyxis AmoebaDB
#> 2 Acanthamoeba castellanii C3 Acanthamoeba castellanii AmoebaDB
#> 3 Acanthamoeba castellanii Ma Acanthamoeba castellanii AmoebaDB
#> 4 Acanthamoeba castellanii str. Neff Acanthamoeba castellanii AmoebaDB
#> 5 Acanthamoeba castellanii str. Neff 2021 Acanthamoeba castellanii AmoebaDB
#> 6 Acanthamoeba culbertsoni A1 Acanthamoeba culbertsoni AmoebaDBMapping PDB IDs to Gene IDs
There is currently no facility in VEuPathDB to convert the PDB IDs to
respective gene IDs. If PDB ID corresponds to a multimer complex and if
you have multiple such PDB ids, it becomes arduous to map them to Gene
IDs manually. To provide solution to this issue, we can first convert
the PDB chains to Uniprot IDs using our own pdb2uniprot()
function and then can use toGeneid() function to obtain
gene IDs.
pdbids <- c("7D2W","4U5A","6E10")
df <- lapply(pdbids, pdb2uniprot) %>% plyr::ldply()
geneids <- toGeneid(inputid = unique(df$attribute),from = "uniprot",to = "ensembl")
## Combining the geneIDs with df
S4Vectors::merge(df,geneids,all=TRUE, by.x="attribute", by.y="UniProt ID(s)") %>% head()
#> attribute entity_id chain_id struct_asym_id unp_start unp_end
#> 1 A0A143ZZR8 1 A A 27 206
#> 2 A0A143ZZR8 1 B B 27 206
#> 3 Q75UY1 1 C C 42 241
#> 4 Q75UY1 1 D D 42 241
#> 5 Q75UY1 1 A A 42 241
#> 6 Q75UY1 1 B B 42 241
#> start.author_residue_number start.author_insertion_code start.residue_number
#> 1 NA 3
#> 2 NA 3
#> 3 NA 2
#> 4 NA 2
#> 5 NA 2
#> 6 NA 2
#> end.author_residue_number end.author_insertion_code end.residue_number
#> 1 181 182
#> 2 181 182
#> 3 NA 201
#> 4 NA 201
#> 5 NA 201
#> 6 NA 201
#> identity coverage query Gene ID
#> 1 1 0.994 7D2W PF3D7_1372300
#> 2 1 0.994 7D2W PF3D7_1372300
#> 3 1 0.901 4U5A <NA>
#> 4 1 0.901 4U5A <NA>
#> 5 1 0.901 4U5A <NA>
#> 6 1 0.901 4U5A <NA>Session
sessionInfo()
#> R version 4.4.1 (2024-06-14 ucrt)
#> Platform: x86_64-w64-mingw32/x64
#> Running under: Windows 11 x64 (build 26200)
#>
#> Matrix products: default
#>
#>
#> locale:
#> [1] LC_COLLATE=English_India.utf8 LC_CTYPE=English_India.utf8
#> [3] LC_MONETARY=English_India.utf8 LC_NUMERIC=C
#> [5] LC_TIME=English_India.utf8
#>
#> time zone: Africa/Khartoum
#> tzcode source: internal
#>
#> attached base packages:
#> [1] stats graphics grDevices utils datasets methods base
#>
#> other attached packages:
#> [1] plyr_1.8.9 dplyr_1.2.1 plasmoRUtils_1.1.2 rlang_1.3.0
#> [5] readr_2.2.0 janitor_2.2.1 BiocStyle_2.32.1
#>
#> loaded via a namespace (and not attached):
#> [1] segmented_2.2-1 fs_2.1.0
#> [3] ProtGenerics_1.36.0 matrixStats_1.5.0
#> [5] bitops_1.1-0 lubridate_1.9.5
#> [7] pRoloc_1.44.1 httr_1.4.8
#> [9] RColorBrewer_1.1-3 doParallel_1.0.17
#> [11] tools_4.4.1 MSnbase_2.30.1
#> [13] utf8_1.2.6 R6_2.6.1
#> [15] lazyeval_0.2.3 withr_3.0.3
#> [17] prettyunits_1.2.0 gridExtra_2.3.1
#> [19] preprocessCore_1.66.0 cli_3.6.6
#> [21] Biobase_2.64.0 textshaping_1.0.5
#> [23] gt_1.3.0 sass_0.4.10
#> [25] topGO_2.56.0 mvtnorm_1.4-2
#> [27] S7_0.2.2 randomForest_4.7-1.2
#> [29] proxy_0.4-29 pkgdown_2.2.1
#> [31] Rsamtools_2.20.0 systemfonts_1.3.2
#> [33] txdbmaker_1.0.1 AnnotationForge_1.46.0
#> [35] dichromat_2.0-1 parallelly_1.48.0
#> [37] limma_3.60.6 rstudioapi_0.19.0
#> [39] impute_1.78.0 RSQLite_3.53.3
#> [41] FNN_1.1.4.1 generics_0.1.4
#> [43] BiocIO_1.14.0 vroom_1.7.1
#> [45] gtools_3.9.5 dendextend_1.19.1
#> [47] GO.db_3.19.1 Matrix_1.7-5
#> [49] MALDIquant_1.22.3 drawProteins_1.24.0
#> [51] S4Vectors_0.42.1 abind_1.4-8
#> [53] lifecycle_1.0.5 yaml_2.3.12
#> [55] snakecase_0.11.1 SummarizedExperiment_1.34.0
#> [57] recipes_1.3.3 SparseArray_1.4.8
#> [59] BiocFileCache_2.12.0 grid_4.4.1
#> [61] blob_1.3.0 promises_1.5.0
#> [63] crayon_1.5.3 PSMatch_1.8.0
#> [65] lattice_0.22-9 beachmat_2.20.0
#> [67] GenomicFeatures_1.56.0 annotate_1.82.0
#> [69] chromote_0.5.1 mzR_2.38.0
#> [71] KEGGREST_1.44.1 pillar_1.11.1
#> [73] knitr_1.51 GenomicRanges_1.56.2
#> [75] rjson_0.2.23 lpSolve_5.6.23
#> [77] future.apply_1.20.2 codetools_0.2-20
#> [79] mgsub_2.0.0 glue_1.8.1
#> [81] pcaMethods_1.96.0 data.table_1.18.4
#> [83] MultiAssayExperiment_1.30.3 vctrs_0.7.3
#> [85] png_0.1-9 gtable_0.3.6
#> [87] kernlab_0.9-33 cachem_1.1.0
#> [89] gower_1.0.2 xfun_0.60
#> [91] prodlim_2026.03.11 S4Arrays_1.4.1
#> [93] coda_0.19-4.1 survival_3.8-6
#> [95] ncdf4_1.24 timeDate_4052.112
#> [97] SingleCellExperiment_1.26.0 iterators_1.0.14
#> [99] hardhat_1.4.3 lava_1.9.2
#> [101] statmod_1.5.2 MLInterfaces_1.84.0
#> [103] ipred_0.9-15 nlme_3.1-169
#> [105] bit64_4.8.2 progress_1.2.3
#> [107] filelock_1.0.3 LaplacesDemon_16.1.8
#> [109] GenomeInfoDb_1.40.1 bslib_0.12.0
#> [111] affyio_1.74.0 irlba_2.3.7
#> [113] rpart_4.1.27 otel_0.2.0
#> [115] colorspace_2.1-3 BiocGenerics_0.50.0
#> [117] DBI_1.3.0 nnet_7.3-21
#> [119] tidyselect_1.2.1 processx_3.9.0
#> [121] bit_4.6.0 compiler_4.4.1
#> [123] curl_7.1.0 rvest_1.0.5
#> [125] httr2_1.3.0 graph_1.82.0
#> [127] SparseM_1.84-2 xml2_1.6.0
#> [129] desc_1.4.3 DelayedArray_0.30.1
#> [131] plotly_4.12.1 bookdown_0.47
#> [133] rtracklayer_1.64.0 scales_1.4.0
#> [135] hexbin_1.28.6 affy_1.82.0
#> [137] rappdirs_0.3.4 stringr_1.6.0
#> [139] digest_0.6.39 mixtools_2.0.0.1
#> [141] rmarkdown_2.31 XVector_0.44.0
#> [143] htmltools_0.5.9 pkgconfig_2.0.3
#> [145] SingleR_2.6.0 sparseMatrixStats_1.16.0
#> [147] MatrixGenerics_1.16.0 dbplyr_2.6.0
#> [149] fastmap_1.2.0 htmlwidgets_1.6.4
#> [151] UCSC.utils_1.0.0 DelayedMatrixStats_1.26.0
#> [153] farver_2.1.2 jquerylib_0.1.4
#> [155] jsonlite_2.0.0 BiocParallel_1.38.0
#> [157] mclust_6.1.3 mzID_1.42.0
#> [159] ModelMetrics_1.2.2.2 BiocSingular_1.20.0
#> [161] RCurl_1.98-1.19 magrittr_2.0.5
#> [163] scuttle_1.14.0 GenomeInfoDbData_1.2.12
#> [165] Rcpp_1.1.2 viridis_0.6.5
#> [167] MsCoreUtils_1.16.1 vsn_3.72.0
#> [169] pROC_1.19.0.1 stringi_1.8.7
#> [171] zlibbioc_1.50.0 MASS_7.3-65
#> [173] listenv_1.0.0 parallel_4.4.1
#> [175] Biostrings_2.72.1 splines_4.4.1
#> [177] hms_1.1.4 igraph_2.3.3
#> [179] QFeatures_1.14.2 reshape2_1.4.5
#> [181] biomaRt_2.60.1 stats4_4.4.1
#> [183] ScaledMatrix_1.12.0 XML_3.99-0.23
#> [185] evaluate_1.0.5 BiocManager_1.30.27
#> [187] tzdb_0.5.0 foreach_1.5.2
#> [189] tidyr_1.3.2 purrr_1.2.2
#> [191] future_1.75.0 clue_0.3-68
#> [193] bio3d_2.4-5 ggplot2_4.0.3
#> [195] rsvd_1.0.5 xtable_1.8-8
#> [197] restfulr_0.0.17 AnnotationFilter_1.28.0
#> [199] easyPubMed_3.1.6 e1071_1.7-17
#> [201] later_1.4.8 viridisLite_0.4.3
#> [203] class_7.3-24 ragg_1.5.2
#> [205] tibble_3.3.1 websocket_1.4.4
#> [207] memoise_2.0.1 AnnotationDbi_1.66.0
#> [209] GenomicAlignments_1.40.0 IRanges_2.38.1
#> [211] cluster_2.1.8.2 globals_0.19.1
#> [213] timechange_0.4.0 caret_7.0-1
#> [215] sampling_2.11