taxastand is one of several R packages that match scientific names against a taxonomy. This vignette compares it with three others that do a similar job, on names with known errors, so that you can judge which suits your data:

Packages that are not compared

Three other packages resolve names by calling a web service instead of matching against a local reference:

One of these may be a good choice when you want the current taxonomy of a particular service and can work online. However, because they are API-based, the same query can give a different answer once the service updates its taxonomy or its matching, and the time taken depends on the network and the server, not on the package. Results can therefore not be reproduced reliably or timed fairly, so they are left out of this benchmark.

How the numbers were made

The numbers come from a benchmark that was run once, on the maintainer’s machine, and saved with the package (the packages compared are not dependencies of taxastand, and taxify needs a large download). All the code that produced them is shown below. Please read the limits before drawing conclusions.

Benchmark Summary

  • On most kinds of error, every package returned the right record for nearly every query. The exceptions, in the share of queries for which the right record was returned: taxify on a misspelled genus (82%) and WorldFlora on short epithets (88%).
  • Results differed for homonyms (the same name published by different authors), depending on whether packages used the authors. For example, WFO has four records named Linum perenne, with the authors L., Lam., Vill. and Guss. Asked for Linum perenne Lam. (checked with only those four records as the reference), taxastand returned only that record. WorldFlora returned all four records and marked the Lam. one as its best match. Over all of the homonym queries, taxastand returned exclusively the right record 100% of the time, U.Taxonstand 81% and WorldFlora 0%. WorldFlora’s best match, which it chooses using the author, was the right record 100% of the time.
  • taxastand found more misspelled genera and misspelled authors than taxify (genus typo: 95% against 82%; author typo: 100% against 93%). U.Taxonstand and WorldFlora were slightly better than taxastand on misspelled genera (99% and 99%).
  • For the nomenclatural differences that taxastand checks explicitly (a missing basionym author, a dropped ex element, a changed infraspecific rank and a Latin spelling variant), taxastand found the right record for at least 100% of the queries. The others did nearly as well on most of these, with some exceptions: taxify on a changed rank (71%) and U.Taxonstand on a dropped ex element (80%) and a changed rank (85%). WorldFlora found the right record for at least 98% of them, but returned other records as well for many (it returned only the right record for 70% to 88%), and its single answer was right for 79% of the dropped ex queries.
  • taxastand does not guess. When several records fit equally well it returns all of them, so on names without an author it gives a single answer less often than packages that always choose one. Its single answers were right every time or nearly so.
  • U.Taxonstand and WorldFlora are much slower than taxastand and taxify, which are similar to each other. Matching 100 queries (after warm-up) took 0.37 s with taxastand using an optimized reference, 0.74 s with taxify, 17.4 s with U.Taxonstand and 126.3 s with WorldFlora. taxastand without an optimized reference took 4.7 s. See the speed tables.

How the comparison works

The same reference for every package. Each package is given exactly the same reference table: the World Flora Online backbone as installed by taxify (1,595,937 species and infraspecific names). taxify can only use its own backbones, WorldFlora reads WFO, and taxastand and U.Taxonstand accept any table, so using WFO lets all four run on identical data and makes the numbers directly comparable.

Queries with known answers. Each query is made by changing a name taken from the reference, so the correct record is known. There are 100 queries for each kind of damage (fewer where the damage did not change the name), drawn at random. The query code is in The code.

kind what changes example
exact Nothing: the full name with its author Mahonia paxii Fedde → Mahonia paxii Fedde
epithet_substitution One letter of the epithet is replaced Gilia anbigua Rattan → Gilia aqbigua Rattan
epithet_deletion One letter of the epithet is deleted Piper oviedoi Urb. → Piper ovidoi Urb.
epithet_swap Two neighboring letters of the epithet are swapped Erica triceps Link → Erica tricesp Link
two_edits Two letters of the epithet are changed Lobelia inflata L. → Lobelia nqlata L.
short_epithet One letter deleted from an epithet of five letters or fewer Quercus robur Asso → Quercus obur Asso
genus_deletion One letter of the genus is deleted Vepris inermis Comm. → Veprs inermis Comm.
author_punctuation Periods removed from the author Croton spinosus L. → Croton spinosus L
author_typo One letter deleted from the author Poa polyphylla Hack. → Poa polyphylla Hck.
no_author_exact The name without its author Pyrus albicans Kit. → Pyrus albicans
no_author_epithet_deletion The name without its author, with a letter of the epithet deleted Elvira martyni Cass. → Elvira martyi
basionym_missing The basionym author (in parentheses) is missing Carex tremula (Ohwi) Ohwi → Carex tremula Ohwi
ex_dropped The ex element of the author is dropped Rosa remensis Desf. ex Bosc → Rosa remensis Desf.
rank_changed The infraspecific rank is changed (var. or subsp.) Phalaris minor var. integra Trab. → Phalaris minor subsp. integra Trab.
spelling_variant A standard Latin spelling variant of the epithet (y/i, ph/f, th/t, rh/r, ae/e) Poa nephelophila Bor → Poa nefelophila Bor

The last four kinds change how the name is written, not how it is spelled. They are differences that taxastand checks for explicitly (see the “Match types” section of ?ts_match_names). A fifth, dropping an in element (Tagawa in Morton queried as Tagawa), could not be made, because WFO has almost no names with an in element. The queries above are drawn at random from the reference, so most of them are for names that only one record has. A separate section looks at names that several records share.

What is measured. Every query is scored by record ID:

  • Found: the right record is among those the package returned.
  • Single answer: some packages choose one answer for each query: taxify always, WorldFlora through WFO.one(), U.Taxonstand through its best match. taxastand does not guess: it returns every record that fits, and its single answer is the record when only one fits. The tables give the share of queries that received a single answer, then the share of those answers that were right (“answered / right”).
  • Speed: the time to match 1 to 1000 queries.

Accuracy: Finding the right record

knitr::kable(
  wide("recall"),
  caption = "Share of queries for which the right record was returned"
)
Share of queries for which the right record was returned
kind taxastand taxify U.Taxonstand WorldFlora
exact 100% 100% 94% 100%
epithet_substitution 100% 99% 100% 100%
epithet_deletion 100% 99% 100% 99%
epithet_swap 100% 100% 100% 100%
two_edits 100% 99% 100% 100%
short_epithet 100% 99% 100% 88%
genus_deletion 95% 82% 99% 99%
author_punctuation 100% 100% 97% 100%
author_typo 100% 93% 95% 100%
no_author_exact 100% 100% 100% 100%
no_author_epithet_deletion 100% 98% 100% 100%
basionym_missing 100% 100% 95% 100%
ex_dropped 100% 97% 80% 100%
rank_changed 100% 71% 85% 100%
spelling_variant 100% 99% 99% 98%

Accuracy: Single answers

single <- function(kinds = names(what_main)) {
  out <- data.frame(kind = kinds, stringsAsFactors = FALSE)
  for (tool in shown) {
    a <- res$accuracy[[tool]]
    i <- match(kinds, a$kind)
    out[[tool_labels[[tool]]]] <- paste0(
      pct(a$pick_answered[i]), " / ",
      ifelse(is.na(a$pick_precision[i]), "-", pct(a$pick_precision[i]))
    )
  }
  out
}
knitr::kable(
  single(),
  caption = paste0(
    "Share of queries that got a single answer / share of those answers ",
    "that were right"
  )
)
Share of queries that got a single answer / share of those answers that were right
kind taxastand taxify U.Taxonstand WorldFlora
exact 100% / 100% 100% / 100% 100% / 94% 100% / 100%
epithet_substitution 91% / 100% 100% / 99% 100% / 99% 100% / 99%
epithet_deletion 94% / 100% 100% / 99% 100% / 100% 99% / 100%
epithet_swap 100% / 100% 100% / 100% 100% / 100% 100% / 100%
two_edits 93% / 100% 100% / 99% 100% / 99% 100% / 99%
short_epithet 82% / 100% 100% / 99% 100% / 100% 88% / 100%
genus_deletion 89% / 100% 86% / 95% 100% / 99% 99% / 100%
author_punctuation 100% / 100% 100% / 100% 100% / 97% 100% / 100%
author_typo 99% / 100% 100% / 93% 100% / 95% 100% / 100%
no_author_exact 94% / 100% 100% / 100% 100% / 100% 100% / 100%
no_author_epithet_deletion 63% / 100% 99% / 99% 100% / 99% 100% / 98%
basionym_missing 100% / 100% 100% / 100% 100% / 94% 100% / 96%
ex_dropped 97% / 100% 100% / 97% 100% / 80% 100% / 79%
rank_changed 100% / 100% 100% / 71% 100% / 84% 100% / 100%
spelling_variant 100% / 100% 100% / 99% 100% / 99% 99% / 99%

Where a package returns several records, taxastand’s result tells you which queries need a decision. On the author-less typo kind (no_author_epithet_deletion) taxastand returned 3.3 records per answered query on average: without an author, many names have several records that fit equally well, and taxastand reports all of them.

Accuracy: Names shared by several records

This section is different from the ones above, so read it separately. Each query here is made from a record whose genus and epithet are also used by at least one other record with a different author (a homonym, or a name that differs only in its rank). Such records were chosen on purpose, because it is where the author, and so the nomenclatural rules, has to decide which record is meant. The results say how each package behaves when a name is shared. They do not say how often that happens: roughly one record in ten in WFO shares its genus and epithet with another. For the other nine in ten, the genus and epithet already point to one record, and the tables above apply.

kind what changes example
homonym A name that several records share, queried with its own author Linum perenne Lam. → Linum perenne Lam.
homonym_typo The same, with a letter of the epithet deleted Oenothera alata Raf. → Oenothera lata Raf.
homonym_basionym_missing A shared name, with the basionym author missing Urtica hirta (Sw.) Sw. → Urtica hirta Sw.
homonym_ex_dropped A shared name, with the ex element of the author dropped Lantana dubia Wall. ex Walp. → Lantana dubia Wall.
homonym_rank_changed A shared name, with the infraspecific rank changed Cornus mas var. alba Weston → Cornus mas subsp. alba Weston
homonym_spelling_variant A shared name, with a Latin spelling variant of the epithet Thymus zygis Pall. → Thymus zigis Pall.
knitr::kable(
  wide("recall", names(what_shared)),
  caption = paste0(
    "Names shared by several records: share of queries for which the right ",
    "record was returned"
  )
)
Names shared by several records: share of queries for which the right record was returned
kind taxastand taxify U.Taxonstand WorldFlora
homonym 100% 100% 81% 100%
homonym_typo 100% 99% 99% 99%
homonym_basionym_missing 100% 95% 52% 100%
homonym_ex_dropped 99% 88% 36% 100%
homonym_rank_changed 96% 34% 47% 100%
homonym_spelling_variant 100% 100% 99% 99%
knitr::kable(
  wide("unambiguous", names(what_shared)),
  caption = paste0(
    "Names shared by several records: share of queries for which only the ",
    "right record was returned"
  )
)
Names shared by several records: share of queries for which only the right record was returned
kind taxastand taxify U.Taxonstand WorldFlora
homonym 100% 100% 81% 0%
homonym_typo 88% 99% 0% 0%
homonym_basionym_missing 97% 95% 46% 0%
homonym_ex_dropped 87% 88% 33% 0%
homonym_rank_changed 92% 34% 0% 0%
homonym_spelling_variant 100% 100% 0% 0%
knitr::kable(
  single(names(what_shared)),
  caption = paste0(
    "Names shared by several records: share of queries that got a single ",
    "answer / share of those answers that were right"
  )
)
Names shared by several records: share of queries that got a single answer / share of those answers that were right
kind taxastand taxify U.Taxonstand WorldFlora
homonym 100% / 100% 100% / 100% 100% / 81% 100% / 100%
homonym_typo 88% / 100% 100% / 99% 100% / 99% 100% / 99%
homonym_basionym_missing 97% / 100% 100% / 95% 100% / 48% 100% / 59%
homonym_ex_dropped 88% / 98% 100% / 88% 100% / 34% 100% / 28%
homonym_rank_changed 96% / 96% 100% / 34% 100% / 45% 100% / 98%
homonym_spelling_variant 100% / 100% 100% / 100% 100% / 99% 100% / 99%

When a name is shared, the packages differ much more than above:

  • taxastand returned the right record for 96 to 100% of the queries. When two records fit equally well (for example “Willd.” against both “Willd. ex Less.” and “Willd. ex DC.”), it returns both instead of choosing, so it answers with a single record less often (as low as 87% for a dropped ex author); when it did answer with one record, that record was right 96 to 100% of the time.
  • WorldFlora returned the right record (99 to 100%), together with the other records that share the name, and marks one as its best match. That best match was right for 28% of the queries with a dropped ex author and 59% with a missing basionym author.
  • taxify returns one record, and it was the right one for 88 to 100% of the queries, except for a changed rank (34%).
  • U.Taxonstand returned the right record for 36 to 99% of the queries, with the lowest results for a dropped ex author (36%) and a missing basionym author (52%).

The comparison is again about the records returned, not about which package is “best”: a package that returns every record that fits gives the user the decision, and one that returns a single record decides for them.

Speed

speed_table <- function() {
  s <- do.call(rbind, res$speed)
  s$tool <- speed_labels[s$tool]
  w <- reshape(s, idvar = "tool", timevar = "n_queries", direction = "wide")
  names(w) <- sub("seconds.", "", names(w), fixed = TRUE)
  rownames(w) <- NULL
  w
}
knitr::kable(
  speed_table(), na = "not run",
  caption = "Seconds to match n queries (best of 3, after warm-up)"
)
Seconds to match n queries (best of 3, after warm-up)
tool 1 10 100 1000
taxastand (optimized) 0.02 0.15 0.37 3.09
taxastand (not optimized) 0.66 1.06 4.70 24.72
taxify 0.01 0.43 0.74 2.50
U.Taxonstand 0.93 2.56 17.36 NA
WorldFlora 0.74 11.15 126.32 NA

U.Taxonstand and WorldFlora were timed only up to 100 queries, because larger runs on a reference of this size take many minutes for those packages.

The “taxastand (optimized)” row are results that used a reference optimized once with ts_optimize_ref() and reused for every call. The “taxastand (not optimized)” row are results that passed the reference names (a character vector) to ts_match_names(), so the reference is parsed again on every call.

The timings are made after a warm-up: before timing, each package was run once on the whole set of queries, so one-time costs are not counted. An optimized reference, for example, is parsed lazily, one genus at a time, so the first time a genus is used costs more than later calls.

Limits of this comparison

  • The queries are synthetic and come from one random sample, with about 100 queries per kind. Differences of a few percent between tools are within what a different sample could change.
  • Each package was run with its default settings (WorldFlora’s acceptedNameUsageID.match = FALSE and WFO.one(Auth.dist = ...) aside; see the code below). Settings such as the thresholds for fuzzy matching can be changed, and a package tuned to your own data could give different results. The defaults differ in kind: taxify allows a fraction of the name to differ and counts a swap of two neighboring letters as one edit, while taxastand allows a fixed number of edits (max_dist) and, by default, counts a swap as two (transpositions = FALSE). How a name is passed in also matters: WorldFlora was given infraspecific names as separate genus, species and infraspecific epithet columns, the way to ask for a match that ignores the rank. Given as one string, a variety that is queried as a subspecies is not found.
  • The nomenclatural kinds and the kinds for shared names were run separately from the others, the last of them on 2026-11-09 (the others on 2026-11-09), on the same machine, with the same reference and package versions. Their queries do not depend on the other kinds, so a single run would have given the same queries.
  • The queries for shared names were chosen because the name is shared (see that section); they show how a package behaves in that case, not how often it happens.
  • Only matching was scored. Resolving synonyms to accepted names, which all of these packages also do, was not.
  • Only plants (WFO) were tested. Results on other groups or on messier real-world lists may differ.
  • Times are from one computer (aarch64-apple-darwin23, R version 4.6.0 (2026-04-24)) and will differ on yours.
  • Versions: taxastand 2.0.0.9000, taxify 0.7.1, WorldFlora 1.14.5, U.Taxonstand 1.4.0. Benchmark run on 2026-11-09.

The code

This is the code that produced the results above, shown unchanged. It is also installed with taxastand (system.file("benchmark", package = "taxastand")). The scripts that run it, data-raw/benchmark-vignette.R (including the kinds and speed arguments used to run some kinds separately) and data-raw/benchmark-combine.R (which merges the runs), are in the source repository.

Reference

The reference is read from taxify’s installed backbone, reduced to species and infraspecific names, and given one row per distinct authored name.

# One row per distinct authored name, with columns:
#   taxonID, canonical, authorship (NA if none), full (canonical + authorship),
#   taxonRank (lower case), taxonomicStatus, acceptedNameUsageID (NA if
#   accepted)
load_reference <- function(source = c("ppg", "wfo"), ppg_path = NULL,
                           ppg_version = "v2.0.0", wfo_backbone = "wfo") {
  source <- match.arg(source)
  if (source == "ppg") {
    if (is.null(ppg_path)) ppg_path <- file.path(tempdir(), "ppg.csv")
    if (!file.exists(ppg_path)) {
      utils::download.file(
        sprintf(
          "https://raw.githubusercontent.com/pteridogroup/ppg/%s/data/ppg.csv",
          ppg_version
        ),
        ppg_path,
        quiet = TRUE
      )
    }
    x <- utils::read.csv(ppg_path, stringsAsFactors = FALSE, na.strings = "NA")
    x <- x[!is.na(x$scientificName), ]
  } else {
    # taxify keeps each backbone as a vectra file in its data directory
    f <- file.path(
      taxify::taxify_data_dir(), wfo_backbone, "latest",
      paste0(wfo_backbone, ".vtr")
    )
    if (!file.exists(f)) stop("taxify backbone not found: ", f)
    b <- as.data.frame(vectra::collect(vectra::tbl(f)))
    x <- data.frame(
      taxonID = b$taxon_id, scientificName = b$canonical_name,
      scientificNameAuthorship = b$authorship,
      taxonRank = tolower(b$taxon_rank),
      taxonomicStatus = tolower(b$taxonomic_status),
      acceptedNameUsageID = ifelse(
        b$accepted_taxon_id == b$taxon_id, NA_character_, b$accepted_taxon_id
      ),
      stringsAsFactors = FALSE
    )
  }
  x <- x[x$taxonRank %in% c("species", "subspecies", "variety", "form"), ]
  auth <- x$scientificNameAuthorship
  has_auth <- !is.na(auth) & auth != ""
  full <- x$scientificName
  full[has_auth] <- paste(x$scientificName[has_auth], auth[has_auth])
  out <- data.frame(
    taxonID = x$taxonID, canonical = x$scientificName,
    authorship = ifelse(has_auth, auth, NA_character_), full = full,
    taxonRank = x$taxonRank, taxonomicStatus = x$taxonomicStatus,
    acceptedNameUsageID = x$acceptedNameUsageID,
    stringsAsFactors = FALSE
  )
  out[!duplicated(out$full), ]
}

Queries

damage() changes one letter; make_queries() builds each kind of query and records the correct answer.

# Damage one letter of `x`: substitution, deletion, or swap of two neighbors.
# `from` is the first position that may be changed.
damage <- function(x, kind, from = 1L) {
  n <- nchar(x)
  if (n < from + 1L) {
    return(x)
  }
  i <- if (n - from > 0L) from - 1L + sample.int(n - from, 1L) else from
  switch(kind,
    substitution = `substr<-`(x, i, i, if (substr(x, i,
      i) == "q") "z" else "q"),
    deletion = paste0(substr(x, 1, i - 1), substr(x, i + 1, n)),
    swap = {
      i <- min(i, n - 1L)
      a <- substr(x, i, i)
      b <- substr(x, i + 1L, i + 1L)
      if (a == b) x else `substr<-`(x, i, i + 1L, paste0(b, a))
    }
  )
}
# Queries made from random species-rank records that have an author, `n` per
# kind of error. Returns a data frame: query, kind, source (row of `ref`) and
# `truth` (a list of the record IDs that count as the right answer: the record
# itself, or, when the query has no author, every record sharing the canonical
# name).
#   exact, no_author_exact     the name as in the reference (without author)
#   epithet_*, genus_deletion  one letter changed, with the author
#   two_edits                  two letters changed in the epithet
#   short_epithet              one deletion in an epithet of five letters or
#                              less
#   author_punctuation         periods removed from the author
#   author_typo                a letter deleted in the author
#   homonym, homonym_typo      a name shared by records with different authors,
#                              queried with its own author (as is, or with a
#                              typo in the epithet): needs the author to decide
# Nomenclatural differences (the form of the name changes, not its spelling).
# They come last, so that the other kinds are the same with or without them,
# and a query that is the same as another reference name is left out, so the
# right record stays unambiguous:
#   basionym_missing           "(Turcz.) Czerep." queried as "Czerep."
#   ex_dropped, in_dropped     "Willd. ex Schkuhr" as "Willd.", "A in B" as "A"
#   rank_changed               "var." queried as "subsp." or the reverse
#   spelling_variant           y/i, ph/f, th/t, rh/r or ae/e in the epithet
# The same four, with a homonym_ prefix, for records that share their genus and
# epithet with another record: here the nomenclatural rules decide which record
# is meant. They are picked because they have competitors, so they show how a
# package behaves when a name is shared, not how often that happens.
make_queries <- function(ref, n = 100, seed = 20261109) {
  set.seed(seed)
  species <- ref$taxonRank == "species" & !is.na(ref$authorship) &
    grepl("^[A-Z][a-z]+ [a-z-]+$", ref$canonical)
  epithet <- sub("^\\S+ ", "", ref$canonical)
  dup_canon <- ref$canonical %in% ref$canonical[duplicated(ref$canonical)]
  auth <- ref$authorship
  not_basio <- !is.na(auth) & !grepl("^[(]", auth)
  infra <- ref$taxonRank %in% c("subspecies", "variety") & !is.na(auth) &
    grepl("^[A-Z][a-z]+ [a-z-]+ (var[.]|subsp[.]) [a-z-]+$", ref$canonical)
  pools <- list(
    default = which(species & nchar(ref$canonical) > 12),
    short = which(species & nchar(epithet) <= 5 & nchar(epithet) >= 3),
    homonym = which(species & dup_canon),
    basionym = which(species & grepl("^[(][^)]*[)]\\s*\\S", auth)),
    ex = which(species & not_basio & grepl(" ex ", auth, fixed = TRUE)),
    `in` = which(species & not_basio & grepl(" in [A-Z]", auth)),
    infra = which(infra),
    foldable = which(species & grepl("y|ph|th|rh|ae", epithet))
  )
  # The same records, but only those that share their genus and epithet
  # (ignoring the rank) with another record, so that the author, and with it the
  # nomenclatural rules, has to decide which record is meant
  key <- sub(" (var[.]|subsp[.]|f[.]) ", " ", ref$canonical)
  shared <- key %in% key[duplicated(key)]
  for (k in c("basionym", "ex", "infra", "foldable")) {
    pools[[paste0(k, "_shared")]] <- pools[[k]][shared[pools[[k]]]]
  }
  by_canon <- split(ref$taxonID, ref$canonical)
  # name parts of record i
  parts <- function(src) {
    list(
      genus = sub(" .*", "", ref$canonical[src]), epi = epithet[src],
      auth = ref$authorship[src]
    )
  }
  dmg <- function(x, kind, from = 1L) vapply(x, damage, "", kind, from = from)
  author_typo <- function(a) {
    vapply(a, function(x) {
      w <- strsplit(x, "(?<=[ .])", perl = TRUE)[[1]]
      k <- which.max(nchar(w))
      w[k] <- damage(w[k], "deletion", from = 2L)
      paste(w, collapse = "")
    }, "")
  }
  # swap var. and subsp.
  swap_rank <- function(x) {
    ifelse(
      grepl(" var. ", x, fixed = TRUE), sub(" var. ", " subsp. ", x,
        fixed = TRUE),
      sub(" subsp. ", " var. ", x, fixed = TRUE)
    )
  }
  # fold one Latin spelling variant (the first that applies) in an epithet
  spelling_variant <- function(x) {
    vapply(x, function(e) {
      for (pat in list(c("y", "i"), c("ph", "f"), c("th", "t"), c("rh", "r"),
        c("ae", "e"))) {
        if (grepl(pat[1], e, fixed = TRUE)) {
          return(sub(pat[1], pat[2], e, fixed = TRUE))
        }
      }
      e
    }, "")
  }
  f_basionym <- function(p) {
    paste(p$genus, p$epi, sub("^[(][^)]*[)]\\s*", "", p$auth))
  }
  f_ex <- function(p) paste(p$genus, p$epi, sub(" ex .*$", "", p$auth))
  f_in <- function(p) paste(p$genus, p$epi, sub(" in .*$", "", p$auth))
  f_rank <- function(p) paste(p$genus, swap_rank(p$epi), p$auth)
  f_spelling <- function(p) paste(p$genus, spelling_variant(p$epi), p$auth)
  kinds <- list(
    exact = list("default", function(p) paste(p$genus, p$epi, p$auth)),
    epithet_substitution = list("default", function(p) {
      paste(p$genus, dmg(p$epi, "substitution"), p$auth)
    }),
    epithet_swap = list("default", function(p) {
      paste(p$genus, dmg(p$epi, "swap"), p$auth)
    }),
    epithet_deletion = list("default", function(p) {
      paste(p$genus, dmg(p$epi, "deletion"), p$auth)
    }),
    two_edits = list("default", function(p) {
      paste(p$genus, dmg(dmg(p$epi, "deletion"), "substitution"), p$auth)
    }),
    short_epithet = list("short", function(p) {
      paste(p$genus, dmg(p$epi, "deletion"), p$auth)
    }),
    genus_deletion = list("default", function(p) {
      paste(dmg(p$genus, "deletion", from = 2L), p$epi, p$auth)
    }),
    no_author_exact = list("default", function(p) paste(p$genus, p$epi)),
    no_author_epithet_deletion = list("default", function(p) {
      paste(p$genus, dmg(p$epi, "deletion"))
    }),
    author_punctuation = list("default", function(p) {
      paste(p$genus, p$epi, gsub(".", "", p$auth, fixed = TRUE))
    }),
    author_typo = list("default", function(p) {
      paste(p$genus, p$epi, author_typo(p$auth))
    }),
    homonym = list("homonym", function(p) paste(p$genus, p$epi, p$auth)),
    homonym_typo = list("homonym", function(p) {
      paste(p$genus, dmg(p$epi, "deletion"), p$auth)
    }),
    # a third element, TRUE, leaves out queries equal to another reference name
    basionym_missing = list("basionym", f_basionym, TRUE),
    ex_dropped = list("ex", f_ex, TRUE),
    in_dropped = list("in", f_in, TRUE),
    rank_changed = list("infra", f_rank, TRUE),
    spelling_variant = list("foldable", f_spelling, TRUE),
    # the same, for names that several records share
    homonym_basionym_missing = list("basionym_shared", f_basionym, TRUE),
    homonym_ex_dropped = list("ex_shared", f_ex, TRUE),
    homonym_rank_changed = list("infra_shared", f_rank, TRUE),
    homonym_spelling_variant = list("foldable_shared", f_spelling, TRUE)
  )
  out <- do.call(rbind, lapply(names(kinds), function(k) {
    pool <- pools[[kinds[[k]][[1]]]]
    if (!length(pool)) {
      return(NULL) # the reference has no names for this kind of query
    }
    src <- pool[sample.int(length(pool), min(n, length(pool)))]
    q <- trimws(kinds[[k]][[2]](parts(src)))
    keep <- if (length(kinds[[k]]) > 2) !q %in% ref$full else TRUE
    data.frame(
      query = q, kind = k, source = src, stringsAsFactors = FALSE
    )[keep, ]
  }))
  out$truth <- lapply(seq_len(nrow(out)), function(i) {
    if (grepl("^no_author", out$kind[i])) {
      by_canon[[ref$canonical[out$source[i]]]]
    } else {
      ref$taxonID[out$source[i]]
    }
  })
  out$original <- ref$full[out$source]
  # a query string can arise from different records or kinds; keep each once
  out[!duplicated(out$query), ]
}

Scoring

# `hits` has one row per record returned: query, id, and `pick` (TRUE for the
# record the tool itself chooses as its answer; at most one per query).
# One row per query: n_hits (records returned), correct (the right record is
# among them), unambiguous (it is the only one returned), picked (the tool chose
# a record) and pick_correct (the chosen record is right).
score_queries <- function(queries, hits) {
  hits <- hits[!duplicated(hits[, c("query", "id")]), ]
  lv <- queries$query
  by_q <- split(hits$id, factor(hits$query, levels = lv))
  picks <- split(
    hits$id[hits$pick], factor(hits$query[hits$pick], levels = lv)
  )
  n_hits <- vapply(by_q, length, 0L)
  correct <- mapply(function(h, t) any(h %in% t), by_q, queries$truth)
  picked <- vapply(picks, length, 0L) > 0
  pick_correct <- mapply(
    function(p, t) length(p) > 0 && p[[1]] %in% t, picks, queries$truth
  )
  data.frame(
    query = queries$query, kind = queries$kind, n_hits = unname(n_hits),
    correct = unname(correct), unambiguous = unname(correct & n_hits == 1L),
    picked = unname(picked), pick_correct = unname(pick_correct),
    stringsAsFactors = FALSE
  )
}

# Per kind of error:
# - recall: share of all queries for which the right record is among those
#   returned; precision: the same share among queries that got an answer
# - unambiguous: share where the right record is the only one returned;
#   mean_hits: average records returned per answered query
# - pick_answered: share of queries for which the tool chose one record;
#   pick_precision: share of those choices that are right
summarize_scores <- function(scores) {
  f <- function(s) {
    answered <- s$n_hits > 0
    data.frame(
      n = nrow(s), answered = mean(answered), recall = mean(s$correct),
      precision = if (any(answered)) mean(s$correct[answered]) else NA_real_,
      unambiguous = mean(s$unambiguous),
      mean_hits = if (any(answered)) mean(s$n_hits[answered]) else NA_real_,
      pick_answered = mean(s$picked),
      pick_precision = if (any(s$picked)) {
        mean(s$pick_correct[s$picked])
      } else {
        NA_real_
      }
    )
  }
  out <- do.call(rbind, lapply(split(scores, scores$kind), f))
  out$kind <- rownames(out)
  rownames(out) <- NULL
  out[, c("kind", setdiff(colnames(out), "kind"))]
}

One adapter per package

Each adapter builds its package’s reference from the shared table and returns the records matched for a set of queries, plus the package’s own single answer.

U.Taxonstand and WorldFlora take the name and the author separately, so this helper splits a query first.

# U.Taxonstand and WorldFlora take the name and the author separately. Split a
# query into the name (genus and epithet, plus the rank and infraspecific
# epithet of an infraspecific name) and the author.
split_query <- function(q) {
  pattern <- "^(\\S+ \\S+ (?:var[.]|subsp[.]|f[.]) \\S+|\\S+ \\S+)\\s*(.*)$"
  m <- regmatches(q, regexec(pattern, q, perl = TRUE))
  name <- vapply(m, `[`, "", 2)
  data.frame(
    name = name, author = vapply(m, `[`, "", 3),
    infraspecific = grepl(" (var[.]|subsp[.]|f[.]) ", name),
    stringsAsFactors = FALSE
  )
}
# Each make_*() takes the reference table and returns a list with `setup_s` (the
# time spent preparing the reference, once) and `run(queries)`.

make_taxastand <- function(ref, prepared = TRUE) {
  tax <- data.frame(
    taxonID = ref$taxonID, acceptedNameUsageID = ref$acceptedNameUsageID,
    taxonomicStatus = ref$taxonomicStatus, scientificName = ref$full,
    stringsAsFactors = FALSE
  )
  start <- proc.time()
  reference <- if (prepared) {
    taxastand::ts_optimize_ref(tax)
  } else {
    tax$scientificName
  }
  setup <- (proc.time() - start)[["elapsed"]]
  list(
    setup_s = setup,
    run = function(q) {
      m <- suppressWarnings(
        taxastand::ts_match_names(q, reference, simple = TRUE)
      )
      m <- m[m$match_type != "no_match", ]
      h <- data.frame(
        query = m$query, id = ref$taxonID[match(m$reference, ref$full)],
        stringsAsFactors = FALSE
      )
      # taxastand does not guess: its answer is the record when only one fits
      n_ids <- tapply(h$id, h$query, function(x) length(unique(x)))
      h$pick <- n_ids[h$query] == 1L
      h
    }
  )
}
make_taxify <- function(ref, backbone = "wfo") {
  setup <- system.time(
    taxify::taxify("Quercus robur", backbone = backbone, verbose = FALSE)
  )[["elapsed"]]
  list(
    setup_s = setup,
    run = function(q) {
      r <- as.data.frame(suppressWarnings(suppressMessages(
        taxify::taxify(q, backbone = backbone, verbose = FALSE)
      )))
      ok <- !is.na(r$taxon_id)
      # taxify returns one record per query: that is its pick
      data.frame(
        query = r$input_name[ok], id = r$taxon_id[ok], pick = TRUE,
        stringsAsFactors = FALSE
      )
    }
  )
}
make_utaxonstand <- function(ref) {
  genus <- sub(" .*", "", ref$canonical)
  setup <- system.time({
    db <- data.frame(
      ID = ref$taxonID, Name = ref$canonical,
      Author = ifelse(is.na(ref$authorship), "", ref$authorship),
      Genus = genus,
      Rank = ifelse(ref$taxonRank == "species", 2, 3),
      ACCEPTED_ID = ref$acceptedNameUsageID,
      stringsAsFactors = FALSE
    )
    db <- U.Taxonstand::nameClean(db)
  })[["elapsed"]]
  list(
    setup_s = setup,
    run = function(q) {
      parts <- split_query(q)
      sp <- data.frame(
        Name = parts$name, Author = parts$author,
        Rank = ifelse(parts$infraspecific, 3, 2), stringsAsFactors = FALSE
      )
      # nameMatch() prints progress bars; discard them
      utils::capture.output({
        r <- suppressWarnings(suppressMessages(
          U.Taxonstand::nameMatch(
            spList = sp, spSource = db, author = TRUE, matchFirst = FALSE
          )
        ))
      })
      # one row per matched record; Name_in_database may be empty on no match
      ok <- !is.na(r$ID_in_database) & r$ID_in_database != ""
      idx <- match(
        paste(r$Submitted_Name, r$Submitted_Author),
        paste(sp$Name, sp$Author)
      )
      # Name_set == 1 marks the best match of each query
      h <- data.frame(
        query = q[idx[ok]], id = r$ID_in_database[ok],
        pick = r$Name_set[ok] == 1, stringsAsFactors = FALSE
      )
      h$pick <- h$pick & !duplicated(cbind(h$query, h$pick))
      h
    }
  )
}
make_worldflora <- function(ref) {
  parts <- strsplit(ref$canonical, " ", fixed = TRUE)
  setup <- system.time({
    wfo <- data.frame(
      taxonID = ref$taxonID, scientificNameID = "",
      scientificName = ref$canonical,
      taxonRank = toupper(ref$taxonRank), parentNameUsageID = "",
      scientificNameAuthorship = ifelse(is.na(ref$authorship), "",
        ref$authorship),
      family = "", genus = vapply(parts, `[`, "", 1),
      specificEpithet = vapply(parts, `[`, "", 2),
      infraspecificEpithet = vapply(parts, function(p) {
        if (length(p) > 2) p[length(p)] else ""
      }, ""),
      verbatimTaxonRank = "", nomenclaturalStatus = "", namePublishedIn = "",
      taxonomicStatus = tools::toTitleCase(ref$taxonomicStatus),
      acceptedNameUsageID = ifelse(
        is.na(ref$acceptedNameUsageID), "", ref$acceptedNameUsageID
      ),
      nameAccordingToID = "", stringsAsFactors = FALSE
    )
  })[["elapsed"]]
  list(
    setup_s = setup,
    run = function(q) {
      parts <- split_query(q)
      # Match one set of queries and pick one record for each. WFO.match() with
      # acceptedNameUsageID.match = FALSE returns the record that matched, not
      # the accepted record it points to, so that matching is scored alone.
      match_wfo <- function(spec, ...) {
        if (!nrow(spec)) {
          return(NULL)
        }
        utils::capture.output({
          r <- suppressWarnings(suppressMessages(
            WorldFlora::WFO.match(
              spec.data = spec, WFO.data = wfo, Authorship = "Authorship",
              acceptedNameUsageID.match = FALSE, no.dates = TRUE,
              verbose = FALSE, counter = 1e9, ...
            )
          ))
        })
        r <- r[!is.na(r$taxonID) & r$taxonID != "", ]
        # WFO.one() chooses one record per query, preferring the best author
        # match (Auth.dist); it expects the columns that WFO.match() adds only
        # when it replaces synonyms by accepted names
        r$New.accepted <- FALSE
        r$Old.status <- ""
        r$Old.ID <- ""
        utils::capture.output({
          one <- WorldFlora::WFO.one(
            r,
            Auth.dist = "Auth.dist", verbose = FALSE, counter = 1e9
          )
        })
        data.frame(
          query = r$key, id = r$taxonID,
          pick = paste(r$key, r$taxonID) %in% paste(one$key, one$taxonID),
          stringsAsFactors = FALSE
        )
      }
      sp <- !parts$infraspecific
      # Species: one name string. Infraspecific names: genus, species and
      # infraspecific epithet in separate columns and no rank, the way to ask
      # WorldFlora for a match that ignores the rank (given as one string, or
      # with the rank, a var. is not found when it is queried as subsp.).
      words <- strsplit(parts$name[!sp], " ", fixed = TRUE)
      rbind(
        match_wfo(data.frame(
          spec.name = parts$name[sp], Authorship = parts$author[sp],
          key = q[sp], stringsAsFactors = FALSE
        )),
        match_wfo(
          data.frame(
            Genus = vapply(words, `[`, "", 1),
            Species = vapply(words, `[`, "", 2),
            Infraspecific = vapply(words, `[`, "", 4),
            Authorship = parts$author[!sp], key = q[!sp],
            stringsAsFactors = FALSE
          ),
          Genus = "Genus", Species = "Species", Infraspecific = "Infraspecific"
        )
      )
    }
  )
}

Timing

# Elapsed seconds (best of `reps`) to match `n` queries, after a warm-up call.
# The queries are shuffled (fixed seed) so that every size is a mix of the
# kinds of error: the query set is ordered by kind, and exact names are much
# cheaper to match than damaged ones.
time_adapter <- function(adapter, queries, sizes = c(1, 10, 100, 1000),
  reps = 3) {
  set.seed(20261109)
  queries <- sample(queries)
  adapter$run(queries[seq_len(min(5, length(queries)))])
  do.call(rbind, lapply(sizes, function(n) {
    q <- rep_len(queries, n)
    q <- unique(q)
    if (length(q) < n) q <- c(q, paste0(q[seq_len(n - length(q))], "x"))
    t <- min(vapply(
      seq_len(reps), function(i) unname(system.time(adapter$run(q))[3]), 0
    ))
    data.frame(n_queries = n, seconds = round(t, 2))
  }))
}

Running it

ref <- load_reference("wfo")
queries <- make_queries(ref, n = 100)
adapters <- list(
  taxastand = make_taxastand(ref),
  taxify = make_taxify(ref),
  utaxonstand = make_utaxonstand(ref),
  worldflora = make_worldflora(ref)
)
for (tool in names(adapters)) {
  hits <- adapters[[tool]]$run(queries$query)
  scores <- score_queries(queries, hits)
  print(summarize_scores(scores)) # found, single answer
  print(time_adapter(adapters[[tool]], queries$query)) # speed
}

Session information

The R session and package versions on the machine that ran the benchmark (not the session that built this vignette), recorded for reproducibility.

#> R version 4.6.0 (2026-04-24)
#> Platform: aarch64-apple-darwin23
#> Running under: macOS Tahoe 26.5.2
#> 
#> Matrix products: default
#> BLAS:   /Library/Frameworks/R.framework/Versions/4.6/Resources/lib/libRblas.0.dylib 
#> LAPACK: /Library/Frameworks/R.framework/Versions/4.6/Resources/lib/libRlapack.dylib;  LAPACK version 3.12.1
#> 
#> locale:
#> [1] C.UTF-8/UTF-8/C.UTF-8/C/C.UTF-8/C.UTF-8
#> 
#> time zone: Asia/Tokyo
#> tzcode source: internal
#> 
#> attached base packages:
#> [1] stats     graphics  grDevices utils     datasets  methods   base     
#> 
#> loaded via a namespace (and not attached):
#>  [1] compiler_4.6.0     WorldFlora_1.14-5  plyr_1.8.9         magrittr_2.0.5    
#>  [5] cli_3.6.6          U.Taxonstand_1.4.0 tools_4.6.0        Rcpp_1.1.2        
#>  [9] jsonlite_2.0.0     vectra_0.14.1      rlang_1.3.0        libgeos_3.11.1-3  
#> [13] taxify_0.7.1