Performs basic proofreading of names in a taxa list.
qc_taxa_names_proof(names, method = "jw", max_distance = 0.13)A data frame with col_tolval values, occurrence (n), and if valid (TRUE/FALSE).
Returns possible differences in a data frame with three columns (qc check, name, potential match(es)). Not all hits are errors but are potential issues that may need to be addressed.
The distance check computes pairwise string distances between names and returns name pairs that are likely duplicates.
Uses Jaro-Winkler (jw) distance by default which performs well for names. Other options are Levenshtein (lv), good for typos, and osa, like Levenshtein but slightly faster.
Good thresholds are jw 0.1 to 0.2, lv and osa <= 2
The checks include:
* **spaces**, leading or trailing, including html white space, or doulble space, or more than 3
* **case**, differences
* **sp** variants; (with/without .) sp and spp, inside next to slash
* **stage** variants; adult, A, pupa, pupae, P, immature, I, imm, juv, juvenile, larva, larvae, L, zoea, myses, mysops?, megalops, megadrile
* **probably**, variants; "?", " prob ", " prob. ", " probably " * add parentheses
* **cf**, variants start, or in string, cf, c.f., cf., c.f
* backslash_dash_underscore
* terrestrial (terr.), megadrile
complex cmplx
all caps
and, &
star
head
possibly, poss, poss.
unknown unk undetermined undet(.), indet, indetermined
large small with space or parentheses
backslash_dash
* **slash, direction** direction; including dash
* **slash, taxa** x/y vs. y/x
* **grp** variants; grp, gr, group, (with/without .) and without and dash and genus group, gp, dash or space before
* **unid** variants; unid, unidentified, unid diff, uid, (with/without .)
diff without unid
* **prob** variants; prob, prob., probably, including "?" (anywhere in text)
* **sensu**
* **parenthetical** text; sensu, prob, inc spec, (with/without .)
* **near** variants; nr n
aff. , f flag
quotes
slash order; c/o vs o/c
with, without, w/, w/o, w/ o, w /, w / o
frag and fragment
Tubificid
* **colon** e.g., Family: Genus
* **patterns** tera$ in Order, idae$ in Family, inae$ Subfamily, and ini$ in Tribe. Look for those patterns not in the expected columns. would need the entire taxa table. Right now only looking at a single vector.
immature, imm, w/ and w/o hair chaetae, hair+pectinate, bifid setae, chaetae
Common authors not in parentheses, e.g., Epler
text mining algorithms (word similarity) Other checks caught:
some not included:
* f. = forma = valid
# Example
proof_issues <- qc_taxa_names_proof(data_taxa_names_issues$FinalID)
# Issues (by type) in data
proof_issues$issues
#> [1] "distance" "case"
#> [3] "stage" "cf"
#> [5] "backslash_dash_underscore"
# different case of same name
proof_issues$case
#> # A tibble: 2 × 3
#> name_lower count original_names
#> <chr> <int> <chr>
#> 1 ablabesmyia rhamphe gr. 2 Ablabesmyia rhamphe Gr., Ablabesmyia rhamphe gr.
#> 2 c/o 2 c/o, C/O
# stage in final id
proof_issues$stage
#> [1] "3 CHIRONOMID PUPAE" "21 Baetid pupae"
#> [3] "Tubificidae unidentified immature"
# cf in final id
proof_issues$cf
#> [1] "Brachycentrus cf. incanus Hagen"
#> [2] "Cricotopus cf. sp. \"Ozarks\" (sensu Epler, 2001)"
# backslash, dash, underscore, or brackets in final id
proof_issues$backslash_dash_underscore
#> [1] "[no specimens present]"
# similar names (potentially many false positives)
head(proof_issues$distance)
#> name_x id_x name_y
#> 1 Tubificidae (imm w/ hair chaetae) 108 Tubificidae (imm w/o hair chaetae)
#> 2 Immature tubif. w/ hair chaetae 103 Immature tubif. w/o hair chaetae
#> 3 Thienemannimyia grp. 40 Thienemannimyia grp
#> 4 Thienemannimyia grp. 40 Thienemannimyia gr.
#> 5 Eukiefferiella sp. 135 Eukiefferiella spp.
#> 6 Eukiefferiella claripennis gr. 51 Eukiefferiella claripennis gp.
#> id_y distance
#> 1 109 0.009803922
#> 2 104 0.010416667
#> 3 41 0.016666667
#> 4 44 0.016666667
#> 5 136 0.017543860
#> 6 52 0.022222222