Performs basic proofreading of names in a taxa list.
qc_taxa_names_proof(names, method = "jw", max_distance = 0.13)A data frame with col_tolval values, occurrence (n), and if valid (TRUE/FALSE).
Returns possible differences in a data frame with three columns (qc check, name, potential match(es)). Not all hits are errors but are potential issues that may need to be addressed.
The distance check computes pairwise string distances between names and returns name pairs that are likely duplicates.
Uses Jaro-Winkler (jw) distance by default which performs well for names. Other options are Levenshtein (lv), good for typos, and osa, like Levenshtein but slightly faster.
Good thresholds are jw 0.1 to 0.2, lv and osa <= 2
The checks include:
* **spaces**, leading or trailing, including html white space, or doulble space, or more than 3
* **case**, differences
* **sp** variants; (with/without .) sp and spp, inside next to slash
* **stage** variants; adult, A, pupa, pupae, P, immature, I, imm, juv, juvenile, larva, larvae, L, zoea, myses, mysops?, megalops, megadrile
* **probably**, variants; "?", " prob ", " prob. ", " probably " * add parentheses
* **cf**, variants start, or in string, cf, c.f., cf., c.f
* backslash_dash_underscore
* terrestrial (terr.), megadrile
complex cmplx
all caps
and, &
star
head
possibly, poss, poss.
unknown unk undetermined undet(.), indet, indetermined
large small with space or parentheses
backslash_dash
* **slash, direction** direction; including dash
* **slash, taxa** x/y vs. y/x
* **grp** variants; grp, gr, group, (with/without .) and without and dash and genus group, gp, dash or space before
* **unid** variants; unid, unidentified, unid diff, uid, (with/without .)
diff without unid
* **prob** variants; prob, prob., probably, including "?" (anywhere in text)
* **sensu**
* **parenthetical** text; sensu, prob, inc spec, (with/without .)
* **near** variants; nr n
aff. , f flag
quotes
slash order; c/o vs o/c
with, without, w/, w/o, w/ o, w /, w / o
frag and fragment
Tubificid
* **colon** e.g., Family: Genus
* **patterns** tera$ in Order, idae$ in Family, inae$ Subfamily, and ini$ in Tribe. Look for those patterns not in the expected columns. would need the entire taxa table. Right now only looking at a single vector.
immature, imm, w/ and w/o hair chaetae, hair+pectinate, bifid setae, chaetae
Common authors not in parentheses, e.g., Epler
text mining algorithms (word similarity) Other checks caught:
some not included:
* f. = forma = valid
# Example Issues
proof_issues <- qc_taxa_names_proof(data_taxa_names_issues$FinalID)
proof_issues$issues
#> [1] "distance" "case"
#> [3] "stage" "cf"
#> [5] "backslash_dash_underscore"
lapply(proof_issues, nrow)
#> $issues
#> NULL
#>
#> $distance
#> [1] 78
#>
#> $space
#> NULL
#>
#> $case
#> [1] 2
#>
#> $stage
#> NULL
#>
#> $unk
#> NULL
#>
#> $cf
#> NULL
#>
#> $backslash_dash_underscore
#> NULL
#>
# Example Master Taxa Lists
proof_MBSS <- qc_taxa_names_proof(data_benthos_MBSS$TAXAID, "jw", 0.13)
proof_MBSS$issues
#> [1] "distance"
head(proof_MBSS$distance)
#> name_x id_x name_y id_y distance
#> 1 APSECTROTANYPUS 212 PSECTROTANYPUS 233 0.02222222
#> 2 DICROTENDIPES 10 MICROTENDIPES 11 0.05128205
#> 3 CLINOTANYPUS 20 NILOTANYPUS 197 0.05808081
#> 4 ORTHOCLADIUS 18 SYNORTHOCLADIUS 157 0.06666667
#> 5 LEPTOPHLEBIIDAE 27 LEPTOPHLEBIA 250 0.06666667
#> 6 HYDROPSYCHE 39 HYDROPSYCHIDAE 65 0.07142857
proof_PacNW_taxaid <- qc_taxa_names_proof(data_benthos_PacNW$TaxaID)
proof_PacNW_taxaid$issues
#> [1] "distance"
head(proof_PacNW_taxaid$distance)
#> name_x id_x name_y id_y distance
#> 1 Microtendipes 84 Dicrotendipes 138 0.05128205
#> 2 Hydropsyche 14 Hydropsychidae 104 0.07142857
#> 3 Cinygmula 5 Cinygma 41 0.07407407
#> 4 Naididae 85 Naidinae 86 0.08333333
#> 5 Rhyacophila blarina 31 Rhyacophila arnaudi 125 0.09958720
#> 6 Rhyacophila hyalinata group 61 Rhyacophila angelita group 207 0.10434473
proof_PacNW_master_taxaid <- qc_taxa_names_proof(TaxaMaster_Ben_BCG_PacNW$TaxaID)
proof_PacNW_master_taxaid$issues
#> [1] "distance" "stage"
proof_PacNW_master_taxaid$stage
#> [1] "Lepidostoma-panel case larvae" "Lepidostoma-turret case larvae"
#> [3] "Lepidostoma-sand case larvae"
head(proof_PacNW_taxaid$distance)
#> name_x id_x name_y id_y distance
#> 1 Microtendipes 84 Dicrotendipes 138 0.05128205
#> 2 Hydropsyche 14 Hydropsychidae 104 0.07142857
#> 3 Cinygmula 5 Cinygma 41 0.07407407
#> 4 Naididae 85 Naidinae 86 0.08333333
#> 5 Rhyacophila blarina 31 Rhyacophila arnaudi 125 0.09958720
#> 6 Rhyacophila hyalinata group 61 Rhyacophila angelita group 207 0.10434473