Loads GLiNER, a named-entity model,
through 'reticulate', and makes pii_scan() use it on the text columns the
rules leave uncertain: those with no finding stronger than low, made of
short values. Up to values distinct values of such a column are tagged
one at a time, without the column's name, so the model's evidence does not
merely repeat the name's. A value counts as a person, a place or an
organization when an entity of that kind covers most of it, and the column
is judged by the share of rows so tagged:
Usage
use_gliner(
model = "urchade/gliner_multi_pii-v1",
threshold = 0.5,
values = 50L,
share = 0.5
)Details
when at least
shareof rows name people, aperson_namefinding is added, ofmediumconfidence when at least 70% of rows do andlowotherwise;when at least
shareof rows name places or organizations, the low-confidenceperson_namefinding of the rules is withdrawn, since place and office names look like person names to them.
The model never decides a single cell, and never gives high confidence.
Independent evaluations put models of this kind at 0.4 to 0.6 F1 on
unfamiliar text, and none covers short cells. On the benchmark in
data-raw/benchmark/ the default model finds names behind question
numbers, in lower case and in US and Indian spellings, and tells village
names from person names; it does not see names written in Devanagari.
The first call installs the Python packages into an environment managed by 'reticulate' and downloads the model, about 1 GB, from Hugging Face. That is the only time piinata goes online. On a laptop CPU the model tags tens of values a second.
Examples
if (FALSE) { # \dontrun{
use_gliner()
pii_scan(data.frame(q15 = c("Ramesh Kumar", "James Smith", "sunita devi")))
use_gliner(FALSE)
} # }