Skip to contents

Loads GLiNER, a named-entity model, through 'reticulate', and makes pii_scan() use it on the text columns the rules leave uncertain: those with no finding stronger than low, made of short values. Up to values distinct values of such a column are tagged one at a time, without the column's name, so the model's evidence does not merely repeat the name's. A value counts as a person, a place or an organization when an entity of that kind covers most of it, and the column is judged by the share of rows so tagged:

Usage

use_gliner(
  model = "urchade/gliner_multi_pii-v1",
  threshold = 0.5,
  values = 50L,
  share = 0.5
)

Arguments

model

A GLiNER model on Hugging Face. The default is the multilingual, Apache-2.0 licensed urchade/gliner_multi_pii-v1.

threshold

The weakest entity score that counts.

values

The most distinct values tagged per column.

share

The share of rows that decides a column.

Value

The settings, invisibly. use_gliner(FALSE) turns the backend off.

Details

  • when at least share of rows name people, a person_name finding is added, of medium confidence when at least 70% of rows do and low otherwise;

  • when at least share of rows name places or organizations, the low-confidence person_name finding of the rules is withdrawn, since place and office names look like person names to them.

The model never decides a single cell, and never gives high confidence. Independent evaluations put models of this kind at 0.4 to 0.6 F1 on unfamiliar text, and none covers short cells. On the benchmark in data-raw/benchmark/ the default model finds names behind question numbers, in lower case and in US and Indian spellings, and tells village names from person names; it does not see names written in Devanagari.

The first call installs the Python packages into an environment managed by 'reticulate' and downloads the model, about 1 GB, from Hugging Face. That is the only time piinata goes online. On a laptop CPU the model tags tens of values a second.

Examples

if (FALSE) { # \dontrun{
use_gliner()
pii_scan(data.frame(q15 = c("Ramesh Kumar", "James Smith", "sunita devi")))
use_gliner(FALSE)
} # }