What is planned, in the order it is planned, and the questions still open. Changes to this page are proposals; open an issue to argue with one.
Using it in a workflow
Researchers call it from R. Projects also need it in the flow of work:
-
Review and exceptions.
pii_review()writes the findings as a file with a decision column for a person to fill in. A committed.piinata.ymlholds what has been reviewed: fields and files that may stay, detectors turned off, thresholds, and extra terms.pii_check()reads it. An exception names a field and a reason, not a blanket switch. - Custom patterns. Users declare their own: a name, a regular expression or a function, an optional validator and a confidence. Study ids, staff codes and local identifiers go here.
-
Git. A pre-commit hook (
language: rin.pre-commit-hooks.yaml) and a GitHub Action that scan the files a commit touches and fail on unreviewed findings. A Python port is on the table if no Python package does the same job, since most hooks run in Python environments. -
Redaction. The levels are: report; drop a field;
replace values with a placeholder of the same type; replace with a
stable pseudonym, so joins and counts still work; and a custom function.
The model is Presidio’s split between an analyzer that returns typed
spans with scores and an anonymizer that applies an operator per type
(replace, mask, hash, redact, encrypt, custom).
pii_scan()is the analyzer;pii_redact()is the anonymizer, andpii_redact_files()writes a clean copy beside the original. Still to do: JSON, and a custom function as a method. -
Originals out of git. Done:
pii_ignore_originals()adds each original to.gitignoreso the clean copy can be committed. It is opt-in and reports what it did; it does not rewrite history, so an original already committed stays in the repository’s past. - URLs and browsing data. A URL can carry personal data in its user, path and query string. A URL detector would parse with a maintained parser and report or strip those parts while keeping the host.
Items 1 to 3 come before redaction: a scanner people cannot quiet is one they remove.
Versions
| Version | Contents | State |
|---|---|---|
| 0.1.0 | first release, on GitHub: reading (archives, notebooks, geographic
files, databases), name, value and structure detectors,
idcheck, dialtone and
checkstring, context, addresses, coordinate pairs, review
and accept, redaction of data frames and files, originals out of git,
use_gliner(), the synthetic and SOTAB benchmarks |
done |
| 0.2.0 | CRAN, once idcheck and dialtone are
there |
|
| 0.3.0 | SurveyCTO, KoboToolbox/ODK, REDCap, SurveyMonkey and Google Forms exports with their form definitions; file-name scanning | |
| 0.4.0 | pre-commit hook and GitHub Action | |
| 0.5.0 | names in Devanagari | |
| 0.6.0 | URL parts |
Open questions
-
Phone numbers. Decided and built: no maintained,
Java-free libphonenumber existed on CRAN (
dialvalidatorwas archived on 2026-03-30 anddialrneeds Java), so the sister package dialtone validates numbers on Google’s libphonenumber metadata (Apache-2.0). piinata uses it when installed: numbers with a country code are always checked, others against the project’sregions; a rejected number stops counting, and a column of valid numbers moves up a grade. -
National ids. Decided and built: the sister package
idcheck validates
identity numbers in C++ (Aadhaar by Verhoeff checksum, GSTIN by
checksum, PAN, voter id, passport and Indian mobile numbers by format).
piinata uses it when installed; a checksum match is high confidence.
CRAN does have two Verhoeff packages,
verhoeffandCheckDigit, but both are GPL and neither is built for scanning a column. -
Person names in values. Without a model, the core
can only say “short, capitalized, mostly distinct strings”, confidence
low, and cannot tell a column of names from a column of village names.
Decided, after a review of GLiNER, Presidio, spaCy, Flair, Stanza,
IndicNER and the R routes:
use_gliner()throughreticulate::py_require()with the Apache-2.0urchade/gliner_multi_pii-v1model, run only on columns the rules leave uncertain and voting on the share of rows, because it manages 2 to 40 cells a second on a CPU. Independent tests put such models at 0.4 to 0.6 F1 on unfamiliar text and none covers short cells, so it is trusted only as far as the benchmark’s decoys show. Not adopted:nametagger(2014 English model, non-commercial license) and Presidio as a backend. -
The name.
piinatais free on CRAN and GitHub.