Four small, open models for Turkish text: three for personal data, one for prompt injection, and the evaluation sets I use to measure them. All of them run on a CPU.
Masking, classification and the benchmarks behind them.
Speeds were measured on an Intel i5-12400F with eight threads and batch size one.
The two classifiers answer what is in a text. The masking model changes the text. A typical pipeline classifies first, picks a masking policy from the result, then masks. The prompt injection classifier sits in front of the LLM: it checks what goes in, whether a user typed it or an assistant retrieved it.
1,125 fictional texts were written from the label definitions alone and labelled twice. The gold set is the 987 where both label sets agree exactly. The texts and both label passes were produced with a language model and were not checked line by line by a person.
Same method: 750 fictional texts, 747 with exact agreement. Both passes used the same language model and guideline, so the 99.6% agreement shows the guideline can be applied consistently. It is not agreement between two independent people.
Tests a prompt injection guard from both sides in one run: 317 attacks from five public Turkish sets, and 3,055 benign rows. The benign rows are the hard negatives that ship with those sets plus the 2,875 texts of the false-alarm set below. No new text was written; every row keeps its source and licence.
How often a prompt injection guard blocks ordinary Turkish business text and task instructions. No new text: collected from the two sets above and cagrigungor’s PII benchmark. Public guards flag 17.8% to 27.8% of it.
The PII model is evaluated on a third-party set, cagrigungor/turkish-pii-masking-benchmark, 1,000 synthetic rows. The prompt injection classifier is measured on four public Turkish sets with each training source held out in turn; see its page.