Turkish KVKK classifier

What it does

Input

hastanın böbrek yetmezliği ilerlemiş, eşi 0532 111 22 33'ten ulaşılabilir

Output

kimlik, iletisim, saglik (special category: yes)

Input

böbrek yetmezliği farkındalık haftası başladı

Output

no label

A category is marked only when the text holds that kind of information about a specific person, even an unnamed one such as “the patient” or “the customer”. General information, rules, announcements and statistics get no label. In the first example a patient’s kidney failure is health data. In the second, an awareness week for kidney failure is not about anyone.

The classifier is meant to sit in front of the masking model: classify the text, pick a masking policy, then mask.

Labels

Results

There are two test sets. The main one is an independent held-out set: 1,125 texts written without looking at the training data, then labelled a second time, blind. The 987 texts where both label sets agree exactly are the gold set (87.7% exact agreement, Cohen’s kappa per label between 0.84 and 1.00). The second is a template-disjoint test of 771 examples from 333 templates that were never used in training. On the 987-text gold set, turkish-kvkk-classifier scores 0.874 micro-F1 and 0.856 macro-F1.

Micro-F1 is F1 computed over all label decisions pooled together. Macro-F1 is the unweighted mean of the per-label F1 scores.

The held-out texts and both label passes were produced with a language model and checked automatically. They were not verified line by line by a person.

Python quick start

import json, torch
from huggingface_hub import hf_hub_download
from transformers import AutoModelForSequenceClassification, AutoTokenizer

MODEL = "halilneed/turkish-kvkk-classifier"
tok = AutoTokenizer.from_pretrained(MODEL)
model = AutoModelForSequenceClassification.from_pretrained(MODEL).eval()
thresholds = json.load(open(hf_hub_download(MODEL, "thresholds.json")))
SPECIAL = {"saglik", "biyometrik", "genetik", "cinsel_hayat", "ceza_mahkumiyeti", "din_felsefi_inanc",
        "siyasi_dusunce", "irk_etnik", "dernek_vakif_sendika", "kilik_kiyafet"}

def classify(text):
    with torch.inference_mode():
        p = torch.sigmoid(model(**tok(text, return_tensors="pt", truncation=True, max_length=320)).logits)[0]
    labels = [model.config.id2label[i] for i, v in enumerate(p.tolist()) if v >= thresholds[model.config.id2label[i]]]
    return labels, bool(set(labels) & SPECIAL)

print(classify("hastanın böbrek yetmezliği ilerlemiş, eşi 0532 111 22 33'ten ulaşılabilir"))

Per-label thresholds were tuned on the dev set and ship in thresholds.json. For the special categories the threshold never goes above 0.5, which favours recall.

Measured speed: 22 ms per text at batch size one on an Intel i5-12400F with eight threads, for texts of 126 characters on average.

Training

• Base model. dbmdz/bert-base-turkish-cased (BERTurk, 110M).
• Method. Multi-label classification, sigmoid with binary cross-entropy, 5 epochs, learning rate 3e-5, bfloat16, up to 320 tokens.
• Data. 36,000 synthetic examples from 3,276 templates. Most are the masking model’s templates: each PII slot maps to a category by a fixed rule, and each template is also marked for the categories its context adds. The rest are contextual sentences that contain no values. 21% of the examples are traps, texts that should get no label.
• Split. Train, dev and test are separated by template hash.

Limitations

• The training data is synthetic. Measure the model on your own organisation’s text before it goes into a critical pipeline.
• The weakest labels are lokasyon (F1 0.74), musteri_islem (0.75) and gorsel_isitsel (0.79). All three are inferred from context and have no sharp boundary. genetik also scores 0.79.
• For the question “is there special-category data here?” recall on the held-out set is 0.884, so roughly one such text in nine is missed.
• The output is a category signal, not a legal assessment. The VERBİS inventory and KVKK compliance remain the user’s responsibility.

Türkçe KVKK kategori sınıflandırıcısı

turkish-kvkk-classifier, Türkçe bir metinde hangi kişisel veri kategorilerinin bulunduğunu belirleyen çok etiketli bir sınıflandırıcıdır: 14 VERBİS kategorisi ve KVKK md. 6’daki 10 özel nitelikli veri kategorisi. Maskeleme modelinden önce çalıştırılır: metin önce sınıflandırılır, sonuca göre maskeleme politikası seçilir, ardından metin maskelenir.

Eğitim verisine bakılmadan yazılmış 987 metinlik bağımsız sette micro-F1 0.874, macro-F1 0.856’dır. CPU’da metin başına yaklaşık 22 ms sürer. Çıktı yalnızca bir kategori tahminidir, hukuki nitelendirme değildir.

Links

• Model card and weights
• Evaluation set
• Turkish PII detection and masking
• Turkish BSEBY classifier
• Turkish prompt injection classifier
• Personal data and AI: a short guide for teams on what to mask and what to keep in-house before text reaches a model
• All models