Run Turkish PII Masking Locally with GGUF

Why run PII masking locally

Support tickets, HR notes and contract excerpts often contain names, TCKN, IBAN, phone numbers and emails. If your masking step calls a remote API, that text leaves your network before it is cleaned. For many teams that is the opposite of the goal.

Local masking keeps the raw text on a laptop or a server you control. The model takes a Turkish instruction plus the text and returns rewritten text with labels such as [AD], [TCKN] and [TEL]. It is not a NER model: it does not return entity spans or character offsets. If you need reversible placeholders ([TCKN_1] with a mapping table), keep a rule layer in front for structured fields and use the model for what rules miss.

See the model page for the full schema, policies and limits. The on-site masking tool is rule-based (TCKN, IBAN, phone, email) and is not the GGUF build. The Hugging Face browser demo runs an ONNX conversion of the base model, also not these GGUF files.

What GGUF is

GGUF is a file format used by llama.cpp and Ollama to run language models without a Python/transformers stack. The companion repo halilneed/turkish-pii-detection-GGUF publishes two builds of v02:

Figures are from the GGUF model card (Ollama 0.40.7, temperature 0, same metric as the main card). There is no Q4_K_M build: the card states that 168M of 268M parameters sit in the embedding table and a 4-bit build was not published without measuring its loss.

Which file? Q8_0 matches the main model's overall 0.944. On the 20-row multi-person slice the card reports Q8_0 at 0.650 vs 0.750 for the main model — so prefer BF16 when you mask texts with several people.

Hardware you need

The GGUF card says the builds run on CPU; no GPU needed (llama.cpp / Ollama).

Published latency and RAM for the main Python model (not the GGUF files): about 600 ms per short sentence and about 1.6 GB RAM on an Intel i5-12400F, fp32, eight threads, batch size one (main model card). The GGUF card does not publish separate latency figures; measure on your own CPU before promising an SLA.

Disk: plan for roughly 300 MB (Q8_0) or 551 MB (BF16) plus Ollama/llama.cpp overhead.

Setup steps (from the GGUF card)

Option A — Ollama

Pull the quantized build:

ollama pull hf.co/halilneed/turkish-pii-detection-GGUF:Q8_0

For deterministic output and the correct stop token, create a named model with the Modelfile from the GGUF repo (keep the Modelfile and .gguf in the same folder):

ollama create turkish-pii -f Modelfile
ollama run turkish-pii "Metindeki tüm kişisel verileri uygun etiketlerle maskele.

Metin: müşteri Ayşe Yılmaz tc 12345678950 tel 0532 111 22 33"
# -> müşteri [AD] tc [TCKN] tel [TEL]

Prompt shape from the card: instruction, blank line, then Metin: …. The instruction sets the policy ("mask everything", "only phone numbers", "keep names", and so on). Policies and the 53-label schema live on the main model card / site page.

Option B — llama.cpp

From the GGUF card:

llama-cli -hf halilneed/turkish-pii-detection-GGUF:Q8_0 --temp 0 -n 512

Use temperature 0 for stable masking output.

Calling it from your app

Python against Ollama's local API (example from the GGUF card):

import json, urllib.request

def maskele(metin, talimat="Metindeki tüm kişisel verileri uygun etiketlerle maskele."):
    body = {
        "model": "turkish-pii",
        "prompt": f"{talimat}\n\nMetin: {metin}",
        "stream": False,
        "options": {"temperature": 0},
    }
    req = urllib.request.Request(
        "http://127.0.0.1:11434/api/generate",
        json.dumps(body).encode(),
        {"Content-Type": "application/json"},
    )
    return json.loads(urllib.request.urlopen(req).read())["response"].strip()

Wire this in before logs, search indexes or any cloud LLM call. Keep return_original-style mapping tables only for your rule layer; the generative model returns masked text, not offsets.

If you still want the original transformers path on the same machine, the main card's CPU example uses torch.float32 without device_map and pins a revision SHA for reproducibility — see /models/turkish-pii-detection/.

Accuracy vs. the full model

Use the card's numbers; do not invent others:

• Q8_0: 0.944 exact match on the 1,000-row benchmark, same headline score as the main model; 98.9% identical outputs on rows the model predicted.
• BF16 GGUF: 0.943; 99.6% identical outputs.
• Weak slices (same story as the main model): suffixed PII, uppercase text; multi-person is where Q8_0 lags.

Exact match on a synthetic benchmark is not entity recall and not a guarantee that your real documents are safe to share. Measure on your own Turkish text before production. Masking alone does not establish irreversible anonymization or KVKK/GDPR compliance; that remains your responsibility. License: Gemma Terms of Use (as stated on the cards).

For choosing which privacy model fits which job, see the first-step guide and the model page above.

FAQ