Turkish BSEBY classifier

Examples

Labels

No label means public information, a general rule, an announcement or a product description.

KVKK is the Turkish data protection law. BSEBY is the Regulation on Banks’ Information Systems and Electronic Banking Services (Official Gazette 15 March 2020, no. 31069). The Secrecy Regulation is the Regulation on the Sharing of Confidential Information (Official Gazette 4 June 2021, no. 31501).

Where I drew the lines

The regulations say what each class is for and do not settle the borderline cases, so I had to draw the lines myself. If your bank’s asset classification guide (BSEBY Article 6/5) draws them differently, evaluate the model against that guide.

• Who owns the authentication data. The regulation says “belonging to the customer”. A text often does not show whether a password belongs to a customer or an employee, so a person’s authentication data is hassas_veri whoever owns it. The password of a system or service account and an API key belong to the institution, not to a person: banka_sirri.
• The value has to be in the text. “müşteri şifresini dolandırıcıya vermiş” (the customer gave their password to a fraudster) is not hassas_veri. Neither is a masked value (****).
• Mother’s maiden name. On its own it is not hassas_veri (BSEBY Article 34/10 bans its use for authentication). It is marked when the text explicitly uses it as an authentication answer.
• Account identifiers. An IBAN, an account number and a full card number are musteri_sirri even when the banking context is not explicit.
• Customers of other businesses. A customer relationship outside banking (e-commerce, telecom, insurance) is personal data, not a customer secret. Debt, instalments and debt enforcement do not count as banking content by themselves.
• Bank secrets. The Secrecy Regulation says only “information that belongs only to the bank” (Article 5/5) and does not define the term further. If the institution is clearly not a bank, the label is not used. If it is unclear, a bank is assumed.
• Legal entities. Information about a customer that is a legal entity is musteri_sirri, not kisisel_veri.

Results

Version 2 (14 October 2026) adds 263 training templates for the weak spots the first version’s page listed: corporate customers, internal information of institutions that are clearly not banks, published bank information, and texts that carry both secrets. Same settings otherwise. v1 stays available as revision="v1".

Both sets are in halilneed/turkish-bseby-classification-benchmark. The held-out set (747 texts) was written on 2 October, before and independently of the v2 templates, so it is the main result. The hard set (202 texts, config hard) targets the same weak spots as the v2 templates; its writer did not see them, but used the same language model and guideline, so read v2’s gain there more cautiously.

On the 747-text held-out set, turkish-bseby-classifier v2 scores 0.960 micro-F1 and 0.952 macro-F1 (v1: 0.944 and 0.939).

“Secret present” means musteri_sirri or banka_sirri. The legal-entity slice is the set of texts whose gold labels include a customer secret and no personal data; a prediction counts only if it gets both right. The template-disjoint test of v2’s own split (1,102 examples) gives micro-F1 0.950.

The texts and both label passes were produced with the same language model (Claude) and the same guideline, and were not verified line by line by a person. Exact agreement was 99.6% on the held-out set and 100% on the hard set. That shows the guideline can be applied consistently; it is not agreement between two independent people.

Python quick start

import json, torch
from huggingface_hub import hf_hub_download
from transformers import AutoModelForSequenceClassification, AutoTokenizer

MODEL = "halilneed/turkish-bseby-classifier"
tok = AutoTokenizer.from_pretrained(MODEL)
model = AutoModelForSequenceClassification.from_pretrained(MODEL).eval()
thresholds = json.load(open(hf_hub_download(MODEL, "thresholds.json")))

def classify(text):
    with torch.inference_mode():
        p = torch.sigmoid(model(**tok(text, return_tensors="pt", truncation=True, max_length=320)).logits)[0]
    return [model.config.id2label[i] for i, v in enumerate(p.tolist()) if v >= thresholds[model.config.id2label[i]]]

print(classify("Demirtepe Yapı A.Ş.'nin nakdi kredi limiti 40 milyon TL'ye çıkarıldı"))
# ['musteri_sirri']

Per-label thresholds were tuned on the dev set and ship in thresholds.json. For the four labels other than kisisel_veri the threshold never goes above 0.5, which favours recall.

Measured speed: 32 ms per text at batch size one on an Intel i5-12400F with eight threads, for texts of 207 characters on average. Text beyond 320 tokens is cut off, so split long documents and take the union of the labels.

Training

• Base model. dbmdz/bert-base-turkish-cased (BERTurk, 110M).
• Method. Multi-label classification, sigmoid with binary cross-entropy, 5 epochs, learning rate 3e-5, bfloat16, up to 320 tokens. The checkpoint was chosen by dev micro-F1. Same settings in v1 and v2.
• Data. 40,000 synthetic examples from 4,786 templates (v1: 4,523; v2 adds 263 for corporate customers, non-bank institutions, published bank information and combined secrets): the 3,276 templates of the KVKK classifier, with personal-data labels derived deterministically and 1,445 of them also marked for banking context, plus 1,247 templates written for this model covering natural- and legal-person customers, internal management and risk, IT and security, authentication data and traps. 16% have no label.
• Split. Train, dev and test are separated by template hash.

Limitations

• The training data and the evaluation sets are synthetic and were produced with a language model. Measure the model on real bank text before it goes into a critical pipeline.
• The weakest label is still hassas_veri (v2: F1 0.881). v2 did not target it: no new templates for token and key formats could be written. In v1, most of what it missed on the held-out set is token and key formats (JWT, private key block, device pairing key, personal access token) and passwords inside a JSON body. Most of its false alarms are masked card numbers and notices such as “sonu 7712 ile biten kartınız” (your card ending in 7712). For values with a fixed format (card number, JWT, key block), use it together with a rule-based check.
• Precision for banka_sirri rose from 0.888 to 0.956 on the held-out set, but is 0.761 on the hard set. In v1 the false alarms fell in three places: technical texts that belong to a corporate customer (a SWIFT MT message, a financial analysis summary), internal information of institutions that are clearly not banks (a factory production report, a foundry’s service account) and published fee schedules. Whether information has been disclosed often depends on a fact outside the text, and the model can only infer it from clues in the text.
• In texts that carry both musteri_sirri and banka_sirri, v2 gets both right in 25 of 27 held-out texts and 27 of 35 hard-set texts (v1: 20 and 18).
• The model looks at one piece of text at a time. A customer secret often arises from a combination of fields (Secrecy Regulation Article 4/4). When you classify tables or records, pass the fields together.
• The labels are a classification signal, not a legal assessment. Compliance with BSEBY and with the duty of secrecy remains the user’s responsibility.

Türkçe BSEBY veri sınıflandırma modeli

turkish-bseby-classifier, Türkçe bir metinde kişisel veri, hassas veri ya da sır kapsamındaki veri (müşteri sırrı, banka sırrı) bulunup bulunmadığını belirleyen çok etiketli bir sınıflandırıcıdır. Bankaların veri envanteri ve veri sızıntısı önleme (DLP) süreçleri için tasarlanmıştır.

Eğitim verisine bakılmadan yazılmış 747 metinlik bağımsız sette v2’de micro-F1 0.960, macro-F1 0.952’dir (v1: 0.944, 0.939); 202 metinlik zor sette 0.933. CPU’da metin başına yaklaşık 32 ms sürer. Etiketler yalnızca bir sınıflandırma tahminidir, hukuki nitelendirme değildir.

Links

• Model card and weights
• Evaluation set
• Turkish PII detection and masking
• Turkish KVKK classifier
• Turkish prompt injection classifier
• Personal data and AI: a short guide for teams on what to mask and what to keep in-house before text reaches a model
• All models