pplx-pii-masking-GGUF
GGUF packaging of
perplexity-ai/pplx-pii-masking,
a PII masking model for conversational data. The model uses a ~600M-parameter
bidirectional Qwen3 encoder with a
perplexity-ai/pplx-embed-v1-0.6b
backbone and two heads:
- Token classification head (1024 → 37): BIOES tags over 9 PII categories
(
private_person, account_number, private_url, private_date,
private_address, private_email, private_phone, other_pii, secret),
decoded with a constrained Viterbi decoder.
- Sensitivity head (1024 → 1): conversation-level sensitivity classifier
on mean-pooled hidden states.
The backbone is provided as an f16 GGUF for llama.cpp. Both heads remain in
fp32 in heads.safetensors. The included Python client applies the heads to
per-token hidden states from llama-server, decodes PII spans, and masks text.
Usage
Install a compatible llama-server build and the Python client dependencies,
then download the repository:
Keep the server running. In a second terminal, enter the downloaded model
directory and run:
The client prints sensitivity, PII spans, and masked text. It includes the
constrained BIOES Viterbi decoder and reads the label list and biases from
pii_head_config.json.
Checkpoint layout and inference
The GGUF stores qwen3.attention.causal = false for bidirectional attention.
Use --pooling none to obtain one 1024-dimensional hidden state per token.
The classification and sensitivity heads are applied separately by the client.
Use llama-server's native /embeddings endpoint for these unnormalized
per-token hidden states:
The client computes token_logits = h @ W_cls.T + b_cls, then decodes the
37 BIOES labels with constrained Viterbi. Sequence sensitivity is
sigmoid(mean_pool(h) @ W_sen.T + b_sen).
Tokenization and character offsets use the supplied tokenizer files.
Limitations
- Inputs must contain between 1 and 4096 tokens. Chunk longer documents
before calling the example client; it sends the entire supplied text to
the server while its local offset tokenizer truncates at 4096 tokens.
- Bidirectional attention requires the whole input in one micro-batch.
Keep
-b and -ub at 4096 for the supported input window.
/v1/embeddings does not support --pooling none; use /embeddings.
- Conversion precision and the inference backend can affect numerical
outputs. Evaluate detection quality and sensitivity thresholds on your
target data; false positives and missed PII are possible.
Files
License
MIT, matching the base model. See LICENSE [blocked].