PII redaction in transcripts

PII redaction removes personal data such as names, card numbers, dates of birth and addresses from transcripts and, optionally, from the recordings themselves. For speech it is a two-stage problem: the recogniser must transcribe the sensitive item correctly before a detector can find it, and the detector's output must then be mapped back onto the audio with word timestamps. Errors in either stage leak data, so redaction should be measured on recall per entity type, not assumed.

What gets redacted#

GDPR's definition of personal data in Article 4(1) is broad: any information relating to an identified or identifiable person. Redaction systems work with a narrower, practical list of entity types. Typical categories are:

• Direct identifiers: names, email addresses, phone numbers, national identification numbers, customer or policy numbers.
• Financial data: payment card numbers, security codes, IBANs and account numbers.
• Location and dates: street addresses, postcodes, dates of birth.
• Special categories: health conditions, medication, religious or political references. These fall under GDPR Article 9 and are much harder to detect by pattern.

Payment card data has its own regime. The PCI Data Security Standard prohibits storing sensitive authentication data, including card security codes, after authorisation, even in encrypted form. A contact centre that records calls where customers read out card details must keep those details out of recordings and transcripts, which is one of the most common reasons redaction is deployed. See contact centre speech analytics.

How a redaction pipeline works#

Detection#

Two families of detector are usually combined:

• Rules and validators for structured items. A regular expression finds a 16-digit sequence; the Luhn checksum confirms it could be a card number. IBANs have a mod-97 check digit defined in ISO 13616. Rules are precise and explainable but only as good as the text they receive.
• Named-entity recognition for names, organisations and locations. Transformer-based taggers, or general language models prompted to find personal data, handle open-ended categories that rules cannot. They are less predictable and need evaluation on your domain.

Open-source frameworks such as Microsoft Presidio package both approaches, with configurable recognisers and anonymisation operators.

import re

def luhn_valid(digits: str) -> bool:
    total = 0
    for i, d in enumerate(reversed(digits)):
        n = int(d)
        if i % 2 == 1:
            n = n * 2 - 9 if n > 4 else n * 2
        total += n
    return total % 10 == 0

def redact_cards(text: str) -> str:
    def replace(m):
        digits = re.sub(r"\D", "", m.group())
        return "[CARD]" if luhn_valid(digits) else m.group()
    return re.sub(r"\b\d(?:[ -]?\d){12,18}\b", replace, text)

print(redact_cards("my card is 4111 1111 1111 1111 thanks"))
# my card is [CARD] thanks

Audio redaction#

Once a token is marked for redaction, its start and end times locate it in the recording, and that span is replaced with silence or a tone. Timestamps from end-to-end models can be off by tens of milliseconds, and the first syllable of a number may begin before the recognised word does, so pad each span on both sides. Where a whole passage is sensitive, for example a customer reading out card details, mute from the first to the last matched token rather than word by word.

Why speech is harder than text#

Redaction tools designed for documents assume the sensitive item is written correctly. In a transcript it often is not.

• Spoken numbers. Callers say four one one one, one one one one, pause, and continue. Whether the transcript contains "4111 1111" or "four one one one" depends on inverse text normalisation, and a digit-based rule will not match words. See numbers and alphanumerics.
• Recognition errors. If one digit is misheard, a Luhn check fails and the rule declines to redact a real card number. If a name is transcribed as a common word, Mrs de Vries becoming Mrs the freeze, an entity model may not tag it.
• Split across turns. A postcode spoken in two parts, with the agent confirming in between, will not match a pattern applied utterance by utterance. Run detection over a window of the conversation, not single segments.
• Spelled-out identifiers. Letters read aloud, such as "D for David", need handling before patterns apply; see structured identifiers.
• Streaming. With partial results, text shown live may reveal digits that a later final result would have redacted. Redact before display, or show only finals.

Measuring redaction quality#

Treat redaction as a detection task and score it per entity type. Recall, the share of real items found, measures leakage. Precision, the share of redactions that were real items, measures damage to the transcript's usefulness. The two errors have different costs: a missed card number is a compliance incident, while an over-redacted product name degrades analytics. Most teams tune for high recall on structured financial data and accept lower precision there.

Build the evaluation set from real calls, labelled by people, and include the hard cases: numbers split across turns, spelled names, accented speech and cross-talk. Aggregate word error rate says little here, because a transcript can have a low WER and still get every account number wrong; entity accuracy is the relevant upstream metric.

Redaction, pseudonymisation and anonymisation#

GDPR distinguishes pseudonymised data, which can still be linked to a person with additional information (Article 4(5)), from anonymous data, which falls outside the Regulation altogether (Recital 26). Redacted transcripts rarely reach the anonymous standard. Context such as a unique complaint, a job title or a rare condition can identify a speaker without any name present, and the Article 29 Working Party's opinion on anonymisation techniques sets a high bar for irreversibility.

Audio is harder still. Muting words removes what was said, not who said it: the remaining voice can identify the speaker, and may be biometric data if processed for that purpose. Treat redacted recordings as personal data and apply retention limits accordingly.

How Reson8 approaches this#

Redaction depends on getting identifiers right in the first place. Reson8's Patterns feature lets a developer declare an expected identifier format, such as INV[0-9]{3}, and biases recognition toward it; on order numbers this reduced WER from 8.0% to 1.5%. Accurate identifiers make downstream rules and checksums more reliable. Reson8 does not retain audio, which limits how many copies of unredacted recordings exist; see the Patterns post and documentation.

Common questions#

This article is general information, not legal advice.

References#

1. Regulation (EU) 2016/679 (General Data Protection Regulation), Articles 4 and 9, Recital 26. EUR-Lex.
2. Article 29 Data Protection Working Party (2014). Opinion 05/2014 on Anonymisation Techniques (WP216).
3. PCI Security Standards Council (2022). Payment Card Industry Data Security Standard, version 4.0. PCI SSC.
4. Microsoft Presidio: data protection and de-identification SDK. GitHub.
5. Luhn, H. P. (1960). Computer for Verifying Numbers. US Patent 2,950,048.
6. ISO 13616-1:2020. Financial services: International bank account number (IBAN).