Redacting logs and tickets before sharing

A typical access log line carries the client IP, a timestamp, the request method and full URL including the query string, the status code, the response size, the referrer and the user agent. The query string alone is frequently enough to identify someone: email addresses used as lookup keys, password reset tokens, order numbers, search terms and, in bad implementations, session identifiers.

Application logs are worse, because they are written by people in a hurry. Stack traces embed request bodies; a debug line dumps a whole user object; an integration logs the provider's response verbatim, card data included. Structured logging helps you find these fields but does nothing to keep them out of the file.

Before sharing, scan for the whole family at once: email addresses, phone numbers, IPv4 and IPv6 addresses, MAC addresses, card numbers, IBANs, UUIDs used as user or session identifiers, JWTs, API keys, coordinates and full URLs. A tool that highlights all of them in the text is faster and far more reliable than reading the file line by line.

A JSON Web Token is three base64url-encoded parts separated by dots. The first two — header and payload — are encoded, not encrypted: anyone can decode them in a second and read the subject, the issuer, the expiry and whatever custom claims you put there, which in practice often means an email address, a user ID and a tenant name. A JWT in a log is both a credential and a personal data leak.

The same applies to provider keys with recognisable prefixes: sk_live_ for Stripe secret keys, ghp_ for GitHub personal access tokens, AKIA for AWS access key IDs, AIza for Google API keys, xoxb- for Slack bot tokens. These prefixes exist precisely so that scanners can spot them — yours and everyone else's.

If a secret has already been in a file that left your machine, redacting the copy you are about to send changes nothing about the copy that already leaked. Rotate the credential first, then share the cleaned file. Treat redaction as hygiene for the future, not as a remedy for the past.

The fastest way to ruin a log file is to replace every sensitive value with the same word. A file where four different customers all became [EMAIL] cannot answer the question the support engineer is actually asking: is this one user retrying, or four users hitting the same bug?

Numbered placeholders solve this. Map each unique value to a stable label — [EMAIL_1], [IP_2], [UUID_3] — and reuse it everywhere that value appears. The structure of the incident survives: you can still count distinct users, follow one session across services and see that the same address appears in the first and the last line.

When the recipient is a parser rather than a person, format-valid fakes work better than labels. A pipeline that expects an IPv4 address chokes on [IP_1] but happily accepts 192.0.2.7 from the RFC 5737 documentation range; the same trick works with 2001:db8:: addresses, example.com email addresses, Luhn-valid test card numbers and IBANs whose check digits have been recomputed. Keep the mapping deterministic so two runs of the same file produce the same output.

A text redactor does nothing for a screenshot. Crop the image to the smallest useful region, then draw opaque shapes over the rest — and export a flattened PNG or JPEG rather than sending the editable file, because layers, shapes and blur can all be undone. Pixelation and blur are not redaction: both have been reversed successfully, most famously in the 2004 case where a swirled face was unwound to identify an offender.

Images also carry metadata. A screenshot from a phone may include the device model and the capture time; a photograph of a screen may include GPS coordinates. Strip EXIF before sending, and remember that office documents keep author names, revision history and comments long after the visible text has been edited.

For PDFs, black rectangles drawn on top of the page leave the text layer intact — the words are still selectable and searchable underneath. Use a tool that removes the content, or export a rasterised copy and accept the loss of quality.

Trim first. Send the twenty lines around the error rather than the whole file; most of the risk is in the part nobody will read. Then scan the excerpt for the full family of identifiers, not just the obvious two, and decide per kind: numbered placeholders for anything the reader needs to correlate, partial masks where the tail matters (the last four digits of a card, the first two octets of an address), removal for everything else.

Check the free-text fields you did not think of: the ticket subject, the file name, the branch name in a stack trace, the hostname in a URL, the company name in a user agent string. Then re-read the cleaned version once as a stranger would, and ask what you could still work out about the customer from it.

With an AI assistant, add one more consideration: unless you are on a contract that says otherwise, assume the text may be retained and may be used to improve the service. That makes the same excerpt a permanent disclosure rather than a temporary one, so the bar for what you paste should be higher, not lower.

Replacing values with placeholders is pseudonymisation. Under Article 4(5) of the GDPR the result is still personal data, and the obligations that come with it — lawful basis, retention limits, the right to erasure — do not disappear because a log line now says [EMAIL_1]. If you keep the mapping table, you keep the ability to re-identify, and so does anyone who gets hold of both files.

Redaction is also not a substitute for logging less. The most reliable way to avoid leaking a field is not to write it to disk: filter secrets at the logging library, log user IDs instead of email addresses, truncate query strings, and set a retention period that actually expires. Cleaning a file before you share it is the last line of defence, not the first.