What to Log, How Long to Keep It, and Why It Matters When It's Too Late

When an incident is discovered — often weeks or months after it began — the first question is always the same: what happened, and how far did it go? The only way to answer is with logs. Yet organizations repeatedly discover, at exactly this moment, that the crucial logs were never collected, were overwritten days ago, or are so voluminous and unstructured that finding the signal is hopeless. Getting logging right before an incident is what makes investigation possible after one.

The two failure modes are opposite and equally common: logging too little to reconstruct events, and logging so much that the cost is crushing and the important lines drown.

What to log

Prioritize the events that answer investigative questions. Authentication — every login, success and failure, with source — reveals compromised accounts and brute-force attempts. Privilege changes and administrative actions show what an attacker did once inside. Access to sensitive data shows what was reached. Network connection logs and DNS queries trace movement and command-and-control. Changes to security controls themselves — disabled logging, altered firewall rules — are red flags. And critically, log the creation and deletion of accounts and the modification of logging itself, because covering tracks is a standard attacker move.

The guiding question is: after a breach, what would I need to have recorded to understand it? Log that, deliberately, rather than everything indiscriminately.

How long to keep it

Retention is a balance of investigative need, compliance requirements and cost. The uncomfortable reality is that breaches often go undetected for a long time, so logs retained for only days or a few weeks may be gone before anyone knows to look. Many mature programs keep security-relevant logs for months to a year, with critical logs longer, sometimes driven by regulatory mandates. Storing older logs cheaply in cold storage, while keeping recent ones instantly searchable, controls cost without sacrificing history.

The point is to make retention a conscious decision matched to how long attackers actually hide, not an accident of default settings.

Making logs usable

Collected logs are only useful if you can search and correlate them. Centralizing logs from many systems into one place — the core idea behind a SIEM — lets you follow an attacker across systems instead of piecing together isolated fragments. Structured, consistent formats and a shared timeline (synchronized clocks, correlation identifiers) turn a pile of text into a reconstructable narrative. And protecting the logs themselves — so an attacker who compromises a system cannot simply erase the evidence — preserves the witness.

None of this is glamorous, and its value is invisible right up until the day it becomes the only thing that matters. Decide what to log by what an investigation needs, keep it as long as attackers hide, centralize and protect it — and when the bad day comes, you will have a witness instead of a blind spot.

A practical logging inventory

Before tuning anything, write down what you have. List every system that matters — identity provider, email, cloud accounts, firewalls, VPN, servers, databases, business applications, endpoint protection — and for each one record whether it produces logs, where they go, how long they are kept and who can read them. The inventory almost always reveals surprises: a critical SaaS tool whose audit log is only available on a higher subscription tier, a firewall writing to local disk that rotates every three days, a database with auditing switched off. Fixing those gaps is usually worth more than any new tool.

Sources teams commonly forget

Some of the most valuable evidence sits outside the usual server logs. Identity provider sign-in logs show impossible travel, unfamiliar devices and consent granted to suspicious applications. Cloud control-plane logs record who created a new access key, opened a storage bucket to the internet or launched machines in an unused region. Email logs show forwarding rules quietly created on a compromised mailbox. DNS logs reveal infected machines calling out to attacker infrastructure. Endpoint telemetry records which process started which. If an attacker's path would cross any of these, and it usually does, they belong in the plan.

Structured logs investigators can use

A log line written for a developer debugging at their desk is often useless to an investigator months later. Prefer structured formats such as JSON with consistent field names: timestamp, source host, user or service identity, action, target resource, result and source address. Use the same name for the same thing everywhere, so that a search for one user returns their activity across every system instead of only the ones that happen to call the field "user". Record outcomes explicitly — success or failure — and include stable identifiers rather than display names that change over time.

Time, clocks and correlation

Reconstructing an incident means putting events from many systems on a single timeline, and that only works if clocks agree. Synchronize every system to a reliable time source and store timestamps in UTC with time-zone information, converting to local time only for display. Where requests pass through several services, propagate a correlation or request identifier so one user action can be followed from the load balancer to the database. A few seconds of clock drift can make an attacker's actions appear to happen in the wrong order, which is enough to send an investigation down the wrong path.

Protecting logs from tampering

An intruder with administrative access to a server can edit or delete the logs stored on it. Ship logs off the host quickly to a central store the source system cannot modify. Restrict who can delete or change retention settings in that store, and alert when anyone tries. Write-once or immutable storage options, offered by most cloud providers, make older logs resistant to deletion even by privileged accounts. Treat the logging pipeline as security infrastructure: a gap in log delivery from a critical system is itself an event worth investigating, because silencing the logs is often an attacker's first move.

Controlling cost without creating blind spots

Log volume grows faster than budgets. Rather than cutting sources indiscriminately, separate tiers. Security-relevant events — authentication, privilege changes, administrative actions, access to sensitive data — stay searchable. High-volume, low-value data such as verbose debug output or health checks can be sampled, summarized or kept only briefly. Archive older data to inexpensive storage that can be restored when an investigation needs it. Review which sources consume the most space each quarter; a single misconfigured application logging every request body can cost more than everything else combined.

Privacy and what not to log

Logs are data, and they can become a liability. Never record passwords, full payment card numbers, authentication tokens, session cookies or secret keys, and be careful with request bodies that may contain personal information. Mask or truncate sensitive fields at the source rather than hoping to filter them later. Apply access controls to logs as you would to the data they describe, and include them in your data-protection records and retention schedule. A log store full of customer personal data that everyone in engineering can search is a breach waiting to happen in its own right.

Detection rules that earn their keep

Collecting logs is the foundation; detection makes them work for you before an incident is discovered by someone else. Start with a small set of high-value rules: many failed logins followed by a success, sign-ins from new countries for privileged accounts, new administrator accounts, disabled security tools, mailbox forwarding rules to external addresses, access keys created outside normal change windows and large data exports. Each rule should state what it detects, why it matters and what the responder should check first. A dozen rules that are understood and acted upon beat hundreds that nobody reads.

Fighting alert fatigue

When every alert is urgent, none are. Review alerts regularly and ask of each one whether it led to action. Tune or retire rules that only produce false positives, add context such as asset owner and user role so triage is faster, and group related events into a single case rather than fifty notifications. Route alerts by severity: genuinely critical signals should wake someone up, while lower-priority findings can wait for a daily review. Measuring how long alerts wait before someone looks at them is a simple, honest indicator of whether the system is working.

A retention policy you can defend

Write the policy down. For each category of log, state how long it is kept in searchable form, how long in archive, who approved the decision and which legal or contractual requirements apply. Typical patterns keep authentication and administrative logs for at least a year, with recent months instantly searchable, while noisy operational logs are kept for weeks. Whatever you choose, the reasoning should be explicit, reviewed periodically and consistent with your privacy commitments, so that you can explain it to an auditor, a regulator or a customer.

Test before you need them

Logging that has never been tested usually fails when it matters. Run a simple exercise: pick a realistic scenario, such as a stolen password used to sign in and download files, and try to reconstruct it from your logs alone. Can you see the sign-in, the device, the files accessed and where they went? Can you do it for an event from three months ago? Gaps found in an exercise are cheap to fix; gaps discovered during a real incident are expensive and sometimes permanent. Fold these checks into your regular tabletop exercises.

A starter plan for small teams

A small organization does not need an expensive platform on day one. Turn on the audit logs your identity provider, email and cloud services already offer, extend their retention to the maximum your plan allows, and forward them to one central place. Add server and firewall logs next. Create five or six alerts for the most dangerous events and make sure someone reads them. Revisit the setup twice a year. This modest foundation answers most investigative questions and can grow into a full SIEM as the organization does.