Data Retention

Version 1.1 · Effective 2026-05-05

# Data Retention Policy

**Living SR&MA Platform**

> **Version**: 1.1
> **Last Updated**: 2026-05-05
> **Pending version**: 1.2 — see the Change Log in §5. The published version number is bumped together with the other legal documents in the Paddle production bundle (`docs/legal-launch-checklist.md` §H), so `_LEGAL_SYSTEM_PAGES` still registers 1.1 / 2026-05-05.
> **Scope**: All personal data and user-generated content held by the Living SR&MA Platform.
> **Regulatory basis**: GDPR Article 5(1)(e) (storage limitation), GDPR Article 17 (right to erasure), and APPI Article 22 (the "endeavor" obligation to delete personal information that is no longer necessary).

This policy defines, by data category, how long the Platform retains data, what triggers deletion, and the legal basis for any extended retention. All retention windows follow the **principle of minimization**.

---

## 1. Core Principles

1. **Minimization**: Each data category is retained only for as long as it is necessary to fulfill the purpose for which it was processed.
2. **Explicit deadlines**: Every category has either a concrete retention window or a stated event that ends retention (for example, deletion of the project it belongs to). "Indefinite," with no stated end at all, is not used.
3. **Automated deletion**: Whenever feasible, deletion is performed by a scheduled cron job rather than ad hoc.
4. **Audit-trail exception**: Records that are subject to evidential or audit obligations are anonymized rather than physically deleted, so the audit trail remains intact while the personal data is unlinked.
5. **User request takes priority**: Even if a category's retention window has not expired, a valid GDPR Article 17 erasure request from the data subject overrides this policy and is honored — subject only to the narrow Article 17(3) carve-outs.

---

## 2. Retention by Data Category

### 2.1 Account Data (`users` table)

| Data | Retention Window | Deletion Trigger | Deletion Method |
|---|---|---|---|
| `email` / `display_name` / `given_name` / `family_name` / `orcid_id` | While the account is active | Account deletion request + 14-day grace period | Anonymize (replace with `_deleted_<hash>`) |
| Profile picture URL (from Google OAuth) | Same as above | Same as above | Same as above |
| `last_login_at` | While the account is active | Same as above | Set to `NULL` |
| `tos_accepted_version` / `tos_accepted_at` | Account lifetime + 7 years (contract-formation evidence) | 7 years after account deletion | Physical delete permitted |
| `signup_type` / `is_honorary` | While the account is active | Account deletion | Physical delete permitted |

### 2.2 Project Content

| Data | Retention Window | Deletion Trigger |
|---|---|---|
| `refs` (bibliographic metadata) | While the project is active and until the owner archives or deletes it | Physical delete on project deletion |
| `decisions` (screening verdicts) | Same as above | Same as above. On a self-deletion request from a single member, only `user_id` is anonymized; the verdict body is preserved for audit-trail integrity. |
| `extractions` (extracted values) | Same as above | Same as above |
| `comments` / `reconciliation_comments` | Same as above | On a self-deletion request, `author_id` is anonymized but the comment body is preserved so that the rest of the team's discussion remains coherent. (Article 17(3)(e) — establishing, exercising, or defending legal claims, plus the legitimate interest of other team members in continuity of record.) |
| PDFs stored in the user's own BYOK Supabase project | While the project is active, subject to the project owner's own Supabase configuration | Deletion is controlled by the project owner in the user's own Supabase project. The Platform does not store PDFs in Operator-controlled storage. |

### 2.3 Communications

| Data | Retention Window | Deletion Trigger |
|---|---|---|
| `email_screening_batches` (invitation tokens) | 30 days after the batch completes | Daily retention cron (§3.3) |
| `token_hash` | Same as above | Same as above |
| Outgoing email log (if retained) | 30 days | Daily cron |
| Support email (if retained) | 3 years (to be redefined when Phase B introduces a formal support ticket system) | n/a today |

**Retention clock for `email_screening_batches`**: the cron measures the 30 days from `completed_at`. Batches that are never completed (status `pending` or `expired`) have no `completed_at`, so for those rows the clock runs from `created_at` instead — otherwise an unused invitation token would be retained indefinitely. In SQL terms the predicate is `COALESCE(NULLIF(completed_at, ''), created_at) < now() - 30 days`.

### 2.4 Audit Trail

The Platform keeps two kinds of audit trail, and they are treated differently. **Operational and security records** identify who performed an administrative action, so the personal data in them expires. **Review-decision records** are the substance of a systematic review and are kept for as long as the project exists.

Earlier versions of this policy referred to a table called `audit_log`. No such table exists; the operational audit trail is stored in `admin_action_log` and `document_audit_log`. The table names below are the real ones.

#### 2.4.1 Operational and security records (`admin_action_log`, `document_audit_log`)

| Data | Retention Window | What Happens at Expiry | Legal Basis |
|---|---|---|---|
| `user_id` (actor) | **2 years** | Replaced with the fixed value `_deleted_retention`, which cannot be traced back to a person | GDPR Article 5(1)(e) (storage limitation) and Article 17 |
| IP address / user agent (`admin_action_log` only) | **2 years** (security audit requirement) | Set to `NULL` | Same as above |
| Action body, target ID, timestamp, `details` | Retained beyond 2 years | Unchanged — the record of *what happened* survives; only *who* is removed | GDPR Article 17(3)(e) (legal claims) and the APPI Article 22 best-effort obligation |

**Deletion is anonymization, not physical deletion.** These tables are append-only: database triggers reject every `DELETE` and reject any `UPDATE` other than the anonymization above (`trg_admin_action_log_immutable` / `trg_admin_action_log_no_delete` / `trg_docaudit_immutable` / `trg_docaudit_no_delete` in SQLite; `admin_action_log_block_update` / `admin_action_log_block_delete` in Postgres). Keeping the row but stripping the actor satisfies storage limitation while preserving a tamper-evident security trail. The daily retention cron (§3.3) performs the anonymization; it is idempotent, so a row that has already been anonymized is not revisited.

The same anonymization is applied immediately — without waiting for the 2-year window — when the actor's own account deletion completes (§3.1). In that case `user_id` is replaced with `_deleted_<hash>` rather than the retention sentinel, so the row still cannot be traced to a person.

**Rationale for the 2-year window**: The Platform is not subject to U.S. SOX (which requires 7 years) or to GCP-equivalent clinical-trial retention (which requires 10 years). A 2-year window therefore strikes a reasonable balance between the need to preserve evidence for security and dispute purposes and GDPR's storage-limitation principle. In the event of a serious incident, the relevant rows are placed under a separate legal hold so that the routine cron does not anonymize them.

#### 2.4.2 Review-decision records (`decision_amendments`, `extraction_edit_history`)

| Data | Retention Window | What Happens at Expiry | Legal Basis |
|---|---|---|---|
| Amendment body, previous and new values, timestamps | **For the lifetime of the project** — no time-based expiry | Nothing; deleted only when the project itself is deleted (§2.2) | GDPR Article 17(3)(e) (establishing, exercising, or defending legal claims) and Article 6(1)(f) (legitimate interests) |
| `changed_by` / `amended_by` (actor) | Same as above | Nothing. On the actor's own account deletion the identifier is replaced platform-wide, so the row no longer points at a named person | Same as above |

**Why these are not expired.** A systematic review is only reproducible if the provenance of every screening and extraction decision survives: which value was corrected, from what, when, and in what order. That chain is what a journal, a peer reviewer, or a court would examine if the review's conclusions were challenged, and it is the record the co-authors themselves rely on. Deleting it after two years would make published reviews unverifiable. These tables are therefore append-only in the strict sense — both `DELETE` and `UPDATE` are rejected by database triggers (`trg_amendments_no_delete` / `trg_edit_history_no_delete`; `migrations/pg/0044`) — and the retention cron does not target them at all.

The personal data they contain is limited to a user identifier, and that identifier is already replaced when the user's account is deleted (§3.1), so an erasure request does not leave a named individual attached to the record.

#### 2.4.3 Status transitions (`ref_status_log`)

| Data | Retention Window | What Happens at Expiry |
|---|---|---|
| Reference ID, previous status, new status, timestamp, actor (`changed_by`) | **2 years** | The whole row — actor included — is physically deleted by the daily retention cron (§3.3) |

This table records the state transitions of a bibliographic record. Since 2026-07-27 it also records the actor who caused each transition (`changed_by`: the acting user's identifier, or the fixed value `system` for automated paths; rows written before that date carry no actor). No separate anonymization step is needed: unlike the operational records in §2.4.1, whose rows are kept and merely stripped of the actor, here the entire row is physically deleted after 2 years, which removes the actor identifier along with everything else. During the retention window the actor column is treated like the identifiers in §2.4.2: it contains a user identifier only, and when the actor's own account is deleted (§3.1) the user record that identifier refers to is anonymized, so the remaining value no longer points at a named person.

#### 2.4.4 Search run log (`search_log`)

| Data | Retention Window | What Happens at Expiry |
|---|---|---|
| Saved-search reference, run timestamp, hit counts, run status | **For the lifetime of the project** — no time-based expiry | Nothing; deleted only when the project itself is physically deleted (§2.2) |

This table records each execution of a project's saved literature-monitoring searches: which saved search ran, when, how many records it returned, and whether the run succeeded. It contains no personal data — only a query reference, a timestamp, hit counts, and a run status. It is the PRISMA-S search provenance for living reviews: the evidence of when each search was run and what it yielded, per update round, which is what makes a published search strategy verifiable. It is therefore retained for the lifetime of the project and deleted only as part of the project's own physical deletion; the retention cron (§3.3) does not target it. Since the saved searches themselves became soft-deleted rather than physically deleted (PR #1318), deleting a saved search no longer removes its run log either.

### 2.5 Disabled AI Usage Metadata and System Logs

| Data | Retention Window | Deletion Trigger |
|---|---|---|
| `llm_usage_log` (model name, token count, project ID, user ID) | **90 days** | Daily retention cron (§3.3) |
| Application logs (Fly.io) | Per Fly.io defaults (approximately 30 days) | Auto-rotated by Fly.io |
| Error and exception logs | 90 days (no separate aggregation today) | Same as above |

**Current implementation status**: the `llm_usage_log` table **does not exist** in either the SQLite or the Postgres schema. It was specified for the BYOK LLM feature (`specs/archive/byok-llm-a-infra.md`), which was deferred and is not part of Phase 2, so no AI usage metadata is collected at all today. Because there is nothing to expire, the retention cron does not carry an entry for it; the 90-day window above becomes operative only if the feature ships, at which point the table is added to the cron's target list.

### 2.6 BYOK Secrets (`project_secrets`)

| Data | Retention Window | Deletion Trigger |
|---|---|---|
| Disabled or future BYOK AI provider keys, if any are registered | While the user or project owner keeps the key registered | (a) Explicit revocation by the project owner, **or** (b) account-deletion request — wiped immediately, without waiting for the 14-day grace period |
| Supabase BYOK configuration (`project_supabase_config`) | While the project is active | Project deletion |

**Rationale for immediate wipe**: API keys are high-risk credentials in the sense of GDPR Article 4(12). To minimize the window during which a compromised credential could be misused, the keys are decrypted and shredded as soon as a deletion request is received, without waiting out the 14-day grace period. If the user later cancels their deletion request, they will be asked to re-register the keys.

### 2.7 Backups

| Data | Retention Window | Notes |
|---|---|---|
| Production database logical backups | Rolling 30-day rotation in Cloudflare R2 | Backups are created with `pg_dump -Fc`, stored in Cloudflare R2 under the default jurisdiction (global edge, region `auto`), and protected by Cloudflare server-side encryption at rest. Backup objects older than 30 days are automatically deleted. |

**Erasure requests vs. backups**: Strictly excising a single user's records from existing backup snapshots is technically infeasible. Anonymized or deleted data may remain in backup objects until the rolling 30-day rotation deletes the relevant backup. The Platform does not selectively erase individual records from existing backup snapshots and does not restore deleted or anonymized data from backups except where necessary for disaster recovery. If a disaster recovery restore occurs, any completed deletion or anonymization requests are reapplied where technically feasible before normal service resumes.

---

## 3. Technical Implementation of Deletion

### 3.1 14-Day Account-Deletion Grace Period

```
Day 0:   The user calls /api/account/delete-request.
         - users.deletion_requested_at = now()
         - users.deletion_scheduled_at = now() + 14 days
         - All API keys in project_secrets are wiped immediately.

Day 0–13: Grace period (the user can call /api/account/delete-cancel).
         - A non-dismissible warning banner is shown on every page.

Day 14:  A scheduled cron job runs hard_delete().
         - Anonymizes users / project_members / comments /
           admin_action_log / document_audit_log / kb_newsletter_sends.
         - In admin_action_log the actor's IP address and user agent are
           set to NULL; in every table the user identifier is replaced
           with _deleted_<hash>.
         - Personal data is no longer recoverable.
```

The bodies of the records themselves are not deleted (see §2.2 and §2.4): what survives is the action, the decision, or the comment, with the person detached from it.

### 3.2 Cron Design

- **Frequency**: Daily at 18:00 UTC, which is 03:00 the following day in Tokyo (JST = UTC+9).
- **Targets**: Rows where `deletion_scheduled_at <= now() AND deletion_scheduled_at IS NOT NULL`.
- **Execution environment**: Cloudflare Workers scheduled job (`cloudflare/subscription-grace`, cron expression `0 18 * * *`). The Cloudflare Free plan allows only five cron triggers per account, so a single daily worker calls each admin endpoint in sequence rather than each job having its own trigger.
- **Entry point**: `POST /api/admin/cron/hard-delete`, authenticated with the `X-Monitor-Secret` header.
- **Default mode**: Dry run; the `--execute` flag (CLI) or `?dry_run=false` (HTTP) is required to perform writes.
- **Failure handling**: Sentry or Supabase alerts are planned. Until those are in place, the cron emails the operator on failure. A failure in one job does not prevent the remaining jobs in the sequence from running.

### 3.3 Retention Cron (audit logs and invitation tokens)

- **Frequency**: Daily at 18:00 UTC = 03:00 the following day in Tokyo. It runs from the same `cloudflare/subscription-grace` worker described in §3.2, as one step in that worker's sequence — no separate cron trigger exists, because of the five-trigger Free-plan limit.
- **Entry point**: `POST /api/admin/cron/retention-purge`, authenticated with the `X-Monitor-Secret` header. The equivalent CLI is `python -m app.jobs.retention_purge [--execute]`.
- **Implementation**: `app/jobs/retention_purge.py`.
- **Targets**:

  | Table | Window | Method |
  |---|---|---|
  | `ref_status_log` (§2.4.3) | 2 years | Physical delete |
  | `email_screening_batches` (§2.3) | 30 days | Physical delete |
  | `admin_action_log` (§2.4.1) | 2 years | Anonymize `user_id`, `ip_address`, `user_agent` |
  | `document_audit_log` (§2.4.1) | 2 years | Anonymize `user_id` |

  `decision_amendments`, `extraction_edit_history`, and `search_log` are deliberately absent: per §2.4.2 the first two are retained for the lifetime of the project, and per §2.4.4 the search run log is likewise retained until the project itself is physically deleted.
- **Portability**: The cutoff timestamp is computed in Python and passed as a bound parameter, so the same predicate works on both SQLite and Postgres; no `interval '2 years'` literals are used.
- **Default mode**: Dry run. Without `--execute` (CLI) or with `?dry_run=true` (HTTP), the job only counts the rows that are past their window and writes nothing.
- **Robustness**: Tables that are absent from the schema are reported as `skipped` rather than raising, and a failure on one table does not stop the others. Anonymization is idempotent: rows already carrying the sentinel are excluded from the next run.
- **Provenance**: Every real execution appends a `cron.retention_purge.executed` entry, with the per-table counts, to `admin_action_log`.

### 3.4 Project Physical Deletion (Article 17)

When a project owner archives a project, the row is soft-deleted (`projects.deleted_at`); a cron physically deletes the project and everything scoped to it after the 30-day grace period (`_hard_delete_project` in `app/routers/projects/core.py`). The owner decision of 2026-08-20 governs what that deletion covers:

- **Data that directly identifies a person is physically deleted.** This includes the invitation contact addresses (`invitations`), the in-app help search log (`help_search_logs`), task assignments (`task_assignments`), newsletter recipient lists (`kb_newsletter_sends`), and every other table scoped to the project. The tables covered are enumerated in `PROJECT_SCOPED_DELETE_ORDER`, and a schema-driven test (`tests/test_project_purge_completeness.py`) fails whenever a newly added project-scoped table is left out of that list.
- **Review-decision correction history is retained in full, free text included.** The rows in `decision_amendments` survive project deletion together with the reviewer's own wording in `prev_reason`, consistent with §2.4.2. Deleting the underlying `decisions` row detaches the amendment (`decision_id` becomes `NULL`), so what remains is the correction itself rather than a link into deleted content.
- **Retaining `user_id` is acceptable.** Account deletion (§3.1) anonymizes the `users` row itself while keeping `users.id`, so a retained identifier no longer resolves to a named person.
- **`admin_action_log` rows are kept and anonymized rather than deleted.** The append-only trigger on that table refuses physical deletion, so project deletion clears the actor's `ip_address` and `user_agent` while leaving the record of the action in place — the same treatment the retention cron applies at two years (§2.4.1).

---

## 4. Conflicts Between User Erasure Requests and This Policy

| Situation | Priority |
|---|---|
| The user requests erasure during the retention window. | The user's request takes priority; the policy's deadline is brought forward. |
| The user requests erasure after the retention window has expired. | The data has already been deleted; the response is `status: already_deleted`. |
| The user asks not to wait the 14-day grace period. | If the reason is reasonable, immediate execution is permitted (operational discretion within the policy). |
| Article 17(3) carve-out is invoked (for example, ongoing legal-claims defense). | The user is informed in writing of the specific carve-out being invoked. Only the data covered by the carve-out is retained; everything else is deleted. |

---

## 5. Change Log

| Date | Version | Change |
|---|---|---|
| 2026-04-22 | 0.1 | Initial draft (created together with `gdpr-compliance.md` Phase A). |
| 2026-04-26 | 1.0 | Promoted to 1.0 alongside the start of Phase A implementation. The body is substantially unchanged from the 0.1 draft. |
| 2026-05-05 | 1.1 | Translated to American English in full (Bundle D, `specs/micro-feedback-bundle-20260505.md` §6.9). No retention-policy changes. |
| 2026-07-25 | 1.2 (pending publication) | Implemented the retention cron (`app/jobs/retention_purge.py` + `POST /api/admin/cron/retention-purge`, wired into the existing daily Cloudflare worker). Rewrote §2.4 around the real table names: operational records (`admin_action_log` / `document_audit_log`) are **anonymized** at 2 years rather than deleted, review-decision records (`decision_amendments` / `extraction_edit_history`) are retained for the project's lifetime under Article 17(3)(e) and legitimate interests, and `ref_status_log` is physically deleted. Corrected §3.2/§3.3 to the actual schedule (daily 18:00 UTC = 03:00 JST, not weekly 02:00 UTC) and fixed the incorrect UTC-to-Tokyo conversion. Noted that `llm_usage_log` does not exist and which timestamp starts the `email_screening_batches` clock. The published version number is bumped with the rest of the legal bundle at the Paddle production flip. |
| 2026-07-28 | 1.2 (pending publication) | Added §2.4.4: the search run log (`search_log`) is retained for the lifetime of the project rather than expired on a timer — it contains no personal data and constitutes the PRISMA-S search provenance for living reviews — so the retention cron deliberately does not target it. Project physical deletion (§2.2) now also removes the project's saved searches (`search_queries`) and their run log (`search_log`), making that the sole deletion path for both tables. |
| 2026-07-28 | 1.2 (pending publication) | Updated §2.4.3: `ref_status_log` now records the actor of each status transition in a `changed_by` column (added 2026-07-27, migration SQLite v2365 / PG 0141, with every write path recording either the acting user's identifier or `system`), so the section no longer claims the table carries no actor column. The 2-year physical delete by the retention cron is unchanged and needs no separate anonymization step, because the whole row — actor included — is removed; until then the actor identifier is handled like the identifiers in §2.4.2 (it stops resolving to a named person once the actor's account deletion completes). |
| 2026-08-20 | 1.2 (pending publication) | Added §3.4: project physical deletion now covers every project-scoped table rather than the subset it previously reached. Recorded the owner decision of 2026-08-20 — directly identifying data is physically deleted, review-decision correction history (`decision_amendments`) is retained free text and all, retaining `user_id` is acceptable because account deletion anonymizes the `users` row itself, and `admin_action_log` rows are retained with `ip_address` / `user_agent` cleared. A schema-driven test now fails whenever a newly added project-scoped table is left out of the deletion list. |