> For the complete documentation index, see [llms.txt](https://docs.catalyx.solutions/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.catalyx.solutions/catalyx-blockchain-manager/canton-network/version-1.11/validator-management/data-retention-and-pruning-policy.md).
# Data Retention & Pruning Policy
## Overview
CatalyX Blockchain Manager provides automated backup, recovery, data retention, and pruning capabilities to help organizations protect their blockchain infrastructure, optimize storage usage, and meet operational and regulatory requirements.
These capabilities are designed to:
* Protect validator and participant data through scheduled backups
* Preserve validator identities and cryptographic material
* Enable disaster recovery
* Optimize storage by removing obsolete data
* Maintain security and audit logs
* Support enterprise data lifecycle management
Although these concepts are closely related, they serve different purposes:
| Feature | Purpose |
| ------------------ | ---------------------------------------------------------------------- |
| **Data Retention** | Defines how long operational logs and audit information are preserved. |
| **Backups** | Protect validator and ledger data for recovery. |
| **Identity Dumps** | Preserve validator identities and cryptographic material. |
| **Pruning** | Removes obsolete ledger data that is no longer required. |
{% hint style="warning" %}
A complete disaster recovery strategy requires **both** scheduled database backups and scheduled Identity Dumps (IDDumps). These backup types protect different components of a validator and cannot replace one another.
{% endhint %}
***
## Data Retention
### Security and Audit Logging
CatalyX Blockchain Manager centralizes security and operational logs using Azure Log Analytics.
Default log retention periods:
| Environment | Log Retention |
| -------------------- | ------------- |
| Mainnet (Production) | 30 days |
| Devnet / Testnet | 7 days |
These logs support:
* Security investigations
* Operational troubleshooting
* Compliance audits
* Incident response
* Infrastructure monitoring
The centralized logging solution includes:
* Azure Activity Logs
* Kubernetes audit logs
* Kubernetes API server logs
* PostgreSQL audit logs (pgaudit)
* Network flow logs
* Microsoft Sentinel integration
* Microsoft Defender integration
### Infrastructure Audit Trail
CatalyX Blockchain Manager maintains an audit trail for infrastructure and operational activities, including:
* Resource creation and deletion
* Infrastructure configuration changes
* Azure role assignments
* Policy modifications
* Kubernetes API activity
* Database authentication
* Database query execution
This information supports operational monitoring, forensic investigations, and compliance reporting.
***
## Backup Strategy
Scheduled backups and Identity Dumps are the foundation of CatalyX's disaster recovery approach for validator infrastructure.
| Backup Type | Includes | Does Not Include | Recommended Usage |
| ------------------------------------------------------- | -------------------------------------------------------------------- | ------------------------------------------------------- | --------------------------- |
| Identity Dump (IDDump) | Validator cryptographic keys, validator identity, wallet credentials | Database contents, ledger state, transaction history | Validator identity recovery |
| Database Backup (PVC/PV Backup) | Validator database, participant database, ledger state | Validator identities and cryptographic keys | Operational recovery |
| External Storage Backup (S3 or other supported storage) | Database dumps and snapshots | Runtime state and identities unless explicitly exported | Long-term disaster recovery |
{% hint style="info" %}
For scheduling database backups and Identity Dumps, viewing backup history, and restoring a validator, see [Validator Backups and Identity Dumps](/catalyx-blockchain-manager/canton-network/version-1.11/validator-management/identity-backup-and-recovery.md). That page covers the full configuration and restore workflow — this page focuses on retention strategy, pruning, and compliance considerations.
{% endhint %}
### Recommended Backup Schedule
| Environment | Database Backup | Identity Dump |
| ----------- | --------------- | ------------- |
| Development | Daily | Weekly |
| Staging | Every 12 hours | Daily |
| Production | Every 6 hours | Daily |
Recommended retention values are documented in [Validator Backups and Identity Dumps](/catalyx-blockchain-manager/canton-network/version-1.11/validator-management/identity-backup-and-recovery.md#best-practices). Organizations should ultimately configure retention according to their own operational and compliance requirements.
***
## Pruning
### What is Pruning?
Pruning permanently removes obsolete ledger data that is no longer required for normal ledger operation.
Unlike backups, pruning is intended to optimize storage rather than provide recovery.
Pruning helps:
* Reduce database size
* Improve storage efficiency
* Remove obsolete ledger history
* Support organizational data lifecycle policies
* Assist with regulatory requirements such as GDPR "right to erasure," where applicable
{% hint style="info" %}
Pruning does not affect active ledger state.
{% endhint %}
### Supported Pruning Types
CatalyX Blockchain Manager supports scheduled pruning for:
* Domain nodes
* Participant nodes
{% hint style="warning" %}
Pruning is supported only for Enterprise deployments using PostgreSQL.
{% endhint %}
### Configuring Pruning
Pruning schedules are configured directly from the **Domain** or **Participant** management pages (see [Domains](/catalyx-blockchain-manager/canton-network/version-1.11/network-and-node-management/domains.md) and [Participants](/catalyx-blockchain-manager/canton-network/version-1.11/network-and-node-management/participants.md)).
| Parameter | Description |
| ---------------- | ------------------------------------------------------------------------ |
| Cron Expression | Defines how frequently pruning is executed |
| Retention Time | Specifies how long data is retained before becoming eligible for pruning |
| Maximum Duration | Maximum execution time allowed for a pruning operation |
Example configuration (for demonstration purposes only — production environments should set retention according to operational and regulatory requirements):
| Setting | Example Value |
| ---------------- | ------------- |
| Cron Expression | `*/1 * * * *` |
| Retention Time | `1 second` |
| Maximum Duration | `20 seconds` |
The same schedule can also be set at the CRD level. See [Validator CRD User Guide](/catalyx-blockchain-manager/canton-network/version-1.11/validator-management/validator-crd-user-guide.md) for the `scheduledPrune` spec fields.
### Monitoring Pruning
After configuring a pruning schedule:
1. Wait for the scheduled execution.
2. Refresh the Pruning Schedule page.
3. Verify that pruning completed successfully.
The UI displays:
* Last execution
* Pruning status
* Pruned records
### Backup Pruning vs. Ledger Pruning
These are different operations and should not be confused.
| Backup Pruning | Ledger Pruning |
| -------------------------------------- | -------------------------------- |
| Removes old backup files | Removes obsolete ledger data |
| Controlled by backup retention setting | Controlled by pruning schedule |
| Frees backup storage | Reduces database size |
| Does not affect ledger state | Does not affect active contracts |
***
## Best Practices
For production deployments, IntellectEU recommends:
* Schedule regular automated database backups.
* Schedule regular Identity Dumps.
* Store backups in external storage whenever possible.
* Periodically validate restore procedures.
* Configure pruning only after defining a suitable retention policy.
* Monitor scheduled backup and pruning jobs.
* Review backup retention periodically to balance recovery objectives and storage utilization.
***
## Compliance Considerations
CatalyX Blockchain Manager provides capabilities that support organizational compliance requirements through:
* Configurable backup retention
* Scheduled pruning
* Security logging
* Infrastructure audit trails
* Validator identity preservation
* Disaster recovery mechanisms
{% hint style="warning" %}
Organizations remain responsible for defining retention policies that satisfy their own regulatory and business requirements.
{% endhint %}
***
## Frequently Asked Questions
What is the difference between a Database Backup and an Identity Dump?
A Database Backup protects the validator database and ledger state. An Identity Dump protects validator identity, cryptographic keys, and wallet credentials. Both are required for complete disaster recovery.
How long are security logs retained?
Mainnet (Production): 30 days. Devnet/Testnet: 7 days.
Can I configure pruning schedules?
Yes. Pruning schedules allow administrators to define the execution schedule, retention period, and maximum execution duration, either from the Domain/Participant management pages or via the CRD.
Does pruning delete active ledger data?
No. Pruning removes only obsolete ledger data that is no longer required for normal ledger operation.
Does pruning affect backups?
No. Ledger pruning and backup retention are independent features.
Can I restore a validator using only an Identity Dump, or only a database backup?
No, in either case. Identity Dumps restore validator identity only — database state must be restored separately. Database backups restore ledger state but do not restore validator identity, cryptographic keys, or wallet credentials. See [Validator Backups and Identity Dumps](/catalyx-blockchain-manager/canton-network/version-1.11/validator-management/identity-backup-and-recovery.md) for the full restore procedure.
Does pruning require downtime?
Pruning is designed as a scheduled maintenance operation and does not require planned validator downtime under normal operating conditions.
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:
```
GET https://docs.catalyx.solutions/catalyx-blockchain-manager/canton-network/version-1.11/validator-management/data-retention-and-pruning-policy.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.