Launchpad itself

BugNotificationRecipient needs pruning

Bug #407288 reported by Stuart Bishop
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Launchpad itself
Fix Released
High
Stuart Bishop
Launchpad itself 3.0

Bug Description

There are now 48 million rows in BugNotificationRecipient.

We should prune old records from this table. Do we need any records older than, say, 30 days?

See original description
Tags: lp-bugs
Stuart Bishop (stub)
Changed in malone:
importance: Undecided → High
Revision history for this message
Björn Tillenius (bjornt) wrote :

Deleting records older than 30 days sounds fine. If we want to be really paranoid, we should add a check to make sure that the corresponding BugNotification record has a not-null date_emailed.

Changed in malone:
status: New → Triaged
description: updated
Revision history for this message
Stuart Bishop (stub) wrote :

I will add to the daily garbage collector a job that removes BugNotificationRecipient records that are older than 30 days and have a not null date_emailed.

Changed in malone:
assignee: nobody → Stuart Bishop (stub)
milestone: none → 3.0
Revision history for this message
Stuart Bishop (stub) wrote :

The table to prune is of course BugNotification, which will cascade deletes to BugNotificationRecipient.

Revision history for this message
William Grant (wgrant) wrote :

I might point out bug #253242 -- that gives a good reason for not doing this. The historical data would be useful.

Revision history for this message
Gavin Panella (allenap) wrote :

Stuart, please can you archive or dump old BugNotification and BugNotificationRecipient data before pruning it?

Revision history for this message
Diogo Matsubara (matsubara) wrote : Bug fixed by a commit

Fixed in db r8342.

Changed in malone:
status: Triaged → Fix Committed
Revision history for this message
Stuart Bishop (stub) wrote :

Pruning BugNotification and BugNotificationRecipient will be a daily task.

Do we just need the historical information from when the bug activity log wasn't rich enough, or is this still an issue and we need to keep the records that are being created now? If I can't throw away a record created today in 30 days time, we should just not do the pruning at all.

Changed in malone:
status: Fix Committed → Triaged
Revision history for this message
Björn Tillenius (bjornt) wrote : Re: [Bug 407288] Re: BugNotificationRecipient needs pruning

On Sat, Aug 08, 2009 at 02:49:36PM -0000, Stuart Bishop wrote:
> Pruning BugNotification and BugNotificationRecipient will be a daily
> task.
>
> Do we just need the historical information from when the bug activity
> log wasn't rich enough, or is this still an issue and we need to keep
> the records that are being created now? If I can't throw away a record
> created today in 30 days time, we should just not do the pruning at all.

We need the historical data only from when the bug activity wasn't rich
enough, so it's a one-off task that needs to be done before pruning the
first time. After that it's ok to prune it without backing up the data.

Revision history for this message
Stuart Bishop (stub) wrote :

A database patch that archives BugNotification to the BugNotificationArchive and BugNotificationRecipient to the BugNotificationRecipientArchive tables has landed.

We should drop these two tables once we have imported the desired information from them or decided not to bother - they are large tables that contribute to lengthy staging restore times and time to create new production database replicas.

Changed in malone:
status: Triaged → Fix Committed
Stuart Bishop (stub)
Changed in malone:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.