Page MenuHomePhabricator
Create Task
Maniphest T197459

Optimize logging table
Closed, ResolvedPublic
Actions

Description

After T184485 and T189596, there were a lot of deleted rows from the logging table on many wikis. This is a summary of the numbers of rows deleted:

wikidatawiki654029578
commonswiki196956373
frwiki84965885
itwiki43086794
enwiktionary40325873
shwiki38929189
dewiki30377762
viwiki29115179
frwiktionary17784536
nlwiki16476995

While the rows have been already physically deleted, optimizing the tables will reclaim (in some cases) a lot of space and clean up the internal purging log (in some cases, fixing slowdowns. eg. dbstore2* T189596#4160236 ).

While this is not high priority, it would be nice to keep track of the optimized databases/hosts to not work twice.

Related Objects

Event Timeline

jcrespo created this task.Jun 15 2018, 10:39 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 15 2018, 10:39 AM
jcrespo triaged this task as Low priority.Jun 15 2018, 10:39 AM
jcrespo updated the task description. (Show Details)
• Marostegui added a comment.Jun 15 2018, 10:46 AM

wikidatawiki and commons were already done as there was a schema change that involved logging table and it rebuilt that table, and it was done right after the deletes finished on that table, so we mark those as done.

jcrespo updated the task description. (Show Details)Jun 15 2018, 10:46 AM
jcrespo updated the task description. (Show Details)Jun 15 2018, 10:51 AM
jcrespo moved this task from Triage to Meta/Epic on the DBA board.Jun 15 2018, 11:28 AM
Nirmos subscribed.Jun 16 2018, 11:56 AM
• Vvjjkkii renamed this task from Optimize logging table to cuaaaaaaaa.Jul 1 2018, 1:03 AM
• Vvjjkkii raised the priority of this task from Low to High.
• Vvjjkkii added projects: CheckUser, Connected-Open-Heritage-Batch-uploads (RAÄ-KMB_1_2017-02), Tamil-Sites, Gamepress, Hashtags, Jade, KartoEditor, Language-2018-Apr-June, New-Editor-Experiences, Mail, TCB-Team (now WMDE-TechWish).
• Vvjjkkii updated the task description. (Show Details)
• Vvjjkkii removed a subscriber: Aklapper.
• Marostegui renamed this task from cuaaaaaaaa to Optimize logging table.Jul 1 2018, 8:06 PM
• Marostegui lowered the priority of this task from High to Low.
• Marostegui removed projects: TCB-Team (now WMDE-TechWish), Mail, New-Editor-Experiences, Language-2018-Apr-June, KartoEditor, Jade, Hashtags, Gamepress, Tamil-Sites, Connected-Open-Heritage-Batch-uploads (RAÄ-KMB_1_2017-02), CheckUser.
• Marostegui updated the task description. (Show Details)
• Marostegui updated the task description. (Show Details)
Stashbot subscribed.Jul 2 2018, 1:15 PM

Mentioned in SAL (#wikimedia-operations) [2018-07-02T13:15:51Z] <marostegui> Optimize viwiki.logging on codfw - T197459

• Marostegui claimed this task.EditedJul 2 2018, 1:15 PM
• Marostegui moved this task from Meta/Epic to In progress on the DBA board.

As I am altering s7, where frwiktionary and viwiki lives, I will take the opportunity to optimize logging table there

Stashbot added a comment.Jul 2 2018, 1:21 PM

Mentioned in SAL (#wikimedia-operations) [2018-07-02T13:21:07Z] <marostegui> Optimize frwiktionary'.logging on codfw - T197459

Stashbot mentioned this in T191316: Schema change to make archive.ar_rev_id NOT NULL.Jul 2 2018, 3:38 PM
Stashbot mentioned this in T192926: Schema change to drop archive.ar_text and archive.ar_flags.
Stashbot mentioned this in T89737: Make several mediawiki table fields unsigned ints on wmf databases.
Stashbot mentioned this in T195193: Schema change for ct_tag_id field to change_tag.

Mentioned in SAL (#wikimedia-operations) [2018-07-02T15:38:26Z] <marostegui> Deploy schema change on dbstore1002:s7 T191316 T192926 T89737 T195193 T197459

Framawiki subscribed.Jul 2 2018, 6:25 PM

@Marostegui Perhaps the description of this task needs an update after your action.

• Marostegui added a comment.Jul 2 2018, 6:30 PM
In T197459#4391362, @Framawiki wrote:

@Marostegui Perhaps the description of this task needs an update after your action.

No because it has not finished across all the hosts of S7 ;)

Framawiki added a comment.Jul 2 2018, 6:58 PM
In T197459#4391404, @Marostegui wrote:
In T197459#4391362, @Framawiki wrote:

@Marostegui Perhaps the description of this task needs an update after your action.

No because it has not finished across all the hosts of S7 ;)

I'm glad I didn't make the change myself :)

Stashbot added a comment.Jul 3 2018, 4:40 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-03T04:40:21Z] <marostegui> Deploy schema change on db1086 T191316 T192926 T89737 T195193 T197459

Stashbot added a comment.Jul 3 2018, 5:26 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-03T05:26:33Z] <marostegui> Deploy schema change on db1094 T191316 T192926 T89737 T195193 T197459

Stashbot added a comment.Jul 3 2018, 6:14 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-03T06:14:24Z] <marostegui> Deploy schema change on db1079 with replication, this will generate lag on s7 on labsdb hosts T191316 T192926 T89737 T195193 T197459

Stashbot added a comment.Jul 3 2018, 7:13 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-03T07:13:34Z] <marostegui> Deploy schema change on db1090:3317 T191316 T192926 T89737 T195193 T197459

Stashbot added a comment.Jul 3 2018, 9:13 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-03T09:13:13Z] <marostegui> Deploy schema change on db1098:3317 T191316 T192926 T89737 T195193 T197459

Ladsgroup subscribed.Jul 3 2018, 9:22 AM
Stashbot added a comment.Jul 3 2018, 9:59 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-03T09:58:59Z] <marostegui> Deploy schema change on s3 codfw primary master (db2043) with replication, this will generate lag on s3 codfw T191316 T192926 T89737 T195193 T197459

• Marostegui updated the task description. (Show Details)Jul 3 2018, 1:57 PM
Stashbot added a comment.Jul 4 2018, 3:55 PM

Mentioned in SAL (#wikimedia-operations) [2018-07-04T15:55:40Z] <marostegui> Optimize dewiki.logging on s5 codfw master with replication, this will generate lag on s5 codfw - T197459

Stashbot added a comment.Jul 5 2018, 6:09 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-05T06:09:48Z] <marostegui> Optimize dewiki.logging on db1100 and dbstore1002:s5 - T197459

Stashbot added a comment.Jul 5 2018, 7:40 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-05T07:40:12Z] <marostegui> Optimize dewiki.logging on db1096:3315 - T197459

Stashbot added a comment.Jul 5 2018, 9:50 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-05T09:50:48Z] <marostegui> Optimize dewiki.logging on db1097:3315 - T197459

Stashbot added a comment.Jul 5 2018, 1:18 PM

Mentioned in SAL (#wikimedia-operations) [2018-07-05T13:18:43Z] <marostegui> Optimize dewiki.logging on db1113:3315 - T197459

Stashbot added a comment.Jul 5 2018, 3:00 PM

Mentioned in SAL (#wikimedia-operations) [2018-07-05T15:00:04Z] <marostegui> Optimize dewiki.logging on db1082 with replication, this will generate lag on s5 on labsdb hosts/script load irssinotifier - T197459

Krinkle awarded a token.Jul 5 2018, 10:54 PM
Stashbot added a comment.Jul 6 2018, 5:01 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-06T05:01:49Z] <marostegui> Optimize dewiki.logging on db1110 - T197459

Stashbot added a comment.Jul 6 2018, 6:22 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-06T06:22:16Z] <marostegui> Optimize dewiki.logging on db1070 (s5 primary master) - T197459

Stashbot added a comment.Jul 6 2018, 6:31 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-06T06:31:25Z] <marostegui> Optimize frwiki.logging on db2039 (s6 codfw master), with replication, this will generate lag on s6 codfw - T197459

Stashbot added a comment.Jul 6 2018, 6:49 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-06T06:49:51Z] <marostegui> Optimize frwiki.logging on dbstore1002 - T197459

Stashbot added a comment.Jul 6 2018, 7:11 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-06T07:11:48Z] <marostegui> Optimize frwiki.logging on db1096:3316 - T197459

Stashbot added a comment.Jul 6 2018, 7:47 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-06T07:47:31Z] <marostegui> Optimize frwiki.logging on db1098:3316 - T197459

Stashbot added a comment.Jul 6 2018, 8:18 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-06T08:18:21Z] <marostegui> Optimize frwiki.logging on db1113:3316 - T197459

• Marostegui updated the task description. (Show Details)Jul 6 2018, 8:49 AM
Stashbot added a comment.Jul 6 2018, 8:53 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-06T08:53:36Z] <marostegui> Optimize frwiki.logging on db11085 with replication (this will generate lag on s6 on labsdb hosts) - T197459

Stashbot added a comment.Jul 6 2018, 10:02 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-06T10:02:46Z] <marostegui> Optimize frwiki.logging on db1093 - T197459

Stashbot added a comment.Jul 6 2018, 10:49 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-06T10:49:33Z] <marostegui> Optimize frwiki.logging on db1088 - T197459

Stashbot added a comment.Jul 6 2018, 11:04 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-06T11:04:36Z] <marostegui> Optimize frwiki.logging on db1061 (s6 primary master) - T197459

Stashbot added a comment.Jul 6 2018, 11:08 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-06T11:08:25Z] <marostegui> Optimize shwiki.logging on db2043 (codfw s3 master), this will generate lag on s3 codfw - T197459

Stashbot added a comment.Jul 6 2018, 11:09 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-06T11:09:49Z] <marostegui> Optimize shwiki.logging on s3 eqiad hosts (one by one) T197459

• Marostegui updated the task description. (Show Details)Jul 6 2018, 11:12 AM
Stashbot added a comment.Jul 6 2018, 11:15 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-06T11:15:34Z] <marostegui> Optimize {itwiki,enwiktionary,nlwiki}.logging on db2035 (codfw s2 master), this will generate lag on s2 codfw - T197459

• Marostegui updated the task description. (Show Details)Jul 6 2018, 11:26 AM
Stashbot added a comment.Jul 6 2018, 11:51 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-06T11:51:20Z] <marostegui> Optimize {itwiki,enwiktionary,nlwiki}.logging on dbstore1002:s2 - T197459

Stashbot added a comment.Jul 6 2018, 12:38 PM

Mentioned in SAL (#wikimedia-operations) [2018-07-06T12:38:47Z] <marostegui> Optimize {itwiki,enwiktionary,nlwiki}.logging on db1105 and db1122 - T197459

Stashbot added a comment.Jul 6 2018, 1:22 PM

Mentioned in SAL (#wikimedia-operations) [2018-07-06T13:22:40Z] <marostegui> Optimize {itwiki,enwiktionary,nlwiki}.logging on db1103:3312 - T197459

Stashbot added a comment.Jul 9 2018, 5:04 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-09T05:04:18Z] <marostegui> Optimize {itwiki,enwiktionary,nlwiki}.logging on db1074 with replication, this will generate lag on labs hosts T197459

Stashbot added a comment.Jul 9 2018, 5:31 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-09T05:31:30Z] <marostegui> Optimize {itwiki,enwiktionary,nlwiki}.logging on db1090 T197459

Stashbot added a comment.Jul 9 2018, 6:13 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-09T06:13:44Z] <marostegui> Optimize {itwiki,enwiktionary,nlwiki}.logging on db1076 T197459

Stashbot added a comment.Jul 9 2018, 6:41 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-09T06:41:19Z] <marostegui> Optimize {itwiki,enwiktionary,nlwiki}.logging on db1066 (s2 primary master) T197459

• Marostegui closed this task as Resolved.Jul 9 2018, 7:05 AM
• Marostegui updated the task description. (Show Details)

This is all done

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits