While doing some repoadmin job I notice that commits such as https://phabricator.wikimedia.org/rEDYMe45d9f76bf6a8ac25ceaac48af94be72cbf957bd which are days old are still on "Importing..." status. I have deactivated the rEDYM repo today, but the commit is from a couple of days ago.
I have experienced this issue in other repositories as well. I'll try to find some more examples if required.
Thank you.
| Restricted Differential Revision |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Aklapper | T196855 Diffusion commits stuck in 'Importing... | |||
| Resolved | Paladox | T196860 Ignore refs/changes/**/**/meta |
This is due to the gerrit upgrade which created notedb commits in each repo as it is migrating commits and other stuff related to changes (in those projects) into the repo's (which is alot of references)
Related to / potential duplicate of T196840: Massive increase of writes in m3 section ?
Mentioned in SAL (#wikimedia-operations) [2018-06-11T17:25:32Z] <twentyafterfour> Phabricator: deploying hotfix (D1067) refs T196840 T196860 T196855
It appears that the history of rMW is still stuck on importing, and no commits are listed after Monday, June 11. Gerrit correctly reports commits after that date: https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/core/. Is there something that is still missing?
I think @mmodell disabled some stuff (phid?, I don't remember) to help ease the db transactions after the ignoring of the refs/.*/.*/meta ignores. That might be?
If there are any others which haven't imported properly please update this ticket. Currently I'm not aware of any others.
What's the rationale behind Diffusion importing repositories at all? It was supposed to be a step towards Phabricator-based code review, but that plan was abandoned - maybe it would be simpler to abandon repo mirroring into Diffusion as well?
Off-topic but :) - It was a repo viewer that we hosted (Gerrit didn't have one until recently, other option was Github) and some people still prefer Diffusion over Gitiles.
@mmodell Commits from the 9th (rMWe0fab761f34c7c10ba33f0ead4969fe981ce9aa4) to the 12th (rMW67ba3f31ed5ed30c0c08bb5a75bb098e3ffbe824) seem to still be stuck on importing.
Found another repository with commits still stuck on importing: rMCSN mediawiki-codesniffer.
Late, but I found another stuck commit: rPHDEP98d9126923d5: Deploy hotfix to ignore refs/changes/**/**/meta, from rPHDEP Phabricator Deployment.
What's the rationale behind Diffusion importing repositories at all?
Moved to T359549: Drop our mirroring of code to Diffusion and empty the repos instead
I guess ./bin/repository importing rYourFavRepo and ./bin/repository reparse yourFavCommitId could be a fun game to play one day
Seems like T196855#4841265 about https://phabricator.wikimedia.org/rPHDEP98d9126923d5db541124f4e784fa240247d891a7 was the only stuck one mentioned in this ticket left. I fixed that by running ./bin/repository reparse --importing --trace rPHDEP98d9126923d5db541124f4e784fa240247d891a7 (cf https://we.phorge.it/book/phorge/article/repository_imports/ )
There are some more out there though, for various unknown reasons.