23/23 models were trained successfully in the 12th round of wikis.

Model evaluation has been completed and below are the backtesting results:

CCing @MGerlach, in case he would like to add comments on the backtesting evaluation.

The conclusion on the backtesting results is that most of the languages look fine besides:

• mnwwiki's precision and recall are bad. Both are nil.
• mnwiki (0.72) and newiki (0.74) have a precision slightly lower than the recommended one (0.75).
• mlwiki's precision (0.69) and recall (0.14) are lower than the recommended one (0.75 and 0.2 respectively).
• mywiki has a low precision (0.63) and very low recall (0.06).

Talked to @MGerlach about these results and agreed mnwiki, newiki, and mlwiki should be deployed but mnwwiki and mywiki shouldn't.

@kostajh, we published datasets for all 21/23 models that passed the evaluation in this round.

Change 935723 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[operations/mediawiki-config@master] GrowthExperiments: Enable backend of link recommendation 10, 11, 12th round wikis

https://gerrit.wikimedia.org/r/935723

I ran this script for adding the link-recommendation task type and populating the excluded sections entries:

for WIKI in mhrwiki miwiki minwiki mkwiki mlwiki mnwiki mrwiki mrjwiki mswiki mtwiki mwlwiki myvwiki mznwiki nawiki nahwiki napwiki ndswiki nds_nlwiki newiki newwiki nnwiki; do
    ORIGIN=`mwscript getConfiguration.php $WIKI --settings 'wgCanonicalServer' --format json | jq --raw-output '.wgCanonicalServer'`
    mwscript extensions/GrowthExperiments/maintenance/changeWikiConfig.php $WIKI \
            --page MediaWiki:NewcomerTasks.json \
            --create-only \
            --json \
            --summary "Growth features configuration boilerplate ([[phab:$PHAB]])" \
            link-recommendation \
            '{ "type": "link-recommendation", "group": "easy" }'
    jq "select(.wiki==\"$WIKI\" and .probability > 0.25) | .section" wiki_sections.jsonl \
        | jq --slurp --compact-output "unique" \
        | mwscript extensions/GrowthExperiments/maintenance/changeWikiConfig.php $WIKI \
            --page MediaWiki:NewcomerTasks.json \
            --json \
            --summary "machine-generated configuration for excluding sections from link recommendations ([[phab:$PHAB]]), feel free to improve" \
            link-recommendation.excludedSections \
            "`cat`"
    echo "$ORIGIN/wiki/MediaWiki:NewcomerTasks.json"
    echo "$ORIGIN/w/index.php?title=MediaWiki:NewcomerTasks.json&diff=next"
    echo "Press <Enter> to continue"
    read # give time for manual verification
done

Change 935723 merged by jenkins-bot:

[operations/mediawiki-config@master] GrowthExperiments: Enable backend of link recommendation 10, 11, 12th round wikis

https://gerrit.wikimedia.org/r/935723

Mentioned in SAL (#wikimedia-operations) [2023-07-11T13:03:28Z] <urbanecm@deploy1002> Started scap: Backport for [[gerrit:935723|GrowthExperiments: Enable backend of link recommendation 10, 11, 12th round wikis (T308135 T308136 T308137)]]

Mentioned in SAL (#wikimedia-operations) [2023-07-11T13:04:58Z] <urbanecm@deploy1002> sgimeno and urbanecm: Backport for [[gerrit:935723|GrowthExperiments: Enable backend of link recommendation 10, 11, 12th round wikis (T308135 T308136 T308137)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet

Mentioned in SAL (#wikimedia-operations) [2023-07-11T13:13:13Z] <urbanecm@deploy1002> Finished scap: Backport for [[gerrit:935723|GrowthExperiments: Enable backend of link recommendation 10, 11, 12th round wikis (T308135 T308136 T308137)]] (duration: 09m 45s)

Status update, as per today all wikis have produced results except for nawiki.

For more context:

• mlwiki and mswiki were missing in wikis.txt file ( cc @kevinbazira ), I've added them and run the recommendations script on demand for these two and they have produced results.
• nawiki the recommendation script has failed to generate results without yielding any errors, but most probably because the wiki has been locked, see https://meta.wikimedia.org/wiki/Proposals_for_closing_projects/Closure_of_Nauruan_Wikipedia

I think we can proceed with the rollout of this round except for nawiki. cc @Trizek-WMF.

Change 948144 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[operations/mediawiki-config@master] GrowthExperiments: enable add a link in 12th round of wikis

https://gerrit.wikimedia.org/r/948144

In T308137#9086803, @Sgs wrote:

Status update, as per today all wikis have produced results except for nawiki.

For more context:

• mlwiki and mswiki were missing in wikis.txt file ( cc @kevinbazira ), I've added them and run the recommendations script on demand for these two and they have produced results.
• nawiki the recommendation script has failed to generate results without yielding any errors, but most probably because the wiki has been locked, see https://meta.wikimedia.org/wiki/Proposals_for_closing_projects/Closure_of_Nauruan_Wikipedia

I think we can proceed with the rollout of this round except for nawiki. cc @Trizek-WMF.

Thank you for conducting the tests. :)

We won't deploy at na.wp. It was locked after I created the lists.

I set the date to deploy on Sept 6th.

Change 948144 merged by jenkins-bot:

[operations/mediawiki-config@master] GrowthExperiments: enable add a link in 12 and 13th round of wikis

https://gerrit.wikimedia.org/r/948144

Mentioned in SAL (#wikimedia-operations) [2023-09-06T20:03:03Z] <taavi@deploy1002> Started scap: Backport for [[gerrit:948144|GrowthExperiments: enable add a link in 12 and 13th round of wikis (T308137 T308138)]]

Mentioned in SAL (#wikimedia-operations) [2023-09-06T20:04:40Z] <taavi@deploy1002> taavi and sgimeno: Backport for [[gerrit:948144|GrowthExperiments: enable add a link in 12 and 13th round of wikis (T308137 T308138)]] synced to the testservers mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet, and mw-debug kubernetes deployment (accessible via k8s-experimental XWD option)

Mentioned in SAL (#wikimedia-operations) [2023-09-06T20:13:20Z] <taavi@deploy1002> Finished scap: Backport for [[gerrit:948144|GrowthExperiments: enable add a link in 12 and 13th round of wikis (T308137 T308138)]] (duration: 10m 16s)