Change 436031 had a related patch set uploaded (by DCausse; owner: DCausse):
[mediawiki/extensions/CirrusSearch@master] Prefix keyword should append its namespace not override the initial ones

https://gerrit.wikimedia.org/r/436031

I am not sure whether it's the same thing as the patch is fixing, but I think we have been mixing two things here:

1. Global query filter, which is applied to all query regardless of query strings, etc. This is what should be set from namespace configuration coming from URL or other places, and I don't think query features should change it. This should also be used to control search behavior (e.g. for Wikidata etc.) and other global things.

2. Namespaces for specific keywords. This should produce a local filter, which can be either AND-ed with filters provided by other filter keywords, or negated, or OR-ed, or combined any other way you like. For some queries, the effect would be the same as modifying the original namespace filter, but for some it won't be.

Separating those I think would clear up the matters somewhat. It can create conflicts - like having disjoint filters in prefix: and in global namespace setup - i.e. you can ask Property: search on Wikidata in query params and then say prefix:Help:Foo in the query string, and in that case the query would fail. I think it's OK.

I am also not sure that if you ask a namespace via prefix: that wasn't given in the URL we should add it - did it work before? If so, we may want to add the prefix-generated namespaces into the global filter, maybe.

Initially I wanted to change how prefix works by disallowing any changes to the original list of namespace requested from the URL. Sadly this caused many components to fail and I had to revert. I completely agree with you that this behavior makes it difficult for us to make some decisions based on the list of namespaces.
Fixing the root cause by changing the behavior of the prefix: keyword seems hard to achieve in a reasonable amount of time. I'm not saying that it's impossible.

The strategy here is to open the initial list of request namespaces to the ones that the prefix keyword wants:

• user requests NS_MAIN but type prefix:Help:Foo, the list of requested namespaces will be [ NS_MAIN, NS_HELP ], the filter generated by the prefix keyword will be : [ 'title': 'Foo', namespace: NS_HELP ].
  • previously we simply overwrote the list of namespaces to [ NS_HELP ], forgetting the original NS_MAIN
  • the patch that was reverted emitted a warning and saying that these namespaces are incompatible and asked the user to align the list of namespaces to the one targeted by the prefix keyword. This caused too much trouble on the current wiki workflows.
• user requests NS_MAIN but type prefix:Foo, the list of requested namespaces will remain unchanged.
• user requests NS_MAIN but type prefix:all:Foo, the list of requested namespaces will be set to null (by cirrus standards it means all namespaces)

After the refactoring the original list of namespaces will still be available and should remain unchanged but the ParsedQuery will have a getter that returns the additional namespaces found in the syntax that the query building components should take into account to generate a valid query.

Overall I think what you suggest is exactly what I plan to do.

user requests NS_MAIN but type prefix:Help:Foo, the list of requested namespaces will be [ NS_MAIN, NS_HELP ]

This makes sense, but this may change behavior with specialized searches like Wikidata ones. Fortunately, prefix: seems to be mostly useless in Wikidata per se, but if we have namespace where prefix: is useful, it would require to do combined search instead of just local namespace search. Maybe not a big deal, as you can always set the namespace in the search box...

So for now we don't really distinguish between the initial namespaces and the namespaces found in the query.
In the current wikibase situation we fallback to cirrus builder anyways in the presence of keywords so this won't be an issue short term.

Later I agree that we will have to make an important decision:

1. should all the decisions (about choosing the appropriate query builders) be based solely on the initial list of namespaces, this means also that the parser should not be able to parse keywords that may change the list of namespaces. E.g. when searching on [ NS_ITEM, NS_PROPS ] on wikibase we will spawn a special parser that is only meant to support syntax supported by your query builder.
   • choose all components based on the initial context i.e. requested namespaces (Parser/QueryBuilders/...) => parse => build query
   • drawback: the syntax may slightly change depending on the initial namespace
   • benefit: guarantee that the parsed query is compatible with your builders, in other words: no need to fallback
2. should we defer all important decisions after parsing, this means we have a single parser and the logic to decide between query builders will be based on the initial set of namespaces + the ParsedQuery (AST).
   • parse the query => decision to choose builders based on inititial context + the parsed query => build query
   • drawback: more complex code to analyze the parsed query, needs a mechanism to fallback
   • advantage: a single syntax

I doubt that from a user POV option 1 is acceptable, it may be very confusing to have a varying syntax depending on the initial set of namespaces.

(This discussion should really go to T189880 )

I doubt that from a user POV option 1 is acceptable, it may be very confusing to have a varying syntax depending on the initial set of namespaces.

I agree. It may also be confusing for us, as it'd be tricky to figure out what is happening in the code in each particular query. I think we need to parse first, and then decide which query builder to use to build this query, in accordance with parsing results.

Change 436031 merged by jenkins-bot:
[mediawiki/extensions/CirrusSearch@master] Prefix keyword should append its namespace not override the initial ones

https://gerrit.wikimedia.org/r/436031