Page MenuHomePhabricator

Smalyshev (Stas Malyshev)
Engineer in Search Platform team

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Saturday

  • Clear sailing ahead.

User Details

User Since
Nov 28 2014, 7:04 AM (517 w, 6 d)
Availability
Available
IRC Nick
Smalyshev
LDAP User
Smalyshev
MediaWiki User
Laboramus [ Global Accounts ]

Recent Activity

Jun 7 2022

Smalyshev updated the task description for T308013: Assign SPDX headers to puppet.git.
Jun 7 2022, 5:22 PM · Patch-For-Review, Infrastructure-Foundations, SRE

Aug 18 2021

Sj awarded T206561: Evaluate Virtuoso as alternative to Blazegraph a Burninate token.
Aug 18 2021, 9:28 PM · Wikidata, Wikidata-Query-Service
Sj awarded T206560: [Epic] Evaluate alternatives to Blazegraph a 100 token.
Aug 18 2021, 9:24 PM · Wikidata, Epic, Wikidata-Query-Service

Jun 10 2021

EgonWillighagen awarded T112151: Support POST for SPARQL query endpoint a Love token.
Jun 10 2021, 6:31 AM · Developer-notice, Discovery-Wikidata-Query-Service-Sprint, Patch-For-Review, Discovery-ARCHIVED, Wikidata, Wikidata-Query-Service

Jun 16 2020

Akuckartz awarded T206561: Evaluate Virtuoso as alternative to Blazegraph a Like token.
Jun 16 2020, 10:04 PM · Wikidata, Wikidata-Query-Service
Akuckartz awarded T206560: [Epic] Evaluate alternatives to Blazegraph a Like token.
Jun 16 2020, 10:03 PM · Wikidata, Epic, Wikidata-Query-Service

May 27 2020

Mahir256 awarded T221917: Create RDF dump of structured data on Commons a Party Time token.
May 27 2020, 2:53 PM · Patch-For-Review, Dumps-Generation, MW-1.34-notes (1.34.0-wmf.10; 2019-06-18), Wikidata-Query-Service, Commons, Wikidata

Mar 11 2020

Smalyshev placed T238153: puppet breakage in the wikidata-query VPS project up for grabs.
Mar 11 2020, 6:19 PM · Wikidata, Wikidata-Query-Service
Smalyshev removed a watcher for Wikidata-Query-Service: Smalyshev.
Mar 11 2020, 6:17 PM

Mar 3 2020

Gamaliel awarded T141602: [Objective Fiscal 19-20/Q4] (9) Provide a Proof of Concept SPARQL endpoint in support of SDoC project a Love token.
Mar 3 2020, 5:56 PM · Discovery-Search (Current work), Epic, Wikidata-Query-Service, SDC General, Commons, Wikidata

Nov 29 2019

Smalyshev added a comment to T239414: Investigate how blank nodes are used and synced between wikibase and wdqs.

Blanks are used as representation of "unknown value" in Wikibase. Also, they are used (completely unrelatedly) in implementation of OWL class that describes "no value" properties (as "no value" is not a value, it is implemented with a predicate and a class instead of usual property predicate).
The class definition itself is useful only for the tools that actually understand OWL semantics. Most tools that query WDQS do not.

Nov 29 2019, 8:45 PM · Wikidata-Query-Service, Wikidata

Nov 19 2019

Ghuron awarded T212826: Create dedicated Updater service in Blazegraph a Like token.
Nov 19 2019, 3:58 AM · Discovery-Search (Current work), Epic, Performance Issue, Wikidata-Query-Service, Wikidata

Nov 11 2019

Smalyshev added a comment to T238002: WDQS Munger should be multi threaded.

Per-item data are mostly independent, so different items can be easily processable in parallel, however that would require splitting the incoming data per item (note that item data not necessarily have item URI as subject - there are statements, references, values, sitelinks, etc.)

Nov 11 2019, 6:37 PM · Wikidata, Wikidata-Query-Service

Nov 6 2019

Smalyshev added a comment to T237502: Provide public "reload entity to WDQS" API.

Actually these WDQS servers are merely reading RCStream

Nov 6 2019, 5:26 PM · Wikidata-Query-Service, Wikidata

Nov 2 2019

Smalyshev assigned T237165: LDF server has 404 errors for JS and CSS resources to Gehel.

Looks like css/js artifacts aren't deployed correctly.

Nov 2 2019, 10:03 PM · Discovery-Search (Current work), SRE, Traffic, Wikidata, Wikidata-Query-Service, Discovery-ARCHIVED

Oct 31 2019

Smalyshev added a comment to T197658: Provide easy script to reset Blazegraph.

I believe munge.sh applies the WDQS data differences documented on the RDF Dump Format page (e. g. merge wdata: and wd:).

Oct 31 2019, 9:02 PM · User-Addshore, [DEPRECATED] wdwb-tech, Discovery-ARCHIVED, Wikidata-Query-Service, Wikidata, Wikibase-Docker-2017+

Oct 15 2019

Smalyshev added a comment to T235540: StackOverflowError when SPARQL query uses same variable name before and after aggregation.

Our fork is in https://github.com/wikimedia/wikidata-query-blazegraph

Oct 15 2019, 5:58 PM · Wikidata, Wikidata-Query-Service

Oct 7 2019

Smalyshev placed T232071: math functions in sparql up for grabs.
Oct 7 2019, 12:19 AM · WDQS-Optimizer

Sep 30 2019

Smalyshev added a comment to T233204: Mixup of unicode characters in Query Service.

The issue is that by default Blazegraph uses tertiary ICU collation level IIRC (I can check specific one) so it ignores some differences like that one - generating same term key for both. It can be switched to Identical but that would generate much larger term keys which would hurt performance and increase storage size.

Sep 30 2019, 6:31 PM · Wikidata, Wikidata-Query-Service

Sep 26 2019

Smalyshev added a comment to T232984: WDQS returns reduced precision for coordinate values.

@seav Please explain the case for millimeter-precision coordinates. Which objects in Wikidata have locations known with millimeter precision?

Sep 26 2019, 10:57 PM · Wikidata-Query-Service, Wikidata
Smalyshev added a comment to T232984: WDQS returns reduced precision for coordinate values.

4 digits is 11m precision, 5 digits is 1m precisions. We could bump the max to 5 digits I presume, but I am not sure which coordinates really have that many significant digits and whether these coordinates indeed are precise within meter or just claim to be so. But changing it wouldn't be very hard - just change COORDINATE_PRECISION in GlobeCoordinateRdfBuilder from 4 to 5.

Sep 26 2019, 10:51 PM · Wikidata-Query-Service, Wikidata

Sep 19 2019

Smalyshev added a comment to T233204: Mixup of unicode characters in Query Service.

Probably related to other issues about Unicode and to ICU collation level. I presume collation level enabled now at Blazegraph confuses these two.

Sep 19 2019, 3:49 AM · Wikidata, Wikidata-Query-Service

Sep 9 2019

Smalyshev added a comment to T232212: QuantityValue quantityUnit contains both Q and P value in Wikidata Query Service - P value is wrong.

I don't think it's worth bothering with depooling, unless the number of affected items is very large, it should be quick enough so nobody should really notice.

Sep 9 2019, 4:48 PM · Wikidata-Query-Service, Wikidata

Sep 7 2019

Smalyshev updated subscribers of T232212: QuantityValue quantityUnit contains both Q and P value in Wikidata Query Service - P value is wrong.
Sep 7 2019, 5:01 AM · Wikidata-Query-Service, Wikidata
Smalyshev updated subscribers of T232212: QuantityValue quantityUnit contains both Q and P value in Wikidata Query Service - P value is wrong.
Sep 7 2019, 5:00 AM · Wikidata-Query-Service, Wikidata
Smalyshev added a comment to T232212: QuantityValue quantityUnit contains both Q and P value in Wikidata Query Service - P value is wrong.

This may happen because value nodes are not updated when data is updated (since they are supposed to be immutable). So if some bad data sneaked in when the problem was there, the bad value (and possibly reference since they behave the same way) nodes are still there. The best way to do it would be:

Sep 7 2019, 5:00 AM · Wikidata-Query-Service, Wikidata

Sep 4 2019

Smalyshev placed T221917: Create RDF dump of structured data on Commons up for grabs.
Sep 4 2019, 5:52 AM · Patch-For-Review, Dumps-Generation, MW-1.34-notes (1.34.0-wmf.10; 2019-06-18), Wikidata-Query-Service, Commons, Wikidata
Smalyshev added a comment to T221631: Dedicated servers on WMCS to test WDQS scalability strategy.

Both evaluating Virtuoso and other solutions (like JanusGraph) would require that. @Gehel should know the details.

Sep 4 2019, 12:55 AM · cloud-services-team (Kanban), Wikidata, Wikidata-Query-Service, Discovery-Search

Aug 29 2019

Smalyshev moved T159723: NotMaterializedException when one branch of UNION binds ?variable and other branch binds ?variableLabel and label service is used from Needs review to Done on the Discovery-Wikidata-Query-Service-Sprint board.
Aug 29 2019, 9:44 PM · Discovery-Wikidata-Query-Service-Sprint, Upstream, Discovery-ARCHIVED, Wikidata, Wikidata-Query-Service
Smalyshev moved T170704: NME when using label service and rdfs:label predicate with the same variable from Needs review to Done on the Discovery-Wikidata-Query-Service-Sprint board.
Aug 29 2019, 9:44 PM · Discovery-Wikidata-Query-Service-Sprint, Upstream, Wikidata-Query-Service, Discovery-ARCHIVED, Wikidata
Smalyshev removed a project from T168876: MWAPI service throws “could not find binding for parameter” if optimizer is not disabled: Patch-For-Review.
Aug 29 2019, 9:42 PM · Discovery-Wikidata-Query-Service-Sprint, WDQS-Optimizer, Upstream, Wikidata-Query-Service, Discovery-ARCHIVED, Wikidata
Smalyshev moved T168876: MWAPI service throws “could not find binding for parameter” if optimizer is not disabled from Backlog to Done on the Discovery-Wikidata-Query-Service-Sprint board.
Aug 29 2019, 9:42 PM · Discovery-Wikidata-Query-Service-Sprint, WDQS-Optimizer, Upstream, Wikidata-Query-Service, Discovery-ARCHIVED, Wikidata
Smalyshev added a project to T168876: MWAPI service throws “could not find binding for parameter” if optimizer is not disabled: Discovery-Wikidata-Query-Service-Sprint.
Aug 29 2019, 9:41 PM · Discovery-Wikidata-Query-Service-Sprint, WDQS-Optimizer, Upstream, Wikidata-Query-Service, Discovery-ARCHIVED, Wikidata
Smalyshev moved T170704: NME when using label service and rdfs:label predicate with the same variable from Backlog to Needs review on the Discovery-Wikidata-Query-Service-Sprint board.
Aug 29 2019, 9:41 PM · Discovery-Wikidata-Query-Service-Sprint, Upstream, Wikidata-Query-Service, Discovery-ARCHIVED, Wikidata
Smalyshev added a project to T170704: NME when using label service and rdfs:label predicate with the same variable: Discovery-Wikidata-Query-Service-Sprint.
Aug 29 2019, 9:40 PM · Discovery-Wikidata-Query-Service-Sprint, Upstream, Wikidata-Query-Service, Discovery-ARCHIVED, Wikidata
Smalyshev moved T165559: HAVING in named subquery results in “non-aggregate variable in select expression” error from Needs review to Done on the Discovery-Wikidata-Query-Service-Sprint board.
Aug 29 2019, 9:39 PM · Discovery-Search (Current work), Upstream, Discovery-ARCHIVED, Wikidata-Query-Service, Wikidata
Smalyshev moved T168741: SELECT * on query with no variables and property path results in NotMaterializedException from Needs review to Done on the Discovery-Wikidata-Query-Service-Sprint board.
Aug 29 2019, 9:39 PM · Discovery-Wikidata-Query-Service-Sprint, Upstream, Wikidata-Query-Service, Discovery-ARCHIVED, Wikidata
Smalyshev moved T173243: UnsupportedOperationException on property path in EXISTS from Needs review to Done on the Discovery-Wikidata-Query-Service-Sprint board.
Aug 29 2019, 9:39 PM · Discovery-Wikidata-Query-Service-Sprint, Upstream, Wikidata, Discovery-ARCHIVED, Wikidata-Query-Service
Smalyshev moved T172113: ConcurrentModificationException on non-grouping query with aggregates in SELECT from Needs review to Done on the Discovery-Wikidata-Query-Service-Sprint board.
Aug 29 2019, 9:39 PM · Discovery-Wikidata-Query-Service-Sprint, Upstream, Wikidata-Query-Service, Discovery-ARCHIVED, Wikidata
Smalyshev moved T173243: UnsupportedOperationException on property path in EXISTS from Backlog to Needs review on the Discovery-Wikidata-Query-Service-Sprint board.
Aug 29 2019, 7:44 AM · Discovery-Wikidata-Query-Service-Sprint, Upstream, Wikidata, Discovery-ARCHIVED, Wikidata-Query-Service
Smalyshev moved T231411: Test new Updater service from Backlog to In progress on the Discovery-Wikidata-Query-Service-Sprint board.
Aug 29 2019, 7:44 AM · Patch-For-Review, Discovery-Search (Current work), Performance Issue, Wikidata-Query-Service, Wikidata
Smalyshev moved T228348: Category graph includes deleted categories from Needs review to Backlog on the Discovery-Wikidata-Query-Service-Sprint board.
Aug 29 2019, 7:44 AM · Discovery-Search (Current work), MW-1.34-notes (1.34.0-wmf.21; 2019-09-03), User-Smalyshev, Wikidata-Query-Service, Wikidata
Smalyshev placed T228348: Category graph includes deleted categories up for grabs.
Aug 29 2019, 7:44 AM · Discovery-Search (Current work), MW-1.34-notes (1.34.0-wmf.21; 2019-09-03), User-Smalyshev, Wikidata-Query-Service, Wikidata
Smalyshev added a comment to T231515: Duplicate blank nodes on edited properties.

Looks like new updater actually handles it better, but we need to verify that.

Aug 29 2019, 7:04 AM · Wikidata-Query-Service, Wikidata
Smalyshev created T231515: Duplicate blank nodes on edited properties.
Aug 29 2019, 7:04 AM · Wikidata-Query-Service, Wikidata
Smalyshev triaged T231411: Test new Updater service as Medium priority.
Aug 29 2019, 6:12 AM · Patch-For-Review, Discovery-Search (Current work), Performance Issue, Wikidata-Query-Service, Wikidata
Smalyshev added a comment to T231411: Test new Updater service.

Loading 1 hour 25 mins of updates from 201908010000 under both updaters shows no differences except ones that can be attributed to edits (since we always load the latest version on old changes). So this first test seems to be a success.

Aug 29 2019, 6:10 AM · Patch-For-Review, Discovery-Search (Current work), Performance Issue, Wikidata-Query-Service, Wikidata
Smalyshev updated the task description for T231411: Test new Updater service.
Aug 29 2019, 6:06 AM · Patch-For-Review, Discovery-Search (Current work), Performance Issue, Wikidata-Query-Service, Wikidata
Smalyshev added a comment to T231411: Test new Updater service.

Procedure for comparing journals:

Aug 29 2019, 5:45 AM · Patch-For-Review, Discovery-Search (Current work), Performance Issue, Wikidata-Query-Service, Wikidata
Smalyshev updated the task description for T168876: MWAPI service throws “could not find binding for parameter” if optimizer is not disabled.
Aug 29 2019, 5:22 AM · Discovery-Wikidata-Query-Service-Sprint, WDQS-Optimizer, Upstream, Wikidata-Query-Service, Discovery-ARCHIVED, Wikidata

Aug 28 2019

Smalyshev moved T168741: SELECT * on query with no variables and property path results in NotMaterializedException from Backlog to Needs review on the Discovery-Wikidata-Query-Service-Sprint board.
Aug 28 2019, 11:02 PM · Discovery-Wikidata-Query-Service-Sprint, Upstream, Wikidata-Query-Service, Discovery-ARCHIVED, Wikidata
Smalyshev moved T172113: ConcurrentModificationException on non-grouping query with aggregates in SELECT from Backlog to Needs review on the Discovery-Wikidata-Query-Service-Sprint board.
Aug 28 2019, 11:02 PM · Discovery-Wikidata-Query-Service-Sprint, Upstream, Wikidata-Query-Service, Discovery-ARCHIVED, Wikidata
Smalyshev moved T165559: HAVING in named subquery results in “non-aggregate variable in select expression” error from Backlog to Needs review on the Discovery-Wikidata-Query-Service-Sprint board.
Aug 28 2019, 11:02 PM · Discovery-Search (Current work), Upstream, Discovery-ARCHIVED, Wikidata-Query-Service, Wikidata
Smalyshev moved T159723: NotMaterializedException when one branch of UNION binds ?variable and other branch binds ?variableLabel and label service is used from Backlog to Needs review on the Discovery-Wikidata-Query-Service-Sprint board.
Aug 28 2019, 11:02 PM · Discovery-Wikidata-Query-Service-Sprint, Upstream, Discovery-ARCHIVED, Wikidata, Wikidata-Query-Service
Smalyshev added a project to T159723: NotMaterializedException when one branch of UNION binds ?variable and other branch binds ?variableLabel and label service is used: Discovery-Wikidata-Query-Service-Sprint.
Aug 28 2019, 10:52 PM · Discovery-Wikidata-Query-Service-Sprint, Upstream, Discovery-ARCHIVED, Wikidata, Wikidata-Query-Service
Smalyshev added a project to T172113: ConcurrentModificationException on non-grouping query with aggregates in SELECT: Discovery-Wikidata-Query-Service-Sprint.
Aug 28 2019, 10:38 PM · Discovery-Wikidata-Query-Service-Sprint, Upstream, Wikidata-Query-Service, Discovery-ARCHIVED, Wikidata
Smalyshev added a project to T168741: SELECT * on query with no variables and property path results in NotMaterializedException: Discovery-Wikidata-Query-Service-Sprint.
Aug 28 2019, 10:37 PM · Discovery-Wikidata-Query-Service-Sprint, Upstream, Wikidata-Query-Service, Discovery-ARCHIVED, Wikidata
Smalyshev added a project to T173243: UnsupportedOperationException on property path in EXISTS: Discovery-Wikidata-Query-Service-Sprint.
Aug 28 2019, 10:35 PM · Discovery-Wikidata-Query-Service-Sprint, Upstream, Wikidata, Discovery-ARCHIVED, Wikidata-Query-Service
Smalyshev added a project to T165559: HAVING in named subquery results in “non-aggregate variable in select expression” error: Discovery-Wikidata-Query-Service-Sprint.
Aug 28 2019, 10:34 PM · Discovery-Search (Current work), Upstream, Discovery-ARCHIVED, Wikidata-Query-Service, Wikidata
Smalyshev updated subscribers of T212826: Create dedicated Updater service in Blazegraph.
Aug 28 2019, 10:32 PM · Discovery-Search (Current work), Epic, Performance Issue, Wikidata-Query-Service, Wikidata
Smalyshev added a comment to T212826: Create dedicated Updater service in Blazegraph.

Testing on wdqs-test shows new Updater is 2x faster than old one. Didn't verify validity yet but speed looks good :)

Aug 28 2019, 10:32 PM · Discovery-Search (Current work), Epic, Performance Issue, Wikidata-Query-Service, Wikidata
Smalyshev added a comment to P8995 Khmer samples.

Mac OS 10.13.6 (High Sierra), Firefox 68.0.2

Aug 28 2019, 4:02 PM · Discovery-Search
Nicolas_Raoul awarded T141602: [Objective Fiscal 19-20/Q4] (9) Provide a Proof of Concept SPARQL endpoint in support of SDoC project a Love token.
Aug 28 2019, 8:01 AM · Discovery-Search (Current work), Epic, Wikidata-Query-Service, SDC General, Commons, Wikidata
Smalyshev created T231411: Test new Updater service.
Aug 28 2019, 7:05 AM · Patch-For-Review, Discovery-Search (Current work), Performance Issue, Wikidata-Query-Service, Wikidata
Smalyshev triaged T230175: Provide search functionality to find all files that have at least 1 structured data statement as Medium priority.
Aug 28 2019, 7:01 AM · MW-1.34-notes (1.34.0-wmf.21; 2019-09-03), Discovery-Search (Current work), Structured-Data-Backlog, SDC General, Wikidata
Smalyshev moved T230175: Provide search functionality to find all files that have at least 1 structured data statement from Needs review to Needs Reporting on the Discovery-Search (Current work) board.
Aug 28 2019, 7:01 AM · MW-1.34-notes (1.34.0-wmf.21; 2019-09-03), Discovery-Search (Current work), Structured-Data-Backlog, SDC General, Wikidata
Smalyshev closed T222306: RDF export generates wrong IDs for federated entities as Resolved.
Aug 28 2019, 6:37 AM · MW-1.34-notes (1.34.0-wmf.19; 2019-08-20), User-Smalyshev, WikibaseMediaInfo, Wikidata-Query-Service, SDC General, Commons, Wikidata
Smalyshev closed T222306: RDF export generates wrong IDs for federated entities, a subtask of T221916: Create RDF export for structured data stored for files, as Resolved.
Aug 28 2019, 6:37 AM · MW-1.34-notes (1.34.0-wmf.21; 2019-09-03), Discovery-Wikidata-Query-Service-Sprint, User-Smalyshev, WikibaseMediaInfo, Wikidata-Query-Service, SDC General, Commons, Wikidata

Aug 27 2019

Smalyshev moved T228348: Category graph includes deleted categories from In progress to Needs review on the Discovery-Wikidata-Query-Service-Sprint board.
Aug 27 2019, 11:51 PM · Discovery-Search (Current work), MW-1.34-notes (1.34.0-wmf.21; 2019-09-03), User-Smalyshev, Wikidata-Query-Service, Wikidata
Smalyshev moved T228348: Category graph includes deleted categories from Next to In review on the User-Smalyshev board.
Aug 27 2019, 11:24 PM · Discovery-Search (Current work), MW-1.34-notes (1.34.0-wmf.21; 2019-09-03), User-Smalyshev, Wikidata-Query-Service, Wikidata
Smalyshev updated subscribers of T228348: Category graph includes deleted categories.

After the patch is merged and deployed, categories DB needs to be re-loaded according to procedure here: https://wikitech.wikimedia.org/wiki/Wikidata_query_service#Categories_reload_procedure

Aug 27 2019, 11:24 PM · Discovery-Search (Current work), MW-1.34-notes (1.34.0-wmf.21; 2019-09-03), User-Smalyshev, Wikidata-Query-Service, Wikidata
Smalyshev added a comment to T228348: Category graph includes deleted categories.

Looks like DELETE SPARQL clauses that the daily dump is generating are wrong... Weird I haven't noticed it.

Aug 27 2019, 11:16 PM · Discovery-Search (Current work), MW-1.34-notes (1.34.0-wmf.21; 2019-09-03), User-Smalyshev, Wikidata-Query-Service, Wikidata
Smalyshev moved T228348: Category graph includes deleted categories from Backlog to In progress on the Discovery-Wikidata-Query-Service-Sprint board.
Aug 27 2019, 10:49 PM · Discovery-Search (Current work), MW-1.34-notes (1.34.0-wmf.21; 2019-09-03), User-Smalyshev, Wikidata-Query-Service, Wikidata
Smalyshev added a comment to T228348: Category graph includes deleted categories.

Looks like there's some problem with deletion handling. E.g. https://en.wikipedia.org/wiki/Category:Delaware_elections,_2006 has been deleted and is listed in enwiki-20190826-daily.sparql.gz dump as deleted, but still present in the database. Strangely enough, the log shows the file was successfully processed - but somehow the results are not there. Will investigate further.

Aug 27 2019, 10:49 PM · Discovery-Search (Current work), MW-1.34-notes (1.34.0-wmf.21; 2019-09-03), User-Smalyshev, Wikidata-Query-Service, Wikidata
Smalyshev merged task T223773: MWAPI requests for external links return fewer than expected into T231390: MWAPI can only match one result per page.
Aug 27 2019, 9:46 PM · Discovery-Wikidata-Query-Service-Sprint, Wikidata, Wikidata-Query-Service
Smalyshev merged T223773: MWAPI requests for external links return fewer than expected into T231390: MWAPI can only match one result per page.
Aug 27 2019, 9:46 PM · Wikidata, Wikidata-Query-Service
Smalyshev added a comment to T223773: MWAPI requests for external links return fewer than expected.

I've created T231390: MWAPI can only match one result per page for handling the multiple values in one result issue, so that we have clearly focused task.

Aug 27 2019, 9:45 PM · Discovery-Wikidata-Query-Service-Sprint, Wikidata, Wikidata-Query-Service
Smalyshev created T231390: MWAPI can only match one result per page.
Aug 27 2019, 9:44 PM · Wikidata, Wikidata-Query-Service
Smalyshev moved T230750: dpkg error when using role::wdqs::labs role from Incoming to Operations/SRE on the Wikidata-Query-Service board.
Aug 27 2019, 9:40 PM · Wikidata, Wikidata-Query-Service
Smalyshev moved T230754: WDQS labs role role::wdqs::labs fails when not finding /srv/wdqs from Incoming to Operations/SRE on the Wikidata-Query-Service board.
Aug 27 2019, 9:40 PM · Wikidata, Wikidata-Query-Service
Smalyshev moved T230755: WDQS labs role role::wdqs::labs creates /srv/wdqs/blazegraph with wrong permissions from Incoming to Operations/SRE on the Wikidata-Query-Service board.
Aug 27 2019, 9:39 PM · Wikidata-Query-Service, Wikidata
Smalyshev moved T230840: Set up proper prefix configuration for RDF export on Commons from Incoming to SDAW on the Wikidata-Query-Service board.
Aug 27 2019, 9:39 PM · Patch-For-Review, WikibaseMediaInfo, Wikidata-Query-Service, SDC General, Commons, Wikidata
Smalyshev moved T230856: RDF dump performance for SDC from Incoming to SDAW on the Wikidata-Query-Service board.
Aug 27 2019, 9:39 PM · Structured-Data-Backlog (Current Work), Dumps-Generation, WikibaseMediaInfo, Wikidata-Query-Service, SDC General, Commons, Wikidata
Smalyshev triaged T230862: Create a way to filter only WB-related changes from Commons recentchanges as High priority.
Aug 27 2019, 9:39 PM · MW-1.35-notes (1.35.0-wmf.10; 2019-12-10), Structured-Data-Backlog (Current Work), Platform Team Workboards (Clinic Duty Team), Patch-For-Review, Structured Data Engineering, MediaWiki-Action-API, Wikidata-Query-Service, SDC General, Commons, Wikidata
Smalyshev moved T230862: Create a way to filter only WB-related changes from Commons recentchanges from Incoming to SDAW on the Wikidata-Query-Service board.
Aug 27 2019, 9:39 PM · MW-1.35-notes (1.35.0-wmf.10; 2019-12-10), Structured-Data-Backlog (Current Work), Platform Team Workboards (Clinic Duty Team), Patch-For-Review, Structured Data Engineering, MediaWiki-Action-API, Wikidata-Query-Service, SDC General, Commons, Wikidata
Smalyshev updated the task description for T222321: Make /entity/ alias work for Commons.
Aug 27 2019, 7:49 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), Discovery-Search (Current work), Wikimedia-Apache-configuration, WikibaseMediaInfo, Wikidata-Query-Service, SDC General, Commons, Wikidata
Smalyshev reassigned T222321: Make /entity/ alias work for Commons from Smalyshev to Gehel.
Aug 27 2019, 7:47 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)), Discovery-Search (Current work), Wikimedia-Apache-configuration, WikibaseMediaInfo, Wikidata-Query-Service, SDC General, Commons, Wikidata
Smalyshev added a comment to T230862: Create a way to filter only WB-related changes from Commons recentchanges.

RecentChanges has many flaws (for example, it is not a reliable stream as timestamps are not sequential and it can't be queried by RC ID - see https://gerrit.wikimedia.org/r/c/mediawiki/core/+/302368) but it is the only way to get change stream for a wiki without setting up Kafka, etc. as I understand. So I imagine until we get containers with all that stuff working we're stuck with RC as the only option to get changes in public.

Aug 27 2019, 7:40 PM · MW-1.35-notes (1.35.0-wmf.10; 2019-12-10), Structured-Data-Backlog (Current Work), Platform Team Workboards (Clinic Duty Team), Patch-For-Review, Structured Data Engineering, MediaWiki-Action-API, Wikidata-Query-Service, SDC General, Commons, Wikidata
Smalyshev added a comment to T230288: Allow CHUNK value to be passed in as an option for munge.sh.

@Addshore 0.3.2 should be up already.

Aug 27 2019, 7:00 PM · Discovery-Wikidata-Query-Service-Sprint, User-Addshore, Wikidata-Query-Service, Wikidata
Smalyshev claimed T228348: Category graph includes deleted categories.
Aug 27 2019, 5:33 PM · Discovery-Search (Current work), MW-1.34-notes (1.34.0-wmf.21; 2019-09-03), User-Smalyshev, Wikidata-Query-Service, Wikidata
Smalyshev triaged T231264: Lexeme tests produce errors on merge as High priority.
Aug 27 2019, 5:17 AM · User-Addshore, Wikidata-Campsite, Wikimedia-production-error (ARCHIVED -- Shared Build Failure), Wikidata Lexicographical data, Wikidata
Smalyshev created T231264: Lexeme tests produce errors on merge.
Aug 27 2019, 5:17 AM · User-Addshore, Wikidata-Campsite, Wikimedia-production-error (ARCHIVED -- Shared Build Failure), Wikidata Lexicographical data, Wikidata

Aug 26 2019

Smalyshev closed T229377: Make WDQS deploy not require chrome tests as Resolved.
Aug 26 2019, 6:36 AM · Discovery-Wikidata-Query-Service-Sprint, Wikidata-Query-Service, Wikidata
Smalyshev closed T230288: Allow CHUNK value to be passed in as an option for munge.sh as Resolved.
Aug 26 2019, 6:35 AM · Discovery-Wikidata-Query-Service-Sprint, User-Addshore, Wikidata-Query-Service, Wikidata
Smalyshev added a comment to T221917: Create RDF dump of structured data on Commons.

I tried to manually dump the mediainfo entries over the weekend, it took 376 minutes for 4 shards (a lot, but less than I expected) and produces 1724656 items. Does not seem to produce significant load on DB so far - but it gives about 20 items/second, which seems to be too slow. If we ever get all files having items, that'd take 4 days to process over 8 shards, probably more since DB access will get slower, right now they are not to slow because there's only 2% of files that have items, so not too many DB queries.

Aug 26 2019, 6:35 AM · Patch-For-Review, Dumps-Generation, MW-1.34-notes (1.34.0-wmf.10; 2019-06-18), Wikidata-Query-Service, Commons, Wikidata

Aug 25 2019

Smalyshev added a comment to T229608: Support SDC URIs in WDQS URI schemes.

@Multichill eventually yes, but since they are not being used anywhere yet it's too early to document them. Once RDF export is properly set up to use these prefixes then we can document them officially.

Aug 25 2019, 8:17 PM · Discovery-Wikidata-Query-Service-Sprint, Patch-For-Review, User-Smalyshev, Wikidata-Query-Service, SDC General, Wikidata

Aug 23 2019

Smalyshev closed T230244: Restore wdqs1009 to its role as auto-deploy testing as Resolved.
Aug 23 2019, 11:40 PM · Discovery-Wikidata-Query-Service-Sprint, Wikidata-Query-Service, Wikidata
Smalyshev triaged T230244: Restore wdqs1009 to its role as auto-deploy testing as Medium priority.
Aug 23 2019, 11:39 PM · Discovery-Wikidata-Query-Service-Sprint, Wikidata-Query-Service, Wikidata
Smalyshev closed T229608: Support SDC URIs in WDQS URI schemes, a subtask of T141602: [Objective Fiscal 19-20/Q4] (9) Provide a Proof of Concept SPARQL endpoint in support of SDoC project, as Resolved.
Aug 23 2019, 12:07 AM · Discovery-Search (Current work), Epic, Wikidata-Query-Service, SDC General, Commons, Wikidata
Smalyshev closed T229608: Support SDC URIs in WDQS URI schemes as Resolved.
Aug 23 2019, 12:07 AM · Discovery-Wikidata-Query-Service-Sprint, Patch-For-Review, User-Smalyshev, Wikidata-Query-Service, SDC General, Wikidata