User profiles for Jonathan K. Kummerfeld
Jonathan K. KummerfeldSenior Lecturer (ie. research tenure-track Asst. Prof.), University of Sydney Verified email at sydney.edu.au Cited by 2876 |
An evaluation dataset for intent classification and out-of-scope prediction
Task-oriented dialog systems need to know when a query falls outside their range of
supported intents, but current text classification corpora only define label sets that cover every …
supported intents, but current text classification corpora only define label sets that cover every …
Improving text-to-sql evaluation methodology
…, JK Kummerfeld, L Zhang, K Ramanathan… - arXiv preprint arXiv …, 2018 - arxiv.org
To be informative, an evaluation must measure how well systems generalize to realistic unseen
data. We identify limitations of and propose improvements to current evaluations of text-to…
data. We identify limitations of and propose improvements to current evaluations of text-to…
A large-scale corpus for conversation disentanglement
Disentangling conversations mixed together in a single stream of messages is a difficult task,
made harder by the lack of large manually annotated datasets. We created a new dataset …
made harder by the lack of large manually annotated datasets. We created a new dataset …
[PDF][PDF] Parser showdown at the wall street corral: An empirical investigation of error types in parser output
Constituency parser performance is primarily interpreted through a single metric, F-score on
WSJ section 23, that conveys no linguistic information regarding the remaining errors. We …
WSJ section 23, that conveys no linguistic information regarding the remaining errors. We …
Spatiotemporal Hierarchy of Relaxation Events, Dynamical Heterogeneities,<? format?> and Structural Reorganization in a Supercooled Liquid
We identify the pattern of microscopic dynamical relaxation for a two-dimensional glass-forming
liquid. On short time scales, bursts of irreversible particle motion, called cage jumps, …
liquid. On short time scales, bursts of irreversible particle motion, called cage jumps, …
Factors influencing the surprising instability of word embeddings
Despite the recent popularity of word embedding methods, there is only a small body of work
exploring the limitations of these representations. In this paper, we consider one aspect of …
exploring the limitations of these representations. In this paper, we consider one aspect of …
A mechanistic understanding of alignment algorithms: A case study on dpo and toxicity
While alignment algorithms are now commonly used to tune pre-trained language models
towards a user's preferences, we lack explanations for the underlying mechanisms in which …
towards a user's preferences, we lack explanations for the underlying mechanisms in which …
Tools for automated analysis of cybercriminal markets
Underground forums are widely used by criminals to buy and sell a host of stolen items,
datasets, resources, and criminal services. These forums contain important resources for …
datasets, resources, and criminal services. These forums contain important resources for …
Leveraging similar users for personalized language modeling with limited data
C Welch, C Gu, JK Kummerfeld… - Proceedings of the …, 2022 - aclanthology.org
Personalized language models are designed and trained to capture language patterns
specific to individual users. This makes them more accurate at predicting what a user will write. …
specific to individual users. This makes them more accurate at predicting what a user will write. …
The eighth dialog system technology challenge
This paper introduces the Eighth Dialog System Technology Challenge. In line with recent
challenges, the eighth edition focuses on applying end-to-end dialog technologies in a …
challenges, the eighth edition focuses on applying end-to-end dialog technologies in a …