User profiles for Jonathan K. Kummerfeld

Jonathan K. Kummerfeld

Senior Lecturer (ie. research tenure-track Asst. Prof.), University of Sydney
Verified email at sydney.edu.au
Cited by 2876

An evaluation dataset for intent classification and out-of-scope prediction

…, A Lee, P Hill, JK Kummerfeld, K Leach… - arXiv preprint arXiv …, 2019 - arxiv.org
Task-oriented dialog systems need to know when a query falls outside their range of
supported intents, but current text classification corpora only define label sets that cover every …

Improving text-to-sql evaluation methodology

…, JK Kummerfeld, L Zhang, K Ramanathan… - arXiv preprint arXiv …, 2018 - arxiv.org
To be informative, an evaluation must measure how well systems generalize to realistic unseen
data. We identify limitations of and propose improvements to current evaluations of text-to…

A large-scale corpus for conversation disentanglement

JK Kummerfeld, SR Gouravajhala, J Peper… - arXiv preprint arXiv …, 2018 - arxiv.org
Disentangling conversations mixed together in a single stream of messages is a difficult task,
made harder by the lack of large manually annotated datasets. We created a new dataset …

[PDF][PDF] Parser showdown at the wall street corral: An empirical investigation of error types in parser output

JK Kummerfeld, D Hall, JR Curran… - Proceedings of the 2012 …, 2012 - aclanthology.org
Constituency parser performance is primarily interpreted through a single metric, F-score on
WSJ section 23, that conveys no linguistic information regarding the remaining errors. We …

Spatiotemporal Hierarchy of Relaxation Events, Dynamical Heterogeneities,<? format?> and Structural Reorganization in a Supercooled Liquid

R Candelier, A Widmer-Cooper, JK Kummerfeld… - Physical review …, 2010 - APS
We identify the pattern of microscopic dynamical relaxation for a two-dimensional glass-forming
liquid. On short time scales, bursts of irreversible particle motion, called cage jumps, …

Factors influencing the surprising instability of word embeddings

L Wendlandt, JK Kummerfeld, R Mihalcea - arXiv preprint arXiv …, 2018 - arxiv.org
Despite the recent popularity of word embedding methods, there is only a small body of work
exploring the limitations of these representations. In this paper, we consider one aspect of …

A mechanistic understanding of alignment algorithms: A case study on dpo and toxicity

…, X Bai, I Pres, M Wattenberg, JK Kummerfeld… - arXiv preprint arXiv …, 2024 - arxiv.org
While alignment algorithms are now commonly used to tune pre-trained language models
towards a user's preferences, we lack explanations for the underlying mechanisms in which …

Tools for automated analysis of cybercriminal markets

RS Portnoff, S Afroz, G Durrett, JK Kummerfeld… - Proceedings of the 26th …, 2017 - dl.acm.org
Underground forums are widely used by criminals to buy and sell a host of stolen items,
datasets, resources, and criminal services. These forums contain important resources for …

Leveraging similar users for personalized language modeling with limited data

C Welch, C Gu, JK Kummerfeld… - Proceedings of the …, 2022 - aclanthology.org
Personalized language models are designed and trained to capture language patterns
specific to individual users. This makes them more accurate at predicting what a user will write. …

The eighth dialog system technology challenge

…, M Adada, M Huang, L Lastras, JK Kummerfeld… - arXiv preprint arXiv …, 2019 - arxiv.org
This paper introduces the Eighth Dialog System Technology Challenge. In line with recent
challenges, the eighth edition focuses on applying end-to-end dialog technologies in a …