Annotator Bias
Data Skeptic23 Marras 2019

Annotator Bias

The modern deep learning approaches to natural language processing are voracious in their demands for large corpora to train on. Folk wisdom estimates used to be around 100k documents were required for effective training. The availability of broadly trained, general-purpose models like BERT has made it possible to do transfer learning to achieve novel results on much smaller corpora.

Thanks to these advancements, an NLP researcher might get value out of fewer examples since they can use the transfer learning to get a head start and focus on learning the nuances of the language specifically relevant to the task at hand. Thus, small specialized corpora are both useful and practical to create.

In this episode, Kyle speaks with Mor Geva, lead author on the recent paper Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets, which explores some unintended consequences of the typical procedure followed for generating corpora.

Source code for the paper available here: https://github.com/mega002/annotator_bias

Jaksot(590)

Time Series Mini Episodes

Time Series Mini Episodes

Time series topics on Data Skeptic predate our current season.  This holiday special collects three popular mini-episodes from the archive that discuss time series topics with a few new comments from Kyle.

25 Joulu 202136min

Forecasting Motor Vehicle Collision

Forecasting Motor Vehicle Collision

Dr. Darren Shannon, a Lecturer in Quantitative Finance in the Department of Accounting and Finance, University of Limerick, joins us today to talk about his work "Extending the Heston Model to Forecast Motor Vehicle Collision Rates."

20 Joulu 202139min

Deep Learning for Road Traffic Forecasting

Deep Learning for Road Traffic Forecasting

Eric Manibardo, PhD Student at the University of the Basque Country in Spain, comes on today to share his work, "Deep Learning for Road Traffic Forecasting: Does it Make a Difference?"

13 Joulu 202131min

Bike Share Demand Forecasting

Bike Share Demand Forecasting

Daniele Gammelli, PhD Student in Machine Learning at Technical University of Denmark and visiting PhD Student at Stanford University, joins us today to talk about his work "Predictive and Prescriptive Performance of Bike-Sharing Demand Forecasts for Inventory Management."

6 Joulu 202140min

Forecasting in Supply Chain

Forecasting in Supply Chain

Mahdi Abolghasemi, Lecturer at Monash University, joins us today to talk about his work "Demand forecasting in supply chain: The impact of demand volatility in the presence of promotion."

29 Marras 202136min

Black Friday

Black Friday

The retail holiday "black Friday" occurs the day after Thanksgiving in the United States. It's dubbed this because many retail companies spend the first 10 months of the year running at a loss (in the red) before finally earning as much as 80% of their revenue in the last two months of the year. This episode features four interviews with guests bringing unique data-driven perspectives on the topic of analyzing this seeming outlier in a time series dataset.

26 Marras 202144min

Aligning Time Series on Incomparable Spaces

Aligning Time Series on Incomparable Spaces

Alex Terenin, Postdoctoral Research Associate at the University of Cambridge, joins us today to talk about his work "Aligning Time Series on Incomparable Spaces."

22 Marras 202133min

Comparing Time Series with HCTSA

Comparing Time Series with HCTSA

Today we are joined again by Ben Fulcher, leader of the Dynamics and Neural Systems Group at the University of Sydney in Australia, to talk about hctsa, a software package for running highly comparative time-series analysis.

15 Marras 202142min

Suosittua kategoriassa Tiede

rss-mita-tulisi-tietaa
utelias-mieli
tiedekulma-podcast
rss-lihavuudesta-podcast
hippokrateen-vastaanotolla
sotataidon-ytimessa
rss-radplus
filocast-filosofian-perusteet
menologeja-tutkimusmatka-vaihdevuosiin
rss-poliisin-mieli
docemilia
rss-bios-podcast
rss-totta-vai-tuubaa
rss-duodecim-lehti
rss-ammamafia
rss-astetta-parempi-elama-podcast
rss-ilmasto-kriisissa
rss-tervetta-skeptisyytta
rss-metsanomistaja-podcast