Annotator Bias
Data Skeptic23 Nov 2019

Annotator Bias

The modern deep learning approaches to natural language processing are voracious in their demands for large corpora to train on. Folk wisdom estimates used to be around 100k documents were required for effective training. The availability of broadly trained, general-purpose models like BERT has made it possible to do transfer learning to achieve novel results on much smaller corpora.

Thanks to these advancements, an NLP researcher might get value out of fewer examples since they can use the transfer learning to get a head start and focus on learning the nuances of the language specifically relevant to the task at hand. Thus, small specialized corpora are both useful and practical to create.

In this episode, Kyle speaks with Mor Geva, lead author on the recent paper Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets, which explores some unintended consequences of the typical procedure followed for generating corpora.

Source code for the paper available here: https://github.com/mega002/annotator_bias

Episoder(590)

Your Consent is Worth 75 Euros a Year

Your Consent is Worth 75 Euros a Year

People who do not want their data tracked and shared online can pay a token for a cookie paywall. But are the websites keeping to their side of the bargain? Victor Morel, a Postdoc candidate at the Chalmers University of Technology joins us to discuss his work around auditing the activities of cookie paywalls. He discussed the findings from his analysis and proffers some solutions to making cookie paywalls more transparent.

7 Nov 202224min

Automated Email Generation for Targeted Attacks

Automated Email Generation for Targeted Attacks

The advancement of generative language models has been a force for good, but also for evil. On the show, Avisha Das, a post-doctoral scholar at the University of Texas Health Center, joins us to discuss how attackers use machine learning to create unsuspecting phishing emails. She also discussed how she used RNN for automated email generation, with the goal of defeating statistical detectors.

31 Okt 202245min

Tribal Marketing

Tribal Marketing

Peter Gloor, a Research Scientist at the MIT Center for Collective Intelligence, takes us on a new world of tribe classification. He extensively discussed the need for such classification on the internet and how he built a machine learning model that does it. Listen to find out more!

24 Okt 202237min

Nano-targetted Facebook Ads

Nano-targetted Facebook Ads

17 Okt 202244min

Debiasing GPT-3 Job Ads

Debiasing GPT-3 Job Ads

We hear about the impeccable achievements of GPT-3 models, but such large generative models come with their bias. On the show today, Conrad Borchers, a Ph.D. student in Human-Computer Interaction, joins us to discuss the bias in GPT-3 for job ads and how such large models can be de-biased. Listen to learn more!

10 Okt 202248min

ML Ops in Production

ML Ops in Production

Moses Guttman from Clear ML joins us to share insights about how organizations leveraging machine learning keep their programs on track. While many parallels exist between the software development life cycle (SWLC) and the machine learning development life cycle, successful deployments of ML in production have demonstrated that a unique set of tools is required. Moses and I discuss the emergence of ML Ops, success stories, and how modern teams leverage tools like Clear ML's open source solution to maximize the value of ML in the organization.

6 Okt 202241min

Ad Network Tomography

Ad Network Tomography

Data sharing in the ad tech space has largely been a black box system. While it is obvious the data is being collected, the data sharing process is obscure to users. On the show today, Maaz Bin Musa and Rishab, both researchers at the University of Iowa, speak about the importance of data transparency and their tool, ATOM for data transparency. Listen to find out how ATOM uncovers data-sharing relationships in the ad-tech space.

3 Okt 202235min

First Party Tracking Cookies

First Party Tracking Cookies

When you accept cookies on a website, you cannot tell whether the cookies are used for tracking your personal data or not. Shaoor Munir's machine learning model does that. On the show today, the Ph.D student at the University of California, discussed the world of first-party cookies and how he developed a machine learning model that predicts whether a first-party cookie is used for tracking purposes.

26 Sep 202235min

Populært innen Vitenskap

fastlegen
rekommandert
fremtid-pa-frys
jss
tingenes-tilstand
rss-rekommandert
tomprat-med-gunnar-tjomlid
vett-og-vitenskap-med-gaute-einevoll
villmarksliv
rss-paradigmepodden
sinnsyn
dekodet-2
rss-nysgjerrige-norge
forskningno
doktor-fives-podcast
nordnorsk-historie
fjellsportpodden
rss-overskuddsliv
tidlose-historier
nevropodden