#84 LAURA RUIS - Large language models are not zero-shot communicators [NEURIPS UNPLUGGED]
Machine Learning Street Talk (MLST)6 Joulu 2022

#84 LAURA RUIS - Large language models are not zero-shot communicators [NEURIPS UNPLUGGED]

In this NeurIPSs interview, we speak with Laura Ruis about her research on the ability of language models to interpret language in context. She has designed a simple task to evaluate the performance of widely used state-of-the-art language models and has found that they struggle to make pragmatic inferences (implicatures). Tune in to learn more about her findings and what they mean for the future of conversational AI.


Laura Ruis

https://www.lauraruis.com/

https://twitter.com/LauraRuis


BLOOM

https://bigscience.huggingface.co/blog/bloom


Large language models are not zero-shot communicators [Laura Ruis, Akbir Khan, Stella Biderman, Sara Hooker, Tim Rocktäschel, Edward Grefenstette]

https://arxiv.org/abs/2210.14986


[Zhang et al] OPT: Open Pre-trained Transformer Language Models

https://arxiv.org/pdf/2205.01068.pdf


[Lampinen] Can language models handle recursively nested grammatical structures? A case study on comparing models and humans

https://arxiv.org/pdf/2210.15303.pdf


[Gary Marcus] Horse rides astronaut

https://garymarcus.substack.com/p/horse-rides-astronaut


[Gary Marcus] GPT-3, Bloviator: OpenAI’s language generator has no idea what it’s talking about

https://www.technologyreview.com/2020/08/22/1007539/gpt3-openai-language-generator-artificial-intelligence-ai-opinion/


[Bender et al] On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?

https://dl.acm.org/doi/10.1145/3442188.3445922


[janus] Simulators (Less Wrong)

https://www.lesswrong.com/posts/vJFdjigzmcXMhNTsx/simulators

Tämä jakso on lisätty Podme-palveluun avoimen RSS-syötteen kautta eikä se ole Podmen omaa tuotantoa. Siksi jakso saattaa sisältää mainontaa.

Jaksot(271)

What Most People Get Wrong About Evolution | Akarsh Kumar

What Most People Get Wrong About Evolution | Akarsh Kumar

A lot of people in AI treat evolution as a dumb fallback, basically random search for when you can't take a gradient. Akarsh Kumar thinks that is wrong. Selection hangs on to partial solutions, so mut...

10 Loka 47min

How a Voice Agent Learns the Rhythm of Conversation — Shawn Wen

How a Voice Agent Learns the Rhythm of Conversation — Shawn Wen

Tsung-Hsien (Shawn) Wen, CTO of PolyAI, tells Tim Scarfe why voice agents are harder than text agents. Voice adds time, and a good conversation depends on adapting to the person on the line, not just ...

1 Loka 1h 10min

Who Checks a Proof No Human Can Read? — Leo de Moura

Who Checks a Proof No Human Can Read? — Leo de Moura

Leonardo de Moura created Lean and co-created Z3. ---This episode is sponsored by Parallel.Parallel, where agents find answers: web search, extraction and deep research APIs built for AI agents.Start ...

30 Syys 1h 14min

When AI Research Starts Moving Faster Than Human Research - Zhengyao Jiang

When AI Research Starts Moving Faster Than Human Research - Zhengyao Jiang

Weco let an AI coding agent rewrite the harness around another agent for eight days: its code, prompts and tools, while the underlying language model stayed fixed. Tim Scarfe asks Weco co-founder Zhen...

26 Syys 43min

How Deep Learning Finally Cracked Messy Tables - Frank Hutter

How Deep Learning Finally Cracked Messy Tables - Frank Hutter

Frank Hutter, co-founder of Prior Labs, talks about TabPFN, a tabular foundation model that makes predictions in a single forward pass, and the research behind it.TabPFN is pre-trained on synthetic da...

23 Syys 1h 53min

Why Scaling Prediction Cannot Create Intelligence - Alexander Mattick

Why Scaling Prediction Cannot Create Intelligence - Alexander Mattick

Alexander Mattick is a researcher at Fraunhofer IIS and a PhD researcher at the University of Technology Nuremberg (UTN), and a regular on Yannic Kilcher's Discord. He first came on MLST in 2022, afte...

21 Syys 2h 14min

How Physical AI Learns Across Language, Video and Action — Ming-Yu Liu

How Physical AI Learns Across Language, Video and Action — Ming-Yu Liu

The car making a left turn at the start of this episode was never filmed. Cosmos 3 generated it. Ming-Yu Liu, who leads the Cosmos research at NVIDIA, explains how one model can describe a video, gene...

15 Syys 25min

Speech Recognition Is Not a Solved Problem — Pavan Kumar Reddy

Speech Recognition Is Not a Solved Problem — Pavan Kumar Reddy

Pavan Kumar Reddy leads audio research at Mistral AI. He joins Tim Scarfe for a deep technical tour of Voxtral — and explains why the frontier of deployed voice is still a cascade of specialised model...

14 Syys 1h 42min