Pre-training language models for natural language processing problems

Pre-training language models for natural language processing problems

When you build a model for natural language processing (NLP), such as a recurrent neural network, it helps a ton if you’re not starting from zero. In other words, if you can draw upon other datasets for building your understanding of word meanings, and then use your training dataset just for subject-specific refinements, you’ll get farther than just using your training dataset for everything. This idea of starting with some pre-trained resources has an analogue in computer vision, where initializations from ImageNet used for the first few layers of a CNN have become the new standard. There’s a similar progression under way in NLP, where simple(r) embeddings like word2vec are giving way to more advanced pre-processing methods that aim to capture more sophisticated understanding of word meanings, contexts, language structure, and more. Relevant links: https://thegradient.pub/nlp-imagenet/

Denne episoden er hentet fra en åpen RSS-feed og er ikke publisert av Podme. Den kan derfor inneholde annonser.

Episoder(316)

Invisible LLM Failures and AI Fluency with Chris Potts (Stanford)

Invisible LLM Failures and AI Fluency with Chris Potts (Stanford)

What happens when a Stanford linguistics professor turns his attention to AI chatbots — and the surprisingly invisible ways humans misunderstand them? Chris Potts joins the show to unpack the hidden f...

20 Jul 41min

Still summer break: back next week

Still summer break: back next week

Still summer break: back next week by Katie Malone

13 Jul 25s

Summer break: back soon

Summer break: back soon

Summer break: back soon by Katie Malone

6 Jul 36s

Interviewing the Linear Digressions Agents (The Agents Season, Episode 11)

Interviewing the Linear Digressions Agents (The Agents Season, Episode 11)

After a five-year hiatus, the podcast that burned out partly over the tedium of writing episode descriptions is back — and using AI agents to handle exactly that task. The season-11 finale turns the l...

28 Jun 37min

Agent Economics (The Agents Season, Episode 10)

Agent Economics (The Agents Season, Episode 10)

What if building more highways made your commute *slower*? That's the paradox at the heart of AI agent economics: even as per-token inference costs have plummeted dramatically over the past two years,...

22 Jun 24min

Agent Trust, Oversight and Control (The Agents Season, Episode 9)

Agent Trust, Oversight and Control (The Agents Season, Episode 9)

Capabilities get all the attention when it comes to AI agents — but what happens when a highly capable agent makes a bad decision in the real world? Trust, oversight, and control are the unglamorous b...

15 Jun 25min

Many Agents, Many Problems (The Agents Season, Episode 8)

Many Agents, Many Problems (The Agents Season, Episode 8)

Whether you work best solo or thrive in a team, you know collaboration is complicated — and it turns out AI agents face the same tensions. This episode dives into multi-agent systems, exploring how ne...

8 Jun 28min

How Do You Evaluate An AI Agent? (The Agents Season, Episode 7)

How Do You Evaluate An AI Agent? (The Agents Season, Episode 7)

Knowing when an AI agent has failed sounds straightforward — until it isn't. Agents have a frustrating habit of finishing confidently while quietly doing the wrong thing, or looping endlessly without ...

1 Jun 31min

Populært innen Teknologi

lydartikler-fra-aftenposten
romkapsel
teknisk-sett
tomprat-med-gunnar-tjomlid
shifter
elektropodden
rss-kunstig-intelligens-med-elisabeth-maren-og-morten
energi-og-klima
nasjonal-sikkerhetsmyndighet-nsm
fornybaren
rss-alt-som-gar-pa-strom
pedagogisk-intelligens
rss-produktledelse-skjar-vekk-skiten
rss-iapodden
rss-bak-skyen
hans-petter-og-co
rss-hvorfor-ble-det-sann
rss-innovasjonslederen
rss-plateprat
rss-ki-til-kaffen