172: Transformers and Large Language Models

172: Transformers and Large Language Models

172: Transformers and Large Language Models


Intro topic: Is WFH actually WFC?

News/Links:


Book of the Show


Patreon Plug https://www.patreon.com/programmingthrowdown?ty=h


Tool of the Show


Topic: Transformers and Large Language Models

  • How neural networks store information
    • Latent variables
  • Transformers
    • Encoders & Decoders
  • Attention Layers
    • History
      • RNN
        • Vanishing Gradient Problem
      • LSTM
        • Short term (gradient explodes), Long term (gradient vanishes)
    • Differentiable algebra
    • Key-Query-Value
    • Self Attention
  • Self-Supervised Learning & Forward Models
  • Human Feedback
    • Reinforcement Learning from Human Feedback
    • Direct Policy Optimization (Pairwise Ranking)



★ Support this podcast on Patreon ★

Avsnitt(185)

Node.js

Node.js

This show covers Node.js: A server-side platform for developing network applications. Books of the Show Jason: You Can Draw in 30 Days: The Fun, Easy Way to Learn to Draw in One Month or Less http://amzn.to/1CsrFsO Patrick: Abyss Beyond Dreams (Peter F. Hamilton) http://amzn.to/1NYmqJZ ★ Support this podcast on Patreon ★

6 Apr 20151h 1min

Unity

Unity

This show covers Unity: A Toolchain and set of three scripting languages primarily for making games. Tools of the show: Jason: Meld http://meldmerge.org/ Patrick: Space Marshals http://www.pixelbite.se/pbsite3/?page_id=7675 ★ Support this podcast on Patreon ★

2 Feb 20151h 11min

Rust

Rust

This show covers Rust: A memory-safe systems programming language. Tools of the show: Jason: This War of Mine Patrick: Vainglory. ★ Support this podcast on Patreon ★

7 Jan 20151h

SIMD

SIMD

This show covers SIMD: A set of languages for fast array operations. Tools of the show: Jason: OpenEmu Patrick: Mint.com. Books of the show: Jason: Emacs Quick Reference Card: http://www.gnu.org/software/emacs/refcards/pdf/refcard.pdf Patrick: The Mote in God’s Eye http://amzn.to/1AwlOaf ★ Support this podcast on Patreon ★

22 Nov 20141h 12min

Funky Languages

Funky Languages

This show covers Funky Languages: esoteric languages that are created mostly for fun. Tools of the show: Jason: Open Broadcaster Software Patrick: Plex. Books of the show: Jason: Impro for Storytellers: http://amzn.to/1sMohKv Patrick: The Android’s Dream (John Scalzi): http://amzn.to/1wHkOL0 ★ Support this podcast on Patreon ★

15 Okt 20141h 11min

Swift

Swift

This show covers Swift, a language developed by Apple for iOS and OS/X. Tools of the show: Jason: Nextdoor Patrick: Waze. Books of the show: Jason: Denial: Why Business Leaders Fail... http://amzn.to/1rvKXbP Patrick: Red Seas Under Red Skies http://amzn.to/1oZplmS ★ Support this podcast on Patreon ★

8 Sep 20141h 2min

Haskell

Haskell

This show covers Haskell, a statically-typed functional language. Tools of the show: Jason: Uber Patrick: Store Coupon Apps. Books of the show: Jason: Start With Why http://amzn.to/1uNg61I Patrick: Childhood's End http://amzn.to/1s8Fmv ★ Support this podcast on Patreon ★

1 Aug 20141h 2min

Databases

Databases

This show covers Databases. Tools of the show: Jason: nvAlt / nvPy Patrick: Arduino. Books of the show: Jason: Negotiating For Dummies http://amzn.to/1jS9Yem Patrick: Steelheart http://amzn.to/1lsRxBv ★ Support this podcast on Patreon ★

26 Juni 20141h 14min

Populärt inom Politik & nyheter

aftonbladet-krim
p3-krim
fordomspodden
rss-krimstad
motiv
flashback-forever
rss-viva-fotboll
aftonbladet-daily
svenska-fall
rss-sanning-konsekvens
rss-vad-fan-hande
grans
rss-krimreportrarna
dagens-eko
rss-frandfors-horna
olyckan-inifran
blenda-2
krimmagasinet
rss-svalan-krim
rss-flodet