172: Transformers and Large Language Models

172: Transformers and Large Language Models

172: Transformers and Large Language Models


Intro topic: Is WFH actually WFC?

News/Links:


Book of the Show


Patreon Plug https://www.patreon.com/programmingthrowdown?ty=h


Tool of the Show


Topic: Transformers and Large Language Models

  • How neural networks store information
    • Latent variables
  • Transformers
    • Encoders & Decoders
  • Attention Layers
    • History
      • RNN
        • Vanishing Gradient Problem
      • LSTM
        • Short term (gradient explodes), Long term (gradient vanishes)
    • Differentiable algebra
    • Key-Query-Value
    • Self Attention
  • Self-Supervised Learning & Forward Models
  • Human Feedback
    • Reinforcement Learning from Human Feedback
    • Direct Policy Optimization (Pairwise Ranking)



★ Support this podcast on Patreon ★

Avsnitt(185)

Java

Java

This show covers Java, a semi-compiled language used heavily in web and mobile development. News: Ouya android gaming console, Is C/C++ Worth It?, Engineered Jellyfish. Tools of the BiWeek: Cygwin, MinGW and Macports, uShare. ★ Support this podcast on Patreon ★

3 Aug 20121h 18min

Build Automation

Build Automation

This show covers build automation, scripts to streamline the build process. News: New Top Level Domains, Anatomy of Freemium, LinkedIn Passwords Leaked, $100 Masters Degree. The tools of the biweek are Apache Cordova (Formerly PhoneGap), Xbox Media Center (XBMC). ★ Support this podcast on Patreon ★

27 Juni 20121h 9min

Lua

Lua

This show covers Lua, an interpreted language designed to be embedded into larger applications. News: Facbeook IPO, Javascript Arduino programming, Brain Controlled Robots. The tools of the biweek are JDBM/BerkeleyDB and Audacity. ★ Support this podcast on Patreon ★

29 Maj 20121h 9min

Interface Description Languages (IDLs)

Interface Description Languages (IDLs)

This show covers Interface Descrption Languages (IDLs), such as thrift and protocol buffers. News: Sony fighting with PSP hackers, Pebble smartphone watch, Pacman runs on 0x10C, UF Computer Science cuts. The tools of the biweek are Sublime Text Editor and Luminance HDR. ★ Support this podcast on Patreon ★

26 Apr 20121h 13min

C++

C++

This show covers C++, a general-purpose programming language. News: Visualizing code to fail faster, the future of the used game market, Prince of Persia source code found. The tools of the biweek are sfxr and DC universe online. ★ Support this podcast on Patreon ★

9 Apr 20121h 15min

Dart

Dart

This show covers Dart, a client-side browser alternative to javascript. Dart currently only runs in Chrome, but dart code can be compiled to javascript and then run in any browser. News: Raspberry Pi, California allows autonomous cars, Stanford online courses. The tools of the biweek are Disk Usage Programs (WinDirStat, Disk usage Analyzer, and Disk Inventory X), and Spelunky. ★ Support this podcast on Patreon ★

20 Mars 201256min

Lisp

Lisp

This show covers Lisp, one of the first functional langauge that is often used for academic purposes, but also has a place in industry. The episode talks about Cuda, how to write a Roguelike, becoming a good programming, and DOS games on Chrome. The tools of the biweek are Google Native Client (NaCl), a way to run native C/C++ code in a browser, and Handbrake, a way to make copies of your video DVDs. ★ Support this podcast on Patreon ★

26 Feb 20121h 6min

Javascript

Javascript

This show covers Javascript, the only client-side scripting language built into most modern browsers. The episode talks about Patrick's move to Silicon Valley, Codecademy, The HP Touchpad Fire sale, and Minecraft's Notch challenging Bethesda to Quake 3. The tools of the biweek are Low-Level Virtual Machine (LLVM), an open source virtual machine and suite of compilers that compile and run code from many languages. and Hamachi, a free virtual private networking solution. ★ Support this podcast on Patreon ★

29 Aug 20111h 25min

Populärt inom Politik & nyheter

aftonbladet-krim
p3-krim
fordomspodden
rss-krimstad
motiv
rss-viva-fotboll
flashback-forever
aftonbladet-daily
svenska-fall
rss-sanning-konsekvens
rss-vad-fan-hande
grans
dagens-eko
rss-krimreportrarna
olyckan-inifran
rss-frandfors-horna
blenda-2
krimmagasinet
rss-flodet
rss-svalan-krim