#547: Parallel Python at Anyscale with Ray
Talk Python To Me6 Touko

#547: Parallel Python at Anyscale with Ray

When OpenAI trained GPT-3, they didn't roll their own orchestration layer. They used Ray, an open source Python framework born out of the same Berkeley research lab lineage that gave us Apache Spark. And here's the twist: Ray was originally built for reinforcement learning research, then quietly faded as RL hit a wall. Until ChatGPT showed up. Suddenly reinforcement learning was back, as the post-training step that turns a raw language model into something genuinely useful. Edward Oakes and Richard Liaw, two founding engineers behind Ray and Anyscale, join me on Talk Python to tell that story. We'll trace Ray from its RISE Lab origins at UC Berkeley to powering some of the largest training runs in the world. We'll talk about what Ray actually is, a distributed execution engine for AI workloads, and how a few lines of Python become work running across hundreds of GPUs. We'll cover Ray Data for multimodal pipelines, the dashboard, the VS Code remote debugger, KubRay for Kubernetes, and where Ray fits alongside Dask, multiprocessing, and asyncio. If you've ever stared at a single-machine Python script and thought, "there has to be a better way to scale this", this one's for you

Tämä jakso on lisätty Podme-palveluun avoimen RSS-syötteen kautta eikä se ole Podmen omaa tuotantoa. Siksi jakso saattaa sisältää mainontaa.

Jaksot(562)

#563: Getting Started with Rust as Python Devs

#563: Getting Started with Rust as Python Devs

Lint the entire CPython code base from scratch. It takes 0.3 seconds. Three blinks of an eye. That is ruff, and it is written in Rust. So are Pydantic, Polars, uv, and Granian. Rust shows up in Python...

16 Syys 1h 10min

#562: DuckLake: The Lakehouse That's Just SQL and Parquet

#562: DuckLake: The Lakehouse That's Just SQL and Parquet

How many files does your query read before it reads any data? On some data lakes, you go through JSON and metadata files first, just to learn which Parquet files matter. DuckLake asks one SQL question...

10 Syys 1h 11min

#561: TonIO, a Multi-threaded Async Runtime for Python

#561: TonIO, a Multi-threaded Async Runtime for Python

How many cores does your machine have, 10, 18? Your async Python code uses just one of them. That isn't a bug in asyncio. That's the design, and optimizing event loops to be faster by 20% doesn't chan...

4 Syys 1h 15min

#560: Building a Research OS: From Django to 30,000 Samples

#560: Building a Research OS: From Django to 30,000 Samples

In 2020, a gastroenterologist in Glasgow did the math on his new research study and came up with 30,000 samples, arriving over two years from three cities and a dozen hospitals. He asked around about ...

26 Elo 1h 2min

#559: 12 Things You Should (and Shouldn't) Do in AWS

#559: 12 Things You Should (and Shouldn't) Do in AWS

Your site is down. It's 3am. Is it a bug, a bill, or a breach? You can't tell yet, and everyone is watching you find out. Matt Lea has spent fifteen years being the person companies call when an outag...

19 Elo 1h 7min

#558: Hyper-Personal Software with Python

#558: Hyper-Personal Software with Python

Every company has one. The little internal tool that Jane built back in 2021, and then Jane left. Nobody understands it, nobody will touch it. There are two unwritten rules around it: don't change it,...

10 Elo 1h 2min

#557: Security of everything at PyCon 2026

#557: Security of everything at PyCon 2026

Security has always been the vegetables of software. Everyone agrees it matters, and somehow it never quite makes it onto the plate. At PyCon US this year, that changed. For the first time ever, secur...

2 Elo 1h 8min

#556: Updates on Django's Async Story

#556: Updates on Django's Async Story

For years, "Django and async" came with an asterisk. The docs themselves warned you off it. Scary performance notes, a story that felt half-finished. Well, that story just got rewritten, literally, an...

26 Heinä 1h 4min