Postgres FM7 Jun 2024

To 100TB, and beyond!

Michael and Nikolay are joined by three special guests for episode 100 who have all scaled Postgres to significant scale — Arka Ganguli from Notion, Sammy Steele from Figma, and Derk van Veen from Adyen. They cover how their setup has evolved, what their plans are for the future, and get into the weeds of some fun and interesting challenges along the way!

Links to some of the things discussed:

Arka Ganguli from Notion https://postgres.fm/people/arka-ganguli
Sammy Steele from Figma https://postgres.fm/people/sammy-steele
Derk van Veen from Adyen https://postgres.fm/people/derk-van-veen
Thank you to yerrysherry on Reddit for the idea! https://www.reddit.com/r/PostgreSQL/comments/1cn8ajh/what_should_we_do_for_episode_100_of_postgres_fm
Lessons learned from sharding Postgres at Notion (October 2021) https://www.notion.so/blog/sharding-postgres-at-notion
Adding Postgres capacity (again) with zero downtime (July 2023) https://www.notion.so/blog/the-great-re-shard
How Figma’s databases team lived to tell the scale (March 2024) https://www.figma.com/blog/how-figmas-databases-team-lived-to-tell-the-scale
Updating a 50 terabyte PostgreSQL database (March 2018) https://www.adyen.com/knowledge-hub/updating-a-50-terabyte-postgresql-database
Part 1: Introduction to Table Partitioning (July 2023) https://www.adyen.com/knowledge-hub/introduction-to-table-partioning
Part 2: Partitioning at Adyen (October 2023) https://www.adyen.com/knowledge-hub/partitioning-at-adyen
Part 3: Maintenance Under Pressure (January 2024) https://www.adyen.com/knowledge-hub/maintenance-under-pressure
Fighting PostgreSQL write amplification with HOT updates (May 2022) https://www.adyen.com/knowledge-hub/postgresql-hot-updates
Tracking HOT updates and tuning FillFactor (October 2022) https://www.adyen.com/knowledge-hub/postgresql-hot-updates-part2
Partitioning your Postgres tables for 20X better performance (upcoming talk by Derk at Posette) https://www.citusdata.com/posette/speakers/derk-van-veen
Figma, including Sammy’s team, is hiring https://www.figma.com/careers
Notion’s engineering team is hiring https://www.notion.so/careers
Adyen’s engineering team is hiring https://careers.adyen.com

~~~

What did you like or not like? What should we discuss next time? Let us know via a YouTube comment, on social media, or by commenting on our Google doc!

~~~

Postgres FM is produced by:

Michael Christofides, founder of pgMustard
Nikolay Samokhvalov, founder of Postgres.ai

With special thanks to:

Jessie Draws for the elephant artwork

Denne episoden er hentet fra en åpen RSS-feed og er ikke publisert av Podme. Den kan derfor inneholde annonser.

Episoder(182)

pgBackRest

Nik and Michael are joined by David Steele to talk all things pgBackRest. Here are some links to things they mentioned: David Steele https://postgres.fm/people/david-steelepgBackRest https://pgbackres...

3 Jul 1h 13min

autovacuum

Nik and Michael discuss autovacuum, including what it does, and the basics of why and how to tune it. Here are some links to things they mentioned: autovacuum https://www.postgresql.org/docs/current/...

29 Mai 46min

pg_flight_recorder

Nik and Michael are joined by David Ventimiglia to discuss pg_flight_recorder, a new tool he created for monitoring a Postgres database from within. Here are some links to things they mentioned: David...

15 Mai 43min

PgQue

Nik and Michael discuss Nik's new project PgQue, a descendent of Skype's PgQ, for running queue-like workloads in Postgres. Here are some links to things they mentioned: Our first episode on Queues in...

8 Mai 47min

pg_wait_tracer

Nik and Michael are joined by Dmitry Fomin to discuss his new tool pg_wait_tracer, as well as changes that could be made to core to allow wait event tracing with lower overhead, and on managed service...

24 Apr 44min

Schema design checklist

Nik and Michael discuss a list of things to check when designing new schema in Postgres. Here are some links to things they mentioned:Use BIGINT in Postgres (blog post by Ryan Lambert) https://blog.ru...

17 Apr 45min

What’s Missing in Postgres?

Nik and Michael are joined by Bruce Momjian to discuss his new talk "What’s Missing in Postgres?" Here are some links to things they mentioned: Bruce Momjian https://postgres.fm/people/bruce-momjianED...

3 Apr 43min

Long-running transactions

Nik and Michael discuss long-running transactions, including when they're harmless, when they cause issues, and how to mitigate those issues. Here are some links to things they mentioned: transaction_...

27 Mar 48min