About Data Pipelines

About Data Pipelines

Lars dove into data pipelines, and emerged bearing arrows and wishing for a lot fewer copies.

What is there to think about regarding data pipelines, what is interesting about them?

Which tools are out there, and why might you want to use them?

Why all this talk about making fewer copies of data?

What does Lars' current ideal pipeline look like, and where does Elixir fit in?

Links

Quotes

  • I've been reading a lot about data pipelines
  • What's so special about data pipelines?
  • There's a lot of special tooling
  • There's a lot of bad, bad tooling
  • Less than optimal tooling
  • Converging on something biggerlk
  • He got me eventually
  • All of your steps in one bucket
  • What tools do you associate with data?
  • I inherited a data pipeline
  • BashReduce
  • Iterate on the L and the T
  • The modern data stack
  • And then you demand more work
  • No unnecessary copies
  • Barely a copy
  • Reconnecting with my Python roots

Tämä jakso on lisätty Podme-palveluun avoimen RSS-syötteen kautta eikä se ole Podmen omaa tuotantoa. Siksi jakso saattaa sisältää mainontaa.

Jaksot(65)

About Ending Things

About Ending Things

The End.LinksLADOKSanne Kalkman - companies should hire junior developersMünchenbryggerietThe art of gatheringDead dog partyNobody wants thisNeon genesis evangelionGhost in the shell: stand alone comp...

20 Tammi 202540min

About the Least Powerful Abstraction

About the Least Powerful Abstraction

Imagine Andreas going around making annoying electronic sounds all the time.Strike that. Andreas and Lars discuss using less power - less fancy abstractions - to make things easier to understand. Andr...

23 Joulu 202442min

About Licenses

About Licenses

How do people learn about licenses?If you entered into software in a certain way, it's easy to assume that everyone is a part-time license attorney. But how do other people pick up license knowledge? ...

25 Marras 202445min

About Learning New Languages

About Learning New Languages

Everyone's favorite idempotent podcast returns to discuss learning new languages and concepts. Can mixing and matching new concepts and syntax help or hinder language adoption? A new concept but a fam...

28 Loka 202442min

About C

About C

Wherein the wonders of C are explored.But first, let Andreas tell you what's so great about Chalmers' approach to teaching computer engineering. Spoiler: starting with Haskell, close to math.The tooli...

14 Loka 202453min

About Defining Functional Programming

About Defining Functional Programming

What is functional programming?Andreas grabs his whiteboard and his Turing machine, and starts from laziness, while Lars thinks of immutability, functions, and data.Is syntax important for being funct...

30 Syys 202437min

About Giving Talks

About Giving Talks

Lars wants a less demanding way to prepare for giving talks, but he doesn't have the time right now.Andreas knows a cheat code for public speaking. Lars uses slides like a blunt instrument.How should ...

2 Syys 202427min

About Developer Experience

About Developer Experience

What are people talking about when they talk about developer experience? Pretty colors in the terminal?What is worth improving, what is not? Lars has thoughts about all of developer experience, not le...

19 Elo 202434min