Sid Anand on Building Agari’s Cloud-native Data Pipelines with AWS Kinesis and Serverless
The InfoQ Podcast9 Kesä 2017

Sid Anand on Building Agari’s Cloud-native Data Pipelines with AWS Kinesis and Serverless

Wesley Reisz talks to Sid Anand, a data architect at cybersecurity company Agari, about building cloud-native data pipelines. The focus of their discussion is around a solution Agari uses that is built from Amazon Kinesis Streams, serverless functions, and auto scaling groups. Sid Anand is an architect at Agari, and a former technical architect at eBay, Netflix, and LinkedIn. He has 15 years of data infrastructure experience at scale, is a PMC for Apache Airflow, and is also a program committee chair for QCon San Francisco and QCon London. Why listen to this podcast - Real-time data pipeline processing is very latency sensitive - Micro-batching allows much smaller amounts of data to be processed - Use the appropriate data store (or stores) to support the use of the dataIngesting data quickly into a clean database with minimal indexes can be fast - Communicate using a messaging system that supports schema evolution More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2rJU9nB You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Want to see extented shownotes? Check the landing page on InfoQ: http://bit.ly/2rJU9nB

Tämä jakso on lisätty Podme-palveluun avoimen RSS-syötteen kautta eikä se ole Podmen omaa tuotantoa. Siksi jakso saattaa sisältää mainontaa.

Jaksot(396)

AI, ML, and Data Engineering InfoQ Trends Report 2026

AI, ML, and Data Engineering InfoQ Trends Report 2026

In this episode of the podcast, members of the InfoQ AI and MIL editorial staff and friends of InfoQ discuss the current trends in the domain of AI, ML and Data Engineering. One of the regular featur...

30 Loka 1h 11min

The Future of Web Apps: Will AI Agents Generate UIs on the Fly?

The Future of Web Apps: Will AI Agents Generate UIs on the Fly?

Rachel Lee Nabors, former React core team member, dives into the architecture and philosophy behind local-first applications, exploring how small language models (SLMs) running directly on-device can ...

19 Loka 47min

What Does It Really Mean To Program With AI: A Conversation with Joy Ebertz

What Does It Really Mean To Program With AI: A Conversation with Joy Ebertz

In this podcast, Michael Stiefel spoke to Joy Ebertz about how the use of artificial intelligence is changing the way architects and developers do their jobs. Focusing on the long term success of the ...

12 Loka 44min

Future Cybersecurity: Hardware Memory Safety, Automated Governance and Post-Quantum Cryptography

Future Cybersecurity: Hardware Memory Safety, Automated Governance and Post-Quantum Cryptography

In this episode, Chris Swan, engineer at Atsign and security track host at QCon London, reflects on the ideas shared during QCon London 2026 and teases topics of the future edition. The discussion exp...

5 Loka 34min

The AI Revolution Fails Without Psychological Safety For Developers: A Conversation with Erin Doyle

The AI Revolution Fails Without Psychological Safety For Developers: A Conversation with Erin Doyle

In this podcast, Michael Stiefel spoke to Erin Doyle about how artificial intelligence tools are transforming the software development process, and how they are affecting software engineers. Architect...

28 Syys 54min

The Future of AI: From Enterprise Adoption to Open Source Sovereignty

The Future of AI: From Enterprise Adoption to Open Source Sovereignty

In this episode, a panel of AI experts, Meryem Arik (DoubleWord), Clara Higuera Cabañes (BBVA), and Jeff Smith (C Proof), demystify the current state of AI in the enterprise. The discussion navigates ...

25 Syys 52min

Securing AI Agents: Identity, Authorisation, and the DPACT Framework

Securing AI Agents: Identity, Authorisation, and the DPACT Framework

In this episode, Sahil Agarwal talks about the critical challenges of identity, authorisation, and security in the age of AI agents. Sahil introduces the DPACT framework (Delegation, Policy, Auditabil...

21 Syys 41min

How Will We Train Developers If AI Does the Routine Work: A Conversation with Scott Hanselman

How Will We Train Developers If AI Does the Routine Work: A Conversation with Scott Hanselman

In this podcast Michael Stiefel spoke to Scott Hanselman about developing new software engineers when artificial intelligence agents are doing most of the work on which junior developers were trained....

14 Syys 36min