Sam Lehman: What the Reinforcement Learning Renaissance Means for Decentralized AI

Sam Lehman: What the Reinforcement Learning Renaissance Means for Decentralized AI

Join Tommy Shaughnessy from Delphi Ventures as he hosts Sam Lehman, Principal at Symbolic Capital and AI researcher, for a deep dive into the Reinforcement Learning (RL) renaissance and its implications for decentralized AI. Sam recently authored a widely discussed post, "The World's RL Gym", exploring the evolution of AI scaling and the exciting potential of decentralized networks for training next-generation models.

The World’s RL Gym: https://www.symbolic.capital/writing/the-worlds-rl-gym



🎯 Key Highlights


The three phases of AI scaling: Pre-training, Inference Time Compute, and the RL Renaissance.

How DeepMind's novel RL approach (using GRPO) created powerful reasoning models with minimal human data.

Understanding "reasoning traces" and how models learn to "think" longer and more effectively.

The potential downsides of human preference data potentially inhibiting model creativity, drawing parallels to AlphaGo.

Exploring the "World's RL Gym" concept: Decentralizing RL through open environments, diverse tasks, and verified data.

Why open, collaborative RL environments might outperform closed-source labs in generating diverse AI strategies.

The critical role of high-quality base models for successful RL fine-tuning.

Future AI architectures: Continuous learning and the potential of modular Mixture-of-Experts (MoE) models.

Current landscape: Open-source vs. proprietary AI, the challenge of model lock-in, and the role of crypto networks.

Debunking recent claims that "RL is dead" and understanding its true impact.



💡 Want to stay updated with the latest in crypto & AI? Hit subscribe and the notification bell! 🔔



🧠 Follow the Alpha


Tommy's Twitter: @Shaughnessy119

Sam's Twitter: @SPLehman

Symbolic Capital’s Twitter: @symbolicvc



🔗 Connect with Delphi


🌐 Portal: https://delphidigital.io/

🐦 Twitter: https://twitter.com/delphi_digital

💼 LinkedIn: https://www.linkedin.com/company/delphi-digital



🎧 Listen on


Spotify: https://open.spotify.com/show/62PR1RigLG2YN5Pelq6UY9?si=18ac7ccf36ab4753

Apple Podcasts: https://podcasts.apple.com/us/podcast/the-delphi-podcast/id1438148082

Youtube: https://www.youtube.com/channel/UC9Yy99ZlQIX9-PdG_xHj43Q



Timestamps


00:00 - Introduction: Sam Lehman, Symbolic Capital & "The World's RL Gym"

01:30 - History of AI Scaling: Pre-training Era

03:30 - Phase 2: Inference Time Compute Scaling

09:30 - Phase 3: The RL Renaissance & DeepMind Moment

14:30 - How DeepMind Trained R1 without Human Preferences

16:30 - AlphaGo Analogy: Human Data Inhibiting Creativity?

20:30 - Generalizability of RL Training: How Far Does It Go?

22:30 - The "Aha Moment": Models Learning to Think Longer

25:30 - Concept: Decentralized RL & The World's Gym

31:30 - Why Decentralize RL? Open Collaboration vs. Closed Labs

35:00 - Understanding Reasoning Traces

39:00 - Current Decentralized RL Projects (Prime Intellect, General Reasoning)

41:30 - Future Architectures: Continuous Improvement & Modular Models

46:30 - Open Source vs. Proprietary AI: Landscape & Challenges

50:30 - The Lock-In Problem with Foundational Models

52:30 - Is AGI Here? Experiences with GPT-4o

56:30 - Investment Focus in Decentralized AI

59:00 - Modular MoE Models & Jensen's HDEE Paper

1:03:00 - Debunking "RL is Dead" Claims

1:06:00 - Importance of Performant Base Models for RL



Disclaimer


This podcast is strictly informational and educational and is not investment advice or a solicitation to buy or sell any tokens or securities or to make any financial decisions. Do not trade or invest in any project, tokens, or securities based upon this podcast episode. The host and members at Delphi Ventures may personally own tokens or art that are mentioned on the podcast. Our current show features paid sponsorships which may be featured at the start, middle, and/or the end of the episode. These sponsorships are for informational purposes only and are not a solicitation to use any product, service or token.

Avsnitt(468)

Andrej Radonjic: Grass's Two Million Data Scraping Users Enabling Crypto x AI on Solana

Andrej Radonjic: Grass's Two Million Data Scraping Users Enabling Crypto x AI on Solana

GRASS (GetGrass.io) is a viral crypto AI project built on Solana that aims to scrape and validate internet data for AI training. The name 'GRASS' was chosen for its memeability and metaphors. The project addresses the scarcity of quality data and the control of web data by a few companies. It also tackles the unfairness of companies scraping data from residential networks without compensating users. GRASS is a network of 2 million devices that scrape and clean web data in real-time. The data collected by GRASS can be used for training specialized AI models, fine-tuning models, and real-time inference. GRASS is a web scraping protocol that aims to democratize access to public web data and enable the creation of AI models. The protocol allows users to download a browser extension or mobile app that scrapes data from websites and contributes it to the GRASS network. The data is then validated and stored on a decentralized network of nodes. GRASS aims to provide transparent and verifiable data, ensuring fairness and preventing bias in AI models. The protocol is built on the Solana blockchain for its speed, scalability, and innovation. Andrej's Twitter - https://twitter.com/0xdrej Chapters 00:00 - Introduction to GRASS 17:00 - Utilizing GRASS Data for AI Models and Inference 26:04 - Amassing Users and the Mission of GRASS 34:26 - Transparency and Privacy in Data Collection 44:36 - Tracking and Authenticating Data Sources 55:05 - Representing Every Region and Culture Disclosures This podcast is strictly informational and educational and is not investment advice or a solicitation to buy or sell any tokens or securities or to make any financial decisions. Do not trade or invest in any project, tokens, or securities based upon this podcast episode. The host and members at Delphi Ventures may personally own tokens or art that are mentioned on the podcast. Our current show features paid sponsorships which may be featured at the start, middle, and/or the end of the episode. These sponsorships are for informational purposes only and are not a solicitation to use any product, service or token.

22 Apr 202457min

Decentralization Unleashed: The Astria Approach to Rollup Sequencing

Decentralization Unleashed: The Astria Approach to Rollup Sequencing

Josh, the co-founder of Astria, discusses the journey from data availability layers to building a shared sequencer network. He explains the concept of shared sequencing and its advantages, such as amortizing the cost of engineering and providing a competitive experience for rollup developers. He also addresses the trade-offs and constraints of using a shared sequencer, including the block time and potential lock-in. Josh highlights the target market for shared sequencers and the potential value accrual in a world where multiple rollups tap into the same shared sequencer. Astria is focused on building a shared sequencer for rollups, which allows for faster and more cost-effective transactions. The market is still evaluating the cost and security trade-offs of shared sequencers versus centralized providers. The architecture of a shared sequencer relies on a distributed network, but it remains to be seen if it can be cost-competitive in the market. There is also an ideological question of where to draw the line between a developer building an app-specific rollup on a centralized sequencer and writing to a base layer. The landscape of optimistic rollups versus ZK rollups is constantly evolving, with ZK technology progressing significantly. Base sequencing refers to rollups that are purely dependent on the block producers of the L1, while shared sequencing involves a separate sequencing layer. Astria's go-to-market strategy involves vertically integrating and building their own rollups on top of the shared sequencer to demonstrate its viability. ⁠Josh's Twitter - https://twitter.com/Jskybowen Chapters 00:00 Introduction and Background 17:14 Target Market for Shared Sequencers 25:11 Value Accrual in a World with Shared Sequencers 32:41 The Evolving Landscape of Optimistic Rollups and ZK Rollups 44:11 The Definition and Challenges of Base Sequencing 50:41 Astria's Go-to-Market Strategy: Vertically Integrating and Building Their Own Rollups Disclosures Disclosures: This podcast is strictly informational and educational and is not investment advice or a solicitation to buy or sell any tokens or securities or to make any financial decisions. Do not trade or invest in any project, tokens, or securities based upon this podcast episode. The host and members at Delphi Ventures may personally own tokens or art that are mentioned on the podcast. Our current show features paid sponsorships which may be featured at the start, middle, and/or the end of the episode. These sponsorships are for informational purposes only and are not a solicitation to use any product, service or token.

18 Apr 202457min

Gui Laliberte and Avichal Garg: Integral's Accounting Suite Driving Backoffice to Zero

Gui Laliberte and Avichal Garg: Integral's Accounting Suite Driving Backoffice to Zero

Integral is a financial operations and accounting platform for companies with Web3 assets. It simplifies financial workflows and provides real-time access to financial information. The platform saves businesses time and frustration by automating tasks such as bookkeeping, payroll, and payments. Integral enables companies to manage their crypto assets, track transactions, and generate financial statements easily. It also facilitates global operations, complex money flows, and multiple asset classes. The platform is part of the larger transformation in capital formation and the redefinition of companies in the Web3 era. Integral is an accounting software for the future of composable Web3 companies. It aims to facilitate the operations of businesses beyond just accounting, including tax, payments, payroll, and financial planning. By providing accurate and real-time data on business finances, Integral enables founders and large corporations to make better decisions and manage their assets more effectively. The platform also offers benefits such as reducing the need for internal controllers, automating processes, and improving efficiency. Integral serves a range of clients, including startups, large brands like Nike, and VC firms. The company is also exploring the use of AI to enhance legibility and provide proactive recommendations. Gui's Twitter⁠ Avi's Twitter Drew's Twitter Chapters 00:00 - Introduction and Background 08:39 - Overview of Integral and its Benefits 31:25 - The Composable Nature of Integral in the Web3 Era 45:13 - Enhancing Legibility and Providing Proactive Recommendations with Integral Disclosures Disclosures: This podcast is strictly informational and educational and is not investment advice or a solicitation to buy or sell any tokens or securities or to make any financial decisions. Do not trade or invest in any project, tokens, or securities based upon this podcast episode. The host and members at Delphi Ventures may personally own tokens or art that are mentioned on the podcast. Our current show features paid sponsorships which may be featured at the start, middle, and/or the end of the episode. These sponsorships are for informational purposes only and are not a solicitation to use any product, service or token. Delphi’s transparency page can be viewed ⁠⁠⁠⁠⁠⁠⁠⁠here⁠⁠⁠⁠⁠⁠⁠⁠.

16 Apr 20241h 5min

Tory Green: io.net’s 500,000 GPUs Powering Crypto x AI on Solana

Tory Green: io.net’s 500,000 GPUs Powering Crypto x AI on Solana

io.net is a distributed training and inference project built on Solana. They aim to solve the problem of the GPU shortage in the AI industry by building a decentralized network that connects underutilized GPUs from multiple sources. Ionet uses clustering technology to combine GPUs from different geographic locations, allowing for more efficient and cost-effective AI compute. They are attracting both web2 and web3 customers, with a focus on inferencing, which makes up the majority of the market. The goal is to decentralize the AI ecosystem and prevent big tech companies from controlling all aspects of AI. io.net is a decentralized AI network that provides GPU compute power for AI workloads. They are focused on solving the compute aspect of decentralized AI and offer a network of choice for users to perform inference, fine-tuning, and training. The team is driven by a sense of urgency and executes quickly, following operational best practices. They have a disciplined go-to-market approach, targeting Series A to seed-generated AI companies. io.net aims to be the currency at the center of decentralized AI and is exploring the possibility of building a decentralized model marketplace and expanding into other areas like gaming and zero knowledge. Tory's Twitter Chapters 00:00 Introduction to Ionet 01:03 Solving the GPU Shortage 13:24 Attracting Web2 and Web3 Customers 27:51 Building a Decentralized Model Marketplace 29:47 The Role of Crypto in Incentivizing Participants 32:37 Easy Onboarding for GPU Workers 34:00 Organic Demand and Onboarding Sales Process 37:20 Choice and Flexibility in Compute Options 43:22 Conquering the Three Key Stakeholders in Decentralized AI Disclosures Disclosures: This podcast is strictly informational and educational and is not investment advice or a solicitation to buy or sell any tokens or securities or to make any financial decisions. Do not trade or invest in any project, tokens, or securities based upon this podcast episode. The host and members at Delphi Ventures may personally own tokens or art that are mentioned on the podcast. Our current show features paid sponsorships which may be featured at the start, middle, and/or the end of the episode. These sponsorships are for informational purposes only and are not a solicitation to use any product, service or token. Delphi’s transparency page can be viewed ⁠⁠⁠⁠⁠⁠⁠here⁠⁠⁠⁠⁠⁠⁠.

8 Apr 202456min

Degen Chain and L3s: What's Next

Degen Chain and L3s: What's Next

The conversation explores the concept of Layer 3 (L3) blockchains and their potential impact on the crypto industry. L3s offer unlimited customizations with near-zero gas fees, allowing for limitless creativity and experimentation. The discussion focuses on Degen, a community built on the L3 chain, and its journey from a tipping functionality on Forecaster to launching its own L3. The founders of Syndicate, the team behind Degen's L3, explain the need for L3s and the benefits they provide to different types of applications. The conversation also touches on the future of L3s and the potential for further customization. Degen Chain, an L3 solution built on top of Ethereum, offers low gas fees and customization options for developers. The ability to customize blockchains is seen as a powerful feature, and Degen Chain aims to pull customizations from startups specializing in that space. L3s are designed for customized functionality, while L2s are for scaling. Interoperability between L3s is a vision that many in the space are pursuing, and there are different approaches to achieving it. The security and value capture of Ethereum are important considerations, but the focus should be on usage and adoption. Degen Chain is just the beginning, and there is excitement about the future development and growth of on-chain communities. Ian's Warpcast Will's Warpcast Jacek's Warpcast Chapters 00:00 Introduction and Disclosures 01:06 Understanding Layer 3 (L3) Blockchains 11:50 Benefits of L3s for Different Types of Applications 29:18 Degen Chain: Low Gas Fees and Customization 31:06 L3s vs L2s: Customized Functionality vs Scaling 32:04 Interoperability: A Vision for L3s 34:32 Usage and Adoption: The Key Metrics 36:30 The Future of On-Chain Communities Disclosures Disclosures: This podcast is strictly informational and educational and is not investment advice or a solicitation to buy or sell any tokens or securities or to make any financial decisions. Do not trade or invest in any project, tokens, or securities based upon this podcast episode. The host and members at Delphi Ventures may personally own tokens or art that are mentioned on the podcast. Our current show features paid sponsorships which may be featured at the start, middle, and/or the end of the episode. These sponsorships are for informational purposes only and are not a solicitation to use any product, service or token. Delphi’s transparency page can be viewed ⁠⁠⁠⁠⁠⁠here⁠⁠⁠⁠⁠⁠.

3 Apr 202459min

Nous Research: Crypto x AI Masterclass, 30 Million Model Downloads, Bittensor Subnets and Launching a Matrix Style World Simulator

Nous Research: Crypto x AI Masterclass, 30 Million Model Downloads, Bittensor Subnets and Launching a Matrix Style World Simulator

In this conversation, Jeff from Nous Research discusses their work on open source models and the BitTensor subnet. They address the challenges of ranking models on platforms like Hugging Face and the need for a more reliable and commercially viable model ranking system. Nous Research has developed Nous Hermes, an open source model that can be fine-tuned to follow specific instructions. They also created the BitTensor subnet, which allows developers to submit their models and rank them against synthetic data generated by GPT-4 and Claude. The goal is to approximate the state-of-the-art models and incentivize the creation of high-quality models. The conversation explores the potential of synthetic data generation, the future of model selection, the world simulator and amorphous applications, the intersection of AI and crypto, concerns and risks in the crypto AI space, the inevitability of AGI, the role of crypto in AI, regulation and over-regulation in crypto AI, the potential interaction between AGI and centralized systems, proof of personhood and verifying authenticity, and Nous Research's mission and focus. Jeffrey's twitter Chapters 00:00 Introduction to Noose Research and Open Source Models 07:12 Overview of Nous Hermes and Fine-Tuning Models 26:30 Challenges with Open Source Models and the Need for BitTensor 32:10 Ranking and Reputation in BitTensor Subnet 34:27 Approaching GPT-4 with BitTensor 37:42 The Potential of Synthetic Data Generation 39:07 The Future of Model Selection 41:29 The World Simulator and Amorphous Applications 44:48 The Intersection of AI and Crypto 48:32 Concerns and Risks in the Crypto AI Space 51:17 The Inevitability of AGI 53:35 The Role of Crypto in AI 56:01 Regulation and Over-Regulation in Crypto AI 58:52 The Potential Interaction Between AGI and Centralized Systems 01:05:24 Proof of Personhood and Verifying Authenticity 01:10:04 Nous Research's Mission and Focus 01:12:26 Attracting AI Developers to Nous Research Disclosures Disclosures: This podcast is strictly informational and educational and is not investment advice or a solicitation to buy or sell any tokens or securities or to make any financial decisions. Do not trade or invest in any project, tokens, or securities based upon this podcast episode. The host and members at Delphi Ventures may personally own tokens or art that are mentioned on the podcast. Our current show features paid sponsorships which may be featured at the start, middle, and/or the end of the episode. These sponsorships are for informational purposes only and are not a solicitation to use any product, service or token. Delphi’s transparency page can be viewed ⁠⁠⁠⁠⁠here⁠⁠⁠⁠⁠.

30 Mars 20241h 15min

ETH Denver Crypto x AI Panel: Casey Caruso, Colin Gagich and Vassilis Tziokas

ETH Denver Crypto x AI Panel: Casey Caruso, Colin Gagich and Vassilis Tziokas

Recorded Live at ETH Denver 2024, this conversation explores the intersection of crypto and AI, discussing the need for decentralized AI models and the challenges of achieving this. The participants highlight the importance of keeping AI open and driven by people, rather than organizations. They discuss the potential of blockchain and Web3 technology to enable new business models and address the limitations of centralized AI companies. The conversation also touches on the role of incentives and tokens in driving innovation in the crypto AI space. The discussion concludes with an examination of BitTensor and the potential for future advancements in crypto AI. The conversation explores various themes related to the intersection of crypto and AI. It discusses the value of outside judgment in rating AI systems, the concept of a hive mind, and exciting use cases in the crypto and AI space. The projects AGI Guild and Morpheus are highlighted, along with the decentralized data warehouse Space and Time. The potential of utilizing unused data in DeFi protocols and the decentralized chat GPT platform MyShell.ai are also discussed. Concerns about OpenAI's dominance, the collaboration between Web2 and Web3, and the concept of zero party data are explored. The conversation also touches on the potential of data DAOs and crowdsourcing, the idea of minimum viable centralization, and the inflection point of decentralized AI and compute. Tangible innovation in ZKML and decentralized data, the importance of edge compute, and the need for decentralized inference are further examined. Casey's Twitter Colin Gagich Vassilis Twitter Chapters 00:00 Introduction and Background 01:54 The Need for Crypto and AI 06:29 Challenges of Decentralizing AI 09:18 Building Better Models with Web3 13:31 Innovation and Value in Crypto AI 19:11 BitTensor and the Future of Crypto AI 24:18 The Value of Outside Judgment 25:14 The Concept of a Hive Mind 25:43 Exciting Use Cases at the Intersection of Crypto and AI 26:33 AGI Guild and Morpheus Projects 27:32 Space and Time: Decentralized Data Warehouse 29:26 Utilizing Unused Data in DeFi Protocols 29:53 Decentralized Chat GPT with MyShell.ai 30:48 Concerns about OpenAI's Dominance 34:00 Web2 and Web3 Collaboration 35:11 The Concept of Zero Party Data 37:30 The Potential of Data DAOs and Crowdsourcing 38:06 Minimum Viable Centralization 40:29 The Inflection Point of Decentralized AI and Compute 43:45 Tangible Innovation in ZKML and Decentralized Data 45:35 The Importance of Edge Compute 46:30 The Need for Decentralized Inference Disclosures Disclosures: This podcast is strictly informational and educational and is not investment advice or a solicitation to buy or sell any tokens or securities or to make any financial decisions. Do not trade or invest in any project, tokens, or securities based upon this podcast episode. The host and members at Delphi Ventures may personally own tokens or art that are mentioned on the podcast. Our current show features paid sponsorships which may be featured at the start, middle, and/or the end of the episode. These sponsorships are for informational purposes only and are not a solicitation to use any product, service or token. Delphi’s transparency page can be viewed ⁠⁠⁠⁠here⁠⁠⁠⁠.

13 Mars 202445min

Ethan Sun and Santiago Santos: Bringing Crypto x AI Experiences to Millions with MyShell.AI

Ethan Sun and Santiago Santos: Bringing Crypto x AI Experiences to Millions with MyShell.AI

In this episode, Tommy and Santiago interview Ethan Sun, the co-founder of MyShell, a decentralized platform for creating and staking AI native applications. They discuss the evolution of AI over the past decade, the importance of consumer-facing applications, and the potential for AI to enhance daily life. Ethan explains how MyShell empowers creators to easily build AI experiences and the role of incentives in the platform's ecosystem. They also explore the future of MyShell, its competition, and the scalability challenges in the AI crypto space. In this conversation, Ethan Sun, the co-founder of MyShell, discusses the future of AI agents and the potential for personalized AI companions. He explores the idea of AI agents that understand individual needs and can perform tasks without explicit instructions. The conversation also touches on the improvement of underlying models and the role of human feedback in reinforcement learning. Ethan shares his thoughts on the differences between OpenAI and Crypto AI and the potential for crypto to empower the open source community. He discusses MyShell's end game and the importance of creators in the platform. The conversation concludes with a discussion on the challenges of incentivizing different parties within the system. Takeaways MyShell is a decentralized platform for creating and staking AI native applications. The focus of MyShell is on consumer-facing applications that enhance daily life. The platform makes it easy for non-technical creators to build AI experiences without coding. Incentives play a crucial role in the MyShell ecosystem, rewarding creators and users for their contributions. The future of AI agents lies in personalized companions that understand individual needs and can perform tasks without explicit instructions. Improving underlying models and gathering human feedback are crucial for enhancing AI agents and reinforcement learning. Crypto AI has the potential to empower the open source community and provide incentives for creators to monetize their work. MyShell aims to create a platform that enables creators to build personalized AI applications and attract a wide range of users. The hardest party to incentivize within the system is the creators, who play a crucial role in unifying models, users, and stakeholders. Chapters 00:00 Introduction and Background 03:01 The AI Stack and MyShell's Focus 07:01 Creating AI Experiences on MyShell 09:47 Easy Creation of AI Experiences 12:45 Applying Incentive Models to AI 16:34 Incentivizing Creators and Users 27:41 Competition and Differentiation 32:41 Web2 vs Web3 and Scalability 39:42 The Future of AI Agents 42:23 Improving Underlying Models and Data 45:04 Creators' Ideas and Applications 48:42 OpenAI vs. Crypto AI 51:09 Crypto's Impact on Open Source 56:02 MyShell's End Game 01:00:08 AGI and Human Feedback 01:04:19 Ratio of Creators to Users 01:06:40 Excitement and Concerns about Crypto AI 01:09:47 Incentivizing Supply and Demand 01:10:16 The Importance of Rules and Restrictions 01:12:04 Blockchain Infrastructure and Deployment 01:14:17 Hardest Party to Incentivize Disclosures Disclosures: This podcast is strictly informational and educational and is not investment advice or a solicitation to buy or sell any tokens or securities or to make any financial decisions. Do not trade or invest in any project, tokens, or securities based upon this podcast episode. The host and members at Delphi Ventures may personally own tokens or art that are mentioned on the podcast. Our current show features paid sponsorships which may be featured at the start, middle, and/or the end of the episode. These sponsorships are for informational purposes only and are not a solicitation to use any product, service or token. Delphi’s transparency page can be viewed ⁠⁠⁠here⁠⁠⁠. As an additional disclosure, Delphi Ventures is invested in MyShell.AI

8 Mars 20241h 13min

Populärt inom Teknik

uppgang-och-fall
market-makers
elbilsveckan
rss-racevecka
rss-elektrikerpodden
rss-uppgang-och-fall
natets-morka-sida
skogsforum-podcast
bli-saker-podden
rss-technokratin
developers-mer-an-bara-kod
rss-veckans-ai
har-vi-akt-till-mars-an
solcellskollens-podcast
mediepodden
rss-laddstationen-med-elbilen-i-sverige
bilar-med-sladd
rss-fabriken-2
hej-bruksbil
rss-bakom-boken