#195 - OpenAI o3 & for-profit, DeepSeek-V3, Latent Space

Our 197th episode with a summary and discussion of last week's big AI news! Recorded on 01/17/2024 Join our brand new Discord here! https://discord.gg/nTyezGSKwP Hosted by Andrey Kurenkov and guest-hosted by the folks from Latent Space Read out our text newsletter and comment on the podcast at https://lastweekin.ai/. Sponsors: The Generator - An interdisciplinary AI lab empowering innovators from all fields to bring visionary ideas to life by harnessing the capabilities of artificial intelligence. In this episode:  - Google and Mistral sign deals with AP and AFP, respectively, to deliver up-to-date news through their AI platforms.  - ChatGPT introduces a tasks feature for reminders and to-dos, positioning itself more as a personal assistant.  - Synthesia raises $180 million to enhance its AI video platform for generating videos of human avatars.  - New U.S. guidelines restrict exporting AI chips to various countries, impacting Nvidia and other tech firms. If you would like to become a sponsor for the newsletter, podcast, or both, please fill out this form. Timestamps + Links: (00:00:00) Intro / Banter (00:04:29) News Preview (00:05:09) Response to listener comments (00:05:58) Sponsor Break Tools & Apps (00:07:01) Google is making AI in Gmail and Docs free — but raising the price of Workspace (00:07:52) Microsoft relaunches Copilot for business with free AI chat and pay-as-you-go agents (00:12:36) Google signs deal with AP to deliver up-to-date news through its Gemini AI chatbot (00:18:08) Mistral signs deal with AFP to offer up-to-date answers in Le Chat (00:18:45) ChatGPT can now handle reminders and to-dos Applications & Business (00:22:53) Palmer Luckey’s AI Defense Company Anduril Is Building a $1 Billion Plant in Ohio (00:28:36) OpenAI is bankrolling Axios’ expansion into four new markets (00:29:39) AI researcher François Chollet founds a new AI lab focused on AGI (00:32:18) Nvidia-backed AI video platform Synthesia doubles valuation to $2.1 billion (00:34:46) Anysphere Raises $105M in Series B (00:40:14) Harvey Valuation of 3 Billion Projects & Open Source (00:46:12) MiniMax-01: Scaling Foundation Models with Lightning Attention (00:51:16) MinMo: A Multimodal Large Language Model with Approximately 8B Parameters for Seamless Voice Interaction (00:53:01) HALoGEN: Fantastic LLM Hallucinations and Where to Find Them Research & Advancements (00:57:03) Titans: Learning to Memorize at Test Time (01:04:38) Transformer2: Self-adaptive LLMs (01:08:15) Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Steps Policy & Safety (01:11:23) Biden administration proposes sweeping new restrictions on exporting AI chips (01:13:56) Biden orders Energy, Defense departments to lease sites for AI data centers, clean energy generation (01:15:00) OpenAI presents its preferred version of AI regulation in a new ‘blueprint’ (01:16:15) More teens report using ChatGPT for schoolwork, despite the tech’s faults Synthetic Media & Art (01:17:55) In AI copyright case, Zuckerberg turns to YouTube for his defense (01:19:53) Outro See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

20 Jan 1h 23min

Our 196th episode with a summary and discussion of last week's* big AI news! *and sometimes last last week's  Recorded on 01/10/2024 Join our brand new Discord here! https://discord.gg/nTyezGSKwP Hosted by Andrey Kurenkov and Jeremie Harris. Feel free to email us your questions and feedback at contact@lastweekinai.com and/or hello@gladstone.ai Read out our text newsletter and comment on the podcast at https://lastweekin.ai/. Sponsors: The Generator - An interdisciplinary AI lab empowering innovators from all fields to bring visionary ideas to life by harnessing the capabilities of artificial intelligence. In this episode: - Nvidia announced a $3,000 personal AI supercomputer called Digits, featuring the GB10 Grace Blackwell Superchip, aiming to lower the barrier for developers working on large models.  - The U.S. Department of Justice finalizes a rule restricting the transmission of specific data types to countries of concern, including China and Russia, under executive order 14117. - Meta allegedly trained Llama on pirated content from LibGen, with internal concerns about the legality confirmed through court filings.  - Microsoft paused construction on a section of a large data center project in Wisconsin to reassess based on new technological changes. If you would like to become a sponsor for the newsletter, podcast, or both, please fill out this form. Timestamps + Links: (00:00:00) Intro / Banter (00:04:52) Sponsor Break Tools & Apps (00:05:55) Nvidia announces $3,000 personal AI supercomputer called Digits (00:10:23) Meta removes AI character accounts after users criticize them as ‘creepy and unnecessary’ Applications & Business (00:16:16) NVIDIA Is Reportedly Focused Towards “Custom Chip” Manufacturing, Recruiting Top Taiwanese Talent (00:21:54) AI start-up Anthropic closes in on $60bn valuation (00:25:38) Why OpenAI is Taking So Long to Launch Agents (00:30:08) TSMC Set to Expand CoWoS Capacity to Record 75,000 Wafers in 2025, Doubling 2024 Output (00:33:10) Microsoft 'pauses construction' on part of data center site in Mount Pleasant, Wisconsin (00:37:23) Google folds more AI teams into DeepMind to ‘accelerate the research to developer pipeline’ Projects & Open Source (00:41:59) Cosmos World Foundation Model Platform for Physical AI (00:48:21) Microsoft releases Phi-4 language model on Hugging Face Research & Advancements (00:50:16) PRIME: Online Reinforcement Learning with Process Rewards (00:58:29) ICLR: In-Context Learning of Representations (01:07:38) Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs (01:11:44) METAGENE-1: Metagenomic Foundation Model for Pandemic Monitoring (01:15:45) TransPixar: Advancing Text-to-Video Generation with Transparency (01:18:03) The amount of compute used to train frontier models has been growing at a breakneck pace of over 4x per year since 2018, resulting in an overall scale-up of more than 10,000x! But what factors are enabling this rapid growth? Policy & Safety (01:23:45) InfAlign: Inference-aware language model alignment (01:28:44) Mark Zuckerberg gave Meta’s Llama team the OK to train on copyrighted works, filing claims (01:33:19) Anthropic gives court authority to intervene if chatbot spits out song lyrics (01:35:57) US government says companies are no longer allowed to send bulk data to these nations (01:39:10) Trump announces $20B plan to build new data centers in the US  See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

13 Jan 1h 46min

Our 194th episode with a summary and discussion of last week's* big AI news! *and sometimes last last week's  Recorded on 12/19/2024 Hosted by Andrey Kurenkov and Jeremie Harris. Feel free to email us your questions and feedback at contact@lastweekinai.com and/or hello@gladstone.ai Read out our text newsletter and comment on the podcast at https://lastweekin.ai/. Sponsors: The Generator - An interdisciplinary AI lab empowering innovators from all fields to bring visionary ideas to life by harnessing the capabilities of artificial intelligence. In this episode: - Google dominates AI news with multiple announcements, including a reasoning model and Project Mariner, an AI browsing agent.  - Anthropic explores alignment faking in LLMs, revealing models may show deceptive compliance under certain conditions.  - Apple observes a trend towards smaller but more efficient language models, bucking previous trends of scaling larger parameter counts.  - Legal drama unfolds as Meta backs Elon Musk's opposition to OpenAI's profit status change, raising concerns about competitive fairness. If you would like to become a sponsor for the newsletter, podcast, or both, please fill out this form. Timestamps + Links: (00:00:00) Intro / Banter (00:02:14) Response to listener comments (00:08:52) News Preview (00:10:01) Sponsor Break Tools & Apps (00:10:55) Google releases its own ‘reasoning’ AI model (00:16:52) Google Gemini can now do more in-depth research (00:21:58) Google DeepMind unveils a new video model to rival Sora (00:27:50) Pika Labs releases AI video generator 2.0 with new features (00:29:51) Google unveils Project Mariner: AI agents to use the web for you (00:34:33) X gains a faster Grok model and a new ‘Grok button’ Applications & Business (00:36:11) AI GPU clusters with one million GPUs are planned for 2027 — Broadcom says three AI supercomputers are in the works (00:43:02) Meta asks the government to block OpenAI’s switch to a for-profit (00:49:36) OpenAI says Elon Musk wanted it to be for-profit in 2017 (00:56:04)  EQTY Lab, Intel, and NVIDIA Unveil 'Verifiable Compute,' A Solution to Secure Trusted AI (00:59:53) Liquid AI just raised $250M to develop a more efficient type of AI model (01:03:19) Hundreds of OpenAI’s current and ex-employees are about to get a huge payday by cashing out up to $10 million each in a private stock sale Projects & Open Source (01:07:45) Phi-4 Technical Report (01:13:04) DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding (01:15:23) Meta AI Releases Apollo: A New Family of Video-LMMs Large Multimodal Models for Video Understanding Research & Advancements (01:16:34) Alignment faking in large language models (01:28:39) Meta AI Introduces Byte Latent Transformer (BLT): A Tokenizer-Free Model That Scales Efficiently (01:36:49) Frontier language models have become much smaller (01:42:28) The Complexity Dynamics of Grokking Policy & Safety (01:46:49) Homeland Security gets its very own generative AI chatbot (01:49:16) Pre-Deployment Evaluation of OpenAI’s o1 Model (01:51:35) Pricing for key chipmaking material hits 13-year high following (01:53:46) Chinese export restrictions — China's restrictions on Gallium exports hit hard Synthetic Media & Art Meta debuts a tool for watermarking AI-generated videos (01:55:27) Outro See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

30 Dec 20241h 59min

Our 193rd episode with a summary and discussion of last week's* big AI news! *and sometimes last last week's  Note: this one was recorded on 12/13, so the news is a bit outdated... will get things back on track soon! Hosted by Andrey Kurenkov and Jeremie Harris. Feel free to email us your questions and feedback at contact@lastweekinai.com and/or hello@gladstone.ai Read out our text newsletter and comment on the podcast at https://lastweekin.ai/. Sponsors: The Generator - An interdisciplinary AI lab empowering innovators from all fields to bring visionary ideas to life by harnessing the capabilities of artificial intelligence. In this episode: - OpenAI launches Sora, a text-to-video model with significant capabilities, and Gemini 2.0 from Google showcasing agentic potential in AI tools.  - Character.ai introduces a teen model to address safety concerns following two tragic incidents linked to addiction and harmful influence.  - The U.S. government sets up a task force to support the rapid development of AI data centers, reflecting the critical need for robust infrastructure.  - A paper from Anthropic reveals that frontier AI systems have reached the capability of self-replication, sparking discussions on future implications and safety protocols. If you would like to become a sponsor for the newsletter, podcast, or both, please fill out this form. Timestamps + Links: (00:00:00) Intro / Banter (00:02:44) News Preview (00:03:43) Response to listener comments (00:09:50) Sponsor Break Tools & Apps (00:11:12) OpenAI has finally released Sora (00:21:16) Google Reveals Gemini 2, AI Agents, and a Prototype Personal Assistant (00:28:23) ChatGPT Advanced Voice Mode adding video and screen sharing input (plus a Santa mode) (00:30:43) Microsoft’s Copilot can browse the web with you using AI ‘Vision’ (00:32:31) Musk’s xAI has launched Grok image generation model (00:35:22) Cognition Labs’ AI Software Engineer Devin Launched for Subscribers (00:40:43) Apple launches its ChatGPT integration with Siri (00:43:23) Reddit’s New AI Search Tool Helps You Find Reddit Answers Without Google Applications & Business (00:46:35) OpenAI Aiming to Eliminate Microsoft AGI Rule to Boost Future Investment (00:53:34) GM halts funding of robotaxi development by Cruise (00:57:08) Largest AI data centre in the world to be built in northwest Alberta (01:02:36) Meta announces 4 million sq ft, 2GW Louisiana data center campus (01:05:22) Google’s future data centers will be built next to solar and wind farms Projects & Open Source (01:08:37) Google DeepMind Just Released PaliGemma 2: A New Family of Open-Weight Vision Language Models (3B, 10B and 28B) Research & Advancements (01:13:51) Training Large Language Models to Reason in a Continuous Latent Space (01:25:37) An Evolved Universal Transformer Memory (01:31:48) APOLLO: SGD-like Memory, AdamW-level Performance (01:37:59) Clio: A system for privacy-preserving insights into real-world AI use Policy & Safety (01:39:47) Character.AI steps up teen safety after bots allegedly caused suicide, self-harm (01:45:22) What Trump’s New AI and Crypto Czar David Sacks Means For the Tech Industry (01:49:03) Frontier AI systems have surpassed the self-replicating red line (01:53:52) Chip war: China launches antitrust probe into US semiconductor giant Nvidia in sign of escalation (01:56:53) White House Creating Task Force on AI Datacenter Infrastructure (02:00:00) US clears export of advanced AI chips to UAE under Microsoft deal, Axios says (02:02:19) Outro See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

23 Dec 20242h 5min

Our 192nd episode with a summary and discussion of last week's* big AI news! *and sometimes last last week's  Note: this one was recorded on 12/04 , so the news is a bit outdated... Hosted by Andrey Kurenkov and Jeremie Harris. Feel free to email us your questions and feedback at contact@lastweekinai.com and/or hello@gladstone.ai Read out our text newsletter and comment on the podcast at https://lastweekin.ai/. Sponsors: The Generator - An interdisciplinary AI lab empowering innovators from all fields to bring visionary ideas to life by harnessing the capabilities of artificial intelligence. The AI safety book “Uncontrollable" which is not a doomer book, but instead lays out the reasonable case for AI safety and what we can do about it. Max TEGMARK said that “Uncontrollable” is a captivating, balanced, and remarkably up-to-date book on the most important issue of our time" - find it on Amazon today! In this episode: OpenAI launches a $200 ChatGPT Pro subscription with advanced capabilities, while Amazon unveils cost-effective Nova multimodal models at the re:Invent conference. Meta releases LLAMA 3.3 70B model, showing significant gains through post-training techniques, and Alibaba introduces QWQ, a reasoning model rivaling OpenAI's O1. Amazon collaborates with Anthropic on a massive AI supercomputer project, and Black Forest Labs eyes a $200 million funding round for growth in AI tools. New research from DeepMind's Genie 2 generates interactive 3D worlds from text and images, progressing AI's understanding of world models and interactive environments. If you would like to become a sponsor for the newsletter, podcast, or both, please fill out this form. Timestamps + Links: (00:00:00) Intro / Banter (00:02:34) Sponsor Break Tools & Apps (00:04:19) OpenAI confirms new $200 monthly subscription, which includes its o1 reasoning model (00:10:40) Amazon announces Nova, a new family of multimodal AI models (00:17:13) ElevenLabs launches GenFM to turn user content into AI-powered podcasts (00:20:21) Google’s new generative AI video model is now available Applications & Business (00:23:56) Elon Musk files for injunction to halt OpenAI’s transition to a for-profit (00:29:40) Amazon Is Building a Mega AI Supercomputer With Anthropic (00:34:15) It Sounds an Awful Lot Like OpenAI Is Adding Ads to ChatGPT (00:38:23) A16z in Talks to Lead $200 Million Round in Black Forest Labs, Startup Behind AI Images on Grok (00:41:10) Bezos Backs AI Chipmaker Vying With Nvidia at $2.6 Billion Value Projects & Open Source (00:45:25) Meta unveils a new, more efficient Llama model (00:50:00) Alibaba releases an ‘open’ challenger to OpenAI’s o1 reasoning model (00:55:21) DeMo: Decoupled Momentum Optimization (00:57:01) PRIME Intellect Releases INTELLECT-1 (Instruct + Base): The First 10B Parameter Language Model Collaboratively Trained Across the Globe (01:03:03) Tencent Launches HunyuanVideo, an Open-Source AI Video Model Research & Advancements (01:09:23) DeepMind’s Genie 2 can generate interactive worlds that look like video games (01:16:43) Language Models are Hidden Reasoners: Unlocking Latent Reasoning Capabilities via Self-Rewarding (01:20:40) Densing Law of LLMs (01:25:59) Monet: Mixture of Monosemantic Experts for Transformers Policy & Safety (01:30:56) Commerce Strengthens Export Controls to Restrict China’s Capability to Produce Advanced Semiconductors for Military Applications (01:37:33) China retaliates against latest US chip restrictions (01:40:52) OpenAI Is Working With Anduril to Supply the US Military With AI (01:43:24) On Targeted Manipulation and Deception when Optimizing LLMs for User Feedback (01:47:52) AI Safety Researcher Quits OpenAI, Saying Its Trajectory Alarms Her (01:51:52) Meta Claims AI Content Was Less than 1% of Election Misinformation (01:55:05) Outro See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

16 Dec 20241h 58min

Our 191st episode with a summary and discussion of last week's big AI news! Hosted by Andrey Kurenkov and Jeremie Harris. Feel free to email us your questions and feedback at contact@lastweekinai.com and/or hello@gladstone.ai Read out our text newsletter and comment on the podcast at https://lastweekin.ai/. Sponsors: The Generator - An interdisciplinary AI lab empowering innovators from all fields to bring visionary ideas to life by harnessing the capabilities of artificial intelligence If you would like to become a sponsor for the newsletter, podcast, or both, please fill out this form. Timestamps + Links: (00:00:00) Intro / Banter (00:02:55) Response to listener comments (00:09:30) Sponsor Break Tools & Apps (00:10:52) OpenAI’s Sora video generator appears to have leaked (00:21:11) Mistral unleashes Pixtral Large and upgrades Le Chat into full-on ChatGPT competitor (00:26:39) Ignite 2024 introduces new AI agents and more for Microsoft 365 Copilot (00:28:50) H, the AI startup that raised $220M, launches its first product: Runner H for ‘agentic’ applications (00:31:20) Anthropic bets on personalization in the AI arms race with new ‘styles’ feature (00:33:42) ElevenLabs now offers ability to build conversational AI agents (00:37:08) Perplexity introduces a shopping feature for Pro users in the U.S. (00:38:49) Google’s Gemini chatbot now has memory (00:43:03) Suno V4 Ai Music Generator Is Out Now And It’s Very Impressive (00:46:28) Introducing FLUX.1 Tools (00:49:51) OpenAI just gave ChatGPT a major 'creativity' upgrade (00:51:26) Runway launches Frames — a new AI image generator that creates custom worlds Applications & Business (00:54:56) OpenAI Email Archives (from Musk v. Altman) (01:02:01) Amazon to invest another $4 billion in Anthropic, OpenAI's biggest rival (01:05:41) Amazon Robots Struggling to Keep Up With Human Workers Projects & Open Source (01:11:27) DeepSeek’s first reasoning model R1-Lite-Preview turns heads, beating OpenAI o1 performance (01:15:30) OpenScholar: The open-source A.I. that’s outperforming GPT-4o in scientific research Research & Advancements (01:18:02) A statistical approach to model evaluations (01:22:08) Scaling Laws for Precision (01:25:10) Cerebras Delivers Record-Breaking Performance with Meta’s Llama 3.1 405B Model Policy & Safety (01:28:01) Sam Altman will co-chair San Francisco mayor-elect Daniel Lurie’s transition team (01:32:21) Biden’s final meeting with Xi Jinping reaps agreement on AI and nukes Synthetic Media & Art (01:33:07) How Did You Do On The AI Art Turing Test? (01:38:27) Outro See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

5 Dec 20241h 42min

Our 190th episode with a summary and discussion of last week's* big AI news! *and sometimes last last week's Hosted by Andrey Kurenkov and Jeremie Harris. Note from Andrey: this one is coming out a bit later than planned, apologies! Next one will be coming out sooner. Feel free to email us your questions and feedback at contact@lastweekinai.com and/or hello@gladstone.ai Read out our text newsletter and comment on the podcast at https://lastweekin.ai/. Sponsors: The Generator - An interdisciplinary AI lab empowering innovators from all fields to bring visionary ideas to life by harnessing the capabilities of artificial intelligence In this episode: * OpenAI's pitch for a $100 billion data center and AI strategy plan outlines infrastructure and regulatory needs, emphasizing AI's foundational role akin to electricity.  * Google's Gemini model challenges OpenAI's dominance, showing strong performance in chatbot arenas alongside generative AI advancements.  * DeepMind's AlphaFold3 gets open-sourced for academic use, while new chips from NVIDIA and Google show significant performance boosts.  * Anthropic and TSMC updates highlight strategic funding, regulation influences, and the complex dynamics of AI hardware and international policy. If you would like to become a sponsor for the newsletter, podcast, or both, please fill out this form. Timestamps + Links: (00:00:00) Intro / Banter (00:02:44) News Preview (00:03:34) Sponsor Break Tools & Apps (00:04:36) OpenAI, Google and Anthropic Are Struggling to Build More Advanced AI (00:16:22) OpenAI Nears Launch of AI Agent Tool to Automate Tasks for Users (00:19:14) Google drops new Gemini model and it goes straight to the top of the LLM leaderboard (00:19:14)  Chinese AI startup takes aim at OpenAI's Sora with image-to-video tool launch (00:20:04) Introducing the Forge Reasoning API Beta and Nous Chat: An Evolution in LLM Inference Applications & Business (00:23:47) OpenAI Discusses AI Data Center That Could Cost $100 Billion (00:26:48) Elon Musk's massive AI data center gets unlocked — xAI gets approved for 150MW of power, enabling all 100,000 GPUs to run concurrently (00:29:34) Newest Google and Nvidia Chips Speed AI Training (00:34:45) Ex-OpenAI CTO Murati’s New Team Takes Shape (00:34:45) Amazon Discussing New Multibillion-Dollar Investment in Anthropic Projects & Open Source (00:37:52) Google DeepMind open-sources AlphaFold 3, ushering in a new era for drug discovery and molecular biology (00:41:29) Near plans to build world’s largest 1.4T parameter open-source AI model Research & Advancements (00:45:38) The Super Weight in Large Language Models (00:55:42) Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Task (01:03:47) Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Models (01:08:14) Contextualized Evaluations: Taking the Guesswork Out of Language Model Evaluations Policy & Safety (01:11:14) The Code of Practice for general-purpose AI offers a unique opportunity for the EU (01:15:38) Three Sketches of ASL-4 Safety Case Components (01:23:05) U.S Department of Commerce finalizes $6.6 billion CHIPS Act funding for TSMC Fab 21 Arizona site , TSMC cannot make 2nm chips abroad now: MOEA (01:26:21) OpenAI to present plans for U.S. AI strategy and an alliance to compete with China (01:30:42) OpenAI loses another lead safety researcher, Lilian Weng (01:33:00) Outro See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

28 Nov 20241h 37min