The Undocumented Web: scraping, private APIs, proxies and “alternative solutions”

The Undocumented Web: scraping, private APIs, proxies and “alternative solutions”

What is the undocumented web? Scott and Wes dive into it, discussing APIs, faking, scraping, automation, proxies as well as tips and tricks for best practices. Kyle Prinsloo’s Freelancing & Beyond — Sponsor Kyle Prinsloo teaches you everything you need to know about freelancing, including how to quit your job, earn a side-income and start taking control of your life. Check it out at https://studywebdevelopment.com/freelancing and use the coupon code “syntax” at checkout to get 25% off. Freshbooks - Sponsor Get a 30 day free trial of Freshbooks at freshbooks.com/syntax and put SYNTAX in the “How did you hear about us?” section. Show Notes 3:05 - What Is the undocumented web? When traditional methods don’t give you what you want Wild West 6:10 - Undocumented APIS Instagram’s private API Use dev tools’ XHR Request React Dev Tools will also have the data Ep 008 - Wes Bos Origin Story Realtor.ca - MLS Mapping Simon Willison’s Zeit Day Talk about XML files Charles Proxy App - also good for debugging requests 17:34 - Faking Faking User Agents Sending headers from requests Free Blackberry wifi on plane 23:52 - Scraping What is craping? Cheerio - jQuery like node package 27:30 - Automation Cron Jobs Filling out forms instantly Headless browsers Puppeteer PhantomJS 33:30 - Proxies Need a fresh IP address Need a different country Side note: Canadians buying stuff in USA M4S1P1 === 41100 API throttling 39:00 - Tips and Tricks Multiple API keys diskDB - Text files as databases Search Github for API keys Inspect Element and Delete Overlays Delete Cookiesokay 44:45 - Stories Photo Voting MLS Listing Top Entrepreneur voting cheating Vanity Plate Checker My Hockey Team Sucks Vehicle auction bidding app youtube-dl ××× SIIIIICK ××× PIIIICKS ××× Scott: Octopath Traveler & OST Wes 1: AvE Youtube Channel Wes 2: Mustie1 Youtube Channel Shameless Plugs Scott’s Level Up Tuts Pro Wes’ Courses Tweet us your tasty treats! Scott’s Instagram LevelUpTutorials Instagram Wes’ Instagram Wes’ Twitter Wes’ Facebook Scott’s Twitter Make sure to include @SyntaxFM in your tweets

Episoder(989)

981: Browsers Are Finally Catching Up (Interop 2026)

981: Browsers Are Finally Catching Up (Interop 2026)

Scott and Wes unpack Interop 2026 and the browser features finally aligning across engines, from container style queries and anchor positioning to scroll-driven animations and view transitions. They b...

23 Feb 51min

980: AI Coding Explained

980: AI Coding Explained

Wes and Scott talk about the state of AI coding in 2026—from editors and models to agents, skills, slash commands, MCPs, and more. They unpack what these things actually do, how they overlap, and how ...

18 Feb 52min

979: WebMCP: New Standard to Expose Your Apps to AI

979: WebMCP: New Standard to Expose Your Apps to AI

Scott and Wes unpack WebMCP, a new standard that lets AI interact with websites through structured tools instead of slow, bot-style clicking. They demo it, debate imperative vs declarative APIs, and s...

16 Feb 16min

978: Should A New Coder Use AI?

978: Should A New Coder Use AI?

Wes and Scott answer your questions about AI agents, learning to code with AI, pagination patterns, skilling up from outdated tech stacks, balancing side projects with family life, real-world hacking ...

11 Feb 1h 2min

977: We built a CSS Challenge platform

977: We built a CSS Challenge platform

Scott and Wes break down how they built SynHax, the real-time CSS Battle app powering the upcoming Mad CSS tournament. From SvelteKit and Zero to diffing algorithms, sync conflicts, and a last-minute ...

9 Feb 41min

976: Pi - The AI Harness That Powers OpenClaw W/ Armin Ronacher & Mario Zechner

976: Pi - The AI Harness That Powers OpenClaw W/ Armin Ronacher & Mario Zechner

Wes and Scott talk with Armin Ronacher and Mario Zechner about PI, a minimalist agent harness powering tools like OpenClaw. They unpack why Bash is “all you need,” the risks of agents, workflow adapta...

4 Feb 57min

975: What’s Missing From the Web Platform?

975: What’s Missing From the Web Platform?

Scott and Wes run through their wishlist for the web platform, digging into the UI primitives, DOM APIs, and browser features they wish existed (or didn’t suck). From better form controls and drag-and...

2 Feb 50min

974: Clawdbot (Moltbot), Agents and the Age of Personal Software

974: Clawdbot (Moltbot), Agents and the Age of Personal Software

Wes and Scott talk about building hyper-specific personal software with AI. They explore personal agents, home automation, JSON-as-a-database, and how LLMs unlock fast, custom apps that reduce frictio...

28 Jan 46min

Populært innen Politikk og nyheter

giver-og-gjengen-vg
aftenpodden
forklart
lydartikler-fra-aftenposten
aftenpodden-usa
popradet
i-retten
stopp-verden
det-store-bildet
fotballpodden-2
rss-gukild-johaug
dine-penger-pengeradet
rss-ness
hanna-de-heldige
nokon-ma-ga
aftenbla-bla
bt-dokumentar-2
e24-podden
frokostshowet-pa-p5
rss-penger-polser-og-politikk