Damian Brady on DevOps for Data Science and Machine Learning - Episode 008

Damian Brady on DevOps for Data Science and Machine Learning - Episode 008

This week, your host, Jeffrey Palermo, interviews Damian Brady. Damian is a Senior Cloud DevOps Developer Advocate at Microsoft, helping customers implement DevOps methods on the Microsoft platform. He’s been with Microsoft for just over a year now and formerly served as a developer for Octopus Deploy.

In this episode, Damian and Jeffrey talk all things data science and machine learning. Damian answers key questions such as: what has been the biggest change in the area of data science since the Azure DevOps release? What does source control look like for data science projects in DevOps? And more. He also explains some of the interesting architectures he has put together for machine learning and walks Jeffrey through the process of his machine learning model from source control, building, packaging, and finally, to deploying. He also gives his recommendations for those who want to go even further with data science after listening to this week’s episode.

Topics of Discussion:

[:52] About today’s guest, Damian Brady.

[1:06] Damian introduces himself and explains his role at Microsoft.

[1:46] Which group Damian is presently on at Microsoft.

[4:14] With the Azure DevOps release, what’s the big change in the area of data science? What is going to be different for people building or running models?

[6:47] For data science projects what does the source control look like?

[8:49] For the Microsoft ML, is there a particular format that the data is stored in, in source control?

[9:09] If the data is large and needs to be versioned, what are the current methods people are using?

[11:06] A word from Azure DevOps sponsor: Clear Measure.

[11:39] Some of the interesting architectures Damian has put together for machine learning.

[16:10] Damian walks Jeffrey through his machine learning model from source control to building, to packaging up the release, to deploying.

[19:20] For this type of model, where would be the physical environment where it’s measuring information?

[20:24] Damian talks firewall rules, permissions, and security.

[23:16] The advantages of using Azure’s IoT Hub.

[24:46] Damian talks about the new open source features that were added with the release.

[28:20] Does Damian still encounter customers who say they don’t want to use Microsoft products because they don’t realize they’re open source?

[29:36] Is it true that VS Code is the most popular editor?

[31:03] One of the huge advantages of using open source.

[31:53] Damian talks build agents.

[33:33] About the new Windows-hosted container build agent.

[35:50] Damian’s recommendation for listeners who want to go further with data science after listening to this week’s podcast!

Mentioned in this Episode:

Azure DevOps

Azure Pipelines

Octopus Deploy

Clear Measure (Sponsor)

Buck Hodges on the introduction to Azure DevOps Services - Episode 001

Donovan Brown on How to Use Azure DevOps Services - Episode 002
Source control in Azure DevOps

Ubuntu

Machine Learning (ML)

Amazon Web Services (AWS)

Azure Cognitive Services

CustomVision.ai

Raspberry Pi
Azure Data Center

.NET Core

Python

GitHub

Azure IoT Hub

ADP Summit
VS Code

Docker Compose

Subversion

Chocolatey

Want to Learn More?

Visit AzureDevOps.Show for show notes and additional episodes.

Follow Up with Our Guest:

Damian Brady’s LinkedIn

Episoder(372)

Mike Martin: Architecting Azure - Episode 268

Mike Martin: Architecting Azure - Episode 268

Mike Martin, an exceptional Senior Cloud Solution Architect and Technical Evangelist at Microsoft. With over two decades of experience in the IT industry, Mike is an expert in coaching and leading teams and architecting, designing, and training systems. As an Azure specialist for ISVs (Independent Software Vendors) and partners, he is the go-to person for all things related to the Microsoft Cloud Platform and Application Lifecycle Management. Mike is known as the perfect hybrid solution with his unique combination of expertise in both development and IT Pro topics.   Mike Martin's involvement in the Belgian Microsoft Azure User Group (AZUG) dates back to January 2012, when he joined the group as a crew member. He has been an active contributor to the community ever since, regularly giving presentations and playing an integral role in organizing events such as ITProceed, Techorama, and the Global Azure Bootcamp (GAB).   Mike’s extensive knowledge and experience in Microsoft Azure have earned him numerous accolades, including being recognized as a Microsoft Azure MVP five times since 2013, with his most recent award in July 2017. He is also a Microsoft Azure Advisor, providing guidance and insights to others in the community.   Beyond his professional achievements, Mike is passionate about giving back to the community and inspiring the next generation of technologists. He takes great pleasure in introducing young people to the world of Microsoft and technology and is always willing to lend a helping hand to those in need.   Topics of Discussion: [5:05] How Mike got into his career, and a few of the highlights he has had over the past 27 years in the field. [9:34] Where he got reborn as a true architect. [10:53] The beauty of being involved from the beginning, and why youngsters may have a challenging time. [13:28] The importance of independent scale. [18:25] Going by the philosophy of KISS: keep it simple, stupid. [22:27] How does Mike coach people in the decision of the level of resilience? [28:40] Functional monitoring when it comes to resiliency. [29:52] The patterns Mike advises to his coders to hook up existing monitoring tools to get that functional level of monitoring. [30:32] Reliable web application and the well-architected framework. [34:46] What is giving Mike hope for the future in programming these days?   Mentioned in this Episode: Clear Measure Way Architect Forum Software Engineer Forum Programming with Palermo — New Video Podcast! Email us at programming@palermo.net. Clear Measure, Inc. (Sponsor) .NET DevOps for Azure: A Developer’s Guide to DevOps Architecture the Right Way, by Jeffrey Palermo — Available on Amazon! Jeffrey Palermo’s Twitter — Follow to stay informed about future events! Serverless360 The Twelve-Factor App Reliable web app patterns Reliable web app patterns — Github Learn About MSAL   Want to Learn More? Visit AzureDevOps.Show for show notes and additional episodes.

23 Okt 202340min

Jeff Julian: IoT with Meadow and Wilderness Labs - Episode 267

Jeff Julian: IoT with Meadow and Wilderness Labs - Episode 267

Jeff Julian is a long-time software developer. He used to run the Geeks with Blogs blogging site when developer blogging was the peak of social media. He’s received multiple Microsoft MVP awards and has spoken at many conferences. He has retired from the software development community and now runs a local farm where he has custom-developed a farm operating system using IoT devices and electronics.   Topics of Discussion: [2:55] Some tips to do conferences right. [4:55] What led Jeff to apply programming to farming? [6:50] Jeff’s decision to buy land, and the challenges that came with purchasing it. [10:31] Becoming aware of Wilderness Labs and Meadow. [12:33] Selling directly to the customer. [12:42] What Meadow is, and some of the things they have automated. [17:15] Driving the fodder system. [20:22] Where and how does the code come in for this automation? [24:46] UptimeRobot and using F7 devices for data collection and IoT projects. [26:22] Using technology for farming and beekeeping. [33:57] IoT devices, sensors, and power consumption. [36:13] How many tiny computers does it take? [38:02] The challenges of IoT devices. [44:15] The heart of the .NET community should be people helping each other learn and grow.   Mentioned in this Episode: YARP Clear Measure Way Architect Forum Software Engineer Forum Programming with Palermo — New Video Podcast! Email us at programming@palermo.net. Clear Measure, Inc. (Sponsor) .NET DevOps for Azure: A Developer’s Guide to DevOps Architecture the Right Way, by Jeffrey Palermo — Available on Amazon! Jeffrey Palermo’s Twitter — Follow to stay informed about future events! Geeks with Blogs Wilderness Labs   Want to Learn More? Visit AzureDevOps.Show for show notes and additional episodes.

16 Okt 202346min

Shawn Wildermuth: Web Service APIs in .NET 7 - Episode 266

Shawn Wildermuth: Web Service APIs in .NET 7 - Episode 266

Shawn Wildermuth has been tinkering with computers and software since he got a VIC-20 back in the early ’80s. He has been a Microsoft MVP, Pluralsight Author, and filmmaker. You can reach him at his blog at Wildermuth.com and he makes films at Twainfilms.com.   Topics of Discussion: [5:34] What got Shawn excited about coding? [9:26] How should developers be thinking about just the concept of an API? What are the categories that they should be aware of? [16:04] Shawn’s first steps in designing an API. [18:37] What are the newer concepts and newer advances that are worth taking a look at? [19:10] Maturing minimal APIs. [24:53] Endpoint filtering. [27:01] Does the core logic need to be aware that it might be in a caching workflow, or does caching as a concept belong to the interface for the application programming? [31:45] Shawn’s favorite method for testing the complete set of web service APIs. [34:59] Helping young developers not get lost along the way of feeling the need to be perfect. [39:25] How developers make the decision of where and how to run the applications they built.   Mentioned in this Episode: YARP Clear Measure Way Architect Forum Software Engineer Forum Programming with Palermo — New Video Podcast! Email us at programming@palermo.net. Clear Measure, Inc. (Sponsor) .NET DevOps for Azure: A Developer’s Guide to DevOps Architecture the Right Way, by Jeffrey Palermo — Available on Amazon! Jeffrey Palermo’s Twitter — Follow to stay informed about future events! Shawn Wildermuth on Next-gen Web Services Twain Films   Want to Learn More? Visit AzureDevOps.Show for show notes and additional episodes.

9 Okt 202343min

Jay Harris: Overcoming Tough Problems in Software Projects - Episode 265

Jay Harris: Overcoming Tough Problems in Software Projects - Episode 265

Jay Harris is a speaker, software consultant, and owner of Arana Software. He has been developing on the web since 1995, when the Blink tag lured him away from Visual Basic 3, and has been awarded as a Microsoft Regional Director, ASPInsider, and Microsoft MVP. Recognizing that the greatest application performance bottleneck is a developer’s time, Jay’s continuing quest is for frameworks, modules, tools, and practices that make developers stronger, fitter, happier, and more productive.   Jay resides in Las Vegas, USA. Follow him on Twitter at @jayharris.   Topics of Discussion: [3:40] Jay gives a shout-out to a phenomenal manager, Larry, who had a profound impact on his career. [5:30] Advice for managing burnout in software development teams. [7:16] The importance of learning how to say no. [10:19] Respecting team limits and honoring downtime is crucial for long-term success. [16:06] Maintaining software team velocity through play and downtime. [18:23] The key to sustainable software delivery is collaboration, compromise, and empowering teams to be self-sufficient. [23:28] Pain points in user interfaces. [30:39] Overcoming the challenges of working with PDFs. [36:49] Jay walks us through the typical code flow.   Mentioned in this Episode: YARP Clear Measure Way Architect Forum Software Engineer Forum Programming with Palermo — New Video Podcast! Email us at programming@palermo.net. Clear Measure, Inc. (Sponsor) .NET DevOps for Azure: A Developer’s Guide to DevOps Architecture the Right Way, by Jeffrey Palermo — Available on Amazon! Jeffrey Palermo’s Twitter — Follow to stay informed about future events! Jay Harris on Distributed DevOps Glenn Burnside Managing Developers   Want to Learn More? Visit AzureDevOps.Show for show notes and additional episodes.

2 Okt 202341min

Jimmy Bogard: .NET 7 and Azure Modernization - Episode 264

Jimmy Bogard: .NET 7 and Azure Modernization - Episode 264

Jimmy is the creator and maintainer of the popular OSS libraries AutoMapper and MediatR. Jimmy is an independent consultant based in Austin, TX. Jimmy has received the “Microsoft Most Valuable Professional” (MVP) award every year since 2009.   Topics of Discussion: [3:45] How do we modernize old software systems? [4:55] Dividing the modernization process into small steps to minimize dependencies and validate changes along the way. [5:01] Does Jimmy have a preferred sequence of work that he has found that makes modernizing a system easier? [7:01] Modernizing legacy ASP.NET web applications with test coverage. [7:24] System web adapters. [12:02] Database migration to Azure using SQL Data Sync and Hangfire. [12:09] Any “gotchas” on the database side? [15:27] What exactly is Hangfire? [17:02] The flexibility of Hangfire in its triggers and scheduling. [23:49] How system web adapters enable easy migration of controllers and actions. [25:16] Second success story for YARP: Yet Another Reverse Proxy. [27:15] What was the thought about observability architectures? [29:02] What are some of Jimmy’s favorite features? [32:08] The team modernized the telemetry system for a large organization, enabling them to query data more efficiently and gain valuable insights. [35:05] Lessons learned and best practices while modernizing.NET applications with Azure DevOps.   Mentioned in this Episode: YARP Clear Measure Way Architect Forum Software Engineer Forum Programming with Palermo — New Video Podcast! Email us at programming@palermo.net. Clear Measure, Inc. (Sponsor) .NET DevOps for Azure: A Developer’s Guide to DevOps Architecture the Right Way, by Jeffrey Palermo — Available on Amazon! Jeffrey Palermo’s Twitter — Follow to stay informed about future events! Architect Tips — Video podcast! YARP: Yet Another Reverse Proxy   Want to Learn More? Visit AzureDevOps.Show for show notes and additional episodes.

25 Sep 202339min

J. Tower: Modernization Strangler Fig Pattern - Episode 263

J. Tower: Modernization Strangler Fig Pattern - Episode 263

Jonathan, or J. as he’s known to friends, is a husband, a father, and the owner of Trailhead Technology Partners, a custom software consulting company with employees all around the world. He is also a Microsoft MVP in .NET and frequently speaks at software meetups and conferences. He doesn’t mind too much because he loves sharing what he’s learned, and it also gives him an excuse to visit any nearby National Parks, a passion of his, proven by the fact that he's currently made it to 56 of the 63 parks.   J. also has a passion for building community and has served on several non-profit boards over the years as a result. Currently, J. sits on the SoftwareGR board, a non-profit trade organization dedicated to building the software industry in West Michigan. He also runs Beer City Code, a software conference, and has served as president on that board for over a decade. J. loves hiking, reading, photography, and trying to see all the best picture nominees before the Oscars ceremony.   Topics of Discussion: [4:18] J starting his own consulting company, Trailhead. [4:55] The two categories that make up software architecture. [5:54] J’s philosophy on when he would rewrite a legacy software system. [10:52] The pros and cons of making small improvements over time. [11:33] What is the strangler fig pattern, and how does that turn into a strategy for a software update? [16:02] Bringing older ASP.NET applications up to .NET7. [19:55] What is a reverse proxy? [22:21] We reference the book Working Effectively with Legacy Code. [25:08] In this process, do both of the applications just get access to everything, or do you have to do something specific? [31:28] Architecturally, does this approach work in modernizing from older or other platform web applications? [34:02] The concept of microfrontends.   Mentioned in this Episodes: Clear Measure Way Architect Forum Software Engineer Forum Programming with Palermo — New Video Podcast! Email us at programming@palermo.net. Clear Measure, Inc. (Sponsor) .NET DevOps for Azure: A Developer’s Guide to DevOps Architecture the Right Way, by Jeffrey Palermo — Available on Amazon! Jeffrey Palermo’s Twitter — Follow to stay informed about future events! Architect Tips — Video podcast! J. Tower on Twitter J. Tower on LinkedIn YARP Basic YARP Sample Strangler Fig Application   Want to Learn More? Visit AzureDevOps.Show for show notes and additional episodes.

18 Sep 202338min

Chris "Woody" Woodruff: Network Programming - Episode 262

Chris "Woody" Woodruff: Network Programming - Episode 262

Chris Woodruff, or as his friends call him, Woody, is a software developer and architect of over 25 years. Woody loves software engineering, especially allowing applications and services to communicate across networks and through Web APIs. He has been a Microsoft MVP in SQL, Data, and C# in the past, along with multiple years of being awarded the AWS Community Builder Award. Woody lives in Grand Rapids, Michigan, where he explores the many breweries in West Michigan and travels with his family. Woody is also a long-time bourbon fan and loves hunting for whiskey bottles.   Topics of Discussion: [4:46] The many positions Woody has held in his career. [7:14] The genesis behind Woody’s new book, Practical Network Programming Using C#. Dive deep into #CSharp12 and #DotNET8. [9:24] The second book Woody is working on co-writing, on the patterns of developer relations. [14:10] The original intent of the internet was to protect the military. [15:22] What is a packet? [21:08] A brief history of web services. [24:00] Who was Roy Fielding? [28:48] Woody talks about using different applications, the WebSocket service, and Dapr. [35:36] You have to know about the transport across the network, as well as how to build the code and architect your application so that it utilizes the network efficiently. [40:14] We can expect the book out by May or June 2024.   Mentioned in this Episodes: Clear Measure Way Architect Forum Software Engineer Forum Programming with Palermo — New Video Podcast! Email us at programming@palermo.net Clear Measure, Inc. (Sponsor) .NET DevOps for Azure: A Developer’s Guide to DevOps Architecture the Right Way, by Jeffrey Palermo — Available on Amazon! Jeffrey Palermo’s Twitter — Follow to stay informed about future events! Architect Tips — Video podcast! Azure DevOps Chris Woodruff's Blog Chris Woodruff LinkedIn Chris Woodruff Twitter   Want to Learn More? Visit AzureDevOps.Show for show notes and additional episodes.

11 Sep 202342min

Jeremy Miller: Parallelism in Software - Episode 261

Jeremy Miller: Parallelism in Software - Episode 261

Jeremy Miller started his career as a “real” engineer but wandered into software because that looked like more fun. Since then, Jeremy has worked in and led software development teams in the computer manufacturing industry, finance, insurance, health care, and banking industries. Lately, Jeremy has been focused on leading software architecture teams and helping mentor other software architects. Having had roles both as an in-house software architect and as a software consultant, Jeremy has a great deal of insight into the challenges that confront companies developing and maintaining enterprise systems over time.   Jeremy is well known for his Open-Source Software tools starting with Structure Map and continuing today to Marten and Wolverine. Jeremy is also a frequent author and technical speaker at software conferences. Jeremy recently helped found JasperFx Software to build a sustainable business around the “Critter Stack” tools.   Topics of Discussion: [6:10] How Jeremy got into open-source development. [6:50] Being a part of the codebetter.com website in the pre-Twitter days. [9:30] What most developers should be aware of in the space of getting code to run or multiple instances to run at the same time and having it come out well. [12:04] What is Marten, and how does it work? [12:26] TPL Dataflow Library is a hidden gem inside of Microsoft. [15:25] The two parts of Marten and how they work together. [17:42] What is a producer-consumer pattern? [20:05] How to implement a queue pattern. [24:04] You should probably have some basic understanding of one level underneath you, but Jeremy thinks you don’t want to work on the thread level yourself. [25:38] Jeremy defines “Critter Stack.” [29:55] Jeremy’s advice for new developers. [32:59] Jeremy talks about the type of customers he is looking to collaborate with.   Mentioned in this Episodes: Clear Measure Way Architect Forum Software Engineer Forum Programming with Palermo — New Video Podcast! Email us programming@palermo.net Clear Measure, Inc. (Sponsor) .NET DevOps for Azure: A Developer’s Guide to DevOps Architecture the Right Way, by Jeffrey Palermo — Available on Amazon! Jeffrey Palermo’s Twitter — Follow to stay informed about future events! Architect Tips — Video podcast! Azure DevOps Jeremy Miller Twitter Jeremy Miller LinkedIn Jeremy Miller Website Jasper FX Marten Wolverine   Want to Learn More? Visit AzureDevOps.Show for show notes and additional episodes.

4 Sep 202336min

Populært innen Politikk og nyheter

giver-og-gjengen-vg
aftenpodden
forklart
aftenpodden-usa
popradet
stopp-verden
fotballpodden-2
dine-penger-pengeradet
det-store-bildet
nokon-ma-ga
frokostshowet-pa-p5
bt-dokumentar-2
rss-dannet-uten-piano
aftenbla-bla
e24-podden
rss-ness
rss-penger-polser-og-politikk
rss-borsmorgen-okonominyhetene
rss-gukild-johaug
rss-garne-damer