Damian Brady on DevOps for Data Science and Machine Learning - Episode 008

Damian Brady on DevOps for Data Science and Machine Learning - Episode 008

This week, your host, Jeffrey Palermo, interviews Damian Brady. Damian is a Senior Cloud DevOps Developer Advocate at Microsoft, helping customers implement DevOps methods on the Microsoft platform. He’s been with Microsoft for just over a year now and formerly served as a developer for Octopus Deploy.

In this episode, Damian and Jeffrey talk all things data science and machine learning. Damian answers key questions such as: what has been the biggest change in the area of data science since the Azure DevOps release? What does source control look like for data science projects in DevOps? And more. He also explains some of the interesting architectures he has put together for machine learning and walks Jeffrey through the process of his machine learning model from source control, building, packaging, and finally, to deploying. He also gives his recommendations for those who want to go even further with data science after listening to this week’s episode.

Topics of Discussion:

[:52] About today’s guest, Damian Brady.

[1:06] Damian introduces himself and explains his role at Microsoft.

[1:46] Which group Damian is presently on at Microsoft.

[4:14] With the Azure DevOps release, what’s the big change in the area of data science? What is going to be different for people building or running models?

[6:47] For data science projects what does the source control look like?

[8:49] For the Microsoft ML, is there a particular format that the data is stored in, in source control?

[9:09] If the data is large and needs to be versioned, what are the current methods people are using?

[11:06] A word from Azure DevOps sponsor: Clear Measure.

[11:39] Some of the interesting architectures Damian has put together for machine learning.

[16:10] Damian walks Jeffrey through his machine learning model from source control to building, to packaging up the release, to deploying.

[19:20] For this type of model, where would be the physical environment where it’s measuring information?

[20:24] Damian talks firewall rules, permissions, and security.

[23:16] The advantages of using Azure’s IoT Hub.

[24:46] Damian talks about the new open source features that were added with the release.

[28:20] Does Damian still encounter customers who say they don’t want to use Microsoft products because they don’t realize they’re open source?

[29:36] Is it true that VS Code is the most popular editor?

[31:03] One of the huge advantages of using open source.

[31:53] Damian talks build agents.

[33:33] About the new Windows-hosted container build agent.

[35:50] Damian’s recommendation for listeners who want to go further with data science after listening to this week’s podcast!

Mentioned in this Episode:

Azure DevOps

Azure Pipelines

Octopus Deploy

Clear Measure (Sponsor)

Buck Hodges on the introduction to Azure DevOps Services - Episode 001

Donovan Brown on How to Use Azure DevOps Services - Episode 002
Source control in Azure DevOps

Ubuntu

Machine Learning (ML)

Amazon Web Services (AWS)

Azure Cognitive Services

CustomVision.ai

Raspberry Pi
Azure Data Center

.NET Core

Python

GitHub

Azure IoT Hub

ADP Summit
VS Code

Docker Compose

Subversion

Chocolatey

Want to Learn More?

Visit AzureDevOps.Show for show notes and additional episodes.

Follow Up with Our Guest:

Damian Brady’s LinkedIn

Episoder(372)

Brian Lagunas: Modern Web - Episode 228

Brian Lagunas: Modern Web - Episode 228

Brian Lagunas is a Microsoft MVP, a Microsoft Patterns & Practices Champion, leader of the Boise .Net Developers User Group (NETDUG), board member of Boise Code Camp, speaker, trainer, and Pluralsight author. He can be found speaking at a variety of developer events around the world. His talks always involve some form of markup (XAML or HTML), as well as how to build well-architected applications with Prism. In his spare time, he authors courses for Pluralsight, blogs, livestreams about various technologies, and manages the Prism Library. The easiest way to find Brian is on Twitter at @BrianLagunas.   Topics of Discussion: [2:43] What triggered Brian to go from the Army into programming? [5:49] Brian started in Java because that’s how new .NET was. [8:22] What is Reveal, and how many code bases do you have to reach all those places? [12:37] What is Brian’s thought about using Blazer vs. JavaScript vs. Typescript? [15:20] How do we bridge the gap between using NPM and Blazer Applications?  [17:31] How does Brian think about the different levels of unit tests of these different types of code, and then what classifications do you create in your test libraries? [21:47] What is App Builder? [24:39] What’s the track record of App Builder? Is it already mainstream? [30:20] What Brian’s team is focused on now is getting that initial application built, generated, out the door, and ready for the developer to implement the logic.   Mentioned in this Episode: Clear Measure Way Architect Forum Software Engineer Forum Architect Tips — Video podcast! Azure DevOps Clear Measure (Sponsor) .NET DevOps for Azure: A Developer’s Guide to DevOps Architecture the Right Way, by Jeffrey Palermo — Available on Amazon! Jeffrey Palermo’s YouTube Jeffrey Palermo’s Twitter — Follow to stay informed about future events! Programming with Palermo - New Video Podcast! programming@palermo.network Brian Lagunas Brian Lagunas Microsoft Profile GitHub Brian Lagunas Brian Lagunas LinkedIn Brian Lagunas Twitter Plural Sight Brian Lagunas Brian Lagunas YouTube   Want to Learn More? Visit AzureDevOps.Show for show notes and additional episodes.

16 Jan 202337min

Marco Rossignoli: Automated Code Coverage Measurement - Episode 227

Marco Rossignoli: Automated Code Coverage Measurement - Episode 227

Marco Rossignoli is a Dev at Microsoft on the .NET Test Platform and Code coverage team. He's also the co-maintainer of the Coverlet Collector NuGet package, which has over 100M downloads.   Topics of Discussion: [1:15] Jeffrey talks about the architect forums he’s hosting and facilitating in 2023. You can register here. [2:53] Marco talks about how he got into code coverage. [6:44] Why is code coverage even useful to measure? [12:40] How does Coverlet work and how is it different from the old ones? How do you run it? [20:30] Is there any difference in how it works between Azure Pipelines or GitHub Actions or TeamCity? [21:40] With multiple test suites running, how does Coverlet support pulling all the results together so that you get the one number of code coverage? [23:40] Report generator merges all of the reports. [25:16] What exactly is Cobertura? [26:02] Marco shares why he is excited about Coverlet and the many opportunities it gives us in the future.   Mentioned in this Episode: Clear Measure Way Architect Forum Software Engineer Forum Architect Tips — Video podcast! Azure DevOps Clear Measure (Sponsor) .NET DevOps for Azure: A Developer’s Guide to DevOps Architecture the Right Way, by Jeffrey Palermo — Available on Amazon! Jeffrey Palermo’s YouTube Jeffrey Palermo’s Twitter — Follow to stay informed about future events! Programming with Palermo - New Video Podcast! programming@palermo.network NuGet Gallery GitHub Coverlet Coverage Marco Rossignoli .Net Coverage Code   Want to Learn More? Visit AzureDevOps.Show for show notes and additional episodes.

9 Jan 202332min

A 2023 Happy New Year and 2022 Review - Episode 226

A 2023 Happy New Year and 2022 Review - Episode 226

Happy New Year to all here in 2023. It’s going to be a great year. It’s a great time to be a programmer. A great time to be building with .NET; you are going to do great things this year. You have what it takes. You are smart, you have great tools, and you have a great team. You are a great leader. This episode is going to be all about remembering what happened this past year at the podcast.   Topics of Discussion:  [1:15] Jeffrey talks about the architect forums he’s hosting and facilitating in 2023. You can register here. [1:46] Huge announcement in Microsoft Developer news including: - Android apps on Windows 11 - ARM processors getting big investments - Microsoft Dev Box — in preview — dev workstation in the cloud - Power Pages websites - Large SKU app service; up to 256GB RAM available for those who need it - Azure Arc, the new name of Hybrid Azure. And a single-node Azure Stack for remote locations but the programming model of Azure — looking forward to testing it at the right time. - Azure Container Apps tooling got better, and it became ready for prime time. Every team should be looking at this. - .NET 7 released. [4:11] What might the default application stacks and environments look like on the platform in 2023? - Windows 11 - Visual Studio 2022 w/ ReSharper - .NET 7 - Onion Architecture - Blazor for interactive applications - .NET service workers for back-end jobs and queue listeners - Entity Framework with Azure SQL — add on other storage services as per application. - Azure App Service for hosting while prototyping Azure Container Apps. - Application Insights with the Open Telemetry NuGet packages. - Azure Pipelines paired with Octopus Deploy (keep an eye on GitHub Actions as they fill out support for scenarios you need). - NordVPN for developer workstation work-from-home or remote Wi-Fi. [9:11] When it comes to developer workstations, desktop computers are still giving the most bang for the buck with power, and only a few laptops do the job really well. I have not reviewed all computers, and there are a lot out there. I can vouch for Alienware R series desktops. Liquid-cooled, so they are really quiet, even under full load. Dell Precision laptops are amazing for software engineers. I really wanted to love the Lenovo P1, but the fan was just too loud when it was under load. And we all know that cooling is so important in laptops. When a laptop gets too hot, your BIOS will slow down the processor to keep it from burning up. Then you no longer have a fast processor. And video calls use a good deal of processor, surprisingly — or not. For super mobile laptops that you can use for programming, I really do like the Microsoft Surface Laptop. I wanted to like the Surface Studio laptop, but they inverted the cooling and the battery placement, so it’s very uncomfortable on my lap and my wrists unfortunately under load. The wrist wrest gets really hot. Normally the battery is under the wrist rest, but Microsoft swapped it on this one, so it’s not fun using it as a laptop on your lap or even on a desk while hot and under load. [13:11] Highlighting some past episodes that will be interesting:  - Highlighting some past episodes over the year that might be interesting. - With Microsoft Orleans providing a new implementation of the Actor design pattern, we have a two-part series interview with Aaron Stannard, the creator of Akka.NET, episodes 172 and 173. - On the IoT front, Wilderness Labs has been trucking along creating system-on-a-chip options that run .NET natively and easily. I interviewed founder and CEO Bryan Costanich. - For those educating themselves for a career in software engineering, my interview with Henry Quillin might be useful. He talks about a programming internship and his education journey, his work earning his Eagle Scout, and how he became a working programmer even as he is just starting university. - More on embedded. Kevin Kirkus was with us in episode 186. He runs a testing team at Intel doing automated testing for their Xeon processor line. The design necessary for testing in this specialized environment gives us all plenty to think about. - For team leaders out there, I interviewed Mark Seemann. He wrote a recent book, Code That Fits In Your Head. He talks about the principles that are in the book. I subsequently bought and read the book, and I wish I had this book earlier in my career. Would have saved me a great deal of time. - On distributed systems, Udi Dahan is always a fascinating gentleman to listen to. Check out episode 192. As the founder and CEO of Particular Software, and the creator of NServiceBus, he is one of the world’s leading experts on distributed systems, microservices, and messaging architectures. - Time-tested ideas are continually useful. I had the pleasure of interviewing Philippe Kruchten. He worked at Rational Software back when they were at the forefront of the software process in the 1990s. He published a paper outlining a framework for emergent, agile architecture. He didn’t call it that. He called it the 4+1 Architecture, but only because it predated the agile manifesto. If you are an architect, and you aren’t aware of this approach to architecture, give episode 195 a listen. - For the Blazor developers, I had Steve Sanderson on in episode 202. Steve is the original designer of Blazor, which has become the new default web application on .NET. He shared about the future of Blazor and WebAssembly. - Because there is so much going on in this space, Daniel Roth also joined me to discuss more Blazor Futures. - GitHub Actions is being talked about quite a bit. While loads of people are using it for builds, people are scratching their heads about where it fits in regarding deployments. Damian Brady, on the GitHub team and a former employee of Octopus Deploy, sheds light on this in episode 206. - Scott Hunter joined me in episode 211. He announced his new role at Microsoft running more of Azure development and .NET. He shared quite a bit behind the scenes regarding Microsoft’s strategy there. - For the UX people. Mark Miller is the Chief Architect of DevExpress, the big UI components company. He has a brilliant user experience mind, and I was able to get him talking in episode 212. - Telemetry. We all need it to keep our software stable in production. The Serilog and AutoFac maintainer, Nicholas Blumhardt, joined me to discuss the fundamentals of modern logging and telemetry. Check out episode 217 for that. - More on the testing front, Eduardo Maltez, a software engineer doing some really interesting full system test work shares his thoughts on what makes tests reliable, stable, and fast — and how to fight brittle tests. Episode 224. - We closed out the year on the security front. With LastPass getting hacked and now Rackspace having a hacking-induced major outage, we all need to take action. Troy Vinson, a multi-certified security professional and certified ethical hacker, gave his perspective on the Rackspace breach and what every .NET team should learn from it.   Mentioned in this Episode: Architect Tips — New video podcast! Azure DevOps Clear Measure (Sponsor) .NET DevOps for Azure: A Developer’s Guide to DevOps Architecture the Right Way, by Jeffrey Palermo — Available on Amazon! Jeffrey Palermo’s YouTube Jeffrey Palermo’s Twitter — Follow to stay informed about future events! Programming with Palermo programming@palermo.network   Want to Learn More? Visit AzureDevOps.Show for show notes and additional episodes.

2 Jan 202318min

Troy Vinson: Learning From the Rackspace Security Breach with Troy Vinson - Episode 225

Troy Vinson: Learning From the Rackspace Security Breach with Troy Vinson - Episode 225

Troy Vinson is a Principal Software Architect at Clear Measure as a CISSP (Certified Information System Security Professional). He is an experienced leader, architect, and problem-solver in Information Systems Security and Software Development technologies and has spent the majority of his career integrating computer science, information science, and cognitive science to assist in software development and the management of information.   Topics of Discussion: [2:39] Is Troy a Certified Ethical Hacker? If so, what does that mean, and what does he see in the divide of focus between security and programming? [5:08] What do we know about the Rackspace security breach? [7:37] How many hosted exchange customers does Rackspace have? [11:01] Having a contingency plan in place and a recovery plan is very important. [14:07] What’s the most basic way that someone could start doing this for themselves? [21:08] Non-malicious use is also a protection against malicious use. [26:09] What is email protection, and how do you use it?  [28:24] What should development teams be thinking about, security-wise, for their custom applications? [32:54] The importance of having a software bill of materials so that you have a policy about which software can be used.   Mentioned in this Episode: Architect Tips — New video podcast! Azure DevOps Clear Measure (Sponsor) .NET DevOps for Azure: A Developer’s Guide to DevOps Architecture the Right Way, by Jeffrey Palermo — Available on Amazon! Jeffrey Palermo’s YouTube Jeffrey Palermo’s Twitter — Follow to stay informed about future events! Programming with Palermo programming@palermo.network Rackspace Ep 161 with Troy Vinson Rackspace Status KnowBe4 Sonar Source Microsoft Security Engineering   Want to Learn More? Visit AzureDevOps.Show for show notes and additional episodes.

26 Des 202237min

Eduardo Maltez: Full-system testing using Selenium - Episode 224

Eduardo Maltez: Full-system testing using Selenium - Episode 224

Eduardo Maltez is a Software Engineer at Clear Measure. He has extensive experience in .NET, including Blazor, SignalR, Azure Service Bus, SQL Server, and all kinds of automated testing. Earlier in his career, he apprenticed under Jeffrey Palermo and is currently building automated full-system tests for a large and complex software ecosystem at one of his clients.   Topics of Discussion: [3:09] How Eduardo got into programming and the steps he took to make it a full-time career. [8:04] How Eduardo makes the decision in his head when he needs to use Selenium vs. other tests. [10:52] Eduardo talks about ShoWorks and the technology components behind it. [16:27] What does it mean when a test is brittle, and what do you do to make tests that are not brittle? [18:41] When a test is brittle, it means you can’t really rely on that test. Brittle tests are unreliable tests, in the end. [23:36] How do you know when to stop a unit test? [25:44] What are Educardo’s go-to methods for finding the right DOM element? [33:26] When it comes to rough edges around Selenium itself, it’s not necessarily something that doesn’t work, but it’s something that definitely always causes just a sudden road bump in the test cycle.   Mentioned in this Episode: Architect Tips — New video podcast! Azure DevOps Clear Measure (Sponsor) .NET DevOps for Azure: A Developer’s Guide to DevOps Architecture the Right Way, by Jeffrey Palermo — Available on Amazon! Jeffrey Palermo’s YouTube Jeffrey Palermo’s Twitter — Follow to stay informed about future events! Programming with Palermo programming@palermo.network ShoWorks Selenium WebDriver Wait Class   Want to Learn More? Visit AzureDevOps.Show for show notes and additional episodes.

19 Des 202237min

David Pine: NET Content Development - Episode 223

David Pine: NET Content Development - Episode 223

David Pine works in Developer Relations at Microsoft, focusing on .NET and Azure developer content. He is recognized as a Google Developer Expert in Web Technologies and is a Twilio Champion. David focuses on the developer community, actively seeking opportunities to share knowledge through speaking engagements around the world. David advocates for open-source, the .NET Foundation, C#, TypeScript, SignalR, Reactive Extensions, Azure, and .NET. He's a founding member and co-host of the On .NET Live show. Follow David on Twitter at @davidpine7.   Topics of Discussion: [2:41] What were the high points along David’s career that steered him in the direction of Microsoft, and how has that evolved? [4:46] Jeffrey’s background in music and how he got into it. [8:20] Does David believe that the age of the JavaScript SPA front end for .NET developers is over? [10:32] David discusses his role as a technical writer, with more on the content developer side. [11:36] David’s show On .NET Live celebrates .NET developers from all over the world. [12:40] Microsoft has been doing a lot of work recently with GitHub actions. How do we monitor their different versions and use them to the best of our abilities? [21:17] What level of maturity is Microsoft’s Orleans, and why does David like using it? [27:10] What are some new samples coming out? [30:05] What exactly is “globbing”?   Mentioned in this Episode: Architect Tips — New video podcast! Azure DevOps Clear Measure (Sponsor) .NET DevOps for Azure: A Developer’s Guide to DevOps Architecture the Right Way, by Jeffrey Palermo — Available on Amazon! Jeffrey Palermo’s YouTube Jeffrey Palermo’s Twitter — Follow to stay informed about future events! Programming with Palermo programming@palermo.network David Pine .NET Fundamentals @DavidPine Twitter GitHub Actions. NET SDK: Hello from the GitHub Actions: Core .NET SDK On .NET Live — YouTube  GitHub .NET Samples .NET GitHub Actions Github.com/dotnet/orleans Learn.microsoft.com/dotnet/orleans/  Learn.microsoft.com/dotnet/fundamentals/ Learn.microsoft.com/dotnet/devops/github-actions-overview Learning Blazor (davidpine.net) Davidpine.net/blog/learning-blazor/   Want to Learn More? Visit AzureDevOps.Show for show notes and additional episodes.

12 Des 202231min

Jeff Fritz: Evolving Cloud Architecture - Episode 222

Jeff Fritz: Evolving Cloud Architecture - Episode 222

Jeff Fritz is an experienced developer, technical educator, and PM on the .NET team at Microsoft. He founded The Live Coders team on Twitch, and regularly livestreams builds of websites and fun applications. You can follow Jeff for more .NET, .NET Core, and Visual Studio content on Twitch and Twitter at @csharpfritz.   Topics of Discussion: [2:41] Jeff talks about surviving the .com bomb and his background as a longtime web developer and technical educator. [3:57] What have been some of the recent developments that Jeff and his team are most excited about at Microsoft? [5:45] Jeff talks about how the application has been growing and how he’s had some bumps in the road. [7:40] How to make video clips searchable and discoverable on the web. [12:12] What made Jeff go for MySQL instead of serverless Azure SQL? [18:01] What’s the duration of the journey from the first line of code to enterprise patterns? [21:09] As we grow applications, we need to figure out a better way to show people what happens when you make a mistake, and to help them through the growth. [27:13] How do you know what’s going to happen at a certain level of production? [22:48] Does Jeff really believe we’re going to end up at Kubernetes? [32:27] Re-architecting your database architecture.   Mentioned in this Episode: Architect Tips — New video podcast! Azure DevOps Clear Measure (Sponsor) .NET DevOps for Azure: A Developer’s Guide to DevOps Architecture the Right Way, by Jeffrey Palermo — Available on Amazon! Jeffrey Palermo’s YouTube Jeffrey Palermo’s Twitter — Follow to stay informed about future events! Programming with Palermo programming@palermo.network Jeff on Twitch KlipTok Fritz's Tech Tips and Chatter   Want to Learn More? Visit AzureDevOps.Show for show notes and additional episodes.

5 Des 202236min

Brady Gaster: Orleans - Episode 221

Brady Gaster: Orleans - Episode 221

Brady is a Principal Program Manager on the Azure Developer Experience team at Microsoft where he works on Orleans, SignalR, microservices, APIs, and integration with Azure service teams in hopes to make it exciting for developers who work on .NET apps to party in the cloud!   Topics of Discussion: [4:17] Moving around a lot gave Brady a lot of insight into all the different ways that we can benefit developers and all the different opportunities we have to make things better. [6:30] The people in Docs hack all the time. [7:01] What is Orleans? [11:40] What’s the best database to use for distributed applications? [21:10] Open telemetry gives us the capability of being able to trace messages that go from one end of your system all the way to the other end of your system through multiple silos and multiple clients. [22:08] The three pillars of observability: logging, distributed tracing (which is really where open telemetry shines), and then metrics. [26:02] How does the traceability side of open telemetry apply to Azure? [28:02] What else should we know about Orleans?   Mentioned in this Episode: Architect Tips — New video podcast! Azure DevOps Clear Measure (Sponsor) .NET DevOps for Azure: A Developer’s Guide to DevOps Architecture the Right Way, by Jeffrey Palermo — Available on Amazon! Jeffrey Palermo’s YouTube Jeffrey Palermo’s Twitter — Follow to stay informed about future events! Programming with Palermo programming@palermo.network “Brady Gaster on SignalR and More” Brady Gaster Brady Gaster GitHub Brady Gaster Rock Paper Orleans Brady Gaster Website   Want to Learn More? Visit AzureDevOps.Show for show notes and additional episodes.

28 Nov 202231min

Populært innen Politikk og nyheter

giver-og-gjengen-vg
aftenpodden
forklart
aftenpodden-usa
stopp-verden
popradet
fotballpodden-2
nokon-ma-ga
dine-penger-pengeradet
det-store-bildet
rss-dannet-uten-piano
frokostshowet-pa-p5
aftenbla-bla
rss-ness
bt-dokumentar-2
rss-penger-polser-og-politikk
e24-podden
rss-borsmorgen-okonominyhetene
rss-gukild-johaug
ukrainapodden