CERN’s Transition to Containerization and Kubernetes with Ricardo Rocha

CERN’s Transition to Containerization and Kubernetes with Ricardo Rocha

Some of the highlights of the show include:


  • The challenges that CERN was facing when storing, processing, and analyzing data, and why it pushed them to think about containerization.
  • CERN’s evolution from using mainframes, to physical commodity hardware, to virtualization and private clouds, and eventually to containers. Ricardo also explains how the migration to containerization and Kubernetes was started.
  • Why there was a big push from groups that focus on reproducibility to explore containerization.
  • How end users have responded to Kubernetes and containers. Ricardo talks about the steep Kubernetes learning curve, and how they dealt with frustration and resistance.
  • Some of top benefits of migrating to Kubernetes, and the impact that the move has had on their end users.
  • Current challenges that CERN is working through, regarding hybrid infrastructure and rising data loads. Ricardo also talks about how CERN optimizes system resources for their scientists, and what it’s like operating as a public sector organization.
  • How CERN handles large data transfers.


Links:


Transcript

Emily: Hi everyone. I’m Emily Omier, your host, and my day job is helping companies position themselves in the cloud-native ecosystem so that their product’s value is obvious to end-users. I started this podcast because organizations embark on the cloud naive journey for business reasons, but in general, the industry doesn’t talk about them. Instead, we talk a lot about technical reasons. I’m hoping that with this podcast, we focus more on the business goals and business motivations that lead organizations to adopt cloud-native and Kubernetes. I hope you’ll join me.



Emily: Welcome to the Business of Cloud Native. I'm your host, Emily Omier, and today I'm here with Ricardo Rocha. Ricardo, thank you so much for joining us.



Ricardo: It's a pleasure.



Emily: Ricardo, can you actually go ahead and introduce yourself: where you work, and what you do?



Ricardo: Yeah, yes, sure. I work at CERN, the European Organization for Nuclear Research. I'm a software engineer and I work in the CERN IT department. I've done quite a few different things in the past in the organization, including software development in the areas of storage and monitoring, and also distributed computing. But right now, I'm part of the CERN Cloud Team, and we manage the CERN private cloud and all the resources we have. And I focus mostly on networking and containerization, so Kubernetes and all these new technologies.



Emily: And on a day to day basis, what do you usually do? What sort of activities are you actually doing?



Ricardo: Yeah. So, it's mostly making sure we provide the infrastructure that our physics users and experiments require, and also the people on campus. So, CERN is a pretty large organization. We have around 10,000 people on-site, and many more around the world that depend on our resources. So, we operate private clouds, we basically do DevOps-style work. And we have a team dedicated for the Cloud, but also for other areas of the data center. And it's mostly making sure everything operates correctly; try to automate more and more, so we do some improvements gradually; and then giving support to our users.



Emily: Just so everyone knows, can you tell a little bit more about what kind of work is done at CERN? What kind of experiments people are running?



Ricardo: Our main goal is fundamental research. So, we try to answer some questions about the universe. So, what's dark matter? What's dark energy? Why don't we see antimatter? And similar questions. And for that, we build very large experiments.



So, the biggest experiment we have, which is actually the biggest scientific experiment ever built, is the Large Hadron Collider, and this is a particle accelerator that accelerates two beams of protons in opposite directions, and we make them collide at very specific points where we build this very large physics experiments that try to understand what happens in these collisions and try to look for new physics. And in reality, what happens with these collisions is that we generate large amounts of data that need to be stored, and processed, and analyzed, so the IT infrastructure that we support, it’s larger fraction dedicated to this physics analysis.



Emily: Tell me a little bit more about some of the challenges related to processing and storing the huge amount of data that you have. And also, how this has evolved, and how it pushed you to think about containerization.



Ricardo: The big challenge we have is the amount of data that we have to support. So, these experiments, each of the experiments, at the moment of the collisions, it can generate data in the order of one petabyte a second. This is, of course, not something we can handle, so the first thing we do, we use these hardware triggers to filter this data quite significantly, but we still generate, per experiment, something like a few gigabytes a second, so up to 10 gigabytes a second. And this we have to store, and then we have large farms that will handle the processing and the reconstruction of all of this. So, we've had these sort of experiments since quite a while, and to analyze all of this, we need a large amount of resources, and with time.



If you come and visit CERN, you can see a bit of the history of computing, kind of evolving with what we used to have in the past in our data center. But it's mostly—we used to have large mainframes, that now it's more in the movies that we see them, but we used to have quite a few of those. And then we transitioned to physical commodity hardware with Linux servers. Eventually introduced virtualization and private clouds to improve the efficiency and the provisioning of these resources to our users, and then eventually, we moved to containers and the main motivation is always to try to be as efficient as possible, and to speed up this process of provisioning resources, and be more flexible in the way we assign compute and also storage.



What we've seen is that in the move from physical to virtualization, we saw that the provisioning and maintenance got significantly improved. What we see with containerization is the extra speed in also deployment and update of the applications that run on those resources. And we also see an improving resource utilization. We already had the possibility to improve quite a bit with virtualization by doing things like overcommit, but with containers, we can go one step further by doing more efficient resource sharing for the different applications we have to run.



Emily: Is the amount of data that you're processing stable? Is it steadily increasing, have spikes, a combination?



Ricardo: So, the way it works is, we have what we call ‘beam’ which is when we actually have protons circulating in the accelerator. And during these periods, we try to get as much collisions as ...

Jaksot(267)

Getting Your Pricing Model Right-ish with Alex Olivier

Getting Your Pricing Model Right-ish with Alex Olivier

In the second episode that I recorded on-site at KubeCon EU in Paris, I spoke with Alex Olivier, CPO and co-founder of Cerbos. This was not a general discussion: It was focused on the process that Cerbos went through to figure out pricing. Here’s what we talked about:The first step of figuring out your pricing is not the number, but rather what you’re charging for. Is it API calls, or amount of data you’re processing, or monthly active users, or monthly active principles… that last one is what Cerbos is charging forWhy it’s important to have a pricing system that allows potential users to be able to roughly estimate for themselves how much using your software is going to cost themYou also want to avoid pricing models that encourage people to look for ways to hack around to find ways to lower their monthly costsWhy your pricing model should be about the value you’re providing, not about how much it costs you to run your systemDiscovering what your price anchors are / what your customers are comparing you toCheck out the full episode for more details! And join us at Open Source Founders Summit for more discussions about the specifics of pricing for open source companies.

10 Huhti 202425min

Nailing Customer Acquisition with Patrick Backman of MariaDB and OpenOcean

Nailing Customer Acquisition with Patrick Backman of MariaDB and OpenOcean

This week, I had a dilemma: should I prioritize the episode where I spoke with one of the MariaDB co-founders, in which we discuss setting up a foundation as a way to ensure that the project continues to be open source in the future, no matter what (relevant given the Redis announcement); or should I prioritize the conversation with one of the founders of Sonatype, one of the oldest companies in the software supply chain security space, in which we talk about the xz debacle. I went with Patrick Backman, general partner at OpenOcean and co-founder of MariaDB, because it’s a little more in my lane. (The conversation with Brian Fox will have to wait for next week!). One of the main things we discussed was the relationship between the MariaDB foundation and the MariaDB company. Including: Why they decided to put MariaDB open source in a foundation, and why they created a separate foundation instead of putting it in an existing foundation The relationship between MariaDB foundation and company today, including the financial relationshipMariaDB was founded by the founders (and some key employees) at MySQL; we also discussed the lessons learned at MySQL that the team then applied at MariaDB. And we talked about customer acquisition, one of the things that Patrick thinks the team had learned at MySQL and therefore had pretty well figured it out at MariaDB. Patrick’s co-founder Monty Widenius is one of the speakers at Open Source Founders Summit — if you want to go into more details on with the lessons from MySQL and MariaDB, as well as lessons from being an investor at OpenOcean, join us in Paris May 27th and 28th at Open source Founders Summit.

3 Huhti 202436min

Ensuring a Project's Long-Term Survival with William Morgan

Ensuring a Project's Long-Term Survival with William Morgan

This week on The Business of Open Source, I have an episode recorded on site at KubeCon EU in Paris with William Morgan, CEO of Buoyant. We had a fabulous conversation, which touched on some touchy subjects, including Buoyant’s slightly changing relationship with Linkerd. But we talked about:Being an open source mercenary, but also being dedicated to making Linkerd a ‘proper’ open source projectFeeling like open source was table stakes for a company in the space Buoyant plays in. This is an under-appreciated reason for being an open source company — you feel like it’s just expected in the market you play in, so you do. Waiting too long (or is it too long?) to commercializeStarting out by selling support, but the problem with that because Linkerd worked well and people kept saying that they didn’t need support because they never had problemsCompeting against Istio, which was backed by the Google engine and how that made Linkerd / Buoyant an underdog (or cockroach). For those of you who haven’t been following Linkerd / Buoyant… Buoyant recently announced that they would be doing edge releases for Linkerd, but not stable releases. We talked about why they made this change and how the ecosystem responded. Check out the full episode!

27 Maalis 202435min

From Project to Profit with Heather Meeker

From Project to Profit with Heather Meeker

This week on The Business of Open Source I talked to Heather Meeker, General Partner of OSS Capital and author of From Project to Profit, How to Build a Business around your Open Source Project. We talked about some things that I entirely agree with, and then there were some points I challenged Heather on — all in all, it was fabulous conversation. Here’s what we covered:Why you should think of your project and product as two different products so you avoid thinking of your open source project as a loss leader and get your incentives rightThe differences between supplementary and complementary products, and how the relationship between project and product is often complementary, even in situations where that relationship is non-obviousWe disagreed about pricing — should COSS businesses have cheaper products than closed-source companies? Why cares about your being open source? Are open source companies more capital efficient? Heather says so, but I’m not convinced. Heather also talked about how they select companies to invest inDo open source companies by definition do a better job at paying attention to user / customer demand? Check out the episode, and check out more about Heather Meeker here: Personal websiteOSS CapitalLinkedIn

20 Maalis 202437min

Delivering Value Quickly in the Observability Space with Pranay Prateek

Delivering Value Quickly in the Observability Space with Pranay Prateek

This week on The Business of Open Source I spoke with Pranay Prateek, co-founder of SigNoz. Pranay talked about why open source is important to SigNoz's business, why it's super important to deliver value quickly, even for an observability product, and why founders shouldn't think of open source just as a distribution model. We also covered: How SigNoz is differentiated in the crowded observability marketWhy Pranay thinks being open source makes it much easier for developers to play around with the project and get to know it; so for them it made intuitive sense that the company that they’d build an open source company Why Pranay also thinks open source enables much deeper integrations, which is critically important for an observability company like SigNozHow one of their first lessons / mistakes was releasing an open source project that didn’t work well on an individual developer’s laptop, because it used too much resourcesThe GTM market, and the challenge delivering value within 30 minutes of trying out the project/product for an observability tool that provides maximum value during an incident — but no one is going to be trying out a new tool during an incident situation Why their first commercial product was a cloud offeringAnd much more! And if you’re interested in more discussions of open source businesses, make sure to join us at Open Source Founders Summit this May.

13 Maalis 202439min

OSFS Special Episode: Being a Strategic Acquisition Target as an OSS Company with Thomas Di Giacomo

OSFS Special Episode: Being a Strategic Acquisition Target as an OSS Company with Thomas Di Giacomo

In this special episode to promote Open Source Founders Summit, I went deep with Thomas di Giacomo about how open source companies can position themselves as attractive acquisition targets for strategic buyers. If you are the founder of an open source company and you have the idea of being acquired even in the back of your mind, this is a must-listen episode. Whether or not you plan to join us May 27th and 28th in Paris, though of course we hope you do join us. By the way, at OSFS Thomas is going to lead a workshop on the topic of being an acquisition target for open source companies. It will be interactive, which means you can ASK QUESTIONS. In this podcast episode, he talked about: Exits 101. You probably know that strategic buyers usually pay more for companies than other types of acquirers, but we talked about different exit strategies and what they entailWhy strategic buyers acquire businesses (in general) but also why you, as the business seller, need to understand every specific potential acquirer’s story and goals so you can see how your company fits into their strategic planStrategic acquisitions are about 1+1=3… so you have to know what your buyer’s “3” isWhy it’s important to be self-aware and know your own goals before you sign any acquisition paperworkAnd tons more… If you want the chance to ask Thomas about strategic acquisitions for OSS companies — as well as to talk about sales strategies, lead generation and more — join us at OSFS 24 in Paris this May 27th and 28th. —> Get your invite here. PS the audio was a little quiet, but so if you’re having trouble hearing turn up the volume, it’s worth it.

12 Maalis 202432min

Buyer-Based Open Core with Zach Wasserman

Buyer-Based Open Core with Zach Wasserman

This week on The Business of Open Source, I spoke with Zach Wasserman, co-founder and CTO of Fleet. This was a fabulous episode for many reasons, but then again I never do crappy episodes, right? The first thing I wanted to call your attention to is that Zach talked about how he’s building an open core business because building an open source business is what he wants to do. When his previous company turned away from open source, Zach left to do consulting around OSquery and Fleet (the project). I always like to talk about how companies / founders need a solid reason for building an open source company… and “this is the kind of company I want to build” is a very good reason. (“Everyone else is doing it” on the other hand, is not a good reason). Everyone puts constraints around the type of company the want to build, and as long as you are intentionally about the decisions, there is nothing wrong about this, business-wise.Second, we talked about the tension that exists between making a great project and still leaving room for a commercial product that people will pay for, and Zach talked through how Fleet uses a buyer-based open core strategy to decide which functionality to put in the enterprise version or in the open core. We also talked about:Leaving his first company, Kolide, when the founders had divergent visions about where the company should goHow his investor arranged a ‘co-founder marriage’ for Zach and his co-founder Mike McNeilHow the transparency aspect of open source can be extremely important, especially for anything in the security spaceLastly, Fleet happens to be a former client of mine. You can check out what Mike, Zach’s co-founder, said about working with me here. And if you’re interested in more conversations like this… but in person!!! you should come to Open Source Founders Summit May 27th and 28th in Paris.

6 Maalis 202437min

The Evolving Relationship between Apache Cassandra and DataStax

The Evolving Relationship between Apache Cassandra and DataStax

Slightly different The Business of Open Source episode today! I spoke with Patrick McFadin and Mick Semb Wever about the relationship between Apache Cassandra and DataStax — how it was at the beginning and how the relationship has evolved over the years. We talked about:— How there was a dynamic around Cassandra where many of the many of the contributors ended up being sucked into the DataStax orbit, simply because it allowed those contributors to work on on Cassandra full-time— How there can be tensions between different stakeholders simply because everyone involved ultimately has their own interests at heart, and those interests are not always aligned. — How it is actually hard to really have open discussions about new features, and how often there can be a new feature dropped in a project that clearly had been developed behind closed doors for some time, and sometimes that created tension in the community— Some open source projects are just too complex to be hobby projects — Cassandra is so complex that you won’t become a code contributor unless you’re working full-time on Cassandra, because that’s the level of skill you need to keep up. — How the relationship between a company and a project often changes as the technology matures. — The importance of addressing tensions between company and community head-on, as adults, when they occur — as well as why you need to remember to treat people as humans and remember that they have good days, bad days, goals and interests. Patrick on LinkedInMick on LinkedIn

28 Helmi 202440min

Suosittua kategoriassa Liike-elämä ja talous

sijotuskasti
mimmit-sijoittaa
psykopodiaa-podcast
puheenaihe
rss-rahapodi
ostan-asuntoja-podcast
pomojen-suusta
rss-startup-ministerio
rss-lahtijat
rss-rahamania
rahapuhetta
rss-neuvottelija-sami-miettinen
kasvun-kipuja
rss-h-asselmoilanen
rss-turvacast
taloudellinen-mielenrauha
leadcast
syo-nuku-saasta
muutosakatemia-coaching-podcast
lakicast