CERN’s Transition to Containerization and Kubernetes with Ricardo Rocha

CERN’s Transition to Containerization and Kubernetes with Ricardo Rocha

Some of the highlights of the show include:


  • The challenges that CERN was facing when storing, processing, and analyzing data, and why it pushed them to think about containerization.
  • CERN’s evolution from using mainframes, to physical commodity hardware, to virtualization and private clouds, and eventually to containers. Ricardo also explains how the migration to containerization and Kubernetes was started.
  • Why there was a big push from groups that focus on reproducibility to explore containerization.
  • How end users have responded to Kubernetes and containers. Ricardo talks about the steep Kubernetes learning curve, and how they dealt with frustration and resistance.
  • Some of top benefits of migrating to Kubernetes, and the impact that the move has had on their end users.
  • Current challenges that CERN is working through, regarding hybrid infrastructure and rising data loads. Ricardo also talks about how CERN optimizes system resources for their scientists, and what it’s like operating as a public sector organization.
  • How CERN handles large data transfers.


Links:


Transcript

Emily: Hi everyone. I’m Emily Omier, your host, and my day job is helping companies position themselves in the cloud-native ecosystem so that their product’s value is obvious to end-users. I started this podcast because organizations embark on the cloud naive journey for business reasons, but in general, the industry doesn’t talk about them. Instead, we talk a lot about technical reasons. I’m hoping that with this podcast, we focus more on the business goals and business motivations that lead organizations to adopt cloud-native and Kubernetes. I hope you’ll join me.



Emily: Welcome to the Business of Cloud Native. I'm your host, Emily Omier, and today I'm here with Ricardo Rocha. Ricardo, thank you so much for joining us.



Ricardo: It's a pleasure.



Emily: Ricardo, can you actually go ahead and introduce yourself: where you work, and what you do?



Ricardo: Yeah, yes, sure. I work at CERN, the European Organization for Nuclear Research. I'm a software engineer and I work in the CERN IT department. I've done quite a few different things in the past in the organization, including software development in the areas of storage and monitoring, and also distributed computing. But right now, I'm part of the CERN Cloud Team, and we manage the CERN private cloud and all the resources we have. And I focus mostly on networking and containerization, so Kubernetes and all these new technologies.



Emily: And on a day to day basis, what do you usually do? What sort of activities are you actually doing?



Ricardo: Yeah. So, it's mostly making sure we provide the infrastructure that our physics users and experiments require, and also the people on campus. So, CERN is a pretty large organization. We have around 10,000 people on-site, and many more around the world that depend on our resources. So, we operate private clouds, we basically do DevOps-style work. And we have a team dedicated for the Cloud, but also for other areas of the data center. And it's mostly making sure everything operates correctly; try to automate more and more, so we do some improvements gradually; and then giving support to our users.



Emily: Just so everyone knows, can you tell a little bit more about what kind of work is done at CERN? What kind of experiments people are running?



Ricardo: Our main goal is fundamental research. So, we try to answer some questions about the universe. So, what's dark matter? What's dark energy? Why don't we see antimatter? And similar questions. And for that, we build very large experiments.



So, the biggest experiment we have, which is actually the biggest scientific experiment ever built, is the Large Hadron Collider, and this is a particle accelerator that accelerates two beams of protons in opposite directions, and we make them collide at very specific points where we build this very large physics experiments that try to understand what happens in these collisions and try to look for new physics. And in reality, what happens with these collisions is that we generate large amounts of data that need to be stored, and processed, and analyzed, so the IT infrastructure that we support, it’s larger fraction dedicated to this physics analysis.



Emily: Tell me a little bit more about some of the challenges related to processing and storing the huge amount of data that you have. And also, how this has evolved, and how it pushed you to think about containerization.



Ricardo: The big challenge we have is the amount of data that we have to support. So, these experiments, each of the experiments, at the moment of the collisions, it can generate data in the order of one petabyte a second. This is, of course, not something we can handle, so the first thing we do, we use these hardware triggers to filter this data quite significantly, but we still generate, per experiment, something like a few gigabytes a second, so up to 10 gigabytes a second. And this we have to store, and then we have large farms that will handle the processing and the reconstruction of all of this. So, we've had these sort of experiments since quite a while, and to analyze all of this, we need a large amount of resources, and with time.



If you come and visit CERN, you can see a bit of the history of computing, kind of evolving with what we used to have in the past in our data center. But it's mostly—we used to have large mainframes, that now it's more in the movies that we see them, but we used to have quite a few of those. And then we transitioned to physical commodity hardware with Linux servers. Eventually introduced virtualization and private clouds to improve the efficiency and the provisioning of these resources to our users, and then eventually, we moved to containers and the main motivation is always to try to be as efficient as possible, and to speed up this process of provisioning resources, and be more flexible in the way we assign compute and also storage.



What we've seen is that in the move from physical to virtualization, we saw that the provisioning and maintenance got significantly improved. What we see with containerization is the extra speed in also deployment and update of the applications that run on those resources. And we also see an improving resource utilization. We already had the possibility to improve quite a bit with virtualization by doing things like overcommit, but with containers, we can go one step further by doing more efficient resource sharing for the different applications we have to run.



Emily: Is the amount of data that you're processing stable? Is it steadily increasing, have spikes, a combination?



Ricardo: So, the way it works is, we have what we call ‘beam’ which is when we actually have protons circulating in the accelerator. And during these periods, we try to get as much collisions as ...

Avsnitt(267)

How Technology Decisions Impact Growth with Misha Bragin

How Technology Decisions Impact Growth with Misha Bragin

This week on The Business of Open Source, I spoke with Misha Bragin, co-founder and CEO of NetBird. This was also the first episode I recorded in 2025, which gives you an idea of how far in advance I’m recording episodes. NetBird has an interesting origin story — it came out of an original idea to make a hardware product, but as Misha and his co-founder were starting to work out the realities of manufacturing, they realized that they would be better off building a software company. Here were some of the take aways from the conversation: Sometimes your users want to pay you! Misha explained that before there was a commercial offering, some open source users proactively reached out asking for one — because they understood that in order for the company to be viable, it needed to have revenue. Just because you get crickets at first doesn’t mean your project sucks. When NetBird’s open source project was first released, it made zero splash.The reality of working in a proprietary piece of software, you can do things quick and dirty. *Obviously you shouldn’t, but let’s face it a lot of people do. But when your code is open, it is also a part of your communication strategy! If you push out code that’s not tested or that’s not well done, you risk reputation damage even if the software works exactly as it should. Misha talked about how choosing Auth0 as an authentication service was a mistake — not because it is a bad service, but because it was not appropriate for their target audience, who are self-hosting. This is a very good reminder of how sometimes choosing the best technology for the job can backfire; you also have to take into account who the target user is and if they are going to be repelled by the choice you’re making. Struggling to figure out how to balance your project development with your product development? Need to supercharge your funnel? You might want to work with me. Want to share your knowledge and learn from your peers who are open source entrepreneurs? You should join us at Open Source Founders Summit May 19th and 20th in Paris.

5 Mars 35min

Open Source and AI Coding Assistants with Ty Dunn

Open Source and AI Coding Assistants with Ty Dunn

This week on The Business of Open Source, I spoke with Ty Dunn, founder of Continue.dev, which is an open source AI code assistant. We had a fabulous conversation that touched on both the AI hype wave and why open source. The first thing I’d like to touch on is why Continue.dev is open source, in other words, what business rationale Ty has for taking that route. Because he makes some great points about building an ecosystem. If building an ecosystem, and encouraging people in your community to build on top of your software, is something that is important to your for either strategic or philosophical reasons, the absolute best way to accomplish this goal is with an open source project. In the case of Continue, being open source, and allowing companies to retain control over their data, is also a differentiation in the market. There was also a phrase in the conversation that I wanted to pull out — Ty talked about monetizing “next to” the open source project. This is precisely how many of the most successful open source companies work. They have a successful open source project but monetize it in a way that is adjacent, rather than directly competing with the project. Check out the full episode for a discussion about the AI hype wave and how it intersects with open source business models. If you want more discussions about open source companies, you shouldn’t miss Open Source Founders Summit this May 19th and 20th — it’s the only event specifically designed for leadership in open source companies. And if you’re struggling with your open source strategy, you should consider working with me.

26 Feb 40min

The CFO's View of Open Source Companies with Eileen Doody and Karen Walker

The CFO's View of Open Source Companies with Eileen Doody and Karen Walker

This week on The Business of Open Source I had a slightly different conversation: I spoke with the CFOs of two open source companies, Sysdig and Percona, to better understand what is different (and what is not) about financial management in open source companies. Karen Walker is the CFO at Sysdig, and Eileen Doody is the CFO at Percona. They both joined me to talk about the CFO role in general and the CFO role in particular at an open source company. Why did I do this episode? Many founders I’ve spoken with are a bit unclear on the role of a CFO — whereas I’ve never spoken with a founder who had trouble understanding what their CTO does. Here’s some takeaways from our conversation: Part of the CFO’s role is about thinking about open source strategically, in terms of how the open source project is going to fit into the company’s overall strategy.Because open source is so ingrained in the company, it doesn’t fit into a single budget line item; it’s impossible to break out and say ‘we spend $X on open source’ because it’s so integrated into everything the company doesHow do you measure your ROI on investment in open source? At Sysdig, two out of three prospects come to the company because of Falco, their open source project. We also talked about the ecosystem effects of having a huge footprint with your open source project; it’s hard to measure the positive influence of having massive brand awareness, but both CFOs are convinced that it is very important to the company. Eileen says that many CIOs now have mandates to look for open source solutions when possible, which was not the case a decade ago. That’s changed the dynamic for a company like Percona that’s based around open source. Another reason I did this episode is because while I usually have founders on the podcast, there are some really important perspectives from other leadership team members. Part of the the role of a CEO is to understand all the other C-level leadership position’s roles and responsibilities, and in my experience the CFO is one of the less well understood roles. In fact, we wrapped up the conversation by talking about how a CFO can be a real strategic partner that’s forward-thinking rather than just the bean-counter that some people expect a CFO to be. A couple things to mention. First of all, if you want to learn more about my consulting work with open source companies, you can do so here. Second, if you want to chance to connect with other founders of open source companies, consider joining Open Source Founders Summit this May 19th and 20th in Paris.

19 Feb 37min

Thinking Hard about your License Choice with Ivan Burazin

Thinking Hard about your License Choice with Ivan Burazin

This week on The Business of Open Source, I spoke with Ivan Burazin, the CEO and co-founder of Daytona. First of all, Daytona was one of the sponsors of the first edition of Open Source Founders Summit, and I had a chance to meet Ivan in person at the event. So a big thank you to him for taking a risk on the first year of the event! But let’s get down to business. We talked about: Why Daytona took an open source approach, even though they originally started out with a pure commercial licensed software. The thought that went into choosing the license — this was a great discussion, because it is a question a lot of people have. Ivan walked through his thought process in choosing a license and how he thinks about difference licenses. The theory about how open source will help Daytona build their business, and the dynamics of how open source adoption leads to commercial success. The importance of technical decisions in an open source project’s growth strategy.The different stakeholders in the community, as well as in the customer community. I really liked the piece when Ivan talked about the individual motivations that people might have for getting involved in a community — instead of thinking about how you get something from the community, think about how others can get something out of the community. For example, you can have someone who wants to get into technical writing who gets that experience by writing documentation for your project. Making it easier for that to happen is in everyone’s best interest. Like this episode? Struggling with your messaging or with your conversions from open source project to commercial customers? You might want to work with me. And if you want to join us at Open Source Founders Summit, apply to join us here.

6 Feb 39min

Bootstrapping an Open Source Company with Ludovic Dubost

Bootstrapping an Open Source Company with Ludovic Dubost

This week on The Business of Open Source, I talked with Ludovic Dubost, founder and CEO of XWiki about the long history of XWiki, which he started in 2003. This was a wide-ranging conversation… here’s some of the things we covered: Why XWiki is an open source company. Ludovic started out with the explicit goal of creating a company, so why start it as an open source company? The reality that starting a software company in Europe means putting yourself at a disadvantage — which doesn’t mean that you shouldn’t do it, but rather that you need to be aware of that fact, and that open source is one tool that European companies can use to  succeed even when they are at a disadvantage. How Ludovic bootstrapped the company, including moving from support contracts to recurring revenue from the SaaS and how the transition has happened over the yearsThe difficulties that come from being small and bootstrappedManaging the balance between distribution and monetizationWhy a community version / enterprise version model did not work for XWikiWhy it’s so important to pay attention to your competitive landscape, because the only way to get customers is by offering something that is better in some way than your competitors — it’s your comparative strengths that matter. Did you enjoy this conversation? Find Ludovic and XWiki at Fosdem on Saturday! And/or join us at Open Source Founders Summit. Are you also interested in the challenges around product management for open source companies? You might want to work with me.

29 Jan 46min

Building an Open Source Company for Long-Term Sustainability with John O'Nolan

Building an Open Source Company for Long-Term Sustainability with John O'Nolan

This week on The Business of Open Source, I spoke with John O’Nolan, the co-founder of ghost.org. Before further ado, John is going to be one of speakers at Open Source Founders Summit 2025, so if you’d like a chance to dive deeper into any of the subjects we talked about on the podcast with him, in person, you should join us in May. There’s a lot of interesting tidbits to pull out from this conversation. First of all, I think it’s interesting that Ghost came about because Wordpress was moving away from its roots as a pure publishing tool and becoming a website builder. John, who was very involved in the Wordpress community at the time, wondered what it would look like if Wordpress went back to its roots and focused on publishing and only publishing. It’s a lesson for founders that sometimes focusing on the small niches left behind as incumbents expand can be huge opportunity. —> It’s worth noting that we recorded this podcast last fall when the drama between Wordpress and WPEngine was exceptionally hot. Ghost is organized as a non-profit, and John also talked about why he made that decision from the beginning. It came down to wanting to make a good salary at a company he had started, but without the goal of becoming fabulously wealthy as a result. We also talked about whether or not a venture-backed company can be ‘responsible’ with respect to their community; and what types of companies tend to be able to manage the tensions between the community needs and the fiduciary duty that you have if you take outside funding. We also talked about the difference in the market between the product and the project, how Ghost manages to expand in spite of not having a dedicated marketing team. We also talked about the difference between building a sustainable business and building a business that gets hockey stick very quickly as well as some of the tension between technology decisions and business decisions. If you want to talk more about these issues — and want to talk directly with John — you should come to Open Source Founders Summit May 19th and 20th, 2025. Get your tickets here.

22 Jan 43min

The Complicated Calculus around Donating a Project to a Foundation with Omri Gazette

The Complicated Calculus around Donating a Project to a Foundation with Omri Gazette

In the last episode of The Business of Open Source recorded at KubeCon Salt Lake City, I spoke with Omri Gazitt, co-founder and CEO of Aserto. Aserto has two open source project that it maintains, one of which it donated to the CNCF. In this episode, we talked about the decision to donate a project to the CNCF — both what the process entailed and what is in for Aserto in having a project at the CNCF. But of course Aserto also has another project, Topaz, which it has not donated to the CNCF. We also talked about why Topaz wasn’t donated to the CNCF. A couple things to pull out of this conversation: The complicated calculus of deciding whether to donate a project to a foundation, and how the dynamics of the market change over the years and you have to think very critically about the specifics of your situation before making the decision to donate to a foundationHow every company has slightly different market pressures — sometimes the market pushed you to donate to a foundation, sometimes the market doesn’t care. The importance of thinking not just about market share when you’re open source, but also how you are going to monetize! It’s possible to have vastly smaller market share but make vastly more money. Why being an open source company does not have to mean that your paid solution has to be cheaper than your competitors. Why you don’t have to start selling into startups — sometimes your best customers will always be either mid-market or enterprise from the very beginning. We talked about the panel I moderated at CloudNative StartupFest at KubeCon. If you missed it, here’s the link to see the replay. We also talked about Adam Jacob’s talk at the same event, which you can see here. If you’re building a company around an open source project and aren’t sure how to manage the relationship between the project and product, you might want to work with me or come to Open Source Founders Summit this May.

15 Jan 24min

A Massive Head Start on Product Development with Open Source with Martin Mao

A Massive Head Start on Product Development with Open Source with Martin Mao

This special episode recorded live at KubeCon Salt Lake City last November is with Martin Mao, CEO and co-founder at Chronosphere.We talked about how M3 was foundational to the early history of Chronosphere, and how the ability to leverage M3, which Martin and his co-founder had written while they were still working at Uber. One of the most important aspects of this story is that since M3 is the foundation Chronosphere is built on, the fact that it was developed over four years at Uber while they were still on Uber’s payroll meant that when they decided to build a company it allowed them to get to market dramatically faster than would have been possible otherwise. Chronosphere’s core platform is a proprietary SaaS product, but still has a significant relationship with two other projects: Perses, which was developed at Chronosphere and donated to the CNCF in 2024; and FluentBit, a CNCF graduated project that was originally developed by Calyptia and became part of Chronosphere when it acquired Calyptia. We talked about: The pros and cons of donating projects to the CNCF, from both the perspectives of the company creating the project and the interests of the community and project itselfWhy Chronosphere’s core platform isn’t open source itselfHow a company can end up getting financial advantages from being the stewards of large open source community, even if the connection doesn’t always seem obviousHow product roadmaps are managed for the two projects versus how it’s managed for Chronosphere’s proprietary products. If you’re building a company around an open source project and aren’t sure how to manage the relationship between the project and product, you might want to work with me or come to Open Source Founders Summit this May.

10 Jan 20min

Populärt inom Business & ekonomi

badfluence
framgangspodden
varvet
rss-borsens-finest
svd-ledarredaktionen
avanzapodden
lastbilspodden
rss-dagen-med-di
borsmorgon
uppgang-och-fall
affarsvarlden
fill-or-kill
rss-svart-marknad
rss-kort-lang-analyspodden-fran-di
rss-inga-dumma-fragor-om-pengar
rss-en-rik-historia
tabberaset
rikatillsammans-om-privatekonomi-rikedom-i-livet
kapitalet-en-podd-om-ekonomi
rss-badfluence