CERN’s Transition to Containerization and Kubernetes with Ricardo Rocha

CERN’s Transition to Containerization and Kubernetes with Ricardo Rocha

Some of the highlights of the show include:


  • The challenges that CERN was facing when storing, processing, and analyzing data, and why it pushed them to think about containerization.
  • CERN’s evolution from using mainframes, to physical commodity hardware, to virtualization and private clouds, and eventually to containers. Ricardo also explains how the migration to containerization and Kubernetes was started.
  • Why there was a big push from groups that focus on reproducibility to explore containerization.
  • How end users have responded to Kubernetes and containers. Ricardo talks about the steep Kubernetes learning curve, and how they dealt with frustration and resistance.
  • Some of top benefits of migrating to Kubernetes, and the impact that the move has had on their end users.
  • Current challenges that CERN is working through, regarding hybrid infrastructure and rising data loads. Ricardo also talks about how CERN optimizes system resources for their scientists, and what it’s like operating as a public sector organization.
  • How CERN handles large data transfers.


Links:


Transcript

Emily: Hi everyone. I’m Emily Omier, your host, and my day job is helping companies position themselves in the cloud-native ecosystem so that their product’s value is obvious to end-users. I started this podcast because organizations embark on the cloud naive journey for business reasons, but in general, the industry doesn’t talk about them. Instead, we talk a lot about technical reasons. I’m hoping that with this podcast, we focus more on the business goals and business motivations that lead organizations to adopt cloud-native and Kubernetes. I hope you’ll join me.



Emily: Welcome to the Business of Cloud Native. I'm your host, Emily Omier, and today I'm here with Ricardo Rocha. Ricardo, thank you so much for joining us.



Ricardo: It's a pleasure.



Emily: Ricardo, can you actually go ahead and introduce yourself: where you work, and what you do?



Ricardo: Yeah, yes, sure. I work at CERN, the European Organization for Nuclear Research. I'm a software engineer and I work in the CERN IT department. I've done quite a few different things in the past in the organization, including software development in the areas of storage and monitoring, and also distributed computing. But right now, I'm part of the CERN Cloud Team, and we manage the CERN private cloud and all the resources we have. And I focus mostly on networking and containerization, so Kubernetes and all these new technologies.



Emily: And on a day to day basis, what do you usually do? What sort of activities are you actually doing?



Ricardo: Yeah. So, it's mostly making sure we provide the infrastructure that our physics users and experiments require, and also the people on campus. So, CERN is a pretty large organization. We have around 10,000 people on-site, and many more around the world that depend on our resources. So, we operate private clouds, we basically do DevOps-style work. And we have a team dedicated for the Cloud, but also for other areas of the data center. And it's mostly making sure everything operates correctly; try to automate more and more, so we do some improvements gradually; and then giving support to our users.



Emily: Just so everyone knows, can you tell a little bit more about what kind of work is done at CERN? What kind of experiments people are running?



Ricardo: Our main goal is fundamental research. So, we try to answer some questions about the universe. So, what's dark matter? What's dark energy? Why don't we see antimatter? And similar questions. And for that, we build very large experiments.



So, the biggest experiment we have, which is actually the biggest scientific experiment ever built, is the Large Hadron Collider, and this is a particle accelerator that accelerates two beams of protons in opposite directions, and we make them collide at very specific points where we build this very large physics experiments that try to understand what happens in these collisions and try to look for new physics. And in reality, what happens with these collisions is that we generate large amounts of data that need to be stored, and processed, and analyzed, so the IT infrastructure that we support, it’s larger fraction dedicated to this physics analysis.



Emily: Tell me a little bit more about some of the challenges related to processing and storing the huge amount of data that you have. And also, how this has evolved, and how it pushed you to think about containerization.



Ricardo: The big challenge we have is the amount of data that we have to support. So, these experiments, each of the experiments, at the moment of the collisions, it can generate data in the order of one petabyte a second. This is, of course, not something we can handle, so the first thing we do, we use these hardware triggers to filter this data quite significantly, but we still generate, per experiment, something like a few gigabytes a second, so up to 10 gigabytes a second. And this we have to store, and then we have large farms that will handle the processing and the reconstruction of all of this. So, we've had these sort of experiments since quite a while, and to analyze all of this, we need a large amount of resources, and with time.



If you come and visit CERN, you can see a bit of the history of computing, kind of evolving with what we used to have in the past in our data center. But it's mostly—we used to have large mainframes, that now it's more in the movies that we see them, but we used to have quite a few of those. And then we transitioned to physical commodity hardware with Linux servers. Eventually introduced virtualization and private clouds to improve the efficiency and the provisioning of these resources to our users, and then eventually, we moved to containers and the main motivation is always to try to be as efficient as possible, and to speed up this process of provisioning resources, and be more flexible in the way we assign compute and also storage.



What we've seen is that in the move from physical to virtualization, we saw that the provisioning and maintenance got significantly improved. What we see with containerization is the extra speed in also deployment and update of the applications that run on those resources. And we also see an improving resource utilization. We already had the possibility to improve quite a bit with virtualization by doing things like overcommit, but with containers, we can go one step further by doing more efficient resource sharing for the different applications we have to run.



Emily: Is the amount of data that you're processing stable? Is it steadily increasing, have spikes, a combination?



Ricardo: So, the way it works is, we have what we call ‘beam’ which is when we actually have protons circulating in the accelerator. And during these periods, we try to get as much collisions as ...

Avsnitt(267)

David and Goliath in the CMS Market with Thomas Schedler

David and Goliath in the CMS Market with Thomas Schedler

This week on The Business of Open Source, I spoke with Thomas Schedler, co-founder and CEO of Sulu. Sulu is a small, bootstrapped company that spun out of an agency; Thomas was recommended by someone to me as a guest, but when I first reached out to him he was skeptical about coming on the show because he wasn’t sure he was a good fit. But there’s a bunch to learn from this episode! Spinning out from an agency —> in my experience, one of the most common ways for open source companies to be bootstrapped, or even to be go on to raise venture capital, is to start out as a web development agency that uses a lot of open source tools.They are working on moving away from services and getting more revenue from the product; and Thomas talked about the advantages that they get from providing both services and developing a product-based business strategy.The CMS market is crowded! And everyone is open source, so being an open source company doesn’t differentiate the product at all. So how does Sulu differentiate itself in the market? Thomas talked about it. I loved that Thomas talked about how important opinions and technology choices are in differentiating themselves on the market. I also loved that Thomas talked about keeping some things out of the product as a way to differentiate themselves from others. Sometimes more features can deliver less value to users. How being small can allow you to move quicker, and that gives Sulu and advantage over the huge players in the CMS market.Why differentiation and knowing your niche is so important when you’re a small company competing again large players. Do you need to differentiate yourself in a crowded market? You might want to work with me.

28 Maj 35min

Open Source Firmware for EV Charging Stations with Marco Möller

Open Source Firmware for EV Charging Stations with Marco Möller

This week on The Business of Open Source I spoke with Marco Möller, CEO and co-founder of Pionix. This was a fabulous conversation about a company that’s in a very different market from the usual open source company: They make firmware for EV charging stations, and are behind the EVerest project, which is part of LF Energy. So who is part of the community open source projects that are so connected to hardware? Here’s what we talked about. There aren’t many hobbyists in the community — but there are a lot of people who have professional interests in the electric vehicle industry. EVerest doesn’t focus on what Marco considers “personal” or “hobbiest” features. This was a great discussion about how understanding the target market, including for the project, leads to different feature decisions. How does Pionix money? It doesn’t yet :), but we talked about various plans the company has for monetization in the future. How being involved with the LF Energy has helped build trust, which is very important for a company that is only going to be selling to large companies and governments. Why Marco thinks that the huge vision he has for creating firmware for EV charging stations is only possible with open source, and how he makes sure that he’s able to fund the large mission without hurting the community. The balancing act with getting investors who specialize in the energy market but who don’t have experience with open sourceStruggling with positioning your project or figuring out how to best monetize? You might want to work with me.

14 Maj 33min

AI-generated Code Copied from Open Source with Julian Coccia

AI-generated Code Copied from Open Source with Julian Coccia

This week on The Business of Open Source, I spoke with Julian Coccia, CTO of ScanOSS, about selling access to data while making open source software. Of course, we also talked about being an open source company that is also deep in the open source world, helping companies understand their reliance on open source code and how open source is used in their codebase. Several things that we talked about: AI and open source code. This is a topic that’s slightly tangential to business, but very interesting to me. If you’re using an AI code generator, you are almost certainly putting verbatim copies of open source code into your codebase, but without realizing you’re doing so and without any understanding of the license requirements. This is a huge compliance risk. 2% of code generated by AI is a verbatim copy of open source code. How the biggest challenges isn’t necessarily monetizing, but getting visibility into the sales funnel. It can feel random and sudden; a company appears out of nowhere ready to sign a contract because they’ve been using the open source project for months. This is the last week to get tickets for Open Source Founders Summit, so if you want to join other leaders of open source companies May 19th and 20th in Paris, get your tickets now!

7 Maj 35min

How to be Successful when Donating a Project to the CNCF with Liz Rice

How to be Successful when Donating a Project to the CNCF with Liz Rice

Today on The Business of Open Source I spoke with Liz Rice, Chief Open Source Officer at Isovalent, which is now part of Cisco. We addressed two subjects: How to be successful as a company that donates their project to the CNCF, and the story of Isovalent’s acquisition by Cisco and the role open source played in that acquisition. We talked about: Trademarks. This is a very important part of what you donate when you donate a project to the CNCF (or other foundations). We talked about what you can and can not do with the name and logo of “your” project when it becomes part of the CNCF, and what that means for the competitive landscape you’re a part of. How to best take advantage of the marketing benefits that being part of the CNCF brings. How to create a link between the CNCF project and the company that donated it. The role that Cilium and eBPF played in Isovalent’s acquisition by Cisco. Why Isovalent’s relationship with open source is valuable to Cisco in and of itself. How open source companies can increase the likelihood that they’ll be able to continue investing in open source post-acquisition. Why it’s so important to find opportunities for collaboration. Want help making the link between your CNCF project and your commercial product? You might want to work with me.

30 Apr 40min

Open Source Manifestos with Vincent Untz

Open Source Manifestos with Vincent Untz

This week on The Business of Open Source I talked about Open Source Manifestos with Vincent Untz, CTO of Centreon. The entire conversation focused on this idea of open source manifestos, which Vincent is going to talk about at Open Source Founders Summit — and I had never heard of before. The idea to create an open source manifesto came about because internally there was a lot of frustration around Centreon’s relationship with open source. A lot of people, especially those who didn’t come from an engineering background, would routinely ask why the company invested so much in open source. At the same time, there was a feeling among the engineering team that the company’s commitment to open source — and the concrete development time spent on open source — was slipping. After creating the manifesto, the doubts about why open source mattered went away immediately — there hasn’t been a single conversation calling into question why open source matters since the manifesto was finished. Now conversations about open source are different. Externally, there’s been a huge change in conversations around open source. People say things like “Centreon is back in the game.” It’s hard to point to concrete financial results as a result of the manifesto, but there have been differences in relationships with partners for sure, and it has helped get more companies to contribute to the project. Want to see what the Centreon Open Source Manifesto looks like? Check it out here.

2 Apr 39min

How a Rebrand Increased Sales with Lukas Gentele

How a Rebrand Increased Sales with Lukas Gentele

This week on The Business of Open Source, I spoke with Lukas Gentele, the CEO and co-founder of LoftLabs. Here’s some of the things we covered: There are many open source projects at LoftLabs. We talked about what the team did differently the second time round, when they’d had the experience of creating the first project under their belt, and why they continued creating additional projects after that. How they make roadmap decisions to decide what goes in the OSS versus what goes in the enterprise editionHow they thought about the benefits to the business from open sourcing vCluster, the second project they released, and whether or not the theory about how it would benefit the business ended up being correct. Whether or not open code is necessary for community building, and why. In particular, we talked about the difference in how people feel ‘ownership’ of open source projects when they contribute to them. The pros and cons of building different brands around your project, your product and your company; and of having different brands around your different projects. We talked about this a lot; LoftLabs has a different company brand from any of the products or projects, but they discovered that if the project and product do not have the same brand, people get so confused that they ultimately do not buy. If you need to figure out how to brand your project and product, you might want to work with me. If you want an opportunity to talk with other founders about branding, community building, and more, you should join us at Open Source Founders Summit May 19th and 20th in Paris.

26 Mars 42min

Fundraising and M&A for Open Source Companies with Daniel Jarjoura

Fundraising and M&A for Open Source Companies with Daniel Jarjoura

This week on the Business of Open Source I spoke with Daniel Jarjoura, an investor at Avolta who specializes in developer-facing companies, and who writes a newsletter on developer-facing startups and also collects data on investments in open source companies. We spoke about the overall state of investment for open source companies and specifically the state of investment in open source companies in Europe. Here’s what we talked about: Differences between venture funding for OSS companies in Europe and in NAHow open source companies were relatively stable in terms of getting venture funding, even as the entire venture funding ecosystem was in free-fall.How it might be true that European investors don’t understand open source companies, but on the other hand plenty of North American investors invest in European companies. Do developers trust open core companies? We had a discussion about whether or not open source companies have a trust problem…How important communication is for open source companies, something that I think is very true — a lot of the problems that open source companies run into when they either adopt an open core model or even when they do a license change. How founders can do a better job communicating with their VCs, including how to do a better job targeting appropriate investors. What do optimizing for in the VC relationship … and spoiler alert, that is for money. You might want a perfect VC who can make introductions, give you advice, give you money, be nice, etc. But the most important thing is the cash; focus on that. The second most important is the introductions. But advice isn’t something you should count on from VCs, and that really isn’t a VCs job, anyway. We finished the conversation with a discussion about M&A for open source companies. If you want good advice about your open source company, you might want to work with me. If you want an opportunity to talk with other founders about things like M&A for open source companies, communicating with VCs, and other topics specific to open source companies, you should join us at Open Source Founders Summit May 19th and 20th in Paris.

19 Mars 43min

Products, consulting, and open source with Andrew Martin

Products, consulting, and open source with Andrew Martin

This week on The Business of Open Source I spoke with Andrew Martin, CEO and founder of Control Plane. Control Plane is ultimately a consulting company, as Andrew introduced it. But the company also created and maintains KubeSec, and also has an enterprise version of Flux CD that it licenses. That gives this conversation a slightly different flavor from the usual. Here’s some of the things we talked about: The business ROI that Andrew expected to get from releasing KubeSec, and whether or not he as actually seen those expected results play outThe difference between bootstrapping a company by offering services and by offering consulting — specifically what the difference between services and consulting is. The problem with “drive-by” feature development in open source projects — when people contribute features but don’t commit to the ongoing maintenance of the projectWhy Control Plane decided to release an enterprise version of FluxCD, and how the enterprise version of Flux is differentiated from the pure open source project. Why underinvesting in marketing, especially in marketing for the open source projects, was a mistakeJoin us at Open Source Founders Summit if you want more conversations about how companies are built around open source projects, from consultancies to large corporations.

12 Mars 36min

Populärt inom Business & ekonomi

badfluence
framgangspodden
varvet
rss-borsens-finest
svd-ledarredaktionen
avanzapodden
lastbilspodden
rss-dagen-med-di
borsmorgon
uppgang-och-fall
affarsvarlden
fill-or-kill
rss-svart-marknad
rss-kort-lang-analyspodden-fran-di
rss-inga-dumma-fragor-om-pengar
rss-en-rik-historia
tabberaset
rikatillsammans-om-privatekonomi-rikedom-i-livet
kapitalet-en-podd-om-ekonomi
rss-badfluence