Live at Start Campus! Omer Wilson & Andrew Bradner: The AI and liquid cooling revolution

Another exclusive episode with Andy Davis and the Inside Data Centre Podcast live from Sines, Portugal! Host Andy Davis is joined by Omer Wilson, Chief Marketing Officer at Start Campus, and Andrew Bradner, General Manager, Cooling Business at Schneider Electric, to chat all things cooling in their new data centre facility in Sines, Portugal! 

The episode takes an overview of the industry in terms of cooling technologies, and how they are adopted and transformed into the industry. From regulation challenges, and energy efficiency and driving down PUE’s, Andrew and Omer talk about these challenges and what we can do to address the demanding needs of an ever-changing landscape across the world. Andy also poses the challenges in data centre design, and how the partnership of Schneider Electric and Start Campus is contributing and future-proofing the digital sector.

Transcript

Copy Transcript Copied

SPEAKER_00 0:14

We talk to the people who power the data center data to give you insider info on everything happening in the host, Andy David.

SPEAKER_03 0:26

Welcome to Inside Data Center podcast. I'm recording this episode live at Start Campus in Sinis, Portugal, where Schneider Electric and Start Campus are discussing how they're building the foundations for sustainable AI. In this episode, I'm joined by Omar Wilson from Start Campus and Andrew Bradner from Schneider Electric. Good afternoon both.

SPEAKER_02 0:48

Hello. Good afternoon.

SPEAKER_03 0:49

Thanks for joining us. Before we go into cooling, we're going to discuss around just really the evolution of cooling, but also some of the implementations that are here at Start Campus, which is which is quite amazing, and we've been lucky enough to see today. Do you want to give a quick introduction into who you are and what your role is in your respective organizations? Andrew, start with you.

SPEAKER_02 1:06

Sure. Andrew Bradner. So I lead the cooling business here at Schneider Electric. Been with the company almost 25 years. Been able to enjoy a very diverse career that saw different types of cooling. So about 10 years ago, I started leading our prefabricated data center business. So again, very different types of cooling that's deployed there. And then the last five years have been leading the cooling business based out of Italy.

SPEAKER_01 1:33

And I'm here. I'm Wilson, Chief Marketing Officer at Start Campus, coming up to six months. But also engaged with the team and the project during last year on an advisory basis. So I know the project very well. And previous to that, I've uh been in the industry almost 15 years. I was uh with a large global operator before that in Eurofination.

SPEAKER_03 1:55

And one point I took from the presentations this morning was how the longevity of the people like yourselves and others in the industry. So it's good to get that viewpoint from where we've been to this amazing journey that we're we're currently on. But from a cooling perspective, you touched on it in your presentations this morning, Andrew. It's changing so much so quickly, and technology is trying to keep up with the demand. So, what have you seen across your career from the evolution in cooling technologies?

SPEAKER_02 2:22

Yeah, it's it's been interesting, right? Um, it's been dominated by air-based cooling, right? We all we all know that. Um, but liquid cooling, which is all the rave now, has been around for a long time. It was used in mainframes back in the 80s, right? So nothing new. And uh Omar, you spent time in in Asia, right? So a lot of people used liquid cooling there 10, 15 years ago really to drive efficiency, low densities, high efficiency. Um, what I would say is changed is really densities from the compute standpoint, right? That's what's driving changes of what type of technology you're adopting. It's not necessarily new, because I said it's been around. Uh, but you know, 20 years ago, the densities were relatively low. The data centers were relatively small, right? A 10 megawatt data center was big. It was like, wow, you're building a 10 megawatt data center. So how you cool it was different, right? And I mentioned, you know, in-row cooling. That was something that was very applicable to enterprise data centers, small and medium data centers, as Colo started. It was great for retail co-location. But when you started talking about higher capacities of data centers, higher densities, you started to look at solutions that did things differently. Regulations drove some of that, right? Some of it was energy efficiency and driving down PUEs. So there was a time that economization was all the rave, right? Direct air economization, indirect air economization, a lot of adiabatic, which you're trying to avoid here at Stark Campus, right? Any water usage. And that also became a challenge. So, how do you eliminate some of that water usage, right? Driving maybe systems that are closed loop chilled water systems, using facility water systems that aren't wasting water, right? So it's been an evolution in larger capacities. Now it's not just larger capacities, but higher density. So how do you do that? Well, liquid or water is a better cooling medium than air, right? It dissipates heat better. So really it's bringing that to be able to keep up with those IT roadmaps. And that's really the evolution I've seen. It's been different types of air, addressing different needs that the data center industry had, right? First, bigger capacity uh campuses, but now it's that density at the rack level that's pivoting what type of cooling is getting deployed.

SPEAKER_03 4:46

Yeah, and I think Omar, when you talk about capacity and scale, we're here at Start Campus, significant scale on the roadmap, significant capacity. So from a cooling perspective, you've got a very unique cooling technology, but how is that going to be applied as you scale? And and also your view on how the industry has evolved over your career.

SPEAKER_01 5:07

Yeah, it's um I'd I'd uh second everything, everything mentioned. You know, when I started back in 2011, we were opening 10, 15 megawatt data centers, and it was vast for us. You know, 30 was unheard of, you know, anything above 30 um was called a campus, you know, rather than a data center. Uh so now we're talking gigawatt scale um sites. Uh so that's that's changed a lot. I think from a from a calling perspective, um, what's you know, from the density perspective, what's happening is um the infrastructure is having to keep up. So that's why, again, start campus and other newly developing data centers, both from the large existing providers and also new providers that are that are uh appearing. Uh the benefits is when you're that's why we talk about AI already, is when you're ground up, you can have that flexibility uh because space is becoming more and more uh premium. Uh it's getting denser in terms of the the the racks and and how much energy is there. The surrounding infrastructure is is growing and getting more complex. So that to be able to fit that economically and efficiently into a day center facility shell and and build model of the campus is becoming pretty critical. Uh, and that's why you know campuses like here on the roadmap, we're we're trying to build for what's coming, what's here now and what could happen next, which is changing rapidly. As you know, we're talking 600 kilowatts, then suddenly we're talking one megawatt racks, and then God knows, in six months or we say, oh, well, actually, one to five megawatt racks. We're gonna, you know, it's it's off the charts. It's gonna, it's gonna balance somewhere. Maybe the limit will be hit in terms of the how how much surrounding infrastructure can be fit, and then it'll stop for a bit. Um, so that part is important, as and as Andrew mentioned here, um, with the the closed loop and the the open loop systems, uh, we're we're pushing, and with the reason for this campus location is a seawater cooling, uh, which means we can um get to a zero water usage uh number uh because we're gonna recycle and transit transmit the seawater to uh cool the liquid cooling uh part of the facility. So that's unique just because of that transition. You know, it was a a coal um fire power plant infrastructure that's decommissioned, and we're able to use those huge pipes and infrastructure for this building. Uh, and that is a story, I think, in the sector, which is gonna be more and more interesting, you know, that transition from old energy to new energy. And AI basically means energy. You know, that's what people sometimes miss. Um you know the industry very much knows it, but maybe the the end user doesn't realize it's all about energy, really, even more than space, I would argue. Uh, because these are the big questions now around how we're gonna fuel it. And as Angie said, it's that density and the old technology is coming back around using liquid cooling, because that's the only way uh that this can be sustainably or efficiently managed in terms of the growth.

SPEAKER_02 8:09

But I think there's one interesting thing. We had this conversation before we were doing the presentations. Just you have to visualize what what Omer's talking about when you talk about space and and the infrastructure to support something. Um, you know, we're in a small room here. Think about a room smaller even than this, or space smaller than this, that's one megawatt of compute. And then start to imagine some of the things that you saw, the switch gear lineups, the UPS on either side, if you have two N or M plus one, at least two in the system, potentially a chiller plant or something else that's 45 feet long on either side, a CDU on either side, and then you saw all that distribution that Omer presented on top or around this one rack, right? That maybe it's a double wide rack now at a megawatt. It's massive what that full infrastructure looks like, both on the cooling and the power side. And I would probably argue maybe more challenges on the power side than the cooling side when you get to those densities, right? And what that means for distributing that type of power.

SPEAKER_03 9:17

And I think it's one point on the demystification of this sector, which always comes up in my conversations around when the rack density increases so much, you start to get those questions around, well, why do we need a facility so big? Because they just see the rack as one megawatt there, rather than like we we've just talked about where a data center used to be one megawatt. It's it's everything, it's all the infrastructure that goes with it, which is required in the space as well, which sometimes is is misunderstood, I think.

SPEAKER_02 9:43

Absolutely.

SPEAKER_03 9:44

And from calling, you you talked about it in your presentations about being agile and having the ability to be flexible depending on the customer and the rack size. So, how how does Schneider Electric manage that challenge when designing Disc to Center?

SPEAKER_02 9:59

Well, a lot of that is working with our partners and our customers, really trying to understand the compute. Because ultimately, all this is determined by the compute. When you talk about how you design something, whether it's the water temperatures, the compute is going to dictate that. Water flow, the compute is going to dictate that. So right now, the industry has by and large designed a little bit more generically. Sometimes you have a tenant that you know what their specifications are, but we don't really know what the compute will be at this point. And that's really where the challenge is. So the flexibility in our offering, right, in terms of what we can do, the water temperatures that we can support, the flow rates that we can support, to be a little bit more adaptive to what compute comes out. Again, the availability of the silicon is driving the timing. I think in the tour we talked about, like how quickly do you have to react to fit out a white space? And it could be that you get a call, hey, we got our allocation, we got to go build now, and and you're scrambling to do that. So sometimes it's hard to predict, uh, but we're trying to design systems that are more flexible, being able to have flexibility in the water temperatures, capacities, flow rates. So there is some future proofing or flexibility in what type of compute comes in. It's you know something that we're working closely with the operators around. Again, it's it's an industry, I think, challenge that we're facing right now, especially when you see those silicon roadmaps that are very aggressive and the densities are changing so much to the point that sometimes you're almost looking at saying, my infrastructure has a a life cycle that's going to be more connected to the actual IT lifecycle. So if you start to predict data center design 10, 15 year infrastructure that the IT might dictate a three to five year infrastructure refresh, at least in maybe parts of it, that's an industry challenge, right? So how do we how do we manage that? And really it's working in the ecosystem, the silicon providers, the relationships and partnerships that we have with NVIDIA designing reference designs, working on their next generation designs, working with our co-location partners and actual tenants and hyperscalers in terms of how do we support this, right? How do we design and look to the future? Uh, and it's really going to be the industry needs to consolidate there in terms of how they approach it.

SPEAKER_03 12:26

And from a start campus perspective, like the importance of this relationship, isn't it, in your your own roadmap, because you've got that timeline and that you know aggressive growth that you're looking to achieve over a period of time. And by having partners like Schneider Electric, it with that insight, it allows you to try and develop the data center for the customer that is unknown at any point.

SPEAKER_01 12:46

Yeah, no, I think what um companies like Schneider give you, exactly as you said, I've always liked the analogy of so wholesale, especially day center operators, or almost like the rainforest, right? Roughly. And within the rainforest, you have all the ecosystems and the you know, the animals at the end users living living. But we are removed a few steps, right? Our customers can be, especially when you get to AI scale data centers, we could be three or four levels removed sometimes. You've got the customer that's providing maybe something to their customer, who then gets to the end customer. What Schneider brings us is they're very they also work with the customers, like uh Andrew said. So that uh that experience and knowledge of what's coming down in terms of cooling technology means um, again, as Andrew said, when we're we're looking at you know uh build-outs and how we we uh get space ready for customers, we need to be conscious of what that three to five year refresh looks like as well. Uh, because again, we it's not our equipment. And again, the customer is putting the equipment into the space, but it impacts the surrounding uh space even more now when we're talking about dense high density and liquid cooling, the surrounding space becomes even more critical. It's not just raised floor or white shell anymore. It's really intertwined with the with the racks and the cooling and the infrastructure. So that planning is very important. So when we're we're now looking at the next build, um, as we did with this one, there was a lot of innovative discussions around how do we make that as modular and as um flexible as we can based on what the customers are gonna do when they come in, um, just because we know um it's changing so rapidly. And that each customer is now looking at the next processor upgrade, which maybe maybe hasn't even come yet. So they put orders on a processor that hasn't even been shipped, and they've got to get that into a data center, and they they're thinking about that density already. So uh it's an interesting time.

SPEAKER_03 14:43

Yeah, and that's one of the biggest challenges, isn't it? Um the unknown. This industry is uh there's a lot of unknowns.

SPEAKER_02 14:49

Yeah, I think you've even demonstrated that here at Stark Campus in terms of how this building was designed in the future buildings. And I would probably guess that even by the time you get to SIN three or four, it's gonna evolve again, right? So the ability and the infrastructure, the the system that you have here gives you that flexibility. Yeah.

SPEAKER_03 15:10

And when you look at the future of cooling, I think a point in your presentation earlier about how the technology is there to call for this density. It is, you know, we are able to do it. So from a future perspective, do you see new technologies coming in or do you see evolution of liquid cooling?

SPEAKER_02 15:25

Well, right now I see probably in the next three to five years evolution, right? I don't think it's gonna be revolution. Um, even if you talk about different liquid cooling technologies, and you can talk about single phase, two-phase, positive pressure, negative pressure, immersion, direct chip, those kind of all exist today, right? Um, where you may see revolution is if quantum really becomes mainstream. Right now, I think it's still a very specific use case. Um, the systems design right now, it's basically in lab environments, right? Academia or uh research, right? And, you know, getting to near, you know, zero Kelvin or absolute zero is hard to do. And you can question whether even there's enough helium, which I think is what they're using right now, available to do it at scale. Um, and that's where I see kind of revolution. Um, as I mentioned, you know, 400 kilowatts of rack has been done already. It's been done in the exascale supercomputers uh that uh we're able to support uh with Cray. One megawatt, there are one megawatt CDUs. The flow rates don't seem to be something that escapes what's possible today. And I think you know, the densities and being able to deal with those hotspots right now are probably best dealt with with a direct chip system. Now that may change over time. Again, more evolution than revolution, right? And I think you're you're seeing that in the market. Talking about, you know, the medium, whether it's water, you know, PG25 or it's some sort of dielectric fluid or refrigerant like a two-phase system, I think is going to be partly based on regulation, right? There are PFAS regulations or dielectrics need to kind of it evolve or improve to be able to go wide scale. And then I think the challenge of just the scale of these deployments, right? You time out your next buildings being over 200 megawatts. That was, you know, unfathomable five years ago, right? Or or 10 years ago. But that scale, some of these technologies have only been deployed in proof of concept or smaller capacities. So can you get a one and a half megawatt, two megawatt, three megawatt, five megawatt two-phase system, right? Is that available? What does a plant have to look like? You know, a a builder like Star Campus, how could they use that with the seawater system that drives efficiencies? Could you even match that type of efficiency with that type of system, right? So for me, it's how do these evolve over time and there may be different use cases, maybe distributed AI and these edge applications, two-phase suddenly becomes, yeah, this is great. You can eliminate water from that system, the fear of leaks, um, easy to deploy. We know how to use DX systems in some form, whether it's thermo siphons or or pump refrigerant, right? That that's technology that's been around for a long time that was used for air cooling. Okay, I know how to do this in small scale. At massive scale, I think that's a different question. And those densities may ask for something else.

SPEAKER_03 18:40

And from start campus perspective, again, like what I what I've been really impressed with while I'm here is obviously the facility itself, the location and everything that goes with it, but the partnership, it's a it's quite unique in the industry that two organizations are coming together like this to open the doors, so to speak. It's unique that doors are open, let alone by two organizations in partnership. So, why is it so important to start campus that you have this relationship with Schneider Electric?

SPEAKER_01 19:04

Yeah, I think it's um you know what we're trying to do here uh is pioneering from a few different aspects, right? Uh seawater cooling has been done before in the industry, never at this scale. Um, the only place it was done at this scale, and that's why we're we're reusing from the coal sector was in the energy sector. So when you're doing something new like that, you again need a partner who is also innovative, who's also following customer changes and and and demands. And uh so that requires it's more than just uh uh a vendor supplier relationship, you know, which is very transactional. We're almost collaborating to then look at what the customers and the markets are doing and solving it together, which is the only way it works. And and it needs to be uh long-term in terms of that planning because again, it's changing so rapidly, and that is has impacts on our timelines. And then you come back to the the one of the pivotal issues in the sector at the moment, which is lead times and delivery times. Again, when you've got a global provider um that has options around supply chain, that's very important because we're we're tied to what the customers need. The customers, like I said, are are delivering their product to one level customer, sometimes two level customers. So then there suddenly becomes a speed of urgency, and then we know the time scale we need to build, and then we need equipment when we need it. So that becomes a partnership uh more than just a transaction, release.

SPEAKER_03 20:39

Yeah, definitely. And I think collaboration is how some of so many of the challenges now in this industry are huge. They need collaboration in order to solve them. And you and I think you've touched on it as well, Andrew. You need those test cases really that you because you it's unique what we're happening. This this period of time is has not happened before. So how can we manage it? How can we progress? How can we move forward? You kind of need these partnerships and collaboration in order to to work out what the best technology is to you know understand how to build them, future-proof them, and everything. So it's uh it's great to see the partnership in action. Thank you both for joining. Thanks for opening the doors for me as well, and everybody else that's here. But thank you both for joining me in this discussion.

SPEAKER_02 21:17

Pleasure, great, it's been a pleasure.