This text was generated using AI and might contain mistakes. Found a mistake? Edit at GitHub
Yeah, welcome to another episode of Software-Architektur im Stream, this time with Kelsey Hightower.
Thanks for taking the time.
And before we get started, I would like to say a few words about how this episode actually happened.
So I was at Container Days in Hamburg, and Kelsey did the keynote there.
And he did something that I thought was very, very interesting.
So he just took questions from the audience and answered them.
And that was very interesting and entertaining.
And the keynote, I enjoyed it a lot.
And I thought it would be a perfect fit for this stream as well, because this is what the stream is about, talking and getting answers to some questions.
And also what I think was a great experience was that he took the time to actually answer all the questions after the talk and spent several hours even talking to a group of people.
And I happened to be part of it.
Actually, I had an appointment right after the keynote and I joined later on.
And that was also a great experience.
And it’s a good, it’s one of those things that I think a lot of attendees of conferences don’t do and should do to talk to the speakers and discuss with them.
And also, it’s something that the speakers should do, like you did, to be open to these discussions.
And this is also why we ended up doing this stream.
So thanks a lot.
Thanks a lot for the keynote.
Thanks a lot for taking the time for the questions and thanks again for also being here on the stream and doing this discussion with me.
Before we start, do you want to say a few words about yourself?
Yeah, I have a few words and I’ll just say just a few.
You know, some people may know me from the Kubernetes days, but my career is 25 years in total before I retired in 2023.
I spent time in enterprise, like a lot of people that may be watching this.
I’ve had fun jobs.
I had what would be considered boring jobs.
I’ve done everything from racking servers inside of a data center to, you know, learning how to read COBOL files and translate them into Python.
So happy to be here and, you know, happy to see what these questions will be like today.
Yeah, so I took some inspiration from the questions at the container days in Hamburg.
And maybe a personal question first.
So as you said, you’re closely associated with the Kubernetes technology.
So why did you get started with Kubernetes?
Oh, I think in, you know, when Kubernetes really kind of got popular, at least my witness, like we’re talking 2013, 2014 timeframe.
So at that moment in time, I think looking backwards, it’s important.
I spent a lot of my career doing the DevOps, like the dawn of DevOps, right?
So this is where around 2009, we started thinking about automation outside of just shell scripting, right?
People were using tools like Puppet, Chef, and Ansible.
And we’re all trying to figure out this concept around promise theory, right?
Declarative config, and then you resolve it on the server, and you try to move to some desired state.
And I think we chased that, I would say, at least for four or five solid years before we realized, it just wasn’t the right level of extraction, right?
We were basically using DSLs to make, you know, domain specific languages to automate things, which is an improvement over shell scripting.
But it wasn’t necessarily enough abstraction to get to this world.
So when we get to around 2012, 2013, Docker and containerization is taking everything over in terms of rethinking what an app is and how it’s bundled.
And so at that point, I’m just now leaving Puppet Labs where I was doing software engineering.
I used to use Puppet in production, and I joined the startup that was behind it.
And I just watched the industry just change, right?
We went from thinking about servers and automation to better abstractions.
And so following that Docker journey, as someone that started using Golang, you know, starting to build my own tools in that space, I find myself at CoreOS, where we were trying to basically take some of the white papers around distributed systems.
If you had a collection of servers that were running Docker containers, what are the missing pieces that you need to make it work?
And so we had our own tools like Fleet that was like a, you know, you can think of it something like Kubernetes, it was a scheduler built on systemd.
So they give it a systemd unit file, and it would just try to find the best server that could fit and run those containers for you.
I think we got as far as we could with our level of experience and the tools available.
And then Kubernetes came out.
And so when Kubernetes gets released, I got involved by like, hey, we’re at CoreOS.
And I looked at Kubernetes.
And the reason why I told you about the previous years of experience, it was immediately clear to me that Kubernetes was the thing that someone with my background would build if they had enough time, and they knew how.
And so I started contributing to the project and overall started giving talks and wrote a book about it eventually.
So we are in the Kubernetes journey for quite some time now.
So what’s the main weakness of Kubernetes in your opinion?
I mean, I think it’s really strong in what it’s designed for.
I think this is where, you know, most people love to see one tool and make it do everything.
If you learn Ruby, you want to use Ruby for web apps, back end, and then you may even try to make a compiler that makes it work for mobile apps.
We tend to do that in software engineering.
So Kubernetes is really good at this idea of saying, hey, you give me a lot of machines, and I will aggregate them in almost in many ways that you treat them like a single machine, right?
This is kind of like the beauty of distributed systems.
It’s good at that.
But I think what people want Kubernetes to be is a Swiss army knife for all of the infrastructure.
And so in some ways, Kubernetes gives a type system to infrastructure.
And what I mean by that is if you were trying to build your own platform without something like Kubernetes, you may use Terraform to stand up cloud resources.
You may write some custom tools to integrate like your own load balancer to back end machines.
You start just gluing things together.
But if I were to observe that as an outsider, it would just look like one big collection of scripts and automation tools.
There will be no objects to think about.
There will be no clean schema to reason about that architecture.
And so what Kubernetes brings for a lot of people is a schema to define things like a running application, a storage volume, a load balancer.
And so for people to see that for the first time, we get much closer to the original visions around declarative infrastructure.
We go from infrastructure as code to infrastructure as data.
And I think it’s a more reasonable system.
So then where does it become weak?
You can’t do imperative things in Kubernetes.
I know people try to make it work, but that’s not what it’s good at.
Because Kubernetes model, it’s more of a static data object that you give to the system.
And the imperative nature of it lives inside the control loops.
But from an API perspective, doing something like turn off this firewall rule, that’s a really weird thing to think about in Kubernetes.
Because let’s say you have a firewall rule.
You extend Kubernetes through CRDs.
And I think that’s the reason why people looked at Kubernetes as the Swiss army knife, because, oh, you can extend it.
So let’s build a service mesh on top.
Let’s build a control plane on top.
Let’s build a configuration management system on top.
Let’s build a CI/CD system on top.
And you can get really far.
But when you start to try to do imperative things like roll out this software over the course of four weeks, one zone at a time, and then observe signals to tell you when to either slow down or to stop, that’s a very imperative workflow.
And this is why you start to see tools like Argo CD, and these various workflow engines that sit on top of Kubernetes, to give you that ability to orchestrate a workflow.
Whereas Kubernetes is really good at orchestrating, in some ways, kind of fixed infrastructure, even if you can out of scale certain types of workloads.
Makes sense.
And I mean, the sort of obvious question after that question is, so what’s the next step after Kubernetes?
What is going to replace Kubernetes eventually?
You know, I think if you think about that question, like what replaces Linux?
What replaces Git?
And so the number one is what’s the value of replacing these?
For most people, if you build something better than Kubernetes, would anyone use it?
Can you get the industry to rally behind it so it also becomes a industry wide standard?
So that’s the hardest part of this equation, is can you get people to agree on something?
And Kubernetes and Linux and Git have gotten lots of people to agree on something.
So that’s the hard part, is can anything else do that?
Assuming that something else could do that, what would you build?
And I think what you’re going to end up building is what we kind of already see.
If you go to like any large hyperscaler like AWS or Microsoft Azure or Google Cloud, what you’ll see is that they have lots of workflow or workload platforms.
Things like serverless, Lambda does not run Kubernetes.
You have things like Dataflow, Dataproc, those are not running Kubernetes necessarily.
And so if you think about the types of workloads people want to run, there’s no guarantee that every workload type needs an entire flexible orchestration platform underneath it.
So in those cases, you will see lots of workload specific platforms emerge.
We’re not done yet.
But as we’re figuring out what those should be, I think Kubernetes will continue to be the experimentation ground.
So if you have a new workload type like we see in AI, right?
AI serving or AI inference.
Well, if you want to figure out how to run that kind of workload, what would you use?
We don’t know yet.
Or we haven’t built something that’s universally accepted by everyone.
So in the short term, you take Kubernetes, you can make some new CRDs to describe what an inference workload is, you package that model in a container, and you extend Kubernetes scheduling abilities to account for things like a GPU or specialized hardware.
That’s where I think Kubernetes still does a good job.
So if you want to replace it, you’ll probably experiment running a new workload for a little while.
And once a really good pattern emerges, that’s when we start jumping into optimized platforms that might say, hey, we don’t need all these layers in Kubernetes.
We don’t need all this flexibility.
So typically, what we do in software engineering is we collapse the layers to build something far more efficient.
And that’s what’s going to come when the time is necessary.
If I recall correctly, there was a similar question in Hamburg during your keynote.
And I think you asked the audience whether anyone is actually at the moment replacing Kubernetes and nobody showed up.
And you said that you were worried about that.
Can you explain that?
Do I remember that correctly?
Yeah, you know, innovation takes a lot of people trying stuff.
And so when DevOps was very popular, it felt like the whole DevOps community was just trying to figure out configuration management, just pick one, Puppet, Chef, Ansible.
And everyone was basically trying the same technique.
And then Docker seemed to come from a whole different wing of the engineering community, the developers.
They’re like, what is this?
What are you guys doing?
You guys are basically on a dead end.
That configuration route was probably considered a dead end.
You were never going to get to where you needed to be on that path.
And then Kubernetes may also have a limitation that people can’t see yet.
And so if everyone thinks in terms of Kubernetes, like some people can’t imagine automation without using Kubernetes.
And I’m like, oh, that’s unfortunate.
This is not the best we could do.
Like, it’s been 10 years now.
Actually, we’re going on like 11, 12.
So I’m hoping that when we hit the 20 year mark, there’s some competition for Kubernetes.
Because if there’s not, that means this is the best we could do was 20 years ago.
And so I’m hoping that people are experimenting with new ways of doing things.
There has to be a better way of doing things than this.
But if no one’s working on that right now, if no one’s learning enough about Kubernetes to realize that there’s a better way, because that’s how Kubernetes was born.
We learned enough from Docker and Docker Swarm and Mesos that we figured there was a better way.
So I’m not saying that Kubernetes shouldn’t be used.
Like, you should totally use it.
It’s really good right now.
It’ll probably be good for another 20 years.
But boy, do I want to see some competition.
Okay, I see.
And there is another aspect.
When you were giving the answer to the original question here in the stream just a few minutes ago, you basically said that it would need quite a few people to agree or a few companies to agree on one standard.
And I have to admit that when Kubernetes was started, and it was done by one of the large companies by Google, and therefore GCP, I was surprised that they managed to convince their competition to also join in and use the very same technology.
So do you think it’s even possible for these kinds of companies that you need to agree on something?
How did that happen with Kubernetes?
So I think people have to be clear.
Most companies like Google, they have much better things to do than think about standardizing Kubernetes.
Like, we’re talking about the company.
They’re making billions of dollars in profit doing ads and other things.
Kubernetes is not something that the leadership, we’re talking CEO level, has time to think about.
And so what tends to happen, and I think in the case of Kubernetes, if you hear about the origin story, Craig McLuckie, and Joe Beda, and Brendan Burns, and Ville, and some people’s names that most people don’t get to learn about, you have people experimenting.
Like, hey, this Docker thing has taken off.
We have a lot of experience with things like Borg, and some of them helped write the white papers that people will go on to read.
They’re experimenting.
They’re trying a new idea to say, hey, what if we were able to bring a better type of orchestration to Kubernetes?
And remember, this similar organization, Google, they also released a container runtime platform called Let Me Contain That For You, written in C++, around the same time Docker came out.
And no one used that.
I don’t even think most people even know what that is.
And so it’s not like Google can make everything work, just to be clear.
But what Kubernetes did well, I think they integrated well.
Day one, you have etcd being used as the backend store, which is basically what other people are doing.
You use Docker as the runtime versus reinventing your own.
And you’re adding the convenience layers that most people didn’t even know they needed.
And so they did a good job day one.
But they also did a good job by bringing people like Red Hat in early, people like Clayton Coleman, who are working on things like OpenShift.
And if you’ve worked on OpenShift, you’ve kind of been playing with these concepts for enterprise customers for a very long time.
People like me just got gravitated to the project because it was open source.
And so I’m joining and eventually become a pretty decent voice in that community.
Me and two other people, not Googlers, we started KubeCon before there was a CNCF.
And so when you think about what kind of happened over time, once the team open sourced a project, and they were very welcoming to people who are not working at Google or working at Red Hat, we all found our way in and contributed to the best of our ability.
And I think the thing that’s probably amazing to people is that Google would allow people to do that part.
And then over time, you start to get a little bit of pressure because now this thing is much bigger than just Google by itself.
It’s now out there.
And then who’s doing the consensus building, not just Google, it’s all the individuals, people like me, people who are doing community management.
It took years.
But once we got to that point, and I think the Linux Foundation did a really good job because the Linux Foundation has so much experience doing similar things for the Linux kernel and other projects, that they brought that same maturity, the same industry building set of techniques, and they brought it to the world of Kubernetes.
And I think that’s the thing that allows it to accelerate.
And there’s also little moments that I think a lot of people don’t realize.
Brendan Burns, one of the co-creators of Kubernetes, co-authors of the book I wrote, Kubernetes Up and Running, he goes to Microsoft early in the lifespan of Kubernetes, like within the first three to four years of Kubernetes.
And now he’s at Microsoft.
Him going to Azure really helps bring in that Kubernetes will be the one.
Remember, they got rid of Docker Swarm and Mesos as container runtimes and said, we’re going to standardize on Kubernetes.
And then it would take several more years before we started getting the likes of Amazon going there.
But before Amazon agrees that this is a thing they should add to their portfolio, we already saw companies like Huawei and smaller cloud providers throughout the world, including Europe, saying, you know what, this is a really great orchestration layer.
Let’s add it.
So at this point, you had so much inertia from the startups, from the customers, that it almost felt inevitable that Amazon would eventually agree.
And they did.
So now you have an industry standard.
So what’s required to do this again is the same level of openness and patience to attract the contributors that may or may not be at your company.
Yeah, I think that’s a great story about outreach and trying to build a community and trying to make sure that everyone is on the same page.
And then it will eventually happen, which also means that for a technology to succeed, it’s not just about the technology, but it’s about how you deal with the people in the community.
There was a question on Mastodon by Rahlhans, I think is their name.
And they ask, why should I care about Kubernetes when developing software that is agnostic to its runtime environment?
And I can sort of relate to that because I could argue that once you have that piece of software, you can just run in a container.
So why would I worry about Kubernetes?
So what’s your take on that?
I mean, I think everyone can forecast the answer.
You shouldn’t have to care.
There’s so many layers to infrastructure, for example, networking.
Most developers don’t really care about networking.
And when I talk about networking, we’re talking about the full process of encapsulation, putting it on the wire, hardware, doing its thing.
Most developers do not think about network infrastructure, router switches, BGP, broadcast protocols, none of that.
Most people just assume that they can make a connection to another application.
And so that’s a good way to think about infrastructure.
When it works, the only thing you should be thinking about is the interface.
What’s my IP bind to a port?
And even in that situation, your libraries are handling a lot of the low level infrastructure things required to even listen to receive traffic.
And so when you extend that to things like deployment, I think this is where things get a little dicey.
Deployments and application lifecycle management is probably the least mature area of our industry, mainly because you can do whatever you want.
You can just take a server and SSH and SCP files there and start it.
It will work.
You can use RPM packaging.
It will work.
You can use a container image or a Lambda with a zip file.
It will work.
And since most of these things will work, you tend to get people doing random things, if anything.
And so I think if you’re a developer, what should you think about?
So let’s say we agree that when it comes to infrastructure, the best contract between the infrastructure and a developer is the interface.
And so then what is the interface of Kubernetes?
Well, it’s roughly the same as the interface to a server.
At the lowest level, you have to make system calls.
You need to bind to a network socket.
And you need things like memory and CPU, right?
Those are still the things that you as a developer needs to know.
What are the requirements of your app at that level?
And then there’s another set of requirements.
Depending on the app you’re building, you probably have some dependencies, like I need a database.
So if we kept it simple, let’s say you have a monolithic application and you need a database.
The question now becomes, how do you express that you need a database?
And this is where we start to get back into interfaces, right?
If you’re doing this configuration management, maybe we just know because you as a developer, you wrote a document.
You might have a help output inside your binary.
So if I run in help and you tell me all your dependencies.
But I think we can have a better way.
So we use metadata.
And the metadata will say things like, I need a config file with these attributes in it.
And I need it mounted in this location.
I need this much memory.
I need this much CPU.
I also plan to listen on port 443.
And I would like my metrics to be pulled from slash metrics.
And if you want to know if my app is running, you can call slash health endpoint.
All of these kind of, you know, configuration data or metadata about the runtime of your app.
Kubernetes allows you to express it in a YAML file or deployment object.
Let’s say you need some storage.
Again, you can go write a doc and hope that the operation team reads it.
Or you can define a section in your manifest that says, I need a volume.
I will prefer SSD with these IOPS.
And I will like that volume mounted here, formatted with ext4, because that’s what my app expects.
And so when you think about all of these kind of dependencies, whether it’s another application or NFS mount.
These have to be expressed somewhere.
So as a developer, I think it is fair to say we need you to express what your app needs, like the low-level instructions of like, in order to use this, you need, like if you wanna use this flashlight, you need two AA batteries, right?
That’s okay.
Now, where I buy the batteries from and what brand I use, that’s implementation detail.
So I don’t think you need to learn how to make a battery in order to build a flashlight.
You should be able to accept the battery and that becomes your interface.
So I think for Kubernetes, the batteries in this analogy are the YAML files.
The thing I think that platform engineers have tripped up on is we tend to expose very low-level interfaces and then we make you learn all of these things where we could abstract that away a little bit.
So instead of the batteries we’re asking you about, we’re asking you to tell us what battery chemistry you want.
Do you want nickel?
And it’s like, no, no, no, that’s way too low and it’s gonna be dangerous if you get the mixture wrong.
And that’s where I think we see a lot of deployment failures.
We’re asking people to mix the battery chemistry and package it when we’re really what we should be doing, like in the case of Kubernetes, we should probably give you a little bit higher level abstraction to say, these are the five things we need to know about your app and its dependencies.
And then we as a team can translate those into the low-level Kubernetes objects or whatever tool we decide to use.
And I think that’s kind of the holy grail.
So then we get back to your original statement or question, you should not have to care about Kubernetes, but the interface between the infrastructure and your app will still be there.
Now it’s up to your team to decide what does that look like?
Is it a raw Kubernetes YAML file or is it a custom data definition that you all decide and agree on that can then be translated into what Kubernetes actually needs to run and deploy your app?
Would you just describe this metadata?
Would you consider the lack of a proper way to express that metadata weakness of Kubernetes?
You didn’t mention it as one of the weaknesses, but I got the impression that there is something left to be desired.
Well, so one thing Kubernetes can do is think for you.
And just like RESTful architectures and HTTP, it doesn’t define what the payload is.
There is a protocol.
So Kubernetes gives you a protocol.
It says, if you want to define the metadata, which you can do with custom resource definition, CRDs, you can host them here.
And then what Kubernetes does for you magically, it has a concept called API machinery that then takes that definition and translate it under the hood into all the REST calls, the logging, the distributed system stuff, the command line tool integrations and interactions.
And then once you have that component, then your control loop, which is something that your operations team should do, can take that definition and do anything like make a database or a composition object, like create a load balancer with SSL certificate with the right firewall rules.
All of that is possible.
So you have the machinery, just like you have the machinery on the internet, right?
You have a protocol for doing things, but you have to decide.
And this is the part where I think people struggle.
This is the things that humans do.
You have to agree on the dialect and what the metadata is, just like spoken languages.
We have to agree on what words and what the words mean.
And if we agree on what the words and what the words means, then we can communicate.
Dev and infrastructure or dev and operations, we don’t have standard communication.
We don’t have the right vocabulary that’s shared.
We only have low level stuff that bubbles up way too high.
So if we were thinking about this in practical terms, so if you’re a software engineer, you want to make this better.
The exercise you can do with someone that is over the platform, you sit together on the whiteboard and you say, okay, let’s not talk about Kubernetes.
Let’s not talk about Docker files.
Let’s just draw out the metadata required for me to give to you that you can then run my app without me being around.
And you might realize that you’re going to probably draw out, let’s say five to seven fields.
And you can just take those fields and you can drop them in a JSON file or a YAML file.
It doesn’t matter.
But once you have that metadata, then what you’ve effectively done is you define a contract.
And so just like any cloud provider or any service with a good API, you now have an API for getting things done.
Will it be perfect?
No.
And when you find things that are missing, you may add additional fields and document what they mean.
So then as a developer, whether the team is using Kubernetes or not, won’t matter.
You just know when I write my app, there are going to be some sane defaults and I can specify the things I need that honor that contract and just give it to the system.
And then if it turns it into Kubernetes objects and runs it, who cares?
One thing that I think you mentioned at container days was that you did some prototype.
I’m not sure where you would use the Linux F binaries and then figure out metadata from that and have Kubernetes configured from that.
That seems to be a solution in this regard.
So how far can you take that?
I mean, how much information is there really in an elf binary to get that done?
Yes, I want to be very patient with this because I think this is where maybe in the infrastructure world, maybe we’re lacking a little imagination.
Okay.
To run an application, we’ve been doing roughly the same process for over 30 years.
You have to target a machine, ARM, Intel.
It is what it is.
That’s not changing right now.
And in order to run an application, it has to talk to the kernel.
That’s what it does.
I want to open a file, that’s a system call.
You want to bind to a network, that’s also a system call.
And so we’ve been doing this same pattern for 30 years.
The app is the truth.
Everything else is guessing and we hope that we’re giving the right thing, but the app is the truth.
The problem is enough binaries, just enough to start an application, right?
Hey, load this part into memory, allow you to make some system calls and be compatible with your kernel slash CPU architecture.
That’s what you can do.
But then if you take it a step further, there are some software things you can do to give more information.
Like if you were using protocol buffers, you would then have definitions of your inputs and outputs that are very rigid.
And then you can machine generate the client and the server side.
So that’s a lot of semantic information that is included outside of the ELF binary.
But just say you don’t have that yet, but let’s just say you’re using protocol buffers.
So let’s say you had the reverse engineer.
You’re stuck on an island in order to get off, you have to run an application, but there are no instructions and there’s no developers.
What would you do?
You would take the binary and you would try to start it forward slash binary name.
And if it were to crash, you would say, why is this crashing?
If you wanna know how the machine can help you, you would just do something like strace, start the binary.
Strace will show you things like, this thing is reaching for a file, Etsy app name, config.json.
Like, wow, this thing is looking for a config file.
Let me just put one there that’s empty and see what happens.
And let’s say you get lucky, you try again and it does same defaults and it keeps going.
And so you can find out it’s depending, oh, it’s trying to bind to port 50.
I don’t know why it’s doing that, but I can’t change this binary right now.
I’m just gonna give it access to bind to port 50 on and on until the application starts.
And so once the application starts, you can try to reverse engineer and send random traffic at it.
And maybe the developers were very nice.
There’s like a JSON schema endpoint.
So you can see all the routes and inputs and outputs like, oh, wow, this is amazing.
They have some standard stuff here.
Oh, I can see the API.
There’s API doc built right in.
So let me just do a curl call to some of these endpoints to see what happens.
And then what you will notice is that the app will start making system calls.
So you might call endpoint call get users.
And you notice that this thing is starting to load things from this, like it has an on file database where it’s looking for JSON file users.
So that’s just like reverse engineering to see how to make an app start.
But you can actually embed more things like, I don’t know, a help flag, forward slash binary dash dash help, dash dash dependencies.
And the dependencies could tell you, I need a database with this schema.
You could encode that in the binary.
And that way the binary would have all this information.
So think about now what will go forward.
What I started, I did a small talk at a conference in London.
And I used a Go binary to demonstrate this.
I actually wrote a prototype to show people.
And so if you took my application, it’s just like a RESTful application, very simple.
And you said, forward slash dash dash deploy.
What it would do is look for kubectl and if they can find it, it would generate a manifest for itself and then apply it to Kubernetes and copy its binary into an empty container on the file.
And it will run.
And then it will map its logs back from Kubernetes and just tell them to send it out.
So if you just say forward slash start, little did you know, it was running remotely on Kubernetes.
This binary on your laptop has copied itself to a cluster and streaming the logs back.
And when you hit control C, I will capture the interrupt and then I would delete the deployment from Kubernetes and then shut down.
And so from that binary, I can encode lots of things.
I can do things to say, if you do forward slash binary dash dash Kubernetes, I just spit the manifest out.
If you say dash dash Docker swarm, I would also spit the manifest out.
So these are things we could easily do to co-locate the metadata versus having that be a continual reverse engineering exercise.
And we’ve been reverse engineering for the last 20 years doing DevOps stuff.
Hey, what Oracle database do you use?
Does this username and password work?
It’s like, why is there no runtime validation on the deployment?
So you can say, hey, as I’m starting up, do I have enough memory?
Do I have enough database connections?
Do I have enough this?
If I don’t, guess what you can do as a developer?
Here are the things I’m missing.
I have these five, but I’m missing my Redis cache.
And I need a Redis cache that is at least version 3.15 or better.
You can do all of that just from one binary.
And then the world looks really different at that point because in many ways app could in theory run itself in terms of at least the initial deployment.
And so going back to where my original thing is, we already have that when you think about what a binary does on a single server.
In that elf binary, there’s enough information for it to start, load, and go into memory.
And now we have things like eBPF where you can actually see the network traffic and some of the payloads depending on how everything is implemented.
So we’re there already, but you have to have the imagination that you can codify this information and establish a contract and then the system around it can do some really nice things.
I think that’s a pretty elegant idea, right?
I mean, it’s not really that complex, but it’s a very smart way of putting in the information into the binary itself.
And it makes a lot of stuff a lot easier.
So I really liked that idea.
And I want to change the subject to something entirely different if that’s fine for you.
So, and that is the zero token architecture.
So you talked about the zero token architecture.
What is that actually?
As someone who’s been on the sidelines for the last two years, I will not claim to be an AI expert or a Gentic AI expert by no means.
But I do advise a lot of companies that are still building products.
So I have to get into the nitty gritty from time to time.
And sometimes I will like download one of these things just to see what it does.
And so when I think about how people are burning tokens, so the concept surrounds itself around tokens.
And so what do tokens do?
You can give a prompt, I call them queries, and then that’s gonna cost you, right?
So if you send, I don’t know, five characters that may represent or map to some amount of tokens.
And then there’s work that needs to get done and that’s gonna cost tokens.
And the response also costs tokens.
And so let’s say we were able to say, here’s a task that costs $1 worth of tokens.
So now we can put a price on it.
And I think that’s kind of interesting to be able to put a price that is not just CPU and memory sitting on the server.
Cause that’s how we used to do it, right?
We would just over provision a server and say this server costs 30 bucks a month to run.
And then we pack with as much things as possible, but we’re not quite sure how much does the kernel take up of those resources, right?
Someone asks you, of that $30, how much of it is the Linux kernel executing system cost?
Most people can’t tell you, right?
Is it $1 or 50 cents?
We don’t know.
But the token economics does a pretty decent job of saying, if you wanna do this task, this is how much it costs in tokens to do it.
Okay, so let’s take that idea.
The next thing we get to is if you run really far with this agentic approach, let’s just use something very simple, like automating a task.
And let’s say the task is about getting, I don’t know, the current stock price.
And if it’s within 10% of what their last stock price was, you wanna buy the stock.
Let’s just keep it very simple.
And so if you were to do this with an agentic loop, that means you’re gonna do a lot of inference or you’re gonna get charged for a lot of inference.
So one step would be evaluating your prompt that you would like to check the stock price.
And if it’s less than 10%, you should buy it.
And that’s a very nice prompt, right?
It’s like a single sentence.
It feels really powerful to have something so smart that it could process that.
But now I’ve gotta infer what you mean, pretty much on every part of the loop or there’s optimizations you can do, but I gotta figure out what you mean.
Then I gotta figure out what tools should I call to allow me to fetch the stock price because you didn’t tell me what tool to use.
You just told me what to do.
So now I gotta burn tokens to decide which tools that I have access to via MCP or built-in, who knows, but I gotta figure that out every time.
Every part of the loop, I gotta go evaluate that.
Again, I could cash my last run, I can cash some of these things, but I’m still probably gonna charge you because I may have to infer what to do.
And then once I do that, I call a tool and data comes back, then I have to process the data.
And then I may have to figure out the next tool, et cetera.
We can go on and on, but either way, there’s a lot of tokens being burnt in a loop to do this task.
Now, once I know how to do the task, why am I inferring anything?
Like when a human writes this script for the first time, let’s say we were to do this in Python, we will be burning a lot of mental tokens to make this work, you know, trial and error, what tool is the best one and evaluate the API, then just do the right thing.
But once you’re done, why would you do that again?
You would save the commands into a script and say, hey, now I want you to specifically go to this endpoint, get this stock price in this format, parse this thing out and call this tool exactly and don’t look for another one.
And so now it’s gonna be very clean, predictable, deterministic and cheap.
We can then take that Python script and run it on a server.
And at that moment, you’re no longer burning tokens to fill that task.
And so when I say zero token architecture, I want people to think about using agentic workflows where it makes sense.
But in cases where it can be predictable, then stop, don’t burn any more tokens to complete the task.
And I’ll say one more thing here.
So when is that not possible?
I think that’s like the other part of this discussion.
As engineers, we gotta know when can you not do something?
I think there’s situations where an LLM reminds me of the first time I use a regular expression.
And it’s like, you wanna parse an email address.
Now, you wanna parse a specific email address, that’s easy, right?
You can just put that in a string and pattern match.
If it’s this email address, or you only care about Gmail domains, right?
You might just say at Gmail, I don’t care what’s in front of it.
You’re gonna probably be wrong because it may not be a valid email address, but you got it.
But a regular expression is like that propagation and this ability to avoid the if statements.
And that regular expression is almost a state machine of its own.
And it can make up for a lot of code because you have this additional embeddable runtime.
And so regular expressions can clean things up, maybe be more precise than if statements.
So when I think about an LLM, there are a lot of tasks like unstructured data.
Someone’s gonna upload a PDF in any random format.
And the data you need is either on page four or on page 50.
And it could be embedded in a JPEG.
When you get to that, that’s a point where you start to say, hey, is there another primitive that can do a better job of me trying to parse pixels or invent image classification on my own?
Like, we don’t wanna do that.
So an LLM for some people has become a big Swiss army knife for pulling things out of unstructured data and then mapping them with some level of precision to the data that you want.
And in that case, I would say you have a hybrid situation where you may have to keep the LLM on the edge as one of the gates that allows you to bring in this fuzzy data.
But then once you do that and you turn it into an adjacent object, then downstream, you should not be burning any more tokens.
So then that particular agent at that point is no longer an agent.
It’s just more of a preprocessor.
And the preprocessor just tries to get you into the most efficient pathway possible.
So that’s what I’m thinking about when I say zero token architecture, we have to just zoom out and just think about the entire thing we’re building.
And I think people are taking the lazy approach of saying, just let the agent do everything and who cares what it costs?
And also, I mean, that is something that you mentioned.
It’s deterministic then, and it’s not probabilistic.
So that also adds some security to it and you can be more sure about the results.
So that’s besides the cost, it’s probably also better in that regard.
So we already discussed operations and Kubernetes.
So what do you think will be the impact of AI on these fields?
I may have a simplistic view of it all, but when I look at AI, I see it as almost like a surface level technology because the thing behind it hasn’t changed.
And I think I have to offer clarification here.
When I think about the old school system administration I used to do, you had a server, has a POSIX like interface.
And all of the tools that we built over the years were this Bash and Corn Shell, Puppet Chef, Ansible, Docker, Kubernetes.
The thing at the bottom is roughly the same.
And so when I look at AI, is AI changing the CPU architecture?
No.
Is it changing the assembly?
No.
You still have the malloc for memory and you can still get buffer overruns.
This is the same stack underneath.
So on the surface, what does AI allow us to do?
And I think what it does, it allows us to use this kind of modeling probabilistic computing technique to pull the strings on existing things.
And it feels like magic, right?
So a plane that can fly itself is still a plane.
It’s the same laws of physics apply.
And now you’re giving AI the ability to drive the plane, but you still need a plane.
And so when it comes to infrastructure, you still need the kernel, you still need the storage, and the same laws of computing physics apply.
And so the AI can only pull the strings that you have at the bottom.
And so until the things at the bottom change, I don’t think of AI as this like… thing you could never imagine, because we’ve trained it.
Like, we, people who have done things, trained it.
And so I think, what does it look like in practicality?
Right now, our apps don’t do very much for the entire lifecycle.
So, garbage collectors do some magic, right?
Like, you can allocate memory, and then it cleans it up using techniques like Mark and Sweep.
Like, this is amazing!
Like, this is cool!
And then it feels like we’ve automated away memory management in a lot of use cases for some particular runtimes.
But we don’t have that kind of situation for a lot of other concerns.
And so what apps have to do now is they just print logs or just crash.
And then we have to jump in, and we’re like, okay, if you see this log message, then just do this.
All of those scenarios can be modeled.
When this happens, then do this.
And so if you like to think of it in magical terms, you will say it’s like, oh, software can become self-healing.
But it’s like, that’s not quite true.
Like, if you want to make it magic, you can say it that way.
But the truth is, we have enough apps across all the companies across the world that you have enough data that you can use statistical analysis to determine what should happen when this error message shows.
And the reason why I know this is true is because that’s how documentation is written.
When people have the same problem, we can now put documents to say, if you see this, then do this.
All of these become either best practices, and now we finally have a tool.
Instead of writing a bunch of if statements, we can now train a model to say, watch these log messages.
And I’m going to now give you access to the steering wheel and the pedals.
And when you see this error, here’s the action you can take because this is the action we’ve been taking for the last 30 years.
Yeah, all software will then feel like something we’ve never seen before.
But if you’re a fundamentalist, then you understand that a self-driving plane is still a plane.
And so I don’t want to devalue the benefits of AI.
I just want to make sure people understand that the fundamentals don’t yet shift.
And this can create some dangers because any time a technology enables speed, you get different behavior.
Like if it takes nine months to sail from the US to Europe, there is no concept of a vacation.
If it takes you nine months to get there, you live there now.
But if you can get there in nine hours, then you can have the concept of a vacation.
And that would change society where people can do a three-day vacation in France because it doesn’t take nine months to get to France.
So a whole industry will spawn from this.
So AI might do the same thing because when you get speed, especially responsive speed based on models, yeah, I expect a whole new set of things and industries to sprout out around that.
So that’s the way I think about how it changes the future, and I only have this level of confidence because I look at just at what layer it sits.
As far as I can see, and I could be wrong, it sits on the surface driving the things below it.
I mean the way that you shaped the answer is pretty much I think about software architecture.
You were talking about the place of AI in a software architecture.
One of the things that I learned at one point is that I’m probably not that much or that good a Linux admin that I should really work as a Linux admin.
And I sort of learned that the hard way in a way.
With AI around, I would be more courageous to try to do it because there is this thing that I can ask for advice specific to my case.
And I’m probably smart enough to pick out what comes out of it works or doesn’t work.
So in a way, I would be, well, brave enough to try to actually do like proper admin work or ops work.
So that means that me as a person who’s not really that qualified in that regard thinks that they can do the job, which on the other side means that those people who actually can do the job are probably not that important anymore.
Would you agree or am I missing something?
So it’s a devaluation of a skill set, I would argue.
And that’s not, I mean, you didn’t cover that at all because you were talking about the software architecture part of things.
So I would like to hear your opinion about this part of the equation.
Yeah, man, I think we don’t want people doing work that they don’t have to do.
Right, that’s just for all the society.
And I do think a lot of work we see in our industry is because it’s necessary sometimes to poor design, like a poorly designed app needs people to be on call and fighting fires way too much.
The user interfaces we have, again, they don’t really deal with lifecycle management.
And so you’re right when there’s an error, you have to jump in and do something.
But even without a guy, let’s just say that the operating system were to evolve where it starts fixing its own errors.
Then you would have less people logging into servers and fixing errors.
We saw this with the cloud.
If you use a serverless platform, they don’t bubble those errors up to you.
You just don’t deal with them.
And then so what happens in that equation?
Well, I hope people allow those humans to do something more important than learning how to deploy an app.
We shouldn’t be doing this for another 30 years.
And if you think about all the other things that a company could be doing, but they’re not, I’m hoping people start to do those things.
So if you think about that space where every team has a backlog.
Every team has had to say no to features due to lack of people being able to even imagine how those features would come to life.
Because they’re still trying to write code for the previous set of ask.
If you streamline all of that, what would be left?
What would that person then be doing?
My guess is we’re going to have another generation of people fixing the underlying thing.
So I think AI might give you cover to stop fighting fires with the thing that isn’t so great.
But that doesn’t mean no one should work on making the thing underneath AI actually great.
So what does a self-healing database look like whether you’re using AI or not?
That’s a whole different thing, right?
Instead of having cloud try to learn how to operate everything, cloud shouldn’t be able to build or manage a database better than me who created the database.
Because I can now introduce these hooks and self-healing capabilities using my own models built into the database.
So now my inference cost is low because I’m not using the LLM to do it.
I’m using a very small domain-specific model that is designed hand-in-hand.
I imagine, if we’re willing, the majority of us will move into that realm of giving AI a better plane to fly.
So I don’t know if it’s going to be as fast because if we accept the current infrastructure and runtimes and then just let AI brute force its way through a bad architecture and design, that’s not good.
So I got a feeling, and I guess I’m maybe being hopeful here, is that the majority of us that have been doing this stuff will evolve to building the underlying layer that is going to be much easier for AI to drive.
So one follow-up question.
I mean, you’re saying that that solution would be better.
And I mean, as a technical person, I would totally agree and to invest in that foundation.
However, I mean, we are in an economic situation and we are living in that society where it’s about the most profitable way and the most efficient way to do things is the one that is actually taken.
So in a way, that will only work out if this is the more efficient way.
So do you think that this will be the more efficient way?
Because if it’s not, then we will have that messy foundation that is somehow sugar-coated by AI and we won’t have proper engineering.
And that would, of course, be a very frustrating place to be in and not a good one, as you mentioned.
So competition has to show up.
So there’s a couple of things where efficiency isn’t paramount, like sovereignty.
We all could agree on one monetary policy and all adhere to it, but we don’t for sovereignty reasons.
We all could just buy our cars from GM.
No BMW, no Mercedes.
That doesn’t make sense, mainly because those two other brands have their own vision and style.
They’re trying to service 9 billion people in the world, and I just don’t think everyone’s going to like the same thing.
So that means we have lots of car companies.
There’s going to be more car companies than there are now in the future.
Why?
Because I think at some point, every country will manufacture their own cars to some degree.
I think this is almost inevitable.
You see Mexico is launching their own car.
And so now that all of these countries are now doing things they could never do before.
So we have the next 50 years for various countries to learn how to build their own airplanes, their own cars, improve the energy infrastructure.
We got a lot of time before we get that to be roughly equalized.
But while that’s happening, all the other countries are going to be trying to build better versions of the thing that they have.
This is not an inefficiency story.
This is more of a capability story where everyone wants to be able to also do this and iterate on their own, have their own sovereignty.
And we saw this play out for the last 30 years in corporate America.
Why is every company writing roughly the same apps over and over again, implementing OAuth?
And every company is building their own version of some author’s login portal, like login portal.
Why are we all not just using a SAS provider?
But there’s a reason for that because you may have some unique requirements, or you may not want to adhere to industry standards because you believe you can push things forward.
I don’t know if that goes away immediately.
So I do think there’s going to be a world where we’re going to suspend efficiency as the ultimate requirement, mainly for sovereignty reasons.
And I think that’s going to be good because it’s going to give people in lots of other places the chance for the first time to come in and contribute new ideas to the field.
And we’ll continue to borrow ideas from each other.
I think that’s the thing that may carry us for the next 30 plus years of people learning how to do the things that other people can do so they don’t have to be dependent on those other people.
I think that’s step one.
And then, of course, if you, through competition, make an operating system that’s self-healing and you decide to open source it, then, yeah, it might end up becoming industry standard.
But you know what I know.
It takes them 10 years for people to leave the thing that they’re currently doing to even adopt that.
So I don’t know.
I just think humans tend to continue to innovate.
And when we stop doing that, when you tell me that that’s all done, I’m hoping at that point, like Star Trek, food is free, replicators are available, and everyone is 3D printing everything that they want anyway.
So now we have to rethink what work means.
Yeah.
Which, I mean, there is one question that I still want to ask, and it’s also, I think, related to what we just discussed.
So is there still a point in trying to become a good programmer?
I mean, with the current situation, it’s along those lines where AI would take over everything.
So why would I bother doing stuff like coding even?
Yeah, I think for the majority of people, the answer is absolutely no.
There is no benefit to becoming a good programmer.
Zero.
Now, can a person that doesn’t know how to program or isn’t good at it, can that person create a new programming paradigm or a new programming language?
I don’t think so.
I think a person that has no idea what they’re doing, hasn’t studied the prior art, isn’t a student of what this means, the relationship between software and machines.
I don’t know if that person will ever be capable of moving the industry forward.
I don’t think they will be.
I also don’t think that that person can fix the errors made by the automation.
And so when I think about that equation, the nice thing about that is people who had no software development skills ever, and it was never on their radar, like a lawyer or someone else in a different profession.
They finally can start building the basic apps that don’t really or shouldn’t really require a great – it shouldn’t require an awesome programmer to build a web app.
Come on.
We should be beyond that for that.
And there’s a lot of things that it will really be nice to just have a web app for your practice.
Sure, we should need the very best engineers to do that.
So now we have enabled a lot of people to now do that for themselves.
This is wonderful.
It almost reminds me of 3D printing.
We’ve enabled a lot of people to do micromanufacturing from home.
This is amazing.
And then when I think the next group of people, what do they do if you’re a good programmer?
If I was a really, really good programmer, I’m going to try to figure out what value do I add to any company by being a good programmer.
I might be able to say, listen, you can use AI to generate all the code, and what I’m going to do is make sure that it can continue to generate great code.
So, for example, as new instruction sets come out, maybe you’re the one that maintains the skills and files to make sure that it prefers AVX 512 instead of less efficient CPU instruction.
And I think those are still going to be things that are being available or a very small few of us, like we see in chip manufacturing.
So I think there’s going to be a world – we haven’t even got to testing, formal verification, like so many things we have not got to with software engineering yet.
And so I think if you’re going to be a really good programmer, boy, does that formal verification space need things.
SQL, is that the best we can do for manipulating, interacting with data?
Maybe it’s time to rethink the thing from the 70s.
So I think if you’re a good programmer, you’re going to have to go downstream to program things that then the API or the AI can control.
So, yeah, I don’t think any good programmer wants to be pushing around JavaScript and HTML for the rest of their career.
So there is a comment from YouTube by – I’m not sure how to say that – Alex K.
VH9WN.
And they say, in a world where capabilities was the motivation to build Kubernetes from the scripting class, AI brings us the capabilities to address the thing we had no time to add.
The thing I think about – so let’s just – I’m going to give AI a lot of credit that it doesn’t deserve right now in this moment.
Let’s say AI could do anything, or I just say it can generate all software with no bugs perfectly.
And so what would happen in that scenario would be if you’re an insurance company, how many features are you going to add?
At some point, you’re going to plateau.
That’s it.
You can’t – unless that insurance company evolves to become an e-commerce company and starts selling stuff too, there’s an evolutionary limit to engineering.
Once you build a bridge in a city that only needs one bridge, I don’t care how fast bridges can be built.
You’re never going to add ten bridges just because you can.
You’re still going to limit yourself to what you need.
I think this is the part that’s getting lost.
Engineering is about constraints.
It’s not about doing everything.
Engineering is the function of doing what you can based on constraints.
There’s only so much land.
There’s only so much attention span.
And so while you could do anything, you should only do the things that are worth doing.
I would agree with that comment to a degree.
There are things that are worth doing that people don’t have the skills to do or the time to do.
That’s right.
You look at an enterprise backlog, the fact that you still don’t have Active Directory integration.
Come on.
That is a solved problem, but you don’t have time to do it.
Great.
Maybe these tools can help.
But I promise you, ten years from now, you’re going to get to a point where you realize we don’t need unnecessary software.
That will be the new challenge.
Why do we have all this unnecessary software?
Companies like Apple have to constrain themselves.
They say, hey, we can make every device we want, but we don’t.
I remember they were doing like a it was like some interview that they did.
Every Apple product that they make, all of them fits on one table.
Think about that.
That means that they’ve made a conscious decision not to make a TV, not to make a car, not to make a flashlight.
Apple could make all of those things, but they choose not to.
So I think software engineers will also realize your goal is to try to build the perfect interface that allows people to manipulate, store, and retrieve data.
That’s what this field is, data and algorithms, and do nothing more.
No bugs, no security vulnerabilities, no confusion on how to use it.
That’s what you’re going to do as a software engineer.
And so I don’t know.
I think this is going to be a very wonderful thing to discuss 10 years from now is when it’s easy to create these things like fashion.
It’s easy to create a T-shirt.
Who should be making T-shirts, everybody or just the people who can sell them?
So we are even a little bit over time.
So and I was trying to make up my mind what the final question would be.
So and I will leave that decision to you.
So it’s your choice to either answer the question, what are you optimistic about in our industry or what worries you?
So which one do you want to answer and what’s your answer?
I’ll go with optimistic.
I’ll go with optimistic because I tend to talk about what I worry about a lot.
But what I’m optimistic about is a lot of people think about technology when it falls into the wrong hands.
I’m optimistic about powerful technology falling into the right hands, and we’ve seen this in open source.
I’ve traveled the world, and you can see what happens when the technology falls into the right hands.
People want to build a better life for themselves.
They want to build a company.
They want to bring some capability to their company or their country, and they can finally do it now because it’s not as cost prohibitive as before.
And so now we have an opportunity to model some very hard problems and allow people to have these tools to solve problems for themselves.
So that means if a person – so so far we’ve been sharing ideas via software that you have to download and run yourself.
Soon people will share ideas that model how they would do it if they were working on the problem, and you will now be able to leverage that.
It could be robotics.
It could be curing diseases.
There is so much upside to this.
Just think about it.
While maybe some of the megacorporations are not interested in figuring out how to feed everyone for free, there is someone that’s going to be very interested in vertical farming that will be able to build a vertical farm with very cheap parts using all of these tools to make sure that they have a high yield and decide to give away the food.
Even if it’s just to their local community.
That is an amazing thing.
You combine that with what we see in energy, the ability that people have realized that you can harvest the sun and turn it into local energy, and you combine it with all of these tools.
Boy, can a person solve problems that were unimaginable just 25 years ago.
So I’m very optimistic that there is a chance if humans decide that you can also do good with this stuff versus chasing harm.
Okay.
Thanks a lot.
Thanks a lot for the optimistic outlook and for answering all the other questions.
And thanks a lot again for Container Days and also to Kubernetes for inviting me to Container Days and making it possible that I went there.
And yeah, have a great weekend.
Thanks again and talk to you soon.
Awesome.
Thanks for having me.