Transcript: America Can’t Build Without Autonomous Construction Equipment

Executive Summary

Kevin Peterson, co-founder & CTO of Bedrock Robotics, joins the show to explain why he transitioned from autonomous vehicles at Waymo to construction robotics. He details how Bedrock is leveraging advanced AI, machine learning, and LLM-like models to automate heavy machinery, addressing critical labor shortages and enhancing safety on the worksite. 

The conversation explores their unique data collection methods, the future of a fully autonomous construction site, and the creation of new tech-focused jobs in the industry.

Key Topics & Timestamps

[00:00] From Waymo to Bedrock: The business case for construction

Kevin Peterson explains that construction offers a strong business case where autonomy’s safety and behavior aspects can be more easily decoupled than in on-road trucking.

[01:13] The economic impact of construction labor shortages

With a labor shortage of half a million workers and an average industry age over 50, automation is critical to offset project delays, rising costs, and a rapidly retiring workforce.

[03:37] What equipment is Bedrock automating?

Bedrock is initially focused on automating heavy excavation machinery, such as excavators, wheel loaders, and bulldozers.

[04:12] Building the tech stack with machine learning and LLMs

The company’s technology stack uses a combination of imitation and reinforcement learning, similar to building an LLM, to learn and reproduce the behaviors of human operators.

[05:45] How Bedrock’s AI adapts to different environments

Bedrock Robotics AI learns to adapt to different digging environments, such as clay versus dirt, by collecting and analyzing data from human operators working in those specific conditions.

[09:05] Hardware advancements enabling faster development

Recent hardware advancements like lower-cost sensors, better cameras, and standard drive-by-wire controls on new equipment have significantly accelerated Bedrock’s development process.

[12:00] Enhancing productivity, safety, and data insights on site

Beyond automation, the platform enhances job sites by providing detailed productivity data, video records for insurance, and advanced safety features to prevent common collisions.

[17:00] How autonomous machines interact with humans and the environment

Using a 360-degree perception system, the machines can detect, track, and safely react to humans, vehicles, and other moving elements on a dynamic job site.

[20:00] Bedrock’s phased approach to full autonomy

The company is taking a phased approach, starting with narrow, high-volume tasks like mass excavation and gradually expanding the scope of work the autonomous systems can handle.

[24:14] Overcoming the challenge of variability in construction work

The biggest challenge is the day-to-day variability of construction, which requires a robust system for communicating changing plans and new tasks to the machines in real time.

[26:00] Creating new jobs: The “StarCraft” of construction site management

New jobs will be created for operators who manage fleets of autonomous machines from a command center, a role compared to playing the video game StarCraft.

[32:00] The vision for a “Bedrock OS” for construction sites

The long-term vision is a “Bedrock OS,” a comprehensive platform allowing customers to monitor, manage, and optimize all equipped machinery and job sites from a single interface.

[34:35] The massive market opportunity for autonomous excavation

The total addressable market (TAM) for excavation in the U.S. is a massive $120 billion, with further growth potential from projects currently stalled by the lack of available labor.

Watch the Full Episode of The Road to Autonomy

Full Episode Transcript

Grayson Brulte: Kevin, it’s great to have you here on the road to autonomy. I’m a big fan of what you and the team are building at Bedrock. We need more automation in the construction industry. Prior to co-founding Bedrock with your team live by Boris is that you were building autonomous vehicles and autonomous trucks at Waymo. Why go from trucks and cars to autonomous construction equipment?

Kevin Peterson: so we’re looking for a, a, a few characteristics, uh, in, in a new venture. The first is business case that works really, really well. Uh, so at at trucking we, you know, the, the, the way the industry works is there’s people who build the trucks is people who. Drive the trucks and, uh, those companies kind of like hold the assets, um, that’s very similar to construction. So the business case works really well. And then, you know, we spent a lot of time in self-driving and one of the challenges with self-driving is, uh, safety. It’s, uh, huge, huge challenge. So we’re looking for a space where autonomy can impact the work that’s being done and accelerate the economy, uh, but where, uh, safety and behavior aren’t, intimately, intimately tied. Not that we don’t care about safety, we care about safety a ton, but, um, here we can kind of decouple ’em a bit.

Grayson Brulte: a hundred percent right on Accelerate economy. That’s why I founded the Council for Economic Resilience to promote the positive benefits of automation, autonomy for the US economy and for the audience here, I wanna highlight some data from the nationals. Association of Home Builders as it relates to the economics of construction. The National Association of Home Builders estimates that there is a shortage of 500,000. That’s a half a million individual shortage of labor in the construction industry leading to construction projects being delayed on average by two months and driving higher carrying costs, which all $2.7 billion a year. And let’s not forget, we’re still in a high interest rate environment. Did you look at that data when you were looking at opportunities to, to go into construction? Wait a second. There was a growing labor shortage. Carrying costs are high. If we automate that, we can have a really positive economic impact on the industry.

Kevin Peterson: Absolutely. Um, the, there’s some crazy stats. The average agent construction’s over 50 years old, and so people are retiring, uh, very, very quickly . the backfill rate is, uh, pretty low. And, you know, younger, the younger generation coming into this industry would rather be behind a desk, uh, maybe behind a computer screen controlling machines, uh, than in the machines. So that’s gonna be a huge problem for the, for the economy. We gotta build data centers. We gotta build roads, we gotta build, , power and uh, houses. And if we’re gonna do that, we need more labor.

Grayson Brulte: We do then we have a lot of power constraints. I mean that that is a growing problem. If you look at your former home Waymo, I was recently in Austin a few weeks ago. And toward the facility, they’re power constraint. They want to go to add more vehicles, but they can’t get the power there. So that, that’s an issue. And if you see what the president’s doing with his executive orders on ai, we’re gonna need data centers. Do you see this autonomous equipment that you’re building going to build data centers? And when I say build data, I mean leveling the ground excavating and getting it ready for the construction to begin.

Kevin Peterson: Yeah, absolutely. Yeah, uh, data centers are, are right on point. , Anywhere where there’s large foundation, uh, where you need to dig ahead of time. Roads the same way, house the same way. , Big power centers the same way. You dig first and then, and then you build and, , we’re, , providing the autonomy to go do that.

Grayson Brulte: What type of equipment are you looking to automate?

Kevin Peterson: So the technology that we’re building automates, , heavy machinery. Generally we’re starting with excavation, so excavators, wheel loaders, uh, and bulldozers. , And then, uh, you know, over time, we’ll, you know. Probably build some foundation models and things like that that can control lots and lots of machines.

Grayson Brulte: So from a, from a technical standpoint, how are you building the technology stack to enable these vehicles to work fully autonomously?

Kevin Peterson: we built two things. One is a, a safety system. It looks like more traditional, perception system that detects objects, detects the terrain around you, understands what’s going on. Uh, and then, , we are using machine learning techniques to, to learn the behaviors, uh, that, that are used. Um, and that’s a combination of imitation learning and, and reinforcement learning.

Grayson Brulte: So behaviors versus on road. Especially if you look at an 18 wheeler going down the freeway versus a construction equipment operating in a job set, behaviors are different. How are you getting that data set? Are, are you setting up test facilities? Are you, is there available data? How are you getting that data set to, to build those behaviors?

Kevin Peterson: so we, , partner with customers, , pretty deeply, uh, and, uh, put sensors, , on the machines, uh, watch the construction occur and then, um, we can reproduce those behaviors.

Grayson Brulte: Wow. So you’re actually, basically, you wanna use the term, you’re putting ’em out there in a supervised manner before you make them autonomous.

Kevin Peterson: we are not directly supervising autonomy in the same way that Waymo does. Uh, what we do is we just record those, be we record the work that’s being done, so we watch people doing the work, uh, and then we can reproduce those behaviors directly while people are doing, you know, this is just, uh, work that’s being done.

Grayson Brulte: So then you’re increasing the overall efficiency of the construction site.

Kevin Peterson: yeah, that’s right. , So once, once we learn how to do those behaviors, we roll them into, uh, the autonomous system and then we, then we launch them.

Grayson Brulte: Let’s, let’s give you an example. So let’s say in Ohio, for example, your equipment is building a, a data center, an AI data center, and then let’s just say, let’s go to Nevada and you’re building a a master plan community. The behaviors that you learned in Ohio are those transferable to Nevada and vice versa.

Kevin Peterson: yeah, absolutely. Yeah.

Grayson Brulte: Wow. And so what is that? Is that just the algorithms that you, that you’ve written or, or what is it that allows that to come to fruition?

Kevin Peterson: we do a handful of things . the, the algorithms that we’ve developed, uh, automatically learn, uh, the different styles of digging. So let’s say you’ve got. You know, sticky clay in Arkansas and you’ve got powdery dirt in , Phoenix. You want to be able to learn how to dig those, those different materials, , by collecting lots and lots of data on how. How people dig in those different environments. We get, , you know, lifetime’s worth of experience very, very quickly. And then we just, , we use machine learning to, to, pull all of those behaviors in, learn every single one. , And then when we’re working in Phoenix, where we’re working in Arkansas,, we just run those behaviors and, uh, the machine understands the environment that it’s, that it’s in, um, and, uh, and adapts to it.

Grayson Brulte: Is that pressure data? Is that vibra vibration data? ’cause if you sit in, in a tractor, and I’ve sat in a. You, you bounce, and then depending on if you’re going into the sticky mud, you, you might be more heavier versus if you’re going into a desert, it might be lighter. Are you putting sensors on that or how are you gathering all the data? Because right now you have fork structure partners. You’re in California, Arizona, Texas, and Arkansas. Completely different environment. So how are you gathering all that data?

Kevin Peterson: Yeah, so, so we put cameras, uh, a lidar and, and, and IMU on the machine. And then we listen to signals coming back from, uh, you know, pressures and things like that. And we watch how the machine work moves. So, uh, let’s say you’re driving it and you hit a rock and the front end of the excavator picks up. , We see that in the data and we can learn from it.

Grayson Brulte: Now, since you mentioned rock, so the environment that you’re operating in is a lot different than trucks. You have dirt, you have dust, you have mud, you have rain, heavy, heavy, torrential rain, and you have big rocks. How were you hardening the, the bedrock system to handle this, I’ll call it this intense environment?

Kevin Peterson: the old way that you, we used to do things was, uh, kind of hand building. Uh, behaviors and detectors, uh, throughout the stack. So you would like take all the lidar data. Uh, look for draw boxes around cars. , Try to detect the dust, actually try to filter that out, and that’s a very, very hard problem. Uh, you know, self-driving car companies have worked on that for a long time, and part of the reason it took a long time is because they’re hand designing every single one of these pieces. What we do today, it looks a lot more like building an LLM where, uh, we, we have input data, we’ve output data, and we’ve got a big model in between. And that model is optimizing. , Everything that you need to detect and it will learn where the dust is and where it’s not, and where that relates to the behavior, uh, that we’re trying to output. And, um, it’s super, super efficient and super, super powerful. So we just learn it directly.

Grayson Brulte: Without that LLM, let’s go back, let’s hit the rewind button and let’s, I’ll, I’ll go old school and use the VCR rewind button and let’s go back five years. Would you have been able to build. Bedrock as fast as you are today if you had the technology that we had five years ago. Is it all the technological breakthroughs that have happened with LLM today? They’re allowing you to do this.

Kevin Peterson: Yeah, I’d say it’s a, it is the last couple years of development that really, really changed the game. Um, I’ve, I’ve looked at construction for. Many years, did self, did self-driving, uh, mine trucks with Caterpillar, uh, sold the company to Cat to do autonomy in construction. And both of those times we looked at it, thought it was very, very hard. Imitation learning and, and those, these kinds of like, uh, LLM techniques, language, things like that, all, uh, uh, let us do a lot, lot more than we could have. And all that’s come about in the last two years.

Grayson Brulte: So you have the old experience from selling a company to to cat. What have been those big changes since that transaction to where it’s today outside of the LLM breakthroughs?

Kevin Peterson: there’s a ton of tailwinds that are, that are helping us out here. Um, hardware costs are coming way down. Uh, camera quality is way, way up. You know, the dynamic range of, of a camera is, is. Huge today compared to where we started. , Vehicles coming off the line now are typically drive by wire, which means you can kind of plug into a, a bus, uh, turn on your software and start controlling them. Uh, you, you contrast that to the, uh, automotive domain where you have to build redundancy into the steering, build redundancy into the brakes. Um, you don’t need to do that in many, for many of these machines. Um, so that all lets us just basically turn on and go, uh, which is fantastic.

Grayson Brulte: Because you have, you have the drive by wire, which is a huge, huge breakthrough. So you have cat and then another big equipment. You, you have deer. Is it similar to trucking where you have to go get the OEM partnership? You’re at Waymo, you very had a very public relationship with Daimler Truck. Or are you able just to go to the, the Deer dealer or the CAT dealer and, and buy the equipment and deploy it? What does that look like from a relationship standpoint?

Kevin Peterson: Right now, we, we buy the machines and, and we upfit them. Uh, so if customers have existing machines, we can, uh, automate those directly. Um. In time, you know, the, the O OEMs are good friends. I, I worked at Caterpillar for a little while. We’ve got good friends over at Deere, uh, uh, and, and all over the industry. So, uh, you know, we’ll see how it evolves. But, , uh, right now we just outfit,

Grayson Brulte: could it evolve into a, say a, a powered by bedrock, partnership at some point in the future?

Kevin Peterson: I think we have to wait and see.

Grayson Brulte: Okay, well, uh, there, there’s a, there’s a, I’ll use the term, there’s a glimmer of hope there. How’s that for you?

Kevin Peterson: Absolutely. I, you know, we’d love to, but uh, we just dunno where that’s gonna go.

Grayson Brulte: bedrock comes outta stealth a few weeks ago with, with four construction partners. How, how did you initially get those partners? Was it pre-existing relationships? Was it, were you pitching them? How did you get those partners to come on board?

Kevin Peterson: people in the space we found are, are super hungry. Um, the, you talked about the labor issues and the, uh, demand for, for this work ., Construction’s growing. And, uh, so when we were starting the company, we, uh, we c we called a bunch of, , friends who knew people in the construction space. And, uh, they introduced us to a bunch of wonderful co partners. , Uh, one of them sent, uh, we’ve been working with from the very beginning. , We, we had a call with them basically the first day we started the company and, , we’ve been working with them ever since. , They’re super excited about the technology and, uh, they see how it’s gonna transform things, um, and they wanna be in there early, so it’s awesome.

Grayson Brulte: So take that one customer. So right now, 94% of construction firms reporting ongoing hiring difficulties. So are you coming and saying, wow, here we are with, with open arms. Is, has it been a very good positive re relationships that you’ve been able to develop just because of the hiring practices?

Kevin Peterson: you know, it’s not just hiring practices . hiring is a, a challenge, uh, but the construction industry itself is growing. There’s a lot of demand for, for. Doing more . the technology that we’re building lets you look at things like productivity. Uh, report how much work you’ve done. Uh, look back at, uh, all of the work that you’ve done. So if you, if you want, uh, if you wanna ensure a project, for instance, you built a foundation, uh, and there’s an issue with it down the road, you can go back and look at video of what you built. Um. Uh, and, and we’re, we also get to, like, connect the, the construction site. So, uh, those benefits are really huge. Safety is also a huge, huge deal. Uh, construction’s a a pretty dangerous job and, um, there’s lots of collisions with these big vehicles. Um, and so, uh, you know, building a system that reduces injuries and, and, uh, fatalities is a big deal.

Grayson Brulte: So let, let’s use an excavator for example. When, when you deploy an your customers or not you, your customer deploys an excavator into their construction site. You mentioned video, is that video recorded? So let’s just say it starts day one and four weeks later it’s done with the job. Is that whole four week of video recorded and archived?

Kevin Peterson: currently record everything.

Grayson Brulte: Because I think of that from an insurance standpoint. If there’s an incident, or, sorry to go back to your Waymo thing, but we saw Waymo’s won multiple court cases. People say, oh, the Waymo hit me, and then Waymo turns over the video and says, actually no, this person jumped on the vehicle. It wasn’t the Waymo vehicle.

Kevin Peterson: Yeah, that’s right.

Grayson Brulte: Do you see a similar situation like that? Because if you look historically in the construction industry, there has been a lot of workers’ comps and there, there, there’s no video there. There’s no data where you can say that the insurance company or your customer can turn over the video data. Here you are, look, we’re not at fault. Do you see that as potentially helping to, helping to lower the insurance costs that your construction customers having to, uh, burden today?

Kevin Peterson: I think that will happen. But of course the, the other thing that we’re gonna do is try to prevent those injuries from ever occurring. Uh, so, uh, the video exists and if there’s like a crack in a foundation, for instance, you can go back and look. So, so that’s part of the insurance cost as well. Uh, for, for accidents, you know, uh, we’ll be able to prevent,, most of them from occurring. You know, some of these things are, uh, harder to, to deal with, but, um, uh, many, many accidents on a construction site are things like, somebody’s very focused on the work that they’re doing. They back up a big machine, they hit a pickup truck behind them. Uh, that’s a pretty easy thing for our software to mitigate. Um, and so all of those, uh, sort of accidents should go away.

Grayson Brulte: So what type of perception range are, is your equipment going to have?

Kevin Peterson: we’re building a system to be able to do the work that’s right in front of it. Uh, it’s got cameras, the cameras can see very, very far, uh, uh, but the machines are mostly moving at low speeds. You know, it’s sort of like three miles an hour to 35 miles an hour. Uh, our team is built systems that can see a kilomet. Uh, so we’ll, we’ll just build what we need to build. Um, right now what we’re focused on is, uh, really, really fine detail right in front of the machine. So if you’re digging and you want to understand, uh, exactly how to, uh, how the earth is moving in front of you. Uh, you’re cutting through, uh, the soil and we’ve got some really cool videos on our website of, of some of this stuff. You’re cutting through the soil. You can actually see, uh, the sand dropping out of the bucket. You can see the clay curling up. And, um, that all goes into, uh, digging really, really well is how, how an operator would, , think about the problem. , So that’s really where we’re focusing our energy. Really good detection, really good understanding of the terrain.

Grayson Brulte: So that data outside of the autonomy stack to me, so let’s say pretend I’m a construction owner, that’s very valuable data for, for me, I can get a really good understanding of the geography and then the terrain. How are your customers, the, the construction owners. Operators, how are they using that? ’cause they’re getting an, an immense set of data. And I ask you that because some construction companies, this with all due respect, can be overwhelmed by the amount of data. That you’re able to provide to them? How are you helping manage that data where they can, I’ll use the, I use the term they can make it actionable to help them increase efficiency in their businesses.

Kevin Peterson: so we provide, uh, statistics. So things like how quickly are you loading a dump truck? Uh, what does the train look like before and after we detect situations, uh, uh, that you might care about? So somebody gets too close to the vehicle, um, uh, somebody is, uh, you know, standing in a hole that they shouldn’t be standing in. Um, we can detect things like that and, uh, give them events,, that they can go back and look at. So it’s really summary information and then all of the deeper information, uh, will be there as well.

Grayson Brulte: Is that detection? ’cause if you have an individual standing there, an individual too close, is that a real, real time detection that goes to an onsite safety manager that can get that notification in real time?

Kevin Peterson: yes, we can do it in real time. We can also inhibit behavior, , uh, so of the machine, so, so that they’re, uh, safe in, in the moment. Um, and then record it as well.

Grayson Brulte: So there you go. So bedrocks for your technology, you’re increasing and enhancing the safety on the job site. How does it work with the, the human interactions? So if you look at a construction site. You have your autonomous equipment and you have human driven equipment, and you also have individuals moving around in vehicles, in some cases, walking. How does your system interact with that envi, with that environment?

Kevin Peterson: we’re, uh, detecting everything that’s happening around the vehicle in 360. Uh, this is, there’s cameras, there’s a lidar, uh, uh, and we’re, we’re tracking everything. So people, trucks, other vehicles. Uh, the terrain itself, , that’s a 360 degree field of view. Uh, we’ve designed it to see, uh, with very high resolution out to, uh, maybe 15 to 20 meters. Uh, and then beyond that, out to, uh, about a hundred meters. We get really, really good detections. Uh, so, um, that’s everywhere around the vehicle all the time. , And, uh, when people are walking near the vehicle, uh, it knows that reacts and, and, , makes a plan. Uh.

Grayson Brulte: Wow. So today, today we’re recording this. This is July 24th, 2025. Bedrock has fully autonomous construction equipment. Out in the field, out in the wild today operating on job sites.

Kevin Peterson: we’re currently testing the equipment on our own sites. Uh, we’re launching soon with a, with a customer. Um, uh, uh, so you know. We care a lot about safety. We’re gonna prototype these things on our own sites. Uh, we’re, we’re data collecting with a bunch of customers, uh, currently. And then, uh, over the next six months, we.

Grayson Brulte: So you’re ramping up your, your job sites. There’s a, a very famous individual that worked in this industry, Fred Flintstone. So you’ve called your company Bedrock. Was it influenced by the Flintstones and is your test site. Is it called the Slate Rock and Great gravel company? Is there is, is there a Flintstones tie here?

Kevin Peterson: we are pretty influenced by the, by the flint. So the name itself didn’t come, uh, straight from that. We, uh, you know, Bedrock’s the foundation, solid foundation that you can build on. Um, that, that was the original, uh, the origin. But, , uh, we do name our machines after, Fred Flintstone and Mr. Slate, uh, are, are, are both machines. So do you know the dog? , So we love Flintstones.

Grayson Brulte: And then you don’t, don’t forget, Barney. Barney was a, Barney Rubble was a big part of the Flintstone.

Kevin Peterson: out there too. Yes,

Grayson Brulte: So you, you’ve got a, let’s call, you have a lot of nomenclature to name your autonomous equipment when you deploy it. You also have names to deploy your sites. When you send out a piece of autonomous equipment to the field, how long on average is it operating? And do you see that time that’s out in the field growing over time?

Kevin Peterson: so I initially we’re focused on narrow scopes, so we’re, uh, you know, mass excavation where you’re moving a lot of dirt. Pretty, pretty soon we should be able to do that, uh, close to human productivity and, uh, all day long. Over time, you know, people do tons and tons of di different things with these excavators. If you look at the distribution of things, kind of a power l from, uh, you know, lots and lots of volume of heavy moving. But, uh. If somebody’s sitting in the, in your, uh, in front of your house working on the pipes, uh, under the, under the road, they’re doing all sorts of kind of wild behaviors, uh, little things that, that you need to learn over time. , So, initially we’ll focus on this, uh, narrow scopes and do a lot of that work. Uh, as we get better and better, we’ll see those scopes grow. Um, and, uh, the goal is to be able to do everything that a person does.

Grayson Brulte: So was the goal just to stay in construction or, or will you eventually move out, which we’ve seen several of your competitors move into the mining sector as well.

Kevin Peterson: Yeah. Mining’s not too different from construction. We see a lot of value in construction where, uh, we want to build houses, power data centers, um, and we wanna accelerate that. So our focus right now is, uh, on that side. But, um, there, there’s really nothing stopping us from, from working on mining.

Grayson Brulte: Do you ever see a scenario where if you’re, you’re building a data center, perhaps you have to clear the land that your autonomous trucks can go in there? Not just level the land, but, but clear the land and then. You can have a different truck that goes in and, and, and levels it and starts to, to do the whole stack. And then on the next stop, next part of that stack is do you eventually go into the, the cement trucks, then to the, the pavers? Do we, can we get a full, it’s called a, a bedrock stack that Mr. Slate would be very proud of. Can we, do we see that ever evolving?

Kevin Peterson: so we’re, we’re going after heavy machines generally, and so anywhere that, uh, you know, these machines are doing lots less work, and, uh, the value of that work is really accelerating. The, the project, we will eventually work, . Uh, that’ll take time, of course. And, uh, you know, some, some of these areas are much harder to automate, uh, than others. You’ve got messy concrete coming out of a concrete pipe. you gotta figure out how to like, move that, uh, move that, uh, implement around, um, where excavation’s a little bit easier. You can, you can just get in there and dig. But the full stack, , is doable and, um, and that’s the dream. Be able to build things as fast as possible.

Grayson Brulte: Could you ever see that stack? Then expanding to onroad, so Deere has a, has a large road paver business. Could you, in those, they’re going two miles, three miles an hour and the, the gentleman that I know very well who runs the road builders, he says, we need automation because it’s such a dangerous job that people do silly things. They drive through there and they, they kill people. Unfortunately, they want automate. Could you ever see your technology going to that, since it is within the construction realm?

Kevin Peterson: absolutely. Yeah. Yeah. Um, I, I think, um, there is a fair amount of automation in, uh, pieces of the pavers. You know, they, they follow a line, uh, they’re now GIS controlled. So, you know, uh, don’t want to wave that aside. There’s lots of good technology there. Uh, doing things like protecting the machines. From collision, uh, is something we certainly could do. Uh, there’s also coordinated effort. You know, you’re driving on a highway, uh, maybe you’ve got a line of cones next to you. There are people walking around those machines. Uh, the, uh, so, uh, you know, automating those other pieces that support the paver and, uh, control the paver, I think, um, is, is really, really important . uh, both both for productivity and also for safety.

Grayson Brulte: Then if you did that, do you have to modify what you’ve built from an autonomy standpoint or can you simply take what you’ve learned and port it very similar to what Waymo did with the self-driving car and then obviously to via, can you start that as a, if you want to call it a base, then you can either build or modify upon that base.

Kevin Peterson: Yeah, that’s right. So the, the algorithms that we’re building, um, are, are very general. So, uh, you know, the process of like, observe the work. And then integrate that into the software. Um, so anywhere that you can do that, we, we should be able to automate . uh, and, and, um, ultimately, you know, we’re, we’re building, software both, uh, on the safety side and then, uh, foundation models that will control lots and lots of different types of machines.

Grayson Brulte: What’s the biggest challenge when you, when you’re automating this big equipment? Is it the controls? Is it the, the weight of the vehicle? Is it being able to protect the, uh, identify what the object is in front of you? Because I know that there has been historically issues. Correctly identifying on construction sites and then grabbing something. What are some of the biggest challenges you have to overcome as you look to automate the construction industry?

Kevin Peterson: I think the most interesting part of construction is the variability of the work. So if you look at, um, if you look at self-driving cars, you know, the, the description of what they’re gonna do is drive from A to B. It’s pretty simple. Telling that machine to do that is, is straightforward. The nuance in how you drive is very challenging. Um, for us, the. the machine is a tool. You’re doing something different every single day. Uh, you know, we’re on sites where they’re changing, changing the scope of work every couple of days. And so you need to be able to communicate with the machine. You need to be able to command it, to do all these different tasks. You need to be able to react in real time, uh, to, uh, to, to everything that’s changing. And so the user interface is really interesting. And then, uh, the, the way that we., Build out the behaviors is important as well. You need to be able to, uh, control them. Uh, you know, maybe you need to be able to literally talk to the machine with natural language. Maybe you need to, uh, use CAD to, uh, tell it how far to dig. It needs to be able to see the things around it and understand and react, uh, to, you know, maybe there’s a tree in the middle of the field that you didn’t know about. Um, and, uh, so you get all these surprises and, and reacting to those surprises is, uh, is, uh, the fun of the game.

Grayson Brulte: who controls that? Is it the construction partners, or do you just have a service layer that controls the vehicle?

Kevin Peterson: there are gonna be people sort of controlling these, these sites. Uh, whether that’s, you know, our customers or we provide a service, we, we haven’t figured out yet. But, uh, yeah, people looking at user interface, um, and people in the field who see issues, uh, and, you know, these are, these are things that people are already doing. Um, so people in the field, uh, you know, looking at issues and, and correcting plans, and then directly communicating with the vehicles.

Grayson Brulte: What you just indirectly said is job creation. You, you said the stat earlier that the average age is 55. Well, if you get a 19, 20, 20, 20 1-year-old that wants to go into construction, they can, they can run the app and run the autonomous machine. That’s a job creation that, that really helps the overall construction industry. I’m curious, let’s say you sign a deal with Acme Construction Company. And they say, Hey Kevin, we need you to go to ar this site in Arkansas and, and deliver the trucks you, you deliver the autonomous truck. What is the first thing that you do when you drop off the autonomous construction equipment at a site for a customer?

Kevin Peterson: we want to slot that machine into the process that they already have. So they’ll go out and do a survey. They make a plan for the cons, for the, for the site. They’ve got drawings, what they’re gonna do. Uh, so we ingest those drawings. So they’re, you know, somebody will. Uh, load those drawings in, uh, there might be some work to, to, uh, translate it into a format that the, that the machine understands. Um, and then, uh, uh, once the job’s set up, you, you push go, uh, the machine starts doing its work. There’ll be a cluster of these machines working together. And so, uh, somebody on that site very likely is gonna be sitting in a, a, you know, sitting behind a computer screen watching these machines do their work, watching their productivity, and, um, uh, uh, I think that’s an awesome job. I think that’s an awesome job. I could sort of think about it like playing StarCraft, uh, and being able to like, control all of the, all of the different, uh, you know, pieces of the game.

Grayson Brulte: it’s a real life video game. Kids have grown up in video games and we go back to the eighties. You’re gotta rot your brain out if you play video games all day. Well, they became some individuals. They became very successful and founded very, very good companies that who’ve gone on to change the world. And they’ve gone on to change the, uh, the movie industry. If you, if you look at what the Unreal Engine’s done, wow. That’s, that’s a, that’s a pretty big incredible breakthrough there. As you ramp up and scale and you’re, I understand the ACME construction site, so you have one human to one machine. How do you see that scaling over time? Could you have a human overseeing, say, 30 machines on the site, 10 machines? How do you see that human to machine ratio changing as it scales on a site?

Kevin Peterson: Yeah, I, I think it’s gonna vary with the sites, uh, you know, um, but, uh, the sort of intelligence of problem solving that the people do today is still gonna exist. Uh. Somebody’s gonna do that behind, uh, behind a desk. And so it is gonna vary a little bit with the complexity of the site. I, I think that ratio is gonna be, gonna be pretty large. You’re gonna be controlling the whole site.

Grayson Brulte: When this happens, this, this this happens all the time in construction. The foreman calls change of plans. This is, this is what we’re doing now. How do you ingest the change of plans into the machine? Is that gonna be the, uh, the operator on site that will, that will put in the new plans?

Kevin Peterson: yeah, that’s right. Is some, somebody is going to, uh, you know, like draw on a map and say, uh, actually I need a trench here that’s three feet deep. Uh, they, they go out, they spray paint on the ground, uh, and, and they say, follow this line, uh, dig out this area. Um, we want to be able to do all of those things so fit seamlessly into, into the process.

Grayson Brulte: So you’re able to do that because of the perception system that developing at Bedrock.

Kevin Peterson: that’s right.

Grayson Brulte: Wow. So you basically, you’re, you’re giving the, you’re giving the, the construction equipment, the ability to see, and then the person is giving it the ability to think, when do you give the construction equipment the ability to think and make decisions?

Kevin Peterson: I mean the, the construction equipment is, you know, if you just sort of like look at the hierarchy of a construction project. . There’s different layers of decision making. There’s things for like, uh, where does all my aggregate come from to build a road, uh, all the way down to, you know, I’m, I’m navigating around, uh, in an excavator and I’m, I’m, I’m digging the material around me. So, um, we’ll do, we’ll do reasoning sort of, uh, for the on the ground work. , Over time, you know, I, I think you, you get to a point where, uh, magic happens, you can actually sort of like plan out the operation of construction site. You get to know. Exactly how long it takes to do a particular job, a particular piece of that job, uh, you can optimize. Uh, you know, if you’re, if you’re an owner and you’re, you’re, you’re running one of these projects, you optimize, uh, the amount of time that it takes because you’ve got a duration that you wanna fit the project into. You can ask, do I need another machine on site with that speed things up. Uh, if I, if I want to take a little bit more time, how do I do that? Can I lower costs? So that’ll be a planner, uh, that, that runs on top of, uh, the rest of the software. Um, so, you know, you can, you can sort of look at this and see, uh, how far it’s gonna go, uh, in time.

Grayson Brulte: cause when you get to that point, you give that construction owner or operator the ability to properly budget time. And that’s the number one thing. We gotta get this done on time. You say, okay, well. Just hypothetical. Say, okay, we have three bedrock autonomous construction equipment running. If we add a fourth one, we can get in on time and that’s gonna do X dollars to our budget. You’re giving all the planning tools that need, and then that’s gonna create su success overall.

Kevin Peterson: That’s right, and you also know. in a really detailed way, the work that that machine can do and the amount of time that it takes to do that, to do that work in a way that you just don’t have today. We know scoop by scoop. How much dirt is going into a truck, today that’s tracked on a piece of paper. Generally, uh, you know, that the truck driver says, I, I did this many loads, and the excavator operator says I loaded this many trucks. Uh, and, uh, if they disagree. There’s a fight or a lawsuit, um, uh, we get that for free automatically, uh, just coming directly off the machine.

Grayson Brulte: So you’re getting all this data off the machine, and at some point, this is a hypothetical, but based on your technology, you could. Automate dump trucks. Then let’s go back to the stack analogy then. Do we, do we do, I see it read a press release, say two years from now, bedrock os for construction sites to manage your entire construction site because all the data you’re getting is, is that where this evolves?

Kevin Peterson: Absolutely. Yeah.

Grayson Brulte: Okay, so, so you’re building it. That’s where like, oh, today, let’s go back to this day. Are the drawings, are those the, is that the fuel that enables your system to go and so when, when the robot ingests the drawings, so the way it goes and then in the future. That becomes the operating system.

Kevin Peterson: the operating system, uh, is, is more than just the drawings. Uh, it, there will be displays where you can see all of the vehicles, see their progress, uh, see what’s happening on the site. , Uh, if you need to radio to a machine, you can do that or, or, uh, you know, uh, communicate. Uh, with, with language, um, , stats will come back. Uh, and you can see the progress of, uh, both the individual machines and, and the site overall optimize it. And so you can imagine that several different kinds of user interfaces and, and, um, pieces that we’ll have to build out.

Grayson Brulte: So basically on a construction site or back at the construction headquarters, there could base be a map. They could see all the pieces of equipment where it’s moving, how. It’s moving and then get the whole data feed essentially. So they, uh, you’re gonna call it command center.

Kevin Peterson: that’s right. Um, uh, command center and, uh, not just that site, you should be able to see, you know, every site that has our equipment on it.

Grayson Brulte: and that’s gonna live with you or will that live with your customers?

Kevin Peterson: customers will have access to that.

Grayson Brulte: So you’re basically the, the, the common denominator that I’ve picked up to this entire conversation is that you’re giving your customers the ability to scale and scale very big.

Kevin Peterson: yeah. that’s right.

Grayson Brulte: And so as we go into a site and you’re allowing your customers to scale and, and truly grow their operations with autonomy because of the growing labor shortage, do you have to pre-AP the site? How, how does, how does that work from a mapping perspective? Or you were able to do it map less?

Kevin Peterson: right now we, we do it map, uh, we’ll do, uh, drone flyovers. You know, this is a common, a common, uh, practice in, in construction is to do drone drone flyers ahead of time and after. Uh, to do before and after, uh, drawings, um, uh, because of the data that we’re getting back, uh, in real time, we, you know, we just build a map online. Uh, I immediately, when we start, uh, we scan the lidar around the site and we get the 3D, uh, terrain. We line that up to drawings and then we go, so, uh, no pre-app required.

Grayson Brulte: I hope your drones are autonomous.

Kevin Peterson: I don’t think drone are allowed to be autonomous right now, but, uh, someday.

Grayson Brulte: Something. We’ll, we’ll get there. Uh, how big of an opportunity do you feel that the construction TAM is for Bedrock?

Kevin Peterson: it, it’s, it’s massive. It’s massive. , There are, uh, $120 billion of, uh, construc of excavation contracts, just excavation, uh, in the United States. There’s millions of vehicles, uh, and, uh, and billions and billions of hours of, of work that goes into these, uh, projects. We’ve seen, uh, multi-billion dollar projects that, uh, went by the wayside because. Uh, they couldn’t, uh, they couldn’t get the people together to do the work. And so, uh, there’s a big tam today, and then that tam is gonna grow.

Grayson Brulte: As the TAM grows. What are you and the team at Bedrock hoping to achieve?

Kevin Peterson: I think the tam growing is an indication that we’re building more. And, uh, you know, I live in the Bay. Uh, we need to build more housing. Uh, I’d love to see that happen. Uh, we’ve talked a bunch about data centers. We need power for those data centers. We’re gonna see those, uh, uh, come together, uh, faster and faster. , The, the whole thesis that we have in the United States. Is, uh, to elevate the work that we do over time. And AI brings that promise. Um, we want to start to do work in the real world so that, uh, uh, we, we just elevate all of the things that we’re doing and go faster and faster. That’s the goal.

Grayson Brulte: the goal is you’re gonna build something special. That, that’s my opinion. Autonomy is the key to scaling construction because the grower la growing labor shortage without autonomy, the construction industry is unfortunately not gonna be able to grow as it dozen, as Kevin rightly said, need more housing in San Francisco, not just San Francisco. There’s various markets around the United States where we need more housing. The population is growing, our tummies are full. We need housing. And autonomous construction equipment will allow us to do that. The future is bright. The future autonomous. The future is Bedrock Robotics. Kevin, thank you so much for coming on the road to Autonomy today.

Kevin Peterson: Thanks for having me. Appreciate.

Key The Road to Autonomy Questions Answered