Alcatraz

Talk · Pete Kelsey · 3DISE 2025 · 25 April 2025 · 09:50 - 10:10 · 28 min

Chapters

What the talk is about

Pete Kelsey of VCTO Labs tells the story of scanning the whole of Alcatraz Island for the US National Park Service, a pro bono project that started with sea level rise and no existing digital assets. Pete explains how a volunteer team combined a survey control network, drone lidar and photogrammetry, Hovermap SLAM, Spot, the Flyability E3 and multispectral imaging in a three-week window, and shows what the data reveals about the 1962 escape and the 1946 Battle of Alcatraz.

Key ideas

  • The project began with a sea-level-rise question and the discovery that the site had no digital assets, so Pete's team set out to scan the whole island inside and out.
  • A survey control network with 60 ground control points came first, so that drone lidar, photogrammetry, terrestrial and SLAM scanning and multispectral imagery could all be tied together; according to Pete, the network closed at about six millimetres.
  • No single tool could do everything: the Hovermap SLAM scanner captured most interiors in the three-week window, Spot carried it into buildings contaminated with lead and asbestos, and the Flyability E3 went into sewers, smokestacks and cisterns.
  • The team slept in cell block D for three weeks to avoid relying on the ferries, and a prototype AMD workstation solved the processing problem caused by the island's poor internet.
  • Combining drone lidar with interior SLAM data gives Alcatraz an X-ray view, and according to Pete the real value comes after capture, in visualization, simulation and analysis such as viewshed studies.
“We make the invisible visible. We give people perspective that no one has ever had before.”
— Pete Kelsey ·
“I'm just a capture guy, but I'm smart enough to know that the real magic comes after.”
— Pete Kelsey ·
“Because I also knew right from the beginning that there was no one tool that could do all of this.”
— Pete Kelsey ·

Questions answered

Why was Alcatraz Island 3D scanned?

According to Pete Kelsey, the project started because of sea level rise: the National Park Service had no existing digital assets, so the team scanned the whole island inside and out.

Which scanners and drones were used to scan Alcatraz?

Pete Kelsey's team used drone lidar with a five-return laser, photogrammetry from a Freefly Astro, Hovermap SLAM (handheld, on a pole and on Spot), the Flyability E3, an eBee for multispectral imagery and terrestrial scanning, all tied to one survey control network.

Did the Alcatraz scan find hidden structures?

Pete Kelsey said there were a few discoveries, but subtle ones about how things were built rather than hidden tunnels.

Full transcriptabout 17 min readShowHide

Transcribed automatically (ElevenLabs Scribe) and lightly edited. It may contain small errors. Click a time to play the video from there.

Host: Pete, welcome.

Pete Kelsey: Thank you. Thank you, thank you. One minute. Voila. Good morning.

Pete Kelsey: Been really excited to come here and talk about this project that has been nothing short of life-changing, and I'll tell you a little bit about why. I got hooked on reality capture maybe 15 years ago. Um, I spent 25 years in AEC work, basically working in, on the tech side of AEC, and immediately realized that reality capture is the ultimate tool for providing context. And how do you put a price on that, right? So I describe it as the ultimate currency, meaning anyone can look at one of the models that we're creating or even just raw data and go, "I get it.

Pete Kelsey: I know what I'm looking at. I understand." To me, that is the most powerful, intoxicating thing that has ever crossed my path. And since then, um, I sort of morphed that into looking for a great story that I knew reality capture could make even better, right? Because it changes, it changes everything, right?

Pete Kelsey: It changes, you know, have a look at this to almost like, come with us, come inside, have this experience with us. And I think that's, that's, that's magic, if you ask me, and, and, and still is all these years later. So I'll go anywhere. I'll do anything. I will try things.

Pete Kelsey: I've done a lot of work with television, um, putting reality capture into situations that, you know, some might think are crazy and, you know, maybe I'll get laughed at. That's okay. Because you know that box they tell you to think outside of? I don't have a box. There is no box.

Pete Kelsey: Screw the box. Okay? So with that... Uh, sh... Okay.

Pete Kelsey: Sorry. Okay. So the genesis of the Alcatraz project came about because of sea level rise. So this is a national park in the United States. It's, it's one of the number one tourist, uh, destinations in the state of California.

Pete Kelsey: Um, so the National Park Service has to-- is entrusted with taking care of this place, keeping it open, keeping it safe, keeping it available, et cetera. So that was the reason, that was how the door, if you will, opened for me and the rest of the team when I said, "No problem." But then I always ask one of the same two or three questions, and one was, "No problem, sea level rise. Tell me about your existing digital assets." And I bet you'll-- this will be familiar.

Pete Kelsey: The answer was, "Digital what?" Right? "Okay, so you don't have any. That means we're gonna start at ground zero. That means I wanna scan the whole place, everything, inside and out."

Pete Kelsey: And they said, "Okay." So before I start showing you some of the data, what few people know about Alcatraz is that it has a really long history, okay? Back to Native Americans. I mean, going back, they've been there since the Ice Age. Uh, but the Gold Rush in the United States, the American Civil War, um, Alcatraz, because of its strategic location in San Francisco Bay, had a lot of value militarily.

Pete Kelsey: So the first sort of large infrastructure was all military. So it was this huge fort with cannons all the way around it. What's the number? A hundred and eight heavy weapons around the site. Now, what I've highlighted here, this is known as the Citadel.

Pete Kelsey: This, um, this is basically the barracks where the soldiers lived, worked, et cetera. Remember that place because it's still there. The cell house of the modern prison was built on top of that. Um, speaking of the modern prison, in 1907, they started building it. 1910, it was finished.

Pete Kelsey: And this is pretty much the Alcatraz that is famous, right? And all these bad characters who got locked up there. Uh, Al Capone, Birdman of Alcatraz, uh, Machine Gun Kelly, Doc Barker. Over there at lower left, um, that's Whitey Bulger. Few people know that he was there in the late '50s, early '60s.

Pete Kelsey: Crazy story in 1946, known as the Battle of Alcatraz, where there was a prison riot and the prisoners took over the cell house. And they called in the Marines, and the Marines came to the island and started mortaring and shooting and, uh, basically trying to put down this riot. And I'll show you some of that evidence in the data. And then, of course, the '62 escape, um, a lot of movies have been made about these three guys. Um, they made it off the island.

Pete Kelsey: Nobody knows if they're alive or dead. The point is they got out, and it remains a really interesting mystery. Once the Park Service said, "Okay, um, yeah, you can scan the whole island," then I started thinking logistically, "Okay, how are we gonna do this?" And one of the immediate challenges was it's an island, and the only way to get there is on a ferry. So you have to take the boat out in the morning and then the last boat, and then nobody's on the island except for one security guy who had overnight.

Pete Kelsey: So I said, "Can we spend the night on the island?" And they said, "You sure you wanna do that?" And I said, "Yes." Um, because, because the other key thing here is they only gave us th... Well, I ha- I had a three-week window to get all this done.

Pete Kelsey: So three weeks relying on the ferries, I knew there was no way that was gonna work. So they put us in D Block, third tier, and we slept in cells for three weeks. And it's just, it was as cool as it sounds, believe me. There's, there's an inner nine-year-old right here that's never really grown up that thinks that's one of the coolest things I've ever done, okay? Um, and a lot of stories about it.

Pete Kelsey: So project like this, for me, because I'm an AEC civil survey geospatial guy, I'm not-- We're not-- The first thing that we're gonna do is establish a survey control network because we're gonna be tying all kinds of disparate data together, right? There's gonna be drone-based lidar, drone-based pho-photogrammetry, terrestrial laser scanning, SLAM lidar, multispectral imagery, and whatever other surprises happen along the way. But it started with that, so I could tie it all together with as high a degree of accuracy as we possibly could. Uh, this is my friend, uh, John with PCL Construction. He came out and just killed it.

Pete Kelsey: We had 60 GCPs. You know, we ran all the math between all of them, and it came down to, like, six millimeters, which we're thrilled, um, to have that kind of fidelity. We flew a lot of drones, a lot of them. Uh, Inspired Flight 1200i with a Phoenix LiDAR Ranger 120. If you've never used a five-return laser, try one.

Pete Kelsey: Change your life. Uh, we flew the Hovermap in the cell house. Why not, right? We were there. Uh, we flew the eBee.

Pete Kelsey: We used the eBee to fly, um, multispectral and some photogrammetry, actually. Um, yeah, that's the Astro by the lighthouse there, the eBee by the lighthouse, and then, uh, the E3 from Flyability. This thing was super cool. Super cool. And why all these tools?

Pete Kelsey: Because I also knew right from the beginning that there was no one tool that could do all of this. It just, it didn't exist, okay? So we knew-- I knew I was gonna have to call in every favor, every friend, beg, borrow, steal, what have you, because this was also a pro bono project. Nobody got paid. I didn't get paid.

Pete Kelsey: None of the team, who you'll meet in a minute, um, it was strictly, "This is gonna be cool. Please come." And everybody did. The drone-based lidar, we could ju- uh, we flew all the, the drone missions from the roof of the cell house because Alcatra- Alcatraz doesn't close. There's seven to ten thousand visitors per day, so we had to work around that.

Pete Kelsey: So we'd work really in the morning, we'd work at night. But, you know... Oh, and the other miracle, if you know San Francisco, look at the weather. It was a miracle. It was literally a miracle because the weather there can be terrible, especially in December.

Pete Kelsey: Photogrammetry. We flew-- This is a Freefly Astro. Super cool drone. Uh, Sony A7RIV. You'll see the data in a minute.

Pete Kelsey: Uh, this is my buddy Jeremiah and the team from Esri who just came and knocked this out of the park. Can't wait to show you that mesh. The E3. We sent this thing into the worst places you can imagine. Sewers, smokestacks, cisterns, crazy places that, you know, weren't safe for us to go, or the Park Service said, "Nobody's-- You know, you're not allowed to go in there."

Pete Kelsey: Like, this is a Civil War-era sewer line, um, that we tried to put this thing in, and we got some good data. We didn't get as far as I would have liked, but I was so impressed with this thing because, of course, it's doing lidar, but it's got its own light source and 4K video all at the same time. Pretty cool. The Hovermap flying in the cell house. You know, if there are five or six of us spending the night- In Alcatraz and we have all this gear with us, we're gonna try stuff, right?

Pete Kelsey: We're gonna take chances. We're gonna in- we're gonna force ourselves to innovate. The Hovermap, boy, if there is a, if there is, gosh, a gold star to the one sensor we used, it, it would be the Hovermap because we, we had two of them, and basically we did all the interiors with the Hovermap. Because again, three weeks, I knew right away SLAM was gonna be the way to go. 'Cause terrestrial laser scanning, granted a lot has changed in the years since this was done, right?

Pete Kelsey: I, and I've seen some of it and it's amazing. But at the time, SLAM was the only solution, and the Hovermap was just amazing. Um, you can put it on a pole that we did there. We put it on Spot. Um, we sent Spot into buildings that are off limits that were, um, contaminated by, uh, lead, asbestos, that type of thing, that we couldn't fly in because if you flew in a space like that, the rotor wash would kick up all that hazardous material.

Pete Kelsey: So Spot was a, an amazing solution for that. Uh, the Hovermap on the backpack, this is one of the latest things from them. I actually didn't believe the, the marketing I saw that, you know, with this RTK antenna on the backpack, all the data drifts and slips, uh, associated with SLAM scanning is gone. And I said, "Show me." And they showed me and it's, it's amazing.

Pete Kelsey: So, uh, a couple of additional miracles happened, uh, because... Well, that made this project successful. And number one absolutely is the team. All of these people came. Nobody got paid.

Pete Kelsey: They said, "When do you want us there? How can we help?" My favorite sort of subtle bit of this is that a lot of these folks, if you look at 'em, might be competitors out in the marketplace, right? But there was none of that. We were-- There was no like, you know, "Don't look at what I'm doing," or, you know, "Keep," uh, it was none of that.

Pete Kelsey: It was let's-- It was more like, "Watch this." Or somebody's struggling with a problem and somebody would be, "Oh, I think I, I think I have an answer for that," or, "I think I have a workaround for that." It was just amazing, and I couldn't be more grateful. So I might be the one up here talking about the project, but these are the people who made it happen. So my endless gratitude to them.

Pete Kelsey: On the software side, post-processing, uh, a-again, pro bono. You know, I had no money. It's like, "Can I have a 30-day license?" And then maybe 30 days more. And everybody was just amazing.

Pete Kelsey: So again, thank you to these folks. Yeah. Yeah. And here we are. Some dear friends now.

Pete Kelsey: I mean, it's the, it's, it's why I love this conference. It's why I love this sense of community. It's just the same thing up here, except rather than being in a beautiful hall, we're out getting dirty scanning. It's great. Okay?

Pete Kelsey: Second sort of miracle. By the second or third day, I realized, there's a locked door or, and who has the key? "Well, they're on holiday." Oh, God, this is never gonna work. So I took my problem to the Park Service and they were amazing.

Pete Kelsey: They handed me the keys to Alcatraz. I'm not making this up. And they basically said, "Don't do anything stupid and don't get hurt." And so that was huge. Again, it-- the project wouldn't have happened without that.

Pete Kelsey: And then finally, the other surprise was... So we're, we're scanning all day long. So people are coming in and out of the office and dumping raw data. And by day one, I'm freaking out because surprise, the Wi-Fi on Alcatraz sucks. It's terrible, okay?

Pete Kelsey: Like dial-up speeds from 20 years ago. So, you know, I'm dying a little bit by the hour. And again, my buddy John said, "You know, I, I got a friend at AMD. Let me make a call." And the heavens opened up and FedEx delivers this prototype AMD supercomputer, 96-core Threadripper PRO, half a terabyte of RAM, 12 terabytes of hard drive space, and the world changed.

Pete Kelsey: I was like, "I think, I think I'm gonna live through this, you know? And I think, I think we're gonna be- actually be able to pull this off." So that was a huge, huge component to that. That's now evolved into, uh, 'cause again, that was a prototype. It's now evolved into, uh, the ThinkStation P8 with the Threadripper PRO in it.

Pete Kelsey: It's just astonishing. And for the geeks in the house, this was so cool. I was like, Control, Alt, Delete, go to the Task Manager, and I squealed. I literally squealed. I'm like, "Guys, you have to come.

Pete Kelsey: Look at this. Look at this. Look at these 96 cores running at 4.68 gigahertz." It was, it was so cool, and that's, that's the nerd in me.

Pete Kelsey: So, um, the data. Ugh, this is a drone-based LIDAR of... Thanks to Burt and NUBIGON, it's been getting all kinds of attention. It was a thrill for me. I took a picture of it that it's out on the big board out in the hall here.

Pete Kelsey: I sent it to the entire team. It's like, "Look at this. This is really great." Um, the media's been running with it. I mean, it's just fabulous.

Pete Kelsey: So drone-based LIDAR, um, from that five-return laser, oh, just stunning. Some of the Hovermap data inside and out, um, the lighthouse. Again, this is, this is what I love about what we all do. We are-- We make the invisible visible. We give people perspective that no one has ever had before.

Pete Kelsey: And again, how do, how do you put a price on that? Okay? Uh, some Hovermap data. That's a still of it. I think there...

Pete Kelsey: Yep, there's a video of the same thing. So colorized, uh, SLAM data. That's the citadel, the oldest part of the fort there underneath. Um, just the 1962 escape. You don't go to Alcatraz and not spend maybe a little extra time on that, right?

Pete Kelsey: So, uh, Jeremy with Hovermap, we, we scanned it all. This is... So they-- This is the, what do they call it? Utility corridor. So they got out of their cells.

Pete Kelsey: They climb up these pipes every night for weeks, and they create this. This is their workshop, and they're building rafts and life jackets and paddles and all this stuff up there. That's actually a crime scene photo from the FBI from '62 or '63. Scanned it all and just, you know, cross-section through some early data 'cause we were dying to see it. Um, and then the hole in the rooftop of the cell house where they actually got out onto the roof.

Pete Kelsey: The Battle of Alcatraz. Remember, the Marines are bombing. Well, there's the north, uh, gun gallery is off-limits to visitors. So ladders, climbing up bars, crawling in there, and the archeology is spectacular. There's graffiti from guards and then the most horrific pornographic graffiti I've ever seen in my life from the rioters who took over the gun gallery.

Pete Kelsey: I wish I could show you. You know, see me at the bar, and maybe I'll show you a picture, but it's oof. This is where the rioters, the last three of them, that-- This is the utility corridor. The Marines are on the roof. They're jackhammering holes through the concrete and dropping hand grenades into that utility corridor, which is where the last of the rioters met their end.

Pete Kelsey: This is building 64. This building nearly killed us. Tons and tons of scans. This was lead, asbestos. This is s- the lowest level is Civil War era.

Pete Kelsey: You can see these huge gun ports, those big squares, and then the modern construction, which is all wooden, is on top of it. And as this progresses, we get into a, a cross-section, again, perspective that no one has seen before. It's just like, "Wow." It never gets old to me, ever. Oh, which by the way, if for all of you who put point clouds and meshes and splats up on LinkedIn, don't stop, okay?

Pete Kelsey: It's the coolest thing ever, and I'm always looking forward to the next one that I just go, "Wow, I gotta get better at this." Okay? Multispectral, one of my favorite new tools. Yeah, it's 2D at the moment, but that near-infrared and near-edge infrared can make the invisible visible. Like, there's some paint striping down here which you can't see with the naked eye, and we're just like, "What's that?"

Pete Kelsey: Turns out it was helipad from way back when, '60s or something like that. So it's designed to work with, um, plants, and the biologists will have this, and hopefully something really interesting will happen. But we did it, and we saw that, and we thought that was great. The photogrammetry from Esri. Talk about squeal.

Pete Kelsey: I saw that chain link fence and squealed 'cause 10 years ago, 'cause I've been at this a long time ago, a chain link fence in a mesh? To me, that's witchcraft, right? But it looks great. It's amazing. So happy with this.

Pete Kelsey: Well, what happens when you start putting things together? Drone-based LIDAR, interior SLAM-based LIDAR, and then you can give it, Alcatraz a bit of an X-ray or a CAT scan. I just... This is where it gets so fun for me is, yeah, I'm just a capture guy, but I'm smart enough to know that the real magic comes after. It's what I've always, what we used to call the VSA, the visualization simulation analysis.

Pete Kelsey: Never gets old. Uh, this just came in from Eric, from the, uh, Esri folks, viewshed analysis, you know, tip of the iceberg about how this data is gonna be repurposed. Uh, this one, again, in that vein of sort of making the invisible visible, here's a modern building with a gabled roof that we can actually remove in the data because it abuts right up to the original gate to Alcatraz when it was a Civil War fort. And there's a really cool sign over the gate which just flashed by there that looks like that. The original name of Alcatraz in Spanish, Alcatraces, from 1857.

Pete Kelsey: Reality Capture is doing some really cool stuff with the data. This just came in yesterday. Uh, this is Twinmotion. Um, and just tip of the iceberg, 'cause I asked them, it's like, "Can, can you get me something I can show at 3DISE? I'd really like to."

Pete Kelsey: They're like, "Okay, we'll put something together." Just wait is all I can say. So do I have to say that this project was as much fun as you can have with your clothes on? It literally was. We were, you know, we're out there with other maps, and I'm like, "Jeremy, it's like don't cross the streams from Ghostbusters."

Pete Kelsey: And somebody took the picture, and we all laughed. The point is, we had a blast. It's, it was as fun as it looks. And you don't go into the mugshot room and scan it and not get your mugshot taken, 'cause we went in there and one of these guys said, "Hey, there's an app, it's like mugshot.com or something, and it'll, it'll take your mugshot."

Pete Kelsey: So we did that, and we thought it was a riot and it was great fun. Um, so thanks for having me. Thanks for your time. I really hope we can all do something cool together going forward. Thanks very much.

Host: Let's give it up for Pete. Fantastic. What an incredible example of the collaboration, and what a result.

Pete Kelsey: Yeah.

Host: Tons and tons of questions came in.

Pete Kelsey: Okay.

Host: I'll take one or two. So Renata is asking, "Did you find any hidden or undocumented structures during the scan?"

Pete Kelsey: It depends on who you ask. Um, hmm. You know what? I, I think it's safe to say yes, there were a few discoveries. It's not like a hidden tunnel or something from the movie The Rock, anything like that.

Pete Kelsey: There were, there were subtle things, like how things were constructed, that kind of thing. So not like Hollywood wow kind of things, but yeah, you can't take on a project like that and not have discovery happen at some level.

Host: Got it. It's part of the, the joy and enjoyment.

Pete Kelsey: Oh, joy.

Host: One more question: Has there been any interest in your data from the gaming/movie industry?

Pete Kelsey: Yes. Yes. Yes. Yes. Uh, one of the first things I told the Park Service is I said, "I guarantee you it's just a matter of time before you hear from Hollywood."

Pete Kelsey: Why? Because you guys know this. They can film the next Alcatraz-type movie and green screen it all, and save millions and millions and millions of dollars from having to do it on site. Just green screen it all. Gaming, stay tuned.

Pete Kelsey: I can't talk much about that, but s- the wheels are turning.

Host: Cool. Ladies and gentlemen, let's give it up for Pete one more time.

Speaker

Pete Kelsey

Pete KelseyLinkedIn

Owner of VCTO Labs

With over 30 years experience in the AECO space I provide technical solutions for storytellers, filmmakers, and creatives that make good stories great. I combine my passion for technology and storytelling to create immersive and engaging experiences using reality capture technology. I have managed…

Hear the next ones live – 3DISE 2027, 20–21 April, Prague

Buy tickets