Every Minute Counts: Practical Applications of 3D Scanning in Film

Talk · Matej Kosovsky · 3DISE 2025 · 25 April 2025 · 14:10 - 14:30 · 21 min

Chapters

What the talk is about

Matej Kosovský, who led Pixomondo's in-house scanning department, walks through the TV series Cada Minuto Cuenta about the 1985 Mexico City earthquake as a case study of how 3D scanning supports a lower-budget virtual production. Matej covers each use of scans, from location scouting, set pieces and miniatures to props, staged extras and alignment scans, and closes with Pixomondo's Tori camera rig and Gaussian splats running on LED volume stages.

Key ideas

  • Visual effects departments each need scan data of different quality and accuracy, so Pixomondo scans on every show and built an in-house capture team to reach correct results faster.
  • Physical set pieces such as a replica hospital walkway are scanned so artists can match dimensions and colour, making the transition to the digital extension seamless.
  • Detailed miniatures of destroyed buildings that would have been thrown away were scanned with photogrammetry and scaled up directly in Unreal Engine, with no remodelling or destruction simulation.
  • According to Matej, a scan of a staged scene sets the ground truth for artists months later, avoiding rounds of back-and-forth that a reference photo would cause.
  • Lidar alignment scans on the blend day before filming overlay the physical build on the digital environment; without that step, misalignment can make a scene unfilmable.
“And we didn't remodel it. We didn't have to do any destruction simulations or anything.”
— Matej Kosovsky ·
“If you scan the reference to begin with, you give that to the artist. It's kind of set in stone, like the ground truth is known”
— Matej Kosovsky ·
“This speed of capturing is crucial for film. We usually face huge time restrictions on locations, and that's why this is a game changer for us.”
— Matej Kosovsky ·

Questions answered

Can Gaussian splats be used on a virtual production LED wall?

Yes. Matej Kosovský said Pixomondo runs Gaussian splats in Unreal in real time on its stages and has already used them in a real production.

Why scan props and sets instead of just photographing them?

According to Matej Kosovský, a photo leaves the artist to interpret it and leads to many review rounds, while a scan fixes the ground truth and saves a lot of time.

What is an alignment scan in virtual production?

Matej Kosovský described it as a lidar scan of the physical build in the volume, taken the day before filming, which is overlaid in Unreal so the digital extension lines up with the real set.

Full transcriptabout 14 min readShowHide

Transcribed automatically (ElevenLabs Scribe) and lightly edited. It may contain small errors. Click a time to play the video from there.

Matej Kosovsky: Hello. Hi. Hello, everyone. Uh, welcome to my talk about, uh, practical application of 3D scanning in the film industry. I'm hoping to inspire some of you because we all know how this technology works, but maybe you don't know how you can use it in the entertainment.

Matej Kosovsky: So I wanna show you how you can do that. Uh, I work for Pixomondo. Uh, it's a visual effects, virtual production, and a technology company. I'm based in Canada, uh, but we operate globally, and we are part of Sony Pictures Entertainment. Uh, we work on a lot of shows and movies across the globe.

Matej Kosovsky: So for the fa-- last five years, I was leading our in-house scanning department as a head of capture because we utilize scans on every show that we work on. Uh, it helps our team go to-- get to the correct results faster. And because of this, we created an in-house department for it. The visual effects require a lot of intensive labors, labor work in, uh, specialized departments, and every department needs a different type of scan data in different quality and accuracy. So today, I wanna show you a case study of one of the projects I worked on.

Matej Kosovsky: Uh, I picked, uh, Cada Minuto Cuenta project, uh, as I believe it's a great example of how scans are supporting today's productions. I worked on this project in 2023, and it was a TV show about an earthquake that happened in Mexico City in 1985. And this earthquake killed thousands of people in the city as many buildings collapsed. And this show re-enacts the events that happened right after the earthquake, especially around hospitals and the victims, uh, i-impacted. We were providing mainly the virtual production support for our team because, uh, our team was tasked to create three full environments for virtual productions, and these were recreations of different neighborhoods that were the most impacted by the earthquake.

Matej Kosovsky: We also deployed operators on set to control the Unreal Engine, uh, during filming. And our focus for scanning was to support that team to get their job done. For timeline, it was a one-week on-location scanning in Mexico City. And here is a video from behind the scenes. You can watch the full video on YouTube.

Matej Kosovsky: There is just a small section, and you can see some of the results of our work, especially in the background.

Video: I think we're pushing the boundaries and we're pushing the limits of what was done to this day with this project. I think we are here in one of the biggest projects with visual effects and virtual production ever done in Latin America. They already had in their minds the ideas of, of, um, building maquettes, and the art department on the show had done a lot of research onto understanding, and they were able to actually build the maquettes of all the destroyed buildings. And with the visual effects team, we were able to actually come in and scan the maquettes, both with LiDAR scans and photogrammetry. We needed to create a lot of detail, but having the scan of the maquettes allowed us to be very true to what happened on the day.

Video: Virtual production i-inherently offers a collaborative environment on set. All creatives involved on the show and visual effects supervisor all have to work together to be able to create these environments, and that's a unique scenario in filmmaking.

Video: When highly creative people are presented with, uh, challenges the way they were on this production, they really rise to the opportunity to be able to tell the story that they want to tell. And I think the team here did it in a beautiful way. They were able to embrace the technology in a way that it was moving.

Matej Kosovsky: So they mentioned that, uh, it was one of the large-scale virtual productions in Latin America, but it was still quite low budget comparing to something like The Mandalorian or Andor that Gabriel was mentioning. So we had to find the creative ways how we can deliver the same type of quality of work, uh, and make it shorter because we didn't have the budget to spend so much labor on it. So I wanna dive in into different sections, uh, throughout the project, and the first was, uh, location scouting. So on the first day on the ground, we went to scout the location. So these would be the areas that we had to recreate the three environments.

Matej Kosovsky: And typically, typically, the location scout would be a small team of up to eight people that goes to the location. Uh, it would be a production designer, art director, and maybe a visual effects supervisor. And we discuss how we're gonna approach filming the scenes that we need to do for the show on that location. We talk about, uh, specific details in each location, and my job or the scanning team's job is to collect photography for references and to do some quick scans on the area, uh, that we are in. So we spend less than 10 minutes on each of these locations, so it's a very fluid process.

Matej Kosovsky: And- You have to just communicate with the team and grab whatever you can in the time that you have while you are there. So this is a neighborhood in Mexico City called Tlatelolco. It was one of the environments that we were rebuilding, and we got scans of few benches in the area and the type of sidewalk that they have. This is just so that people in VFX, when they are creating the environments, they have understanding and some kind of reference of the ground truth. And some areas were changed and some were still original from nineteen eighty-five, so some of the areas were still, uh, looked the same way that we had to recreate.

Matej Kosovsky: The next big thing is, uh, set pieces. So filming on locations can be very limiting. You are tied to a specific time of day. Uh, you can only work certain amount of hours. You are weather dependent, so if it's raining it-- and you can't have rain, then you need to skip a day.

Matej Kosovsky: Uh, you need location security. And it's obviously difficult to destroy, uh, real places if you need to do some destruction, which we had to do in this, uh, example. We had a sequence in a hospital with this walkway on the left side, uh, where the characters were discussing, uh, something on the day. And they f- they closed down that part of hospital because this is a real, opera-- active, uh, hospital in Mexico still to this day. And they could only close it for like two days to get the basic sequence done.

Matej Kosovsky: But then we had to destroy it and film more, so they created a replica of that walkway on-- to be filmed in virtual production and on a stage. So when you recreate a replica like this, we have to create extensions that is digitally recreated in the virtual production. So you have this physical prop in the middle where you have the actors, and that's, uh, created by the art department team on the client side, on production side. And our team in Canada or Germany or other places, they need to match the dimension, the color, so that they can make the extension in the digital environment. And if you have just a photo, the artist has very little information to work with.

Matej Kosovsky: So a scan helps them to get to the correct result faster and to make sure that it will be actually seamless when we transition between the physical one and the digital extension. This, uh, is another example of a set piece. So even if we have set pieces that were created for the show that we are not digitally extending directly, so this might be a normal piece that's in the physical set, and we don't make the digital copy of it. We still scan them if we can because it gives us a lot of information from the production's view. So these are usually hero props, what we call, which is the highest detail or the highest quality of props that they create.

Matej Kosovsky: And it has a lot of surface detail. So on the left side, you can see the, the concrete blocks have a specific detail that the production is matching to the historical, uh, accuracy. And if we are creating similar elements, we might take the texture or some details from these sets and make a variation based on that. So any information that we can gather on set helps our team globally to create digi-digital, uh, variations of it. Miniatures.

Matej Kosovsky: So miniatures were used on this project quite a lot, but usually the miniatures are not very, um-- They are not usually left untouched in the final picture. And in this case, they wanted to film the miniatures for the background scenes, but then they decided to not go that route and go with us with virtual production, and they were gonna be just thrown away. And instead of doing that, we actually used them to create our Unreal assets. So we scan them with highly detailed photogrammetry, and this way we could recreate the full miniatures that a team spent a lot of time and research to match the destruction that actually happened on the day. And we just scaled up the asset into our Unreal scene.

Matej Kosovsky: And we didn't remodel it. We didn't have to do any destruction simulations or anything. We used the scans fro-- of the maquettes, of the miniatures directly in Unreal. So in the end, the red zone is the physical set, and the cutout is our digital extension. So both on the left and the right side are scanned miniatures just scaled up and then touch up with a little bit of detail so that it holds up on the screen.

Matej Kosovsky: As you can see, there was a lot of debris around. Uh, even here you can see they are used to blend the practical sets and the virtual sets that we have. So it's kind of hides the transition between the two. So our production created a lot of these debris elements. They were various sizes, maybe from small to very large, may-- up to three meters.

Matej Kosovsky: And we scan them in bulk so that we can use them anywhere in the environments and fill in the details in the backgrounds as well very quickly. And usually, how that would go is that we would have a warehouse set up with all the props and all the elements that we need to scan. Uh, we would be doing a photogrammetry with a cross polarized setup. It's very efficient, it's very quick, and you can access all the angles. So the props would be sep-- maybe every two meters, they will be separated, kind of like social distancing during COVID.

Matej Kosovsky: And that will allow us to go quickly around the prop from any angle, and then we switch them. So maybe we have five at a time set up, and in a day we can scan maybe-- one person can scan sixty props or so. So props are the most common things to be scanned. They are usually unique to the show. Uh, whether you are doing a sci-fi show of things that don't exist or a time-specific period like this project, it's very hard to find or recreate accurate reference, uh, assets for, for that.

Matej Kosovsky: So you have to take whatever the production created. And here is a interesting example on the right side with, uh, covered victims. So these were actually, uh, extras that were pulled on the side during, uh-- between takes, and the production designer just put them on the f-- on the blankets, just asked them to lie down, cover them, stage them, and then we had to quickly, quickly scan them. It took maybe thirty seconds to do the scan. Uh, we did few variations of that, and we tried to do it so quick because it was very uncomfortable for the people to lie down underneath.

Matej Kosovsky: It was a very hot day. Uh, and you might ask, "Why not just take a photo?" So if you take a photo, if you send it a f-- a few months later down the line, someone in different country, an artist is working on it, and he will interpret that photo in his view. And then when the vi-visual effects supervisor sees whatever he created, he might have some notes. So they go back and forth and back and forth until the visual effects supervisor is happy.

Matej Kosovsky: Then the visual effects supervisor shows it to the client, which is the production designer who staged this to begin with, and it might be different. So they go back all the way to the artist, and you have more iterations there. If you scan the reference to begin with, you give that to the artist. It's kind of set in stone, like the ground truth is known, and the artist has some more information to work with. He doesn't have to guess.

Matej Kosovsky: And that will save you a lot of time, of course. And you still have creative freedom because you might need to change it, but you da-- do that as a conscious choice on the creative end. And the last, uh, use case that we have for scans is, uh, virtual production support scans. These are mainly LiDAR-based, and they are not used in the final picture. Uh, we scan every set that is filmed on our stages, and this data is always there if we need it further in post-production.

Matej Kosovsky: The most common type of, uh, virtual production support scan is, uh, alignment scan. This is a scan that, uh, scans the physical build. You can see the walkway on the right side. So when we have a piece of set in the stage like this, we need to make the digital extension, as I mentioned before. And we need to align the physical, real asset that's placed inside of the volume to the dig-digital one.

Matej Kosovsky: And if we scan it, we actually can take the scan into our Unreal environment and overlay it on top. So then we can match the digital one we already created to match perfectly with the physical one. And that will ensure that when you look through the camera, there is no gap and continuous transition between the two assets, between the physical and the digital one. And this is usually done one day before filming. We call it a blend day, and it is to ensure that we have the latest position, and it's accurate because sometimes if you do it three days before, the one day before, they still move it, and then you have to redo it again.

Matej Kosovsky: And after the scan is captured, it needs to be processed immediately, sent to the team so they can adjust their Unreal environment, and then the next morning it's-- the filming starts. And it's a simple in theory, but if you don't have this step, sometimes you can't film at all because if you have the gap and misalignment between the two assets, uh, you can't film it because it would-- you would just have to replace everything in post anyway. And we are trying to make, uh, in-camera visual effects finals with this. So all of this was done in two thousand twenty-three, and since then, as we've seen many times in the last two days, uh, a lot has changed in our industry, especially with the new technologies like Gaussian splats and NeRF. We can do better reconstructions and faster.

Matej Kosovsky: Uh, in the last year, my focus was on improving the efficiency of capturing the environments that we work on, and today, I would like to showcase our Tor-Tori solution in this video.

Video: Setting new standards for how environments are captured and recreated, Pixo's 3D reconstruction technology uses Gaussian splats to transform real world and synthetic data into accurate digital twins. One of the advanced rigs built by the Pixo team is a versatile camera array called Tori. By taking photos simultaneously in different directions, Tori helps you cover a location quickly. The rig accurately captures all angles of a location to archive and store a slice of reality as a digital twin. The scale this flexible technology creates is endless, unlocking the potential for anyone to efficiently capture and customize their own unique scenes and share cutting-edge transmedia content.

Matej Kosovsky: So I didn't know if we were gonna have enough time for questions, so I prepared in advance and anticipated some of the questions. So most of them that people asked me in the last few days was, "Are we running Gaussian splats in virtual production wall?" Yes, we do. And, "Have we used Gaussians in a real production already?" Yes, we did.

Matej Kosovsky: And, "Was this captured with a Tori rig?" No, it was not. But this was. And we can run the Gaussian splats in Unreal in real time on any of our stages. This is in, uh, Stage seven in Culver City in Los Angeles.

Matej Kosovsky: An interesting thing might be that we-- Why do we need an array of like sixty-one megapixel cameras and that resolution? Is that even necessary? That might be another question. Well, this wall has, uh, thirty-two different segments. Each of them is 4K.

Matej Kosovsky: So it's actually more resolution that the sphere has in Las Vegas, and you don't run the full resolution and the full on the whole wall all the time. You only run whatever the camera sees. But the movies require a lot of accuracy and a lot of, uh, flexibility in the quality of the data that they get. So we always try to push the boundaries higher and higher, and currently, the Gaussians are optimized to be lower resolution, but it will, it will get there. And this is a-another example.

Matej Kosovsky: Actually, this was captured by one person under twenty minutes using a three-camera Tori setup, and this is crucial for us. This speed of capturing is crucial for film. We usually face huge time restrictions on locations, and that's why this is a game changer for us. Uh, I have truly enjoyed the, all the sh- presentations that you guys had, uh, in the last two days and learned from you. I hope that you got something from my presentation as well, and thank you for your attention.

Host: Ladies and gentlemen, Matej Kosovský. Good job.

Matej Kosovsky: Thank you.

Host: I really enjoyed how you handled the Q&A yourself. Love that. Uh, if the audience is going to have any further questions, are you going to still be around?

Matej Kosovsky: Yes, of course. Of course.

Host: Perfect. So make sure to ask further questions to Matej. Thanks a lot.

Speaker

Matej Kosovsky

Matej KosovskyLinkedIn

Digital Twin Technical Director at PIXOMONDO

Currently Head of Capture at Pixomondo, I lead the internal scanning department, merging VFX, virtual production, and innovation. With a blend of technical expertise and creative vision, I bring digital worlds to life—from on-set scanning to post-production. I thrive on collaboration and new tech breakthroughs.

Hear the next ones live – 3DISE 2027, 20–21 April, Prague

Buy tickets