Jeffrey Martin, co-founder and CEO of Mosaic, traces a career from gigapixel and 360 photography, through 360cities.net and the hard lessons of the Sphericam camera, to building 360-degree mobile mapping cameras shipped to more than 40 countries. Jeffrey shows how 360 imagery feeds Gaussian splats and point clouds, compares photogrammetry with lidar, proposes an AI model that understands every pixel of a city, and announces a free research dataset of 1.3 million images of Prague.
“It's an inherently, uh, geographic medium, a 360-degree image. It can only describe one place.”
“I would say 360 video was kind of a solution looking for a problem.”
“In general, I would say that photogrammetry is, is eventually, uh, going to win”
Jeffrey Martin expects photogrammetry to eventually win: images from cheap cameras can give denser, even sub-millimeter point clouds, while lidar remains a useful supporting technology with more failure modes and higher cost.
Yes. Jeffrey Martin showed a Gaussian splat of a Prague square captured in about ten seconds with a Mosaic camera and processed in Nerfstudio, and credited global shutter sensors for capture at high speed.
Yes. According to Jeffrey Martin, Mosaic released 1.3 million RTK-positioned images of central Prague, one every meter, free for non-commercial research under a Creative Commons share-alike license.
Transcribed automatically (ElevenLabs Scribe) and lightly edited. It may contain small errors. Click a time to play the video from there.
Jeffrey Martin: Thank you. Hi, everybody. Uh, thank you to, uh, Michal Gula and all, all the other organizers of the conference. It's been great so far. So, um, I'm here to tell you about my general experiences in my life and career and, uh, what has led me to, uh, to where I am today.
Jeffrey Martin: Um, my name is Jeffrey Martin. I am the co-founder, CEO of Mosaic. Uh, we make, uh, cameras and associated hardware and software for, uh, mapping and 3D scanning. Um, so I came from, uh, the world of 360 degree photography. Um, I don't know if there's a video.
Jeffrey Martin: Okay. Um, you can turn down the audio if you want. Um, so I came from the world of 360 photography. Uh, is anyone here a 360 photographer? Few.
Jeffrey Martin: Cool. All right. Some of you I've known for a while. Um, so, uh, I discovered 360 photography in, like, 2003. Got my first digital camera, um, and I was like, "Wow, you can like, you know, join images together."
Jeffrey Martin: These are not like film images. These are fungible. They are, you know, malleable, and you can, uh, just, uh, join together as many as you want. You can make a 360, or you can like, you know, zoom in and take, like, you know, you can take an infinite number of photos almost and join them all together. So that's what I did, actually.
Jeffrey Martin: Um, started doing gigapixel photography. This is Tokyo. Uh, shot this in 2014 from, um, the top of the, uh, Tokyo Tower. Uh, here is a taxi driver. I don't think he saw me, but I saw him.
Jeffrey Martin: Um, so what I like about gigapixel photography is, uh, it's an extraordinary amount of information, um, but you can show it and share it online, uh, in its full quality , which, uh, is not really the case with 3D scanning, uh, which is still has-- There's sort of a gap in the technology between the, um, capture and, and the delivery, uh, which we'll get to later. But with, uh, gigapixel photography, it's kind of, um, you know, mature enough, both the capture, the stitching, post-production, and web display. Um, so you have like a, you know, full, uh, continuum from, like, the information that you, that you gather and, and what you can show. So that to me is great. Um, and 360 photography and gigapixel photography is, is now a mature technology as such.
Jeffrey Martin: Uh, I don't think 3D scanning is mature yet in this way, but it's getting there. Um, so besides doing this stuff, uh, shot lots of these. I like to-- Each time I go to a new city, I like to try to find a, you know, tall building and shoot one of these things. Um, also started, uh... This was Prague 360.
Jeffrey Martin: I started this in 2005. Uh, the, uh, you can see at the top right, or it's pretty small, there's the QTVR of the week. Who knows what a QTVR is? One person. Andy.
Jeffrey Martin: Hey, Andy. Uh, so a QTVR was a 360 degree photo back in the early 2000s. It was a QuickTime VR, uh, started by Apple. Um, that was at the time the-- one of the only web plugins that was, uh, uh, possible to install and to in or- in order to let you see a 360 degree photo online. So, um, this website, this was before Google Maps, um, and Google Street View.
Jeffrey Martin: So the original website was in Flash. Does anyone remember Flash? That yucky, like, web plugin. Um, so originally it used Flash, um, and we had to find a map. Finding a map was hard.
Jeffrey Martin: Um, so I thought this would like sell itself. I thought I would like shoot panoramas of Prague, hotels and bars and, like, it would sell itself, and every restaurant in town would want to... No, that was not the case. Nothing sells itself. Uh, nobody even knew what this was.
Jeffrey Martin: No one knew what 360 imagery was. They'd never seen one before. Um, but, uh, as a, as a platform, it took off and it, it, it grew into a company. Um, it helped when Google Street View came out and people became familiar with the technology. Uh, it became 360cities.
Jeffrey Martin: net, which is, uh, still online. It's a, uh, licensing platform. It's a publishing and licensing platform for extremely high quality, high resolution 360 degree images. The images are licensed to, uh, advertising agencies, um, uh, journalism, uh, education, and things like that. Um, in, uh, 2015, I started a company called Sphericam.
Jeffrey Martin: Um, 2015 was kind of an interesting time. Uh, it was after Facebook bought Oculus, and then all of a sudden, everyone-- all anyone could talk about was VR. Um, now, you know, it's all AI. Everything's AI. Everyone's doing AI.
Jeffrey Martin: You gotta put AI into everything. Even if you-- doesn't have anything to do with AI, you gotta- You know, ram it in there somehow. So it was the same with, uh, VR in, in, in 2015. Uh, so, you know, you've got these goggles and, uh, you know, this is gonna be bigger than the iPhone. Uh, everyone's gonna wanna be, you know, everyone's gonna be using this, and you need s- you need content for VR, so you need cameras, you need 360-degree cameras.
Jeffrey Martin: That's gonna be the next big thing in photography. You're gonna take a 360 photo, and you'll be able to look around as if you were there. Um, that actually didn't turn out to be the case, and nobody really wants to, um, you know, wear VR headsets and look at 360 photos, not at least as many people as we thought there would be. But at the time, this was a, this was a very big hype. Um, did a Kickstarter campaign, raised half a million dollars on Kickstarter, went to CES, uh, raised some additional money from investors, and started a real company.
Jeffrey Martin: We had this, uh, magical device. It was, uh ... looked and felt magical. However, the ultimate, um, engineering of that device turned out to be much harder than, uh, anyone imagined. Um, you can catch me for a beer later, and I'll tell you about the whole slow-motion train wreck that was Sphericam.
Jeffrey Martin: It was a, you know, um, a horrendously painful learning experience. Um, it ultimately did not result in a mass-produced, uh, prosumer-grade spherical video camera, as we had hoped. Uh, we did build and ship a number of them, but it didn't turn out to be economically viable based on what we had designed for many, many reasons. Um, so that company didn't survive. Uh, but it was, uh, kind of ...
Jeffrey Martin: Uh, I wasn't done with it, and I didn't feel like, uh, just because, you know, the use case of 360 imaging for VR didn't turn out to be economically viable or, or the, you know, product itself, um, I, I ... I've always known that the medium of 360 imaging is, is important and very useful. Uh, and ever since the beginning, I've found it to be a, you know ... It's an inherently, uh, geographic medium, a 360-degree image. It can only describe one place.
Jeffrey Martin: It cannot describe any other. If you take a picture of, you know, the Eiffel Tower, you know, you could probably take that photo from a few different places and have the same thing. But a 360-degree photo doesn't work that way. By definition, it is one place. So it's a geographic kind of medium.
Jeffrey Martin: And, um, as I figured out how to reboot Sphericam into a, into a viable company, it was like, you know, um, let's build something that people want. Let's build something that has an actual, you know, business use case today. Let's not build something for consumers 'cause that's very difficult. Um, s- you know, selling something to businesses, uh, uh, is, is ... you know, can be, uh, less of a challenge for many reasons.
Jeffrey Martin: Um, so mobile mapping, uh, I think was kind of the obvious choice. I'd been in touch with the, uh, Google Maps, Google Street View team, uh, already for quite a long time and understood a lot of their, you know, desires and requirements. Um, and, you know, mobile mapping has been around for a long time, since the 1980s. Uh, there's been TeleAtlas, TomTom, uh, CycloMedia has been driving around with a, you know, giant fisheye lens sticking through the roof of their car in the, in the, you know, early to mid-'80s, I think. Um, and that, that's something that has been, uh, I think all ...
Jeffrey Martin: for a very long time, um, a growing medium and something that's been very useful. But it's still, um, you know ... Even today, we're encountering companies that are moving into the 21st century in terms of their technology. They're moving away from, you know, someone walking around with a clipboard taking notes to, you know, automated data collection of, uh, laser data, image data, and, and, and, you know, doing this, you know, surveying and inspection in, in the office. Um, so again, 360-degree video, uh, it's kind of, kind of an obvious evolution of 360-degree photography, but, um, the way we thought we were going to consume it, like 10 years ago, that you'll have this 360 video, and you'll have a headset, and you can, you know, watch it however way w- That's, that's not how it's turned out to be.
Jeffrey Martin: Um, the actual use case has been you can shoot a 360 video, and then you can frame it. You can, you know ... You just pick the, um, spot where you want the camera, and then you can point the camera later in post-production, and you can create a normal video that can be shared on normal platforms, but you can do the post-production later. So, um, I would say 360 video was kind of a solution looking for a problem. Uh, but it, it has found that, but it's taken many years to do that.
Jeffrey Martin: Um, so on to, on to Mosaic. Uh, we founded Mosaic. We have been, um, successfully, uh, building, designing, building, shipping, um, 360-degree cameras to more than 40 countries worldwide. And the initial use case of this was like a Google Street View type of device. And, you know, when we're driving around, we're, you know, doing lots of testing all the time in Prague.
Jeffrey Martin: We're driving around. Everyone's like, "You're Google." And we're like, "No, we're not, not Google. You're not gonna be in Google Maps tomorrow. Sorry."
Jeffrey Martin: But, um, you know, uh, it's, it's always the first thing that people think. Um, but besides the 360-degree images themselves, these cameras are very, uh, am- amenable and useful for creating, uh, all kinds of 3D data very quickly and, you know, easily. So- Especially when you view the data from the, you know, perspective where it was shot. Obviously, the car is traveling on, uh, you know, near the surface of the ground, which is also the, uh, kind of a human scale, uh, perspective. The 3D data can be, uh, very rich, detailed, uh, with, with, with high fi-fidelity.
Jeffrey Martin: Um, it allows people to collect 3D data where drones are not allowed or where, um, you know, it might take a lot of permission to, to fly a drone. Um, and it's possible to get many of the same types of data that you would get from drones, but without the permissions and at a, at a, you know, much lower cost, much higher speed. Um, we've also been busy making NeRFs and 3D Gaussian splats. Uh, here's a NeRF-- uh, this is a 3D Gaussian splat actually, uh, of, uh, Vaclavske Namesti in Prague. So this small dataset was collected in about 10 seconds using our, our camera.
Jeffrey Martin: Um, yes, you're not supposed to drive here, but I-- you know, whatever. I- ... uh, I was careful. We're always careful. Um, but so this was-- this data was collected in about 10 seconds.
Jeffrey Martin: Um, it was aligned, run through Nerfstudio, and we have this amazing, um, 3D Gaussian splat. You can see the things on the roof there, those thin objects, the shiny objects. You, you know the story. Um, it's pretty amazing, and it's great at this conference to see the kind of tool chains for, for this to be, um, you know, evolving quickly 'cause we need that. It's, uh...
Jeffrey Martin: Um, so back to the subject of, uh, gigapixels and how you can actually experience the full fidelity of what you're collecting. I think Gaussian splats are that thing, and that's why everyone's so excited about it, 'cause finally you don't have to reduce your, uh, model from, um, you know, a billion triangles to, like, 10,000 triangles to fit it onto a, you know, um, mobile experience. You can, you know, get something very close to what you actually captured and deliver it to people in, in really amazing quality. And, you know, look at these trees. Look at the background.
Jeffrey Martin: It's, it's just so cool. And, you know, I think this is really, um, helping to mature the overall tool chain a lot. Um, so again, with, uh, you know, our capture hardware, we can, you know, capture all the, all the things that you need. Um, meshes, textured meshes, point clouds. Um, we can extract features from the images or from the point clouds.
Jeffrey Martin: Um, a-and as we move towards, like, AI, ev- you know, everyone's shoving AI into everything. Um, but w- uh, one thing, and I've talked about this idea with a few friends for, like, more than a year now, so, um, I, I wanna just be open with it and see if someone can do it. So, um, we've got large language models. Um, and yes, we've got AI for, you know, detecting things in point clouds or images, and that's fine, and it's good. Um, but what, what I'm really interested in the idea of is, like, if you take something like a large language model, which is a, you know, distillation of, like, everything on the internet.
Jeffrey Martin: So do something similar with images of, of a, of a whole city or region. And, you know, just like, you know, imagine when you first started using, um, ChatGPT or something, you didn't really know what to ask it. And after, like, a year or so, people are getting used to, you know, figuring out, like, what it is useful to ask. So imagine if you, you know, had a way to understand everything about a place, like, every possible detail about a place. What would you ask it?
Jeffrey Martin: I don't know. Uh, here are a couple ideas. Like, um, you know, where are the oldest cars in this town? Or, you know, um, you know, show me the places with poor visibility due to vegetation. Or I don't know what.
Jeffrey Martin: Like, there are gonna be a lot of interesting things that you can query of a place if you are able to, you know, um, ingest the, the, the full, you know, dataset, every pixel of, of a place. Like, every pixel of a place from, from every perspective. And, uh, I hope someone builds that 'cause that's, I think, waiting to be built, and someone's gonna do it. I'm just-- uh, I hope they do. I'm l-looking forward to that.
Jeffrey Martin: Um, photogrammetry versus LiDAR. Of course, we, uh, integrate LiDAR with our cameras. Um, there are strengths and weaknesses. In general, I would say that photogrammetry is, is eventually, uh, going to win, although LiDAR is-- I'm, I'm sort of-- you know, come from the world of photography. I think LiDAR is a good supporting actor, but it's, um, it has so many fail-failure modes in the operation and, and processing of it that, uh, it's, it's, it's expensive.
Jeffrey Martin: It was the original way to collect 3D data, but I think photogrammetry is kind of, you know, catching up. You can create point clouds with images or with LiDAR. Uh, we can do both. Our customers do both. We have some customers who create point clouds from the images, and it's, uh, much, much cheaper, um, especially for the hardware.
Jeffrey Martin: It can be-- cost more in the processing or take longer, but sometimes that doesn't matter. Um, you can often get a much denser point cloud f-from images than you can with, uh, even very expensive LiDAR. You can get sub-millimeter, extremely dense, uh, point clouds using just, like, you know, cheap cameras. So it seems like it's catching up. Um, but of course, LiDAR is always extremely useful still, and it will also continue to evolve.
Jeffrey Martin: Um- So one thing that we have done as we've tested our cameras is we've, uh, obviously you can't drive around your office neighborhood every day forever. You go crazy. So we mapped the whole city of Prague. We did this with, uh, our high-resolution camera. We did it with RTK.
Jeffrey Martin: We have centimeter-accurate positioning. We have one image every meter of most of the center of Prague. I'm pretty sure this is a denser data set of imagery, uh, than exists anywhere, and, uh, we've released it for free for research purposes. It is under Creative Commons share-alike licensing, so if you want to do something with the data that is not commercial, you can grab all one point three million images and do stuff with it, and please do. Please do something with it and tell us about it.
Jeffrey Martin: Um, you know, everyone who has, you know, learned how to do this stuff, uh, couldn't do it without things being available, and it's, you know, I, I want to, you know, give something back, and so we're, we're making this available to everyone. I hope you get a chance to try something with it. Um, I don't know how much time I have left. Is there-- Okay. Um, this is a example of the data set.
Jeffrey Martin: So this is one camera output. It's about one hundred and twenty by ninety degrees. Again, it's, uh, one image every meter. It wasn't tr-- um, captured at a constant frame rate, but it was triggered once every meter using RTK, so, um, that will give you a better density and, you know, accuracy of the images. Um, just a couple of, uh, quick pet peeves.
Jeffrey Martin: The word distortion. I've always been using fisheye images. Uh, people say they're distorted or the, you know, concept of distortion is I think maybe is a little bit of a misnomer. Um, distortion rever- refers to the deviation from your mathematically ideal, um, lens model. So pretty much any lens is, is distorted, although, uh, a fisheye image is-- fi- fisheye image is, um, has an equal resolution, uh, across the frame, whereas a wide-angle, uh, image, uh, gets kind of stretched out, which in some cases could, uh, mean that it is more distorted.
Jeffrey Martin: So that's just, uh, one kind of little pet peeve I would like to get out of the way. Rolling shutter versus global shutter. Uh, our came-- our cameras use global shutter sensors, which, um, makes this not an issue. That's why we're able to capture data, uh, at a very high speed, and why we were able to capture that, um, Gaussian splat in, in, in ten seconds. Um, kind of run out of time here, so I'm gonna skip through that and, uh, say thank you very much.
Jeffrey Martin: Good to be here.
Host: Ladies and gentlemen, let's give it up for Jeffrey. Thank you so much for guiding us through.
Jeffrey Martin: Thank you.
Host: Thanks so much. Thank you.

Founder and CEO at Mosaic
I’m Jeffrey Martin, CEO of Mosaic and a pioneer in 360° imagery and mobile mapping. From founding 360cities to creating record-breaking panoramas and VR cameras, I now lead Mosaic, building cutting-edge tools like Mosaic X and Xplor for mapping, AI, VFX, and digital twins.