The World at Eye Level: Mapping at Human Scale to Solve AGI

Talk · Jeffrey Martin · 3DISE 2026 · 6 May 2026 · 10:30 - 10:40 · 21 min

Chapters

What the talk is about

Jeffrey Martin, founder and CEO of Mosaic, looks at world models in the sense of embodied cognition: how brains perceive and act so well with little energy, and what machines would need to do the same. From Helmholtz and a 1963 kitten experiment to optical illusions, Jeffrey explains perception as prediction, argues that today's video data lacks what AI needs, and presents Mosaic's new body-worn capture device for building human-scale training datasets.

Key ideas

  • Jeffrey means world models in the sense of embodied cognition, not generative video: how a brain without language perceives, thinks and acts in the world.
  • Experiments such as the 1963 kitten carousel suggest vision only develops when coupled to body movement; the brain predicts top-down and uses the senses for error correction.
  • According to Jeffrey, YouTube-style video lacks geometric ground truth, multimodal and time-synchronized data, and consistent capture coupled to a human body.
  • Robotics has no large repository of training data the way LLMs had trillions of words, so Jeffrey sees human-scale capture devices as key to building it.
  • Mosaic's new body-shaped device combines RTK, PPK, lidar and six global shutter cameras to capture synchronized multisensor data usable for Gaussian splats and AI training.
“even a rat brain is, is tiny, um, and can do things that we haven't figured out and haven't solved.”
— Jeffrey Martin ·
“if you try to build a brain in a jar, it's not gonna work.”
— Jeffrey Martin ·
“I think scanning the world at scale in the proper way can help us unlock this mystery of the mind”
— Jeffrey Martin ·

Questions answered

How does Mosaic's new capture device compare with XGRIDS?

Jeffrey called XGRIDS a fantastic product and said the Mosaic device targets larger-scale captures with RTK, PPK, lidar and six global shutter cameras with one-inch sensors, so its cameras are higher spec.

How heavy is the Mosaic backpack capture system?

According to Jeffrey, it weighs about five kilos and is an enterprise-grade device priced like Mosaic's other camera systems, paying for itself after two or three jobs.

Can structured GIS context support AI reasoning?

Jeffrey thinks it probably can: priors are beliefs that are hopefully true, and injecting that knowledge into an embodied creature or machine can help.

Full transcriptabout 13 min readShowHide

Transcribed automatically (ElevenLabs Scribe) and lightly edited. It may contain small errors. Click a time to play the video from there.

Jeffrey Martin: Hi, everybody. We have more Gaussian splats for you. So, um, I know a few of you people here. Uh, some of you I've known for a very long time, uh, and there are a lot of interesting, great people that I'm meeting for the first time here. Um, some of you know me as, um, experimental photographer.

Jeffrey Martin: Some of you know me as, um, person running, uh, Mosaic. We make geospatial camera systems for capturing scenes like these. So for the past few months, I've been thinking a lot about how what we're doing intersects with what other people do-- are doing, especially in the, in the space of, of AI and of, uh, especially the, the AI that is not the LLM stuff, but the other stuff. Um, we, uh, we call it world models. There are generally a couple different things that we mean when we talk about world models, so I just wanna clarify.

Jeffrey Martin: Um, I'm not talking about generative video like Sora, um, or even about things like Marble from World Labs, which is a kind of immersive, uh, type of deal. I'm talking more about things relating to, uh, embodied cognition and what allows our minds to develop and function so well using, uh, remarkably few resources and so little energy. Um, just put it into context, a, a rat brain has maybe 200 million neurons and, uh, maybe a trillion synapses, and GPT-5 has, I don't know, 350 billion parameters or something like that. It's not apples to apples comparison, but, um, basically even, even a rat brain is, is tiny, um, and can do things that we haven't figured out and haven't solved. Like, nobody in the world has figured out how it is that the, the brain of an animal, uh, without using language is, is able to, to, um, perceive, think, and act in, in the, in the world in remarkably sophisticated ways.

Jeffrey Martin: So th-that's what I have been studying over the past few months to try to understand better. Um, and people have been thinking about this for a long time. For well over a century, people have been trying to figure out, like, you know, what it is our mind does. What, what do our eyes do? What, what, what exactly is going on?

Jeffrey Martin: Um, you have a baby who can do nothing, and then they eventually recognize faces. They have object permanence. They understand gravity. They, uh, you know, understand physics to the extent that they can throw things and then understand where they're gonna go. So there's something extremely sophisticated going on.

Jeffrey Martin: Um, and there are a lot of theories that are, you know, pretty plausible by now. Uh, so I would like to, I would like to get into those, and then I will get into how that intersects with, with, uh, what we're doing and what we're looking at here. Um, so, uh, Hermann von Helmholtz, he invented the ophthalmoscope, which is, um, um, a, a machine for looking, uh, in, into the eyeball, um, for ophthalmologists. Um, he said, "The sensations of the senses are tokens for our consciousness. It is left to our intelligence to learn how to comprehend their meaning."

Jeffrey Martin: So this was in 1867. So already someone is proposing that we're not actually seeing the world. We are using our eyes to see something, and then our brains are turning that into the thing that we perceive. And I think everyone's heard this idea before. Uh, it probably isn't very new.

Jeffrey Martin: Um, but it's, it's, it's really important. So fast-forward to 1963. We can, uh-- We're experimenting with kittens. So we've got, uh-- These kittens were harmed. Um, well, the one on the left was.

Jeffrey Martin: Um, so we've got two kittens in a carousel, and, uh, one is, one is walking and able to use his body, and the other one is, uh, kind of tied up, bundled up, and, uh, is not using its, its body. So as the, as the two kittens walk around, they both see the same thing. But one, one of the kittens is not using its body, uh, to correlate what it's seeing. And, and what happens is that, um, the visual system of the, of the kitten on the left doesn't develop. The kitten on the left is, is functionally blind.

Jeffrey Martin: Its eyes work, but it doesn't-- it can't-- it, it, it-- they're not connected to the rest of its brain, so it's functionally blind. So there's something going on where, uh, the eyes aren't the whole story, but they're connected to the body and, and there's, um, there's a process going on which require-- requires the body, um, for the eyes to actually work. Um, so generally the idea is that the brain is not recording, the brain is not doing this 3D reconstruction of the world all the time and then building up this model that we use to, to interact with the world. This is almost certainly not what's happening at all. There's very much a top-down-- You can call it a hallucination, you can call it a assumption.

Jeffrey Martin: There's, there's a top-down kind of idea of what we think is happening all the time, and we're only using our senses to do some error correction, and we're acting as a part of this error correction in, in order to understand what's, what's really going on. So, um, action and perception are really sitting right next to each other. Um, there are a lot of optical illusions to, to sort of show this. There-- You know, we get a lot of glimpses into what we think is happening from, uh, optical illusions, from people who have brain damage, from people who have other conditions such as schizophrenia, autism, things like that. Uh, this is a very classic optical illusion.

Jeffrey Martin: I'm sure you've all seen it. Uh, it doesn't matter how many times you look at it, uh, square A and square B look different. They look different, right? They're not different. They are the same.

Jeffrey Martin: Like, I promise you they're the same, and it doesn't matter how much you understand the fact that they are the same color. They look like different colors, and that's because your brain is just, uh, giving you this top-down, fairly reasonable assumption, and it, you know, almost all the time it's correct, uh, that square A and square B are, are different colors 'cause you know what a checkerboard is. You've internalized it deeply and, and that's just a fact, and it doesn't-- really doesn't matter how many times you see this. Um, there's other optical illusions like the-- you remember the blue dress, gold dress internet thing f- some years ago. Um, there's the hollow face illusion.

Jeffrey Martin: There's, there's a lot, a lot of optical illusions that kind of can give you a general idea that your brain is in fact making some very heavy predictions all the time, and even when your senses tell you that it's, uh, it's not that way, you, you still sort of see it that way. So, um, this classical idea of, uh, there's the world, you sense the world, and then your brain assembles it, this is probably not right. So you've got this cycle. Um, your brain's generating predictions. That is the main job of your brain.

Jeffrey Martin: Um, and this is much easier, it's much simpler, it's much less resource intensive than, you know, this bottom-up generating an entire picture of the world. You're generating predictions. The world deliver, delivers signals that, um, confirm or deny what's going on. Um, and these prediction errors go back to the brain to, um, to give you the, the hopefully true picture of, of what's happening. Um, the whole thing is hierarchical.

Jeffrey Martin: So, um, the system learns its own priors. You've got priors that are these kind of basic ideas of what's happening, and you've got precision weighting that decides what errors matter most. Sometimes you can just discard most of the information that's happening. In fact, most of the time most people are doing that. If you're driving home on a familiar route, you're not really needing to pay much attention at all to what's going on.

Jeffrey Martin: Uh, if you're driving in a, you know, foggy night on an unfamiliar road, uh, you're probably really tired afterwards 'cause you have to pay such extreme attention and actually reconstruct the world, you know, ad hoc from scratch all, you know, at every moment. So the general theory about what is probably going on is that we've got this loop that uses the same neurons. We've got perception, attention, and action. Um, when you imagine something, you're often using the same neurons as, as when you're doing it for real, and this has all been studied quite extensively. Um, action's not a response to the world.

Jeffrey Martin: Action is how you select the next sample of what you're doing, and therefore what you're, what you're thinking about. Um, so generally you can't really remove the, the body, the self, uh, from, from this, uh, mechanism. Um, if you try to build a brain in a jar, it's not gonna work. Just like with the kitten, if you take away the ability of the kitten to use its body, its eyes don't work. So the body is very, very central to, um, the way we're, we're, we're able to, to use our minds.

Jeffrey Martin: Um, and also the, the world itself is, is part of that. You can't really take away, uh, someone's body or, or the world outside and expect a, a mind to develop. They're very much connected to each other, um, which is generally not how a lot of, um, robotics and stuff like that is happening or, or how, you know, AI is, is, is being solved right now. Um, we're sort of trying to take the mind and, and put it into a world without having them develop together. So when we're talking about solving world models or solving embodied cognition, I'm talking about, you know, how can we build a machine that can act and perceive in the world the way we do?

Jeffrey Martin: You probably need some training data and the available training data like, okay, let's take YouTube. YouTube's got like, I don't know what, a billion hours every hour is being uploaded or something like that. It is, uh, you know, extensive and, and, um, it's, it's quite staggering amount of content, but it doesn't have a lot of stuff that could be pretty useful for, for training these things. It doesn't have a-- any kind of geometric world truth. It's not multimodal.

Jeffrey Martin: It's not time synchronized. It's sometimes captured at human scale, sometimes it's not. Sometimes it is, uh, you know, not moving the way the body moves, and it's not really coupled to, to a person in any way. Um, so what we're f- what we're thinking about here is, is how, how to build capture devices that come closer to, to this kind of data, which I think is going to be very important in the next step of solving this, this aspect of, of AI. Um, so what am I, what am I doing here and who am I and why am I talking about this?

Jeffrey Martin: So I, uh, I went through my photos collection and l- f- found all the pictures of me over the years. You can see by the, you know, lack of gray hair how long I've been doing this. I've been building various types of cameras attached to bodies, uh, creating human-scale data for, uh, a very long time. So the first picture's from 2008, I think. I, I showed this at a PanoTools conference.

Jeffrey Martin: A couple of you in the audience were, were there for that. Um, so I'm not a electrical engineer or a software engineer. I'm just, uh, someone with persistence in, uh, in building a human body scale camera system that can capture the world. So this turned into a project called Sphericam in t- 2015, 2016. Um, some of you are familiar with that project.

Jeffrey Martin: It, uh, was a small VR camera. Um, and that morphed into Mosaic, which is, uh, who we are today, and we've shipped cameras to 50 countries worldwide. Our cameras work really well. We have a lot of very happy customers doing, uh, geospatial and 3D, um, data collection for, for many different applications. Um, so about the training data, about how to build robots and how this is gonna come about.

Jeffrey Martin: Um, one of the co-founders of World Labs observed that robotics doesn't have a large repository of training data. It's not like LLMs. LLMs had all the words already there. We had trillions of words just there for the taking. Um, that is not the case for this.

Jeffrey Martin: Um, so that's why I would like to show our next product. Um, it's shaped like a body 'cause the body's the best shape, and it's great for capturing data at the human scale. Um, this is not the same as the amazing Gaussian splat datasets that we've been looking at that were made with, um, giant professional cameras. This is more-- We can build a th- a thousand of these and walk around and, and capture, uh, synchronized multimodal, multisensor data that can make these amazing Gaussian splats that I showed you earlier. Um, but it is with the idea that this can be part of building large datasets that can help to solve the next chapter of AI.

Jeffrey Martin: Thank you very much.

Host: Jeffrey, thank you so much.

Jeffrey Martin: Thank you.

Host: Thank you so much. We've got the questions from the audience. Thank you so much. So, uh, let's tackle them. The first one, uh, you talked a lot about how to understand our minds and, uh, brain.

Host: So can you elaborate on that? How understanding our minds can help us improve scanning?

Jeffrey Martin: Uh, well, I think it's maybe the other way around. I think scanning the world at scale in the proper way can help us unlock this mystery of the mind, which, to be clear, it, it, it's very much unsolved. I outlined some theories about the mechanisms, but what exactly is happening and, and how we can replicate that, we, we still don't know. But what's definitely really important is the visual aspect. This is extraordinarily important for any animal that doesn't use language, uh, to understand the world and do sophisticated things.

Jeffrey Martin: So I think it starts with scanning to help us understand the mind.

Host: Thank you so much. Uh, question about your solution. So where does your product stand against competition like XGRIDS? What is your unique selling point?

Jeffrey Martin: Um, so I don't wanna go into a like spec war or anything like that. XGRIDS is a fantastic product. Uh, they have a good and enthusiastic user base. They have a good, uh, post-processing pipeline. Um, this device is, is for larger scale types of captures, so it incorporates, uh, RTK, PPK, LiDAR.

Jeffrey Martin: It has six global shutter cameras- Uh, with one-inch sensors and great optics. I would say that would be the, like, you know, hardware specification-wise, the, the biggest difference between a system like XGRIDS, um, is that the cameras themselves are, um, quite a bit more higher spec and, and sophisticated. Um, and this can be for the, um, for the reasons that I was outlining before, I think it can be very important for this type of training data.

Host: Mm-hmm.

Jeffrey Martin: For generating Gaussian splats, I think it can help to generate maybe a higher level of detail if that is necessary.

Host: Cool. Let's take a look at one more question. How do you see enterprise systems of record influencing hierarchical prediction system? Can constructed-- Uh, I'm sorry. Can structured GIS context support AI reasoning?

Jeffrey Martin: Oh, thank you, Arkadiusz. I think so. Um, I think generally understanding the world as you perceive it and getting priors that are... You know, priors are basically like beliefs that you have which are hopefully true. So if you are able to kind of inject this knowledge, uh, into an embodied creature or machine, um, I, I, I think that probably can help, yes.

Host: Okay. Thank you so much. Last question: How heavy is the backpack or the, the, the product, and how much does it cost?

Jeffrey Martin: The backpack is, um, I think it's about five kilos. It seems to be about as heavy as my daily carry backpack when I have a laptop and a camera in it. You're welcome to try it out later. Uh, it costs similar to our other camera systems, so it's kind of an enterprise pricing. It's a few tens of thousands of euros.

Jeffrey Martin: Um, again, it's, it's a kind of enterprise-grade device, not a consumer-grade device, so for professionals who are doing this type of work, it's the kind of device that pays for itself after, you know, two or three such jobs. That's the general context of how you can think about that pricing.

Host: Perfect. And, uh, can people try it on at your stand?

Jeffrey Martin: Yes.

Host: Perfect. All right. Ladies and gentlemen, Jeffrey Martin. Jeffrey-

Jeffrey Martin: Thank you.

Host: Thank you so much.

Speaker

Jeffrey Martin

Jeffrey MartinLinkedIn

Founder and CEO at Mosaic

I’m Jeffrey Martin, CEO of Mosaic and a pioneer in 360° imagery and mobile mapping. From founding 360cities to creating record-breaking panoramas and VR cameras, I now lead Mosaic, building cutting-edge tools like Mosaic X and Xplor for mapping, AI, VFX, and digital twins.

Hear the next ones live – 3DISE 2027, 20–21 April, Prague

Buy tickets