Spatial Computing with Cathy Hackl

Cathy Hackl is a visionary thinker, Apple Vision Pro programmer, and co-writer of the forthcoming book, Spatial Computing: An AI-Driven Business Revolution.

Motley Fool host Deidre Woollard connected with Hackl for a discussion about:

  • How corporations, like Lockheed Martin and Lowe’s, are already leveraging spatial computing.
  • The present hurdles in creating applications for the Vision Pro.
  • Virtual air rights, digital fashion, and concerns regarding the future of spatial computing.

If you want access to full episodes of all The Motley Fool’s complimentary podcasts, visit our podcast center. For beginners in investing, take a look at our beginner’s guide to investing in stocks. A complete transcript follows the video.

This video was recorded on Feb. 24, 2024.

Cathy Hackl: Once we begin to have these devices that are scanning the physical world in real time, we start to establish what I term large vision models. Models of the physical world that comprehend our world and are refreshed almost instantly. That’s when it starts to become truly captivating and somewhat worrying, from a privacy viewpoint, to be honest.

Mary Long: I’m Mary Long and that’s Cathy Hackl, a visionary thinker, author, and an Apple Vision Pro programmer. Deidre Woollard connected with Hackl for a discussion about spatial computing. They talk about how this technology could revolutionize meetings, manufacturing, and remodeling a kitchen, what the commencement of mobile can teach us about this computing revolution, and the distinction between the technology and value in the Apple Vision Pro.

Deidre Woollard: Let’s discuss spatial computing. This is all about the Vision Pro. Apple wants us to believe this is the future of how we interact with technology. What is your perspective?

Cathy Hackl: I want to start by stating that I believe we need to take a step back, we’re going to talk about the excitement, of course, but not frame that conversation spatial computing around just one single device, and this is why. Because when you start to think about spatial computing, I think a lot of individuals are considering it in the context of only the Apple Vision Pro or some people might use it interchangeably with the term mixed reality. I am of a varying mindset. I don’t think that this is the same. Just like the internet is not the same thing as our mobile phone, I don’t think special computing is the same as one single device or one single technology.

What I mean by that is that spatial computing is an evolving form of computing that is 3D-centric and that employs technologies like AI, computer vision, mixed reality, and other technologies to mix virtual experiences in someone’s experience of the physical world into a human’s experience of the physical world. That being said, through spatial computing, we’re also enabling technology, computers, robots, devices, hardware to begin to navigate the physical world with us. It’s a fusion of various new technologies and about human-to-human communication, but also human-to-computer interaction and it is not about one single thing.

I would even go so far as to say this is a new field of technology and it is as groundbreaking as mobile computing has been for all of us. Spatial computing I think people need to take a step back and realize what we’re talking about here is the future of computing, the future of how we as humans will interact with technology in new ways, and not just one device or one single technology or even the metaverse and I know we’re going to get into that.

Deidre Woollard: I have to inquire, have you experienced or utilized the Vision Pro and how does that correlate with this overarching theory of spatial computing?

Cathy Hackl: Certainly have utilized the device. I am personally an Apple Vision Pro programmer, so I actually had an opportunity to experiment with the device before many people did. I just couldn’t discuss it publicly. Obviously, lots of NDAs and stuff, but now everything’s out in the open. I had experimented with it before. Obviously, when it was released to the public, I went and bought my personal unit and everything. I’ve definitely experienced the device. I think the device is magical. I think a lot of individuals, when they first experienced the device, they have this “aha” moment of awe of seeing technology in a new way in most of the people that I have noted doing the reviews or doing their demos.

I definitely have witnessed that throughout my career in technology when people experiment with these new technologies. That being said, I think the device, and I’ve mentioned this before, I think there is $3,500 worth of technology in the device. As of now, I don’t think there’s $3,500 worth of value to the regular consumer. There is value there for programmers like myself. Perhaps for enterprises not yet, but because it’s really early. It is a fantastic product. I think it is doing a lot of great things. It is extremely powerful, and another thing I think a lot of individuals don’t realize is how much artificial intelligence is being used in this device. It’s a version 1. It is the right product at the right time, to be honest, but it’s not a mass-market product yet.

Deidre Woollard: I love that you made that distinction between the value of it and the value to consumers. I’m curious, though, since you were a developer on this, how does developing for this vary with other experiences you’ve had before?

Cathy Hackl: This one is a bit different because, obviously, it’s really new hardware that very few people had access to. Even the development process, you have to have a computer that has at least an M2 chip, so M3, M2. I think that in itself prices out maybe some developers from creating for this. If you want to develop utilizing Unity, you have to have a Unity license. There are certain things I think right now that might make it tough for maybe a young kid in their dorm to develop for this. They could still develop using some of the native, let’s say AR Kit or some of the native Apple part of the Apple ecosystem. But if you want to develop something really robust, you’re going to need certain capabilities. So I would say that.

I will say, though, because the device is so amazing and truly blends that physical world and virtual experience together, the possibilities of what we can develop are truly mind-blowing. As a developer, it is both an exciting challenge and an opportunity to start to create experiences that have never been created, that nobody’s really thought of, and that take full advantage of that value that is in the device, in the technology within the device. I think once we have more developers creating amazing content, we’ll get to that point where there is going to be more value for the consumer, for the mass market. We’re just not there yet with that part, but that’s where the work of someone like myself, it’s facial dynamics, comes into play. We need to create this content, we need to push the limits of the hardware and the technology and truly create mind-bending experiences.

Deidre Woollard: Fantastic. Well, I’m curious about that because thinking about when the iPhone came out and there was not a lot of development. You had a few cool things you could do, but it was really two years later that ecosystem developed. Do you see a similar timeline happening here?

Cathy Hackl: A hundred percent. This is version 1 and I would say this is like the beginning of mobile. It’s really early. First of all, there might be a lot of apps available for the Vision Pro. That doesn’t mean that discovering them is easy or that finding them is easy within the Vision Pro ecosystem when you’re wearing the device, and that doesn’t mean that all these apps are using the full capability of what the device can do in spatial computing. I want to be very clear about that. It is early. We’re all testing and learning, even with the companies that I’m already working with, the early adopters. It’s about exploring the potential of what this can become. It is about what are the early learnings that you can start to take when you’re creating these new apps and new experiences.

Because if you look at spatial computing from the perspective that it is the evolution of computing and that it is what comes after mobile computing and you start to think about it’s not about just the device, but it’s about every surface or the physical world around you becoming a spatial interface, then it starts to get interesting. Another thing I’ve noticed with the companies I’m working with is that they’re also thinking long-term, and I tell them this. When you start to think about an iPhone 16, an iPhone 17, a Vision Pro 2, a Vision Pro 3 and other hardware that the ecosystem is going to throw at us, then it starts to get more interesting. Especially when you start to think about iPhones with more spatial capabilities, more spatial video, maybe some other things you might be able to do with the phone that you couldn’t do before, that’s where I think the companies and the mass market and more brands are going to be like, OK, so now maybe it’s something we should pay attention to.

That’s when you start to not be only about 400,000 headsets being sold, but about millions or billions of people with these devices at hand that it starts to have these facial capabilities. A lot of us are playing the long game here. That long game is moving really fast, though. So I do have to say that. When people asked me, do I put it at 5-10 years? I can’t tell you. I don’t think anyone really can tell you. I would have to have a vision into every one of these companies’ road maps to truly tell you is it five years from now, is it 10 years from now? I think we’ll have to see and there’s a lot of different parts that have to fall into place. This is the other part, which I think is interesting and people forget, is spatial computing has four components. It has the hardware, which obviously we as tech business people, we love overindexing on hardware, taking it apart, talking about all this stuff.

There’s the software. The software is where there’s going to be a lot of work that needs to be done to create this content and create value and experiences. The connectivity. When you start to think about all these devices, if everyone’s going to end up wearing glasses and all these devices connecting on edge and the types of connectivity we’re going to need. Another level of not even 5G, we’re going to need to take 6G and whatever comes next. Then there’s also the part of the data, all the information that these devices need to operate because they’re literally scanning the physical world almost in real time and then all the data they are going to produce. That obviously has a lot of implications. Long answer, but that’s what I think right now.

Deidre Woollard: [laughs] Well, as someone who invests in data center REITs, I’m like, “OK, I see something happening here that I’m interested in.” I also want to talk about so many people focus with spatial computing on the consumer aspect like games and videos and things like that, but there are a lot of business applications here, too. I feel like people are ignoring a little bit of that part. But with spatial computing, thinking about for me from the real estate aspect, you have tremendous capability for business, digital twins, building information modeling. Talk to us a little bit about some of the business applications.

Cathy Hackl: Yeah. I think that, obviously, a lot of people want to focus on consumer because that’s a little bit more exciting to talk about, shopping and all that. But I think from the business perspective, there’s already use cases that have astonishing results. I’ll give you an example, Lockheed Martin, for example, used the Microsoft HoloLens, which is in essence a special computing device, and they achieved a 93% reduction in costs on the creation of one part of the manufacturing process for the Orion space vehicle. That’s a 93% reduction in cost because they use spatial computing. So there’s already a few use cases out there that show you that this technology can be extremely powerful. Spatial computing plus AI, you’re starting to talk about revolutionizing a lot of different processes, whether it is manufacturing, whether it is real estate.

With this technology, you can actually put the device on and show someone in the physical world what that is going to look like, what that building is going to look like, where the exits are going to be. You’re going to be able to walk through a virtual building really, not in virtual reality, but in the physical world, seeing everything around you. That to me is extremely powerful. It’s also going to allow you to tour a place that you might not be able to go. You can do that in virtual reality, and there’s the Zillow app already in the Vision Pro.

But right now it’s more like 360 photos that you’re walking around, so it’s not that sense of presence just yet. But when we start to think about that, I think real estate from decorating your house, and Lowe’s is already doing that in their Vision Pro with the kitchen. They have a kitchen demo where you can decorate your demo. But actually seeing what something could look like in the physical world I think is really exciting, or understanding from a construction perspective what the physics of the space could do to the design, the physical design of that house.

I think there’s a lot that’s about to be impacted and because the devices, but because the physical world becomes a spatial interface, the physical world becomes where we see technology. We no longer have to see it in these little rectangles that we carry around with us or on our computers, it starts to be all around us. So I think that that opens up opportunities for real estate, for finance, for education, manufacturing, of course, HR, you name it. Just like mobile computing has impacted most industries, I believe spatial computing as the evolution of computing, of future compute, will impact almost every industry.

Deidre Woollard: Well, I want to talk a little bit about this co-presence idea because everybody wants a better meeting experience. When Meta first debuted some of the meeting experiences with the individuals with the legs, everybody loved to make fun of that. But it seems like we are getting to a place where maybe we’re getting closer to that. I was looking at what [Alphabet‘s] Google is doing with Project Starline, it feels like we’re getting closer to actually making these remote meetings feel a little more like we’re really together.

Cathy Hackl: I will tell you, I had a demo of Project Starline last year at the Code Conference, and I was blown away. You’re sitting in front of someone, you’re not wearing a device. It’s done through AI and cameras and you’re seeing someone through a screen, but the screen is showing you a version of them in 3D, and I was so impressed by the fidelity. I could see someone’s pores more than I would normally see them in person. I was like, “Wow, this is crazy.” Or I remember the person I was doing the demo with had an apple and I literally just wanted to grab the apple from them because they moved the apple forward like they were giving me the apple, of course, I couldn’t grab it, but those sorts of things. Something that really stuck with me in the presence part was the pores, but also I was wearing this ring, and you could see the detail on the ring. He could see the detail on the ring in such a way that was even better than eyesight. I don’t know how to explain it.

Deidre Woollard: Wow.

Cathy Hackl: The detail was so clear that he said, OK, this could be really powerful, from a presence perspective. Right now, for example, with the Vision Pro, you can use your personas, which is like a 3D representation of you. It’s starting to get a little better. The first couple of weeks very uncanny. It’s still very uncanny, but it’s starting to get a little bit better. I think they’re starting to improve that. I think we’ll start to get a little bit more high fidelity, let’s say, of how we look in these devices. But the idea of presence, I think that’s where it starts to get interesting. That’s where it starts to also become something of value and of interest to the mass market because one of the most powerful things I’ve been able to do on my device is look at the video I took during the holidays with my parents. My dad is turning 80 this year. He’s getting older. He’s very healthy.

But I was like, I’m just going to shoot as much as I can on my iPhone so I can have these facial videos of him reading Llama Llama Red Pajama to my kids, stuff like that that’s special. When I put the device on and I can see these experiences in 3D, in spatial video, it’s powerful. It’s powerful and I’m like, is this the future of the family photo? Is this the future of how we might retain some of those memories? So that’s where that idea of presence in real time but presence also after the fact I think could be very powerful and could become one of the reasons that someone like my dad might eventually, in a couple of years, get one of

Leave a Reply

Your email address will not be published. Required fields are marked *