A former neurotechnology researcher at Apple has published the whole truth about the Vision Pro

Sterling Crispin, a former Apple employee, published in his Twitter account a long post about Vision Pro. This text is of particular interest because it contains personal experience of using the product, unlike the standard advertising propaganda created by marketers and PR-specialists with their visual effects.

Some excerpts from the text are even more impressive than the impressions left by yesterday's presentation. Translation of the tweet:

"I spent 10% of my life developing VisionPro while I was working at Apple as a neurotechnology prototype researcher in the Technology Development Group. This is the longest time I have ever worked on a single project. I am proud and relieved that it has finally been announced. I've been working on AR and VR for ten years now, and in many ways this is the culmination of the entire industry in one product. I'm grateful to have helped make this a reality, and I'm available to consult and take calls if you're looking to enter this space or refine your strategy.

The work I was doing contributed to the seminal development of Vision Pro, mindfulness experiments, products ▇▇▇▇▇▇, and more ambitious neurotechnology research. For example, predicting what you're going to click on something before you do it, essentially mind reading. I worked there for 3.5 years and left at the end of 2021, so I can't wait to see how the last two years have come together. I'm very interested in what made the list and what will be released later.

In particular, I am proud to have contributed to the initial vision, strategy and direction of ▇▇▇▇▇▇ for Vision Pro. The work I've done as part of a small team has helped give the green light to this product category, and I think it could one day have a significant global impact.

Most of the work I did at Apple is under NDA and covers a wide range of topics and approaches. But some things have come to light through patents, which I can quote and paraphrase below.

In general, much of my work involved determining the mental state of the users based on data from their bodies and brains when they were in an immersive experience.

So, the user is in mixed reality or virtual reality, and AI models try to predict whether you are experiencing curiosity, mind wandering, fear, attention, recalling past experiences, or some other cognitive state. This data can be obtained through measurements such as eye tracking, electrical activity in the brain, heartbeat and rhythms, muscle activity, blood density in the brain, blood pressure, skin conductance, etc.

Many tricks have been used to make specific predictions possible, as detailed in several patents I appear in. One of the coolest results was predicting what the user was going to click on something before they did it. It was tremendous work that I'm proud of. Your pupil reacts before you press the button, partly because you expect something to happen after you press it. So you can create biological feedback to the user's brain by tracking the user's eye behavior and redesigning the user interface in real time to create more of that anticipatory pupil response. It's a crude brain-computer interface through the eyes, but very cool. And I'd prefer it over invasive brain surgery any day.

Other tricks for determining cognitive state include quickly flashing visual images or sounds that the user may not perceive and then measuring their reaction to them.

Another patent details the use of machine learning and body and brain signals to predict how focused, relaxed, or good you are at learning. And then updating the virtual environment to reinforce those states. Imagine an adaptive immersive environment that helps you learn, work, or relax by changing what you see and hear in the background.

All of these details are available in patents, and they were carefully written to make sure nothing leaked out. There were a ton of other things I was involved in, and hopefully many of them will eventually see the light of day.

Many people have been waiting for this product for a long time. But this is still one step forward on the road to VR. And it will take until the end of this decade for the industry to fully realize the grand vision of this technology.

Again, I'm open to consulting and taking calls if your business wants to enter this space or improve its strategy. Mostly, I'm proud and relieved that this is finally being announced. It's been more than five years since I started working on this project, and I've spent a good part of my life on it, as has an army of other designers and engineers. My hope is that the whole will be greater than the sum of the parts, and Vision Pro will blow your mind.