The Virtual Patient

Precisely this openness is what gives Gründer’s vision its bold quality, because the healthcare metaverse currently resembles a smorgasbord of apps from different providers that are not connected with each other. “That’s why it’ll take at least a decade before we see a shared platform,” he predicts. But until that happens, business can still be conducted successfully in the parallel universes of the world today.

Together with Doob, a German 3D specialist that went through T-Systems’ accelerator program, Gründer and his colleagues have already built numerous metaverses for companies in the healthcare sector. “We create walk-in spaces for our clients in the virtual world,” he says. “They’re either completely new or some type of replica of hospitals or offices.” Users can enter them only as their avatars, of course. To get an avatar, they have a scan made of their face and place it on one of a range of different bodies on offer. If they do not want a ready-made selection, they can opt for Doob’s full-body scan. Roughly the size of a telephone booth, the scanner generates a digital twin of the person in around 30 seconds. The German national soccer team, for example, has already used the Doob scanner for purposes of its own.

Generally speaking, however, the people who have themselves scanned by Doob are healthcare experts. They recently included consultants from Barmer Ersatzkasse, one of Germany’s major health insurance companies, for which T-Systems and Doob have created virtual sales spaces. “Many health insurance companies are slowly losing touch with the generation below around 25 years of age,” notes Gründer in sketching their predicament. Barmer wants to win them back with the help of its branch office in the metaverse. “It makes Barmer stand out,” he says, “and offers potential young customers an intriguing and appealing alternative to conventional insurance consultations.”

Wheelchairs, too, have gone through Doob’s scanner. “Anyone who needs a wheelchair sets off on an odyssey to a host of different providers,” explains Gründer. “But now, they and their advisors can generate avatars and meet in a virtual showroom to choose a suitable model together.” All kinds of different “spaces” can be transferred to the virtual realm. Together with Essen University Medical Centre (UME), Gründer and his colleagues have sent an entire magnetic resonance imaging (MRI) system into the 3D world of the metaverse. UME’s aim was to train its employees. “With hospital personnel no longer occupying the cost-intensive high-tech system for their training, the virtual MRI system pays off in about a week already,” he says. In any event, Gründer sees education and training for doctors and other healthcare professionals as key fields of activity that can be shifted to the metaverse with an eye to cost and user benefits.

These are the kinds of individual applications currently available in the healthcare metaverse, says Gründer. In order to transform them into his vision of a shared meta-village accessible to everyone, policymakers first have to prepare the ground. “If a platform of this type is to arise in Europe, potential providers will need greater access rights to healthcare data,” he says. If the platform is located in Germany, patients could well give it the benefit of the doubt. “As far as German consumers are concerned, it’s surely better to have sensitive healthcare data stored on servers in their own country than with international tech companies.”

Policymakers are not the only players needed to enable a breakthrough for the metaverse. “The biggest problem in my opinion right now are the patients,” says Gründer. In fact, very few of them think of the metaverse when they have health problems or need medical advice. And even then, some would ask why they should go online if they need an appendix removed in real life. “We therefore have to do a lot more explaining of what the metaverse actually is and how each of us can benefit from using it.”

Not least of all, Gründer considers the requisite equipment to pose a hurdle for large groups of potential users. Devices currently on the market are expensive, not always easy to operate, and in some countries not available at all. Business settings are therefore prime candidates for the use of VR headsets, he notes, especially for online events and training. And, in fact, some companies—not least of all the BPV Group—have already entered this new world and are handling their entire hiring process and life-cycle management via VR or AR systems.

Gründer is convinced that the headset problem will gradually resolve itself. As time goes by, the accessibility bottleneck will widen and enough accessories will go onto the market. As he concludes, “Anyone who starts using the devices for personal reasons—even if that’s only for computer games—will later take their headsets into the offices and virtual consulting rooms of the healthcare metaverse.”

Böckenfeld now knows a good deal more about the “healthverse,” as she calls it. That includes knowing that a comprehensive healthverse is far off in the future. “But everything that is still fiction in one moment can become reality in the next,” she says. And adds, “Technology is developing exponentially these days.” In fact, much of what might have been dismissed as science fiction several years ago is now possible thanks to VR technology. In August 2022, Brazilian surgeons succeeded in separating a pair of conjoined twins in a 27-hour operation. Virtual replicas of the twins had previously been made from CT and MRI scans, and they enabled surgeons to practice the unusual procedure over a period of months in a near-realistic setting. The team was supported by numerous international experts who could enter the virtual space from anywhere in the world. Noor ul Owase Jeelani, one of the surgeons, was so amazed by the VR technology that he described it in a post-operation interview with the BBC as “man-on-Mars stuff.”

Examples like this show the potential the virtual world offers for healthcare in particular. It transcends spatial and temporal limits and brings experts together to develop solutions that benefit patients. All of that is already reality. But Böckenfeld criticizes the fact that the healthverse currently consists primarily of a large number of individual applications that are not connected with each other. “What’s missing is interoperability—the healthcare ecosystem as a whole is not interconnected.”

Granted, international tech companies would love to create a platform to host all the applications and virtual reality solutions. But in Böckenfeld’s view, that would be only the second-best option. “It would be better to entrust an independent global organization with providing the healthverse’s digital infrastructure to all the players,” she says. Even though that in itself wouldn’t solve all the problems facing an all-encompassing “healthcare metaverse.”

Like many innovative projects in the healthcare sector, the global healthverse would also be powered to a high degree by data. “That’s problematic because each country has different rules on how to handle data,” she explains. Not only that—each patient’s data are generally distributed across numerous sites. They are stored with different doctors, at hospitals or medical centers, and on smartphones and wearables. “In the future, however, every user will have to see themselves as the owner of their data, and consolidate them in a single place.”

That type of place could be a “digital wallet,” says Böckenfeld. The data gathered there would reflect the complete life cycle of the respective owner. It would represent the essence of what Böckenfeld calls their digital identity. “The identity could be used to create a digital twin, for example, which in turn can help predict what would be beneficial in a specific situation or perhaps harmful.” These data are what would make it meaningful to enter the global healthverse in the form of one’s avatar. Not least of all, the data are what the healthcare apps need in order to create value for their patients.

Bartl notes that the term “metaverse” has multiple layers of meaning. He himself views it primarily as a merging of the real world with data. “That process yields new and impressive forms of interactive data visualizations ,” he says, “which can be used as the foundation for collaboration and decision-making.” Given the high importance Bartl ascribes to the metaverse, it’s no wonder the main thing he wants from it is more. ”The more data we have, the better the apps we can develop.” And the more data patients bring to their treatments, the more effectively the apps can be put into practice. In a sense, he says, the metaverse is like “a space to be filled with data.”

But Bartl contrasts this narrow definition of the metaverse with a broader one, namely the vision of an all-encompassing parallel digital world. Such visions are found largely in the entertainment world, he notes. The questions there are along the lines of whether it’s possible to attend concerts virtually, or embark on virtual journeys. The focus is on shared experiences in a virtual space. In a healthcare context, however, the idea of an all-encompassing metaverse makes no sense. “In a world with specific health-related problems, metaverse applications need to focus in targeted form on particular illnesses or treatment methods,” he explains. “And I don’t think that’s going to change in the near future.”

Brainlab has shown with numerous apps that commercial success is possible within the narrow definition of the metaverse. The Munich-based company specializes in technologies designed to improve the work involved in surgical operations. One example would be the Virtual Technical Guides from its subsidiary Level Ex. The underlying app provides a three-dimensional interactive model of the person to be operated on. By also showing the instruments that can be used, it helps OR personnel prepare for the procedure. The technology is soon expected to be used during the surgery itself, with 3D model visualizations of the next steps. But do these properties actually make it a metaverse app? Without a doubt, says Bartl. “Users interact in real time with a 3D model, and the app allows parallel access by external specialists.” Interaction and collaboration: there they are again—Bartl’s two defining criteria for the metaverse.

Metaverse applications can also be helpful in earlier stages of preparation for surgery, for example in radiology. Brainlab is developing 3D anatomical models from MRI and CT scans. With the help of holograms viewed in augmented reality (AR) headsets, doctors can show patients how operations will proceed in realistic detail. “A deeper level of understanding lowers patient anxiety,” says Bartl in explaining the benefits for all concerned. “It promotes acceptance and instills confidence in the recommended procedure.”

Once generated, the 3D model can also be used later during the operation itself. Critical structures requiring special attention, such as neural pathways, can be superimposed in the eyepieces of OR microscopes. Established practice since the turn of the millennium, this process has nothing to do with the metaverse, however. Bartl notes it is used by thousands of clients. Brainlab is currently working on the ability to take the entire anatomical hologram into the OR—ideally enriched with even more useful data.

Despite the comprehensive imaging options for 3D models and the different ways of interacting with them, not all experts are open to these types of solutions. “Many doctors continue to work with 2D cross-sectional images and use them as the bases for their decisions,” says Bartl. A real breakthrough for the metaverse as defined by Brainlab would therefore be to have three-dimensional understanding integrated into the training at medical schools. “Unfortunately that’s only rarely the case. Usually the residents only encounter 3D representations of patient imaging data very late in their clinical training, often in fact with our apps.”

Bartl hopes that will change in the near future and that metaverse applications will soon be shaping medical training as well. In his view, this type of integration would not only improve university medical education but also democratize it. “Metaverse apps will make the relevant expertise available everywhere, not only to students at specialized and often expensive institutions,” he predicts. This is a vision that would improve the chances of successful treatment for a great number of patients—regardless of what happens with the all-encompassing version of a metaverse platform for everyone.