The Virtual Patient

Pre­cise­ly this open­ness is what gives Gründer’s vision its bold qual­i­ty, because the health­care meta­verse cur­rent­ly resem­bles a smor­gas­bord of apps from dif­fer­ent providers that are not con­nect­ed with each other. “That’s why it’ll take at least a decade before we see a shared plat­form,” he pre­dicts. But until that hap­pens, busi­ness can still be con­duct­ed suc­cess­ful­ly in the par­al­lel uni­vers­es of the world today.

Togeth­er with Doob, a Ger­man 3D spe­cial­ist that went through T‑Systems’ accel­er­a­tor pro­gram, Grün­der and his col­leagues have already built numer­ous meta­vers­es for com­pa­nies in the health­care sec­tor. “We cre­ate walk-in spaces for our clients in the vir­tu­al world,” he says. “They’re either com­plete­ly new or some type of repli­ca of hos­pi­tals or offices.” Users can enter them only as their avatars, of course. To get an avatar, they have a scan made of their face and place it on one of a range of dif­fer­ent bod­ies on offer. If they do not want a ready-made selec­tion, they can opt for Doob’s full-body scan. Rough­ly the size of a tele­phone booth, the scan­ner gen­er­ates a dig­i­tal twin of the per­son in around 30 sec­onds. The Ger­man nation­al soc­cer team, for exam­ple, has already used the Doob scan­ner for pur­pos­es of its own.

Gen­er­al­ly speak­ing, how­ev­er, the peo­ple who have them­selves scanned by Doob are health­care experts. They recent­ly includ­ed con­sul­tants from Barmer Ersatzkasse, one of Germany’s major health insur­ance com­pa­nies, for which T‑Systems and Doob have cre­at­ed vir­tu­al sales spaces. “Many health insur­ance com­pa­nies are slow­ly los­ing touch with the gen­er­a­tion below around 25 years of age,” notes Grün­der in sketch­ing their predica­ment. Barmer wants to win them back with the help of its branch office in the meta­verse. “It makes Barmer stand out,” he says, “and offers poten­tial young cus­tomers an intrigu­ing and appeal­ing alter­na­tive to con­ven­tion­al insur­ance consultations.”

Wheel­chairs, too, have gone through Doob’s scan­ner. “Any­one who needs a wheel­chair sets off on an odyssey to a host of dif­fer­ent providers,” explains Grün­der. “But now, they and their advi­sors can gen­er­ate avatars and meet in a vir­tu­al show­room to choose a suit­able model togeth­er.” All kinds of dif­fer­ent “spaces” can be trans­ferred to the vir­tu­al realm. Togeth­er with Essen Uni­ver­si­ty Med­ical Cen­tre (UME), Grün­der and his col­leagues have sent an entire mag­net­ic res­o­nance imag­ing (MRI) sys­tem into the 3D world of the meta­verse. UME’s aim was to train its employ­ees. “With hos­pi­tal per­son­nel no longer occu­py­ing the cost-inten­sive high-tech sys­tem for their train­ing, the vir­tu­al MRI sys­tem pays off in about a week already,” he says. In any event, Grün­der sees edu­ca­tion and train­ing for doc­tors and other health­care pro­fes­sion­als as key fields of activ­i­ty that can be shift­ed to the meta­verse with an eye to cost and user benefits.

These are the kinds of indi­vid­ual appli­ca­tions cur­rent­ly avail­able in the health­care meta­verse, says Grün­der. In order to trans­form them into his vision of a shared meta-vil­lage acces­si­ble to every­one, pol­i­cy­mak­ers first have to pre­pare the ground. “If a plat­form of this type is to arise in Europe, poten­tial providers will need greater access rights to health­care data,” he says. If the plat­form is locat­ed in Ger­many, patients could well give it the ben­e­fit of the doubt. “As far as Ger­man con­sumers are con­cerned, it’s sure­ly bet­ter to have sen­si­tive health­care data stored on servers in their own coun­try than with inter­na­tion­al tech companies.”

Pol­i­cy­mak­ers are not the only play­ers need­ed to enable a break­through for the meta­verse. “The biggest prob­lem in my opin­ion right now are the patients,” says Grün­der. In fact, very few of them think of the meta­verse when they have health prob­lems or need med­ical advice. And even then, some would ask why they should go online if they need an appen­dix removed in real life. “We there­fore have to do a lot more explain­ing of what the meta­verse actu­al­ly is and how each of us can ben­e­fit from using it.”

Not least of all, Grün­der con­sid­ers the req­ui­site equip­ment to pose a hur­dle for large groups of poten­tial users. Devices cur­rent­ly on the mar­ket are expen­sive, not always easy to oper­ate, and in some coun­tries not avail­able at all. Busi­ness set­tings are there­fore prime can­di­dates for the use of VR head­sets, he notes, espe­cial­ly for online events and train­ing. And, in fact, some companies—not least of all the BPV Group—have already entered this new world and are han­dling their entire hir­ing process and life-cycle man­age­ment via VR or AR systems.

Grün­der is con­vinced that the head­set prob­lem will grad­u­al­ly resolve itself. As time goes by, the acces­si­bil­i­ty bot­tle­neck will widen and enough acces­sories will go onto the mar­ket. As he con­cludes, “Any­one who starts using the devices for per­son­al reasons—even if that’s only for com­put­er games—will later take their head­sets into the offices and vir­tu­al con­sult­ing rooms of the health­care metaverse.”

Böck­en­feld now knows a good deal more about the “health­verse,” as she calls it. That includes know­ing that a com­pre­hen­sive health­verse is far off in the future. “But every­thing that is still fic­tion in one moment can become real­i­ty in the next,” she says. And adds, “Tech­nol­o­gy is devel­op­ing expo­nen­tial­ly these days.” In fact, much of what might have been dis­missed as sci­ence fic­tion sev­er­al years ago is now pos­si­ble thanks to VR tech­nol­o­gy. In August 2022, Brazil­ian sur­geons suc­ceed­ed in sep­a­rat­ing a pair of con­joined twins in a 27-hour oper­a­tion. Vir­tu­al repli­cas of the twins had pre­vi­ous­ly been made from CT and MRI scans, and they enabled sur­geons to prac­tice the unusu­al pro­ce­dure over a peri­od of months in a near-real­is­tic set­ting. The team was sup­port­ed by numer­ous inter­na­tion­al experts who could enter the vir­tu­al space from any­where in the world. Noor ul Owase Jee­lani, one of the sur­geons, was so amazed by the VR tech­nol­o­gy that he described it in a post-oper­a­tion inter­view with the BBC as “man-on-Mars stuff.”

Exam­ples like this show the poten­tial the vir­tu­al world offers for health­care in par­tic­u­lar. It tran­scends spa­tial and tem­po­ral lim­its and brings experts togeth­er to devel­op solu­tions that ben­e­fit patients. All of that is already real­i­ty. But Böck­en­feld crit­i­cizes the fact that the health­verse cur­rent­ly con­sists pri­mar­i­ly of a large num­ber of indi­vid­ual appli­ca­tions that are not con­nect­ed with each other. “What’s miss­ing is interoperability—the health­care ecosys­tem as a whole is not interconnected.”

Grant­ed, inter­na­tion­al tech com­pa­nies would love to cre­ate a plat­form to host all the appli­ca­tions and vir­tu­al real­i­ty solu­tions. But in Böckenfeld’s view, that would be only the sec­ond-best option. “It would be bet­ter to entrust an inde­pen­dent glob­al orga­ni­za­tion with pro­vid­ing the healthverse’s dig­i­tal infra­struc­ture to all the play­ers,” she says. Even though that in itself wouldn’t solve all the prob­lems fac­ing an all-encom­pass­ing “health­care metaverse.”

Like many inno­v­a­tive projects in the health­care sec­tor, the glob­al health­verse would also be pow­ered to a high degree by data. “That’s prob­lem­at­ic because each coun­try has dif­fer­ent rules on how to han­dle data,” she explains. Not only that—each patient’s data are gen­er­al­ly dis­trib­uted across numer­ous sites. They are stored with dif­fer­ent doc­tors, at hos­pi­tals or med­ical cen­ters, and on smart­phones and wear­ables. “In the future, how­ev­er, every user will have to see them­selves as the owner of their data, and con­sol­i­date them in a sin­gle place.”

That type of place could be a “dig­i­tal wal­let,” says Böck­en­feld. The data gath­ered there would reflect the com­plete life cycle of the respec­tive owner. It would rep­re­sent the essence of what Böck­en­feld calls their dig­i­tal iden­ti­ty. “The iden­ti­ty could be used to cre­ate a dig­i­tal twin, for exam­ple, which in turn can help pre­dict what would be ben­e­fi­cial in a spe­cif­ic sit­u­a­tion or per­haps harm­ful.” These data are what would make it mean­ing­ful to enter the glob­al health­verse in the form of one’s avatar. Not least of all, the data are what the health­care apps need in order to cre­ate value for their patients.

Bartl notes that the term “meta­verse” has mul­ti­ple lay­ers of mean­ing. He him­self views it pri­mar­i­ly as a merg­ing of the real world with data. “That process yields new and impres­sive forms of inter­ac­tive data visu­al­iza­tions ‚” he says, “which can be used as the foun­da­tion for col­lab­o­ra­tion and deci­sion-mak­ing.” Given the high impor­tance Bartl ascribes to the meta­verse, it’s no won­der the main thing he wants from it is more. ”The more data we have, the bet­ter the apps we can devel­op.” And the more data patients bring to their treat­ments, the more effec­tive­ly the apps can be put into prac­tice. In a sense, he says, the meta­verse is like “a space to be filled with data.”

But Bartl con­trasts this nar­row def­i­n­i­tion of the meta­verse with a broad­er one, name­ly the vision of an all-encom­pass­ing par­al­lel dig­i­tal world. Such visions are found large­ly in the enter­tain­ment world, he notes. The ques­tions there are along the lines of whether it’s pos­si­ble to attend con­certs vir­tu­al­ly, or embark on vir­tu­al jour­neys. The focus is on shared expe­ri­ences in a vir­tu­al space. In a health­care con­text, how­ev­er, the idea of an all-encom­pass­ing meta­verse makes no sense. “In a world with spe­cif­ic health-relat­ed prob­lems, meta­verse appli­ca­tions need to focus in tar­get­ed form on par­tic­u­lar ill­ness­es or treat­ment meth­ods,” he explains. “And I don’t think that’s going to change in the near future.”

Brain­lab has shown with numer­ous apps that com­mer­cial suc­cess is pos­si­ble with­in the nar­row def­i­n­i­tion of the meta­verse. The Munich-based com­pa­ny spe­cial­izes in tech­nolo­gies designed to improve the work involved in sur­gi­cal oper­a­tions. One exam­ple would be the Vir­tu­al Tech­ni­cal Guides from its sub­sidiary Level Ex. The under­ly­ing app pro­vides a three-dimen­sion­al inter­ac­tive model of the per­son to be oper­at­ed on. By also show­ing the instru­ments that can be used, it helps OR per­son­nel pre­pare for the pro­ce­dure. The tech­nol­o­gy is soon expect­ed to be used dur­ing the surgery itself, with 3D model visu­al­iza­tions of the next steps. But do these prop­er­ties actu­al­ly make it a meta­verse app? With­out a doubt, says Bartl. “Users inter­act in real time with a 3D model, and the app allows par­al­lel access by exter­nal spe­cial­ists.” Inter­ac­tion and col­lab­o­ra­tion: there they are again—Bartl’s two defin­ing cri­te­ria for the metaverse.

Meta­verse appli­ca­tions can also be help­ful in ear­li­er stages of prepa­ra­tion for surgery, for exam­ple in radi­ol­o­gy. Brain­lab is devel­op­ing 3D anatom­i­cal mod­els from MRI and CT scans. With the help of holo­grams viewed in aug­ment­ed real­i­ty (AR) head­sets, doc­tors can show patients how oper­a­tions will pro­ceed in real­is­tic detail. “A deep­er level of under­stand­ing low­ers patient anx­i­ety,” says Bartl in explain­ing the ben­e­fits for all con­cerned. “It pro­motes accep­tance and instills con­fi­dence in the rec­om­mend­ed procedure.”

Once gen­er­at­ed, the 3D model can also be used later dur­ing the oper­a­tion itself. Crit­i­cal struc­tures requir­ing spe­cial atten­tion, such as neur­al path­ways, can be super­im­posed in the eye­pieces of OR micro­scopes. Estab­lished prac­tice since the turn of the mil­len­ni­um, this process has noth­ing to do with the meta­verse, how­ev­er. Bartl notes it is used by thou­sands of clients. Brain­lab is cur­rent­ly work­ing on the abil­i­ty to take the entire anatom­i­cal holo­gram into the OR—ideally enriched with even more use­ful data.

Despite the com­pre­hen­sive imag­ing options for 3D mod­els and the dif­fer­ent ways of inter­act­ing with them, not all experts are open to these types of solu­tions. “Many doc­tors con­tin­ue to work with 2D cross-sec­tion­al images and use them as the bases for their deci­sions,” says Bartl. A real break­through for the meta­verse as defined by Brain­lab would there­fore be to have three-dimen­sion­al under­stand­ing inte­grat­ed into the train­ing at med­ical schools. “Unfor­tu­nate­ly that’s only rarely the case. Usu­al­ly the res­i­dents only encounter 3D rep­re­sen­ta­tions of patient imag­ing data very late in their clin­i­cal train­ing, often in fact with our apps.”

Bartl hopes that will change in the near future and that meta­verse appli­ca­tions will soon be shap­ing med­ical train­ing as well. In his view, this type of inte­gra­tion would not only improve uni­ver­si­ty med­ical edu­ca­tion but also democ­ra­tize it. “Meta­verse apps will make the rel­e­vant exper­tise avail­able every­where, not only to stu­dents at spe­cial­ized and often expen­sive insti­tu­tions,” he pre­dicts. This is a vision that would improve the chances of suc­cess­ful treat­ment for a great num­ber of patients—regardless of what hap­pens with the all-encom­pass­ing ver­sion of a meta­verse plat­form for everyone.