ChatGPT: Promise and Peril

Many large com­pa­nies are look­ing to intro­duce Chat­G­PT for pur­pos­es such as doing research, brain­storm­ing, or gen­er­at­ing texts. Some are already using the lan­guage-based model, includ­ing in cus­tomized form. They are hop­ing arti­fi­cial intel­li­gence (AI) can reduce employ­ee work­loads and increase pro­duc­tiv­i­ty. But the chal­lenges asso­ci­at­ed with improv­ing busi­ness process­es and train­ing per­son­nel in the pro­fes­sion­al use of their new dig­i­tal co-work­er are some­times underestimated.

The first step, as every­one would agree, is to acquire a thor­ough under­stand­ing of the chatbot’s basic prin­ci­ples, abil­i­ties, and lim­its. Here, an inter­dis­ci­pli­nary team of data ana­lyt­ics and AI experts from the Porsche Con­sult­ing man­age­ment con­sul­tan­cy answers 11 burn­ing ques­tions about Chat­G­PT and offers solu­tions based on expe­ri­ence with clients.

Dana Ruiter: “Chat­G­PT is what’s known as a chat­bot — basi­cal­ly a lan­guage-based model that uses arti­fi­cial intel­li­gence to sim­u­late lan­guage as used by peo­ple. Chat­G­PT has learned how to answer ques­tions in ways that close­ly resem­ble human respons­es. Its answers already seem very nat­ur­al. But we shouldn’t deceive our­selves: it is a lan­guage model, not a knowl­edge model. And please note that pro­vid­ing fac­tu­al­ly cor­rect infor­ma­tion is not the aim of this model. That being said, large-scale lan­guage mod­els do dis­play aston­ish­ing abil­i­ties to abstract. They can han­dle a wide array of tasks with­out hav­ing been trained to do so by devel­op­ers. These large mod­els have made quite a splash not only in AI research cir­cles, but also in the busi­ness world.”

Dana Ruiter: “There’s a pre-train­ing peri­od, in which the sys­tem uses enor­mous amounts of data from the inter­net to learn how human lan­guage is struc­tured. Then there’s a peri­od of fine-tun­ing when it learns how to per­form cer­tain tasks. ChatGPT’s devel­op­ers showed it a vast num­ber of ques­tion-answer pairs, from which it learned how to sim­u­late answers to ques­tions. The final learn­ing peri­od con­tin­ues to this day. Human feed­back tells the sys­tem which answers it gen­er­at­ed were use­ful and which weren’t. That hap­pens every time some­one enters a ques­tion in Chat­G­PT and then rates the answer as good or bad. User input is enabling the sys­tem to for­mu­late its answers in ever more suit­able ways.”

Dana Ruiter: “Large lan­guage mod­els have a spe­cial qual­i­ty: The more com­plex they become, mean­ing the more train­ing data and neu­rons they acquire, the more inde­pen­dent abil­i­ties they are able to dis­play. They can sud­den­ly do amaz­ing things they haven’t been explic­it­ly trained for. For exam­ple, I could ask Chat­G­PT to trans­late an infor­mal Ger­man email into Shake­speare­an Eng­lish. And it can actu­al­ly do that, although it sure­ly didn’t see that ques­tion dur­ing its train­ing peri­od. Lan­guage mod­els can do this because of the dif­fer­ent lev­els of abstrac­tion they learn. Sim­i­lar to a brain, actu­al­ly. The low­est level has very basic infor­ma­tion like the mean­ings of words, for instance. On inter­me­di­ate lev­els the words are placed into con­text, and the ulti­mate level con­sists of abstrac­tion. If you ask the sys­tem to trans­late a casu­al Ger­man email into Shake­speare­an Eng­lish, it’s already famil­iar on the abstract level with the con­cepts of Ger­man, Shake­speare­an Eng­lish, and trans­la­tion, so it knows how to for­mu­late the text. This is quite impres­sive, from a sci­en­tif­ic stand­point as well, and was hard­ly con­sid­ered pos­si­ble just sev­er­al months ago.

But here, too, cau­tion is the name of the game. If the sys­tem doesn’t com­plete­ly under­stand a term, or if a term inad­ver­tent­ly sends it off in the wrong direc­tion dur­ing a task, it can pro­duce incor­rect results. These are known in the field as ‘hal­lu­ci­na­tions.’ They make it hard to apply gen­er­a­tive AI mod­els on a sta­ble basis.”

Dana Ruiter: “Many fields will ben­e­fit great­ly from gen­er­a­tive AI. There’s a lot of poten­tial espe­cial­ly in areas with high per­son­nel costs such as health­care and law, and also in admin­is­tra­tion. The impor­tant thing here is to auto­mate repet­i­tive tasks and to let experts con­cen­trate on mat­ters that require their direct atten­tion and knowl­edge. Let’s take med­ical exams, for exam­ple: doc­tors can use speech recog­ni­tion sys­tems to fill out files in advance, which gives them more time to actu­al­ly talk with their patients. It’s essen­tial, of course, that experts ver­i­fy every­thing at the end and decide what actions should then be taken.

The legal pro­fes­sion can also ben­e­fit from gen­er­a­tive AI. The mod­els can already col­lect infor­ma­tion rel­e­vant to a spe­cif­ic ques­tion from mass­es of legal doc­u­ments, and sum­ma­rize it as well. Freed from this type of rou­tine work, lawyers can apply their ana­lyt­i­cal skills to more com­plex issues.

For the media the prob­lem with gen­er­a­tive AI is that it’s not cur­rent­ly con­nect­ed with world events and is there­fore not in a posi­tion to auto­mat­i­cal­ly pro­duce news reports that are truly inno­v­a­tive and fac­tu­al­ly accu­rate. That of course doesn’t stop crim­i­nal inter­ests from using AI to gen­er­ate false news and flood social media with it.”

Dana Ruiter: “It’s cru­cial to keep in mind that Chat­G­PT is a lan­guage model, not a knowl­edge model. Sci­en­tif­ic work con­sists of observ­ing the world and its prop­er­ties, and eval­u­at­ing the knowl­edge there­by acquired in light of the pre­vi­ous canon of knowl­edge. Chat­G­PT can do nei­ther of these things: it can’t observe the world, nor can it repro­duce the exist­ing canon of knowl­edge in fac­tu­al­ly accu­rate form. What it’s very good at, how­ev­er, is sim­u­lat­ing the style of a sci­en­tif­ic paper. This can be help­ful in the for­mu­la­tion stage of a paper once the research results are in and the lit­er­a­ture search has been done. But it does not replace the sci­en­tif­ic work per se.”

Dana Ruiter: “Results can be manip­u­lat­ed with the help of what’s called prompt engi­neer­ing. Once again, the prob­lem lies in the system’s lack of trans­paren­cy and the output’s lack of robust­ness. The sys­tem isn’t trans­par­ent because it’s unclear which terms in the prompts have which effects on the sen­tences gen­er­at­ed. It’s pure trial and error. And the lack of trans­paren­cy means these mod­els are not robust: a prompt that pro­duces the desired result one time can, with slight­ly mod­i­fied input, lead to unde­sir­able results oth­er­wise. That makes it espe­cial­ly hard to inte­grate gen­er­a­tive AI mod­els into pro­duc­tion sys­tems. After all, no one wants the sys­tem to start spout­ing hate speech or other unde­sired con­tent due to a trig­ger in the user input.”

Kevin Lin: “Chat­G­PT is often used as a search machine right now, but I think its main appli­ca­tion will be pro­cess­ing doc­u­ments in auto­mat­ed form. One exam­ple would be clas­si­fy­ing com­plaints in order to con­nect cus­tomers with the right advi­sors or com­pa­ny rep­re­sen­ta­tives. Anoth­er would be iden­ti­fy­ing and clear­ly sum­ma­riz­ing rel­e­vant admin­is­tra­tive infor­ma­tion, and yet anoth­er would be improv­ing indi­vid­ual writ­ing styles. All these skills are rel­e­vant to many dif­fer­ent sec­tors and can eas­i­ly gen­er­ate sub­stan­tial added value. Solu­tions based on gen­er­a­tive AI will be par­tic­u­lar­ly appeal­ing to fields with high per­son­nel costs or a short­age of work­ers, such as health­care, the legal pro­fes­sion, or admin­is­tra­tive areas.”

Kevin Lin: “The work­ing world will change. The good thing here is that spe­cial­ized work will still need to be done by qual­i­fied indi­vid­u­als. Chat­G­PT can­not replace how a physi­cian inter­acts with patients or how a lawyer for­mu­lates an air­tight con­tract. The abil­i­ty to inte­grate spe­cial­ized exper­tise with knowl­edge about the world in gen­er­al, and to link it with sit­u­a­tion­al and social­ly aware actions, will remain the domain of well-trained human spe­cial­ists for quite a while. Repet­i­tive tasks that con­sis­tent­ly fit a cer­tain pat­tern, how­ev­er, will be replaced to an increas­ing extent. For ser­vices like pro­cess­ing com­plaints or per­form­ing admin­is­tra­tive tasks, Chat­G­PT will offer con­sid­er­able lever­age in terms of increas­ing effi­cien­cy and requir­ing fewer albeit high­ly qual­i­fied personnel.

Gen­er­a­tive AI will elim­i­nate some tasks in the future, that’s clear. Employ­ees will there­fore need not only the spe­cial­ized skills they already com­mand, but also ever greater famil­iar­i­ty with both the abil­i­ties and lim­its of these gen­er­a­tive mod­els. There’s a big need here for fur­ther train­ing under expert instruc­tors and lead­ers, right at com­pa­nies. That’s the only way to ensure that gen­er­a­tive AI can be applied in advan­ta­geous and depend­able ways.”

Dana Ruiter: “Lots of peo­ple con­fuse Chat­G­PT with some type of search engine. But that’s not what it is. Chat­G­PT has only learned to sim­u­late answers to ques­tions. The goals of its train­ing do not include ensur­ing that these answers are fac­tu­al­ly accu­rate. Nor can Chat­G­PT ensure that the sen­tences it gen­er­ates always meet the expec­ta­tions of the peo­ple who entered the prompts. Here’s an exam­ple: Let’s say I use Chat­G­PT to cre­ate a tool that helps clients for­mu­late emails in a more pro­fes­sion­al style. I test my tool, and it seems to work well because the email mes­sages it refor­mu­lates sound com­pe­tent and polite. How­ev­er, when I hand over the tool to real users who enter all kinds of infor­ma­tion, they com­plain that it neglects parts of their orig­i­nal texts or invents new con­tent. In other words, there’s no guar­an­tee that the sys­tem will always act as antic­i­pat­ed. This is why high­ly sen­si­tive pro­duc­tion sys­tems often stick with rule-based process­es, which pro­vide cer­tain guardrails for clean and cor­rect out­put. In the future we’ll be see­ing more hybrid sys­tems that com­bine gen­er­a­tive AI with rule-based mod­els in order to make the results more robust.”

Fabi­an Schmidt: “Chat­G­PT from Ope­nAI is cur­rent­ly just a chat­bot based on a lan­guage model. This type of tech­nol­o­gy is not unique. And there are plen­ty of com­peti­tors. In addi­tion to closed-source mod­els such as Bard from big tech com­pa­nies like Google, there are also spe­cial­ized region­al solu­tions from start-ups. In 2022, the Porsche Con­sult­ing man­age­ment con­sul­tan­cy advised Aleph Alpha, an AI start-up from the south­west­ern Ger­man city of Hei­del­berg, in con­nec­tion with the Ger­man Entre­pre­neur Award (Deutsch­er Grün­der­preis). Lumi­nous, the lan­guage model from Aleph Alpha, is espe­cial­ly rel­e­vant for the Ger­man mar­ket and has a Euro­pean focus. Its mod­els are host­ed in Ger­many and meet the high Euro­pean data pro­tec­tion stan­dards. How­ev­er, there are two major prob­lems with many closed-source solu­tions. Their mod­els are often not eas­i­ly adapt­able, and data and model sov­er­eign­ty are fre­quent­ly in exter­nal hands. It’s espe­cial­ly dubi­ous if they’re out­side Europe and there’s no way of con­trol­ling how the data are processed. That’s why it’s often a good idea for com­pa­nies to devel­op their own solu­tions using open-source mod­els. Some solu­tions already exist for large-scale lan­guage mod­els, such as LLaMA (Meta) and BLOOM (Hug­ging Face). They’re host­ed local­ly and can be adapt­ed at will, which in turn allows full con­trol of the model and the data. If small­er lan­guage mod­els are suf­fi­cient — for more basic tasks such as clas­si­fy­ing cus­tomer com­plaints or spot­ting trends on social media — then there’s an entire range of open-source lan­guage mod­els such as BERT, XLM, and Camem­BERT. Por­tals like Hug­ging Face pro­vide free access to these mod­els, along with sup­port func­tions to facil­i­tate their adap­ta­tion to spe­cif­ic use cases.”

Fabi­an Schmidt: “As con­sul­tants, the first thing we do with our clients is see whether there’s already a con­crete prob­lem that AI should solve. Or whether the goal is to use gen­er­a­tive AI to dis­cov­er new and inno­v­a­tive AI solu­tions. In the lat­ter case it helps to have a work­shop with spe­cial­ists or to use Porsche Consulting’s Inno­Lab in Berlin. As soon as we’re clear on the con­crete use cases, the deci­sion­al process­es for how to imple­ment them can begin.

One ques­tion that needs to be answered right away is whether it even makes sense to auto­mate a par­tic­u­lar task with gen­er­a­tive AI. If the answer is yes and an AI solu­tion will save money and time, the next step is to see whether there’s a good ‘off-the-shelf’ solu­tion already on the mar­ket. It’s impor­tant to note that these solu­tions don’t have to come from big play­ers like Microsoft or Google. Their solu­tions tend to be based on very broad usabil­i­ty, which might not be right for a given use case. Small­er com­pa­nies and estab­lished start-ups fre­quent­ly pro­vide very good solu­tions for spe­cif­ic use cases and sec­tors. To gain a good overview here, we rec­om­mend doing mar­ket analy­ses that focus on fac­tors like data pro­tec­tion, tech­ni­cal sophis­ti­ca­tion, and scalability.

If a com­pa­ny can­not find a ready-made solu­tion, how­ev­er, it needs to devel­op its own. At Porsche Con­sult­ing we work with our clients to clar­i­fy the data land­scape. Our team’s experts then anonymize and process the data. Only then do we apply the actu­al model to the use case and refine it togeth­er with the client in mul­ti­ple feed­back rounds. For com­pa­ny-spe­cif­ic devel­op­ments, we place a pre­mi­um on data and model con­trol, because no one wants their data to fall into exter­nal hands. Open-source mod­els that are local­ly host­ed and eas­i­ly adapt­able often offer the surest road to solid com­pa­ny-spe­cif­ic developments.”

Direct con­tact to Porsche Consulting’s Arti­fi­cial Intel­li­gence and Data Ana­lyt­ics divi­sion: ai@porsche-consulting.com