ChatGPT: Promise and Peril

ChatGPT is coming to workplaces. But who commands sufficient knowledge of this language model’s functions, abilities, applications, and risks to make good decisions on whether and how to use it in professional settings?


Digital language models like ChatGPT mine the internet to generate texts that sound as though they were written by people, for purposes of research, classification, processing, and reformulation in different styles. Porsche Consulting/Thomas Kuhlenbeck

Many large com­pa­nies are look­ing to intro­duce Chat­G­PT for pur­pos­es such as doing research, brain­storm­ing, or gen­er­at­ing texts. Some are already using the lan­guage-based model, includ­ing in cus­tomized form. They are hop­ing arti­fi­cial intel­li­gence (AI) can reduce employ­ee work­loads and increase pro­duc­tiv­i­ty. But the chal­lenges asso­ci­at­ed with improv­ing busi­ness process­es and train­ing per­son­nel in the pro­fes­sion­al use of their new dig­i­tal co-work­er are some­times underestimated.

The first step, as every­one would agree, is to acquire a thor­ough under­stand­ing of the chatbot’s basic prin­ci­ples, abil­i­ties, and lim­its. Here, an inter­dis­ci­pli­nary team of data ana­lyt­ics and AI experts from the Porsche Con­sult­ing man­age­ment con­sul­tan­cy answers 11 burn­ing ques­tions about Chat­G­PT and offers solu­tions based on expe­ri­ence with clients.

Expert answers from Dana, Fabian, and Kevin

Dr. Dana Ruiter, Analytics Consultant at Porsche Consulting: ai@porsche-consulting.comPorsche Consulting
Fabian Schmidt, Associate Partner at Porsche Consulting: ai@porsche-consulting.comPorsche Consulting
Kevin Lin, Head of AI & Data Analytics at Porsche Consulting: ai@porsche-consulting.comPorsche Consulting

1. What would be the best way of describing ChatGPT?

Dana Ruiter: “Chat­G­PT is what’s known as a chat­bot — basi­cal­ly a lan­guage-based model that uses arti­fi­cial intel­li­gence to sim­u­late lan­guage as used by peo­ple. Chat­G­PT has learned how to answer ques­tions in ways that close­ly resem­ble human respons­es. Its answers already seem very nat­ur­al. But we shouldn’t deceive our­selves: it is a lan­guage model, not a knowl­edge model. And please note that pro­vid­ing fac­tu­al­ly cor­rect infor­ma­tion is not the aim of this model. That being said, large-scale lan­guage mod­els do dis­play aston­ish­ing abil­i­ties to abstract. They can han­dle a wide array of tasks with­out hav­ing been trained to do so by devel­op­ers. These large mod­els have made quite a splash not only in AI research cir­cles, but also in the busi­ness world.”

2. How does ChatGPT “know” so much? And who “feeds” it?

Dana Ruiter: “There’s a pre-train­ing peri­od, in which the sys­tem uses enor­mous amounts of data from the inter­net to learn how human lan­guage is struc­tured. Then there’s a peri­od of fine-tun­ing when it learns how to per­form cer­tain tasks. ChatGPT’s devel­op­ers showed it a vast num­ber of ques­tion-answer pairs, from which it learned how to sim­u­late answers to ques­tions. The final learn­ing peri­od con­tin­ues to this day. Human feed­back tells the sys­tem which answers it gen­er­at­ed were use­ful and which weren’t. That hap­pens every time some­one enters a ques­tion in Chat­G­PT and then rates the answer as good or bad. User input is enabling the sys­tem to for­mu­late its answers in ever more suit­able ways.”

3. Is the system clever enough to generate real “new content,” or is it simply recombining text passages that already exist?

Dana Ruiter: “Large lan­guage mod­els have a spe­cial qual­i­ty: The more com­plex they become, mean­ing the more train­ing data and neu­rons they acquire, the more inde­pen­dent abil­i­ties they are able to dis­play. They can sud­den­ly do amaz­ing things they haven’t been explic­it­ly trained for. For exam­ple, I could ask Chat­G­PT to trans­late an infor­mal Ger­man email into Shake­speare­an Eng­lish. And it can actu­al­ly do that, although it sure­ly didn’t see that ques­tion dur­ing its train­ing peri­od. Lan­guage mod­els can do this because of the dif­fer­ent lev­els of abstrac­tion they learn. Sim­i­lar to a brain, actu­al­ly. The low­est level has very basic infor­ma­tion like the mean­ings of words, for instance. On inter­me­di­ate lev­els the words are placed into con­text, and the ulti­mate level con­sists of abstrac­tion. If you ask the sys­tem to trans­late a casu­al Ger­man email into Shake­speare­an Eng­lish, it’s already famil­iar on the abstract level with the con­cepts of Ger­man, Shake­speare­an Eng­lish, and trans­la­tion, so it knows how to for­mu­late the text. This is quite impres­sive, from a sci­en­tif­ic stand­point as well, and was hard­ly con­sid­ered pos­si­ble just sev­er­al months ago.

But here, too, cau­tion is the name of the game. If the sys­tem doesn’t com­plete­ly under­stand a term, or if a term inad­ver­tent­ly sends it off in the wrong direc­tion dur­ing a task, it can pro­duce incor­rect results. These are known in the field as ‘hal­lu­ci­na­tions.’ They make it hard to apply gen­er­a­tive AI mod­els on a sta­ble basis.”

4. Which fields can benefit especially rapidly from ChatGPT? Can the media — whether online, print, or TV — make use of automated news reports? What about contracts for lawyers, medical reports for doctors, or marketing and sales materials for agencies?

Dana Ruiter: “Many fields will ben­e­fit great­ly from gen­er­a­tive AI. There’s a lot of poten­tial espe­cial­ly in areas with high per­son­nel costs such as health­care and law, and also in admin­is­tra­tion. The impor­tant thing here is to auto­mate repet­i­tive tasks and to let experts con­cen­trate on mat­ters that require their direct atten­tion and knowl­edge. Let’s take med­ical exams, for exam­ple: doc­tors can use speech recog­ni­tion sys­tems to fill out files in advance, which gives them more time to actu­al­ly talk with their patients. It’s essen­tial, of course, that experts ver­i­fy every­thing at the end and decide what actions should then be taken.

The legal pro­fes­sion can also ben­e­fit from gen­er­a­tive AI. The mod­els can already col­lect infor­ma­tion rel­e­vant to a spe­cif­ic ques­tion from mass­es of legal doc­u­ments, and sum­ma­rize it as well. Freed from this type of rou­tine work, lawyers can apply their ana­lyt­i­cal skills to more com­plex issues.

For the media the prob­lem with gen­er­a­tive AI is that it’s not cur­rent­ly con­nect­ed with world events and is there­fore not in a posi­tion to auto­mat­i­cal­ly pro­duce news reports that are truly inno­v­a­tive and fac­tu­al­ly accu­rate. That of course doesn’t stop crim­i­nal inter­ests from using AI to gen­er­ate false news and flood social media with it.”

5. ChatGPT seems to be sparing students a lot of routine work like doing research, compiling sources, or even formulating entire papers. Will digital assistants mean they don’t have to spend time in libraries? And how will this change the face of research and science?

Dana Ruiter: “It’s cru­cial to keep in mind that Chat­G­PT is a lan­guage model, not a knowl­edge model. Sci­en­tif­ic work con­sists of observ­ing the world and its prop­er­ties, and eval­u­at­ing the knowl­edge there­by acquired in light of the pre­vi­ous canon of knowl­edge. Chat­G­PT can do nei­ther of these things: it can’t observe the world, nor can it repro­duce the exist­ing canon of knowl­edge in fac­tu­al­ly accu­rate form. What it’s very good at, how­ev­er, is sim­u­lat­ing the style of a sci­en­tif­ic paper. This can be help­ful in the for­mu­la­tion stage of a paper once the research results are in and the lit­er­a­ture search has been done. But it does not replace the sci­en­tif­ic work per se.”

6. ChatGPT users can select different styles and also the degree of in-depth treatment. And if the output doesn’t meet their expectations, they can tell ChatGPT to improve or refine it. What does this tell us about the results — how malleable and how arbitrary are they?

Dana Ruiter: “Results can be manip­u­lat­ed with the help of what’s called prompt engi­neer­ing. Once again, the prob­lem lies in the system’s lack of trans­paren­cy and the output’s lack of robust­ness. The sys­tem isn’t trans­par­ent because it’s unclear which terms in the prompts have which effects on the sen­tences gen­er­at­ed. It’s pure trial and error. And the lack of trans­paren­cy means these mod­els are not robust: a prompt that pro­duces the desired result one time can, with slight­ly mod­i­fied input, lead to unde­sir­able results oth­er­wise. That makes it espe­cial­ly hard to inte­grate gen­er­a­tive AI mod­els into pro­duc­tion sys­tems. After all, no one wants the sys­tem to start spout­ing hate speech or other unde­sired con­tent due to a trig­ger in the user input.”

7. In which fields will ChatGPT be used first and most effectively?

Kevin Lin: “Chat­G­PT is often used as a search machine right now, but I think its main appli­ca­tion will be pro­cess­ing doc­u­ments in auto­mat­ed form. One exam­ple would be clas­si­fy­ing com­plaints in order to con­nect cus­tomers with the right advi­sors or com­pa­ny rep­re­sen­ta­tives. Anoth­er would be iden­ti­fy­ing and clear­ly sum­ma­riz­ing rel­e­vant admin­is­tra­tive infor­ma­tion, and yet anoth­er would be improv­ing indi­vid­ual writ­ing styles. All these skills are rel­e­vant to many dif­fer­ent sec­tors and can eas­i­ly gen­er­ate sub­stan­tial added value. Solu­tions based on gen­er­a­tive AI will be par­tic­u­lar­ly appeal­ing to fields with high per­son­nel costs or a short­age of work­ers, such as health­care, the legal pro­fes­sion, or admin­is­tra­tive areas.”

8. Which qualifications and jobs will ChatGPT change or even render obsolete?

Kevin Lin: “The work­ing world will change. The good thing here is that spe­cial­ized work will still need to be done by qual­i­fied indi­vid­u­als. Chat­G­PT can­not replace how a physi­cian inter­acts with patients or how a lawyer for­mu­lates an air­tight con­tract. The abil­i­ty to inte­grate spe­cial­ized exper­tise with knowl­edge about the world in gen­er­al, and to link it with sit­u­a­tion­al and social­ly aware actions, will remain the domain of well-trained human spe­cial­ists for quite a while. Repet­i­tive tasks that con­sis­tent­ly fit a cer­tain pat­tern, how­ev­er, will be replaced to an increas­ing extent. For ser­vices like pro­cess­ing com­plaints or per­form­ing admin­is­tra­tive tasks, Chat­G­PT will offer con­sid­er­able lever­age in terms of increas­ing effi­cien­cy and requir­ing fewer albeit high­ly qual­i­fied personnel.

Gen­er­a­tive AI will elim­i­nate some tasks in the future, that’s clear. Employ­ees will there­fore need not only the spe­cial­ized skills they already com­mand, but also ever greater famil­iar­i­ty with both the abil­i­ties and lim­its of these gen­er­a­tive mod­els. There’s a big need here for fur­ther train­ing under expert instruc­tors and lead­ers, right at com­pa­nies. That’s the only way to ensure that gen­er­a­tive AI can be applied in advan­ta­geous and depend­able ways.”

9. What is ChatGPT not capable of?

Dana Ruiter: “Lots of peo­ple con­fuse Chat­G­PT with some type of search engine. But that’s not what it is. Chat­G­PT has only learned to sim­u­late answers to ques­tions. The goals of its train­ing do not include ensur­ing that these answers are fac­tu­al­ly accu­rate. Nor can Chat­G­PT ensure that the sen­tences it gen­er­ates always meet the expec­ta­tions of the peo­ple who entered the prompts. Here’s an exam­ple: Let’s say I use Chat­G­PT to cre­ate a tool that helps clients for­mu­late emails in a more pro­fes­sion­al style. I test my tool, and it seems to work well because the email mes­sages it refor­mu­lates sound com­pe­tent and polite. How­ev­er, when I hand over the tool to real users who enter all kinds of infor­ma­tion, they com­plain that it neglects parts of their orig­i­nal texts or invents new con­tent. In other words, there’s no guar­an­tee that the sys­tem will always act as antic­i­pat­ed. This is why high­ly sen­si­tive pro­duc­tion sys­tems often stick with rule-based process­es, which pro­vide cer­tain guardrails for clean and cor­rect out­put. In the future we’ll be see­ing more hybrid sys­tems that com­bine gen­er­a­tive AI with rule-based mod­els in order to make the results more robust.”

10. What alternatives are there to ChatGPT? Are competitors on the horizon?

Fabi­an Schmidt: “Chat­G­PT from Ope­nAI is cur­rent­ly just a chat­bot based on a lan­guage model. This type of tech­nol­o­gy is not unique. And there are plen­ty of com­peti­tors. In addi­tion to closed-source mod­els such as Bard from big tech com­pa­nies like Google, there are also spe­cial­ized region­al solu­tions from start-ups. In 2022, the Porsche Con­sult­ing man­age­ment con­sul­tan­cy advised Aleph Alpha, an AI start-up from the south­west­ern Ger­man city of Hei­del­berg, in con­nec­tion with the Ger­man Entre­pre­neur Award (Deutsch­er Grün­der­preis). Lumi­nous, the lan­guage model from Aleph Alpha, is espe­cial­ly rel­e­vant for the Ger­man mar­ket and has a Euro­pean focus. Its mod­els are host­ed in Ger­many and meet the high Euro­pean data pro­tec­tion stan­dards. How­ev­er, there are two major prob­lems with many closed-source solu­tions. Their mod­els are often not eas­i­ly adapt­able, and data and model sov­er­eign­ty are fre­quent­ly in exter­nal hands. It’s espe­cial­ly dubi­ous if they’re out­side Europe and there’s no way of con­trol­ling how the data are processed. That’s why it’s often a good idea for com­pa­nies to devel­op their own solu­tions using open-source mod­els. Some solu­tions already exist for large-scale lan­guage mod­els, such as LLaMA (Meta) and BLOOM (Hug­ging Face). They’re host­ed local­ly and can be adapt­ed at will, which in turn allows full con­trol of the model and the data. If small­er lan­guage mod­els are suf­fi­cient — for more basic tasks such as clas­si­fy­ing cus­tomer com­plaints or spot­ting trends on social media — then there’s an entire range of open-source lan­guage mod­els such as BERT, XLM, and Camem­BERT. Por­tals like Hug­ging Face pro­vide free access to these mod­els, along with sup­port func­tions to facil­i­tate their adap­ta­tion to spe­cif­ic use cases.”

11. How can I introduce generative AI solutions at my company?

Fabi­an Schmidt: “As con­sul­tants, the first thing we do with our clients is see whether there’s already a con­crete prob­lem that AI should solve. Or whether the goal is to use gen­er­a­tive AI to dis­cov­er new and inno­v­a­tive AI solu­tions. In the lat­ter case it helps to have a work­shop with spe­cial­ists or to use Porsche Consulting’s Inno­Lab in Berlin. As soon as we’re clear on the con­crete use cases, the deci­sion­al process­es for how to imple­ment them can begin.

One ques­tion that needs to be answered right away is whether it even makes sense to auto­mate a par­tic­u­lar task with gen­er­a­tive AI. If the answer is yes and an AI solu­tion will save money and time, the next step is to see whether there’s a good ‘off-the-shelf’ solu­tion already on the mar­ket. It’s impor­tant to note that these solu­tions don’t have to come from big play­ers like Microsoft or Google. Their solu­tions tend to be based on very broad usabil­i­ty, which might not be right for a given use case. Small­er com­pa­nies and estab­lished start-ups fre­quent­ly pro­vide very good solu­tions for spe­cif­ic use cases and sec­tors. To gain a good overview here, we rec­om­mend doing mar­ket analy­ses that focus on fac­tors like data pro­tec­tion, tech­ni­cal sophis­ti­ca­tion, and scalability.

If a com­pa­ny can­not find a ready-made solu­tion, how­ev­er, it needs to devel­op its own. At Porsche Con­sult­ing we work with our clients to clar­i­fy the data land­scape. Our team’s experts then anonymize and process the data. Only then do we apply the actu­al model to the use case and refine it togeth­er with the client in mul­ti­ple feed­back rounds. For com­pa­ny-spe­cif­ic devel­op­ments, we place a pre­mi­um on data and model con­trol, because no one wants their data to fall into exter­nal hands. Open-source mod­els that are local­ly host­ed and eas­i­ly adapt­able often offer the surest road to solid com­pa­ny-spe­cif­ic developments.”

Direct con­tact to Porsche Consulting’s Arti­fi­cial Intel­li­gence and Data Ana­lyt­ics divi­sion:


Read the next topic“A Priceless Treasure”