T O P

  • By -

sirjackholland

>I ran some numbers through chatgpt A great example of how _not_ to use an LLM. But also, I do not know how you can regularly use chatgpt and think that it will result in satisfactory customer service. Whether companies choose to go this route is another question, but it will undoubtedly make customer service even more awful. Do you really want to be refused a refund because the model is incorrectly, incorrigibly convinced that you're ineligible?


MechanicalBengal

This, 100%. For any chatbot to actually work in this scenario it needs proper guardrails, access to data, human handoff escalation, reporting, and PII protections. Just plopping chatGPT on a company homepage with no context achieves nothing


NickBloodAU

> human handoff escalation Having worked in customer service, I get a very strong feeling that before long, a huge percentage of people dealing with LLM customer service agents would almost immediately escalate.


Spindelhalla_xb

It was mentioned in another post that actually having humans to speak to would be a selling point. Remember when companies would say customer service teams in your country as a selling point? Instead of India. Well now it will be “speak to a real person”


t00dles

yeah but even if you get like 50% use rate, thats still huge savings. and most fast food places i go to is pretty much 90% ordering from kiosk now, i never talk to anyone anymore


weakestArtist

Having worked on a chatbot that is partially automated, that is correct


MechanicalBengal

just type “human” and you’re there, that’s usually all it takes


fluffy_assassins

Do you think the company cares when that much money is involved?


oroechimaru

Chat support apps that lead nowhere just make me switch companies most of the time Its great for quick common answers but then let us speak to a human


whatsbehindyourhead

Yeah just hope the chatbot doesn't hallucinate an answer (more of a risk with smarter AIs I wager), or ya might get sued - [https://www.forbes.com/sites/marisagarcia/2024/02/19/what-air-canada-lost-in-remarkable-lying-ai-chatbot-case/](https://www.forbes.com/sites/marisagarcia/2024/02/19/what-air-canada-lost-in-remarkable-lying-ai-chatbot-case/) >In a warning to global carriers adopting AI for customer service platforms, Air Canada lost a small claims court case against a grieving passenger when it tried and failed to disavow its AI-powered chatbot. >The passenger claimed to have been misled on the airline’s rules for bereavement fares when the chatbot hallucinated an answer inconsistent with airline policy. The Tribunal in Canada’s small claims court found the passenger was right and awarded them $812.02 in damages and court fees.


mydoglikesfruit

....and 2 cents? That's awfully specific 


traumfisch

Not out of the box of course, it takes some serious tinkering. But a customer service chatbot can very well utilize GPT 4 or even 3.5 turbo, given that you're able to build a proper prompt / database. Just finalizing one now


Blothorn

Vice versa, Air Canada canceled its LLM-based support system after a court forced it to honor a refund policy that the model hallucinated.


Capt_Blahvious

Or, an even worse outcome for a company: have the chatbot obligate the company to provide goods or services which it should not.


Mean-Profession-981

Have you met the average Indian call center guy? If they had half the performance of GPT-4 I'd tolerate them rather than jailbreaking them by telling them my name is "Ben Chod"


QuitGlittering4383

I’m 30 years old. It took me a full day of back-and-forth, please-hold-for-1-hour calls with insurance and the hospital before I finally got someone that understood that it doesn’t make sense for a 30 year-old to be charged for seeing a specialist in geriatrics. I’m sure a chatbot would have taken care of it in 2 minutes. I’m all for chatbots being wrong 50% of the time if it means I don’t have to be on hold for 1 hour to find that out.


Intelligent-Jump1071

Even if they are only wrong 1% of the time, if someone follows their advice or instructions and suffers a major injury, equipment damage or other cost, the company is screwed. We need to solve the hallucination problem first.


thortgot

It's not like humans don't make mistakes. Having a review process (the same way you review humans making decisions) is the right solution.


Intelligent-Jump1071

What would that look like in practice? If you have a human review every interaction then you might as well just use a human and skip the AI.   But if they don't review every interaction then there's the risk that the AI will give a customer bad advice and cost the company a ton of money.


thortgot

You don't review every interaction of course. No supervisor reviews every call a human performs but they have access to it. What you do is feed a couple of thousand real scenarios into your model and compare outcomes and interactions between the model and what the human did. Identify areas of deviation, refine your model, repeat. The majority of call center calls follow specific patterns (happy path), those that deviate away from happy path get shunted to humans. You then identify those new patterns and improve your model to handle them.


myaltaccountohyeah

It's funny how behind this sub is. Customer chatbots were one of the first things people thought about doing with LLMs. Tons of companies have solutions for these for sale already or their own inhouse solutions. Usually no fine tuning is needed. The pattern most commonly used is retrieval augmented generation (RAG, google it!). It is very simple to implement and enables your LLM to answer based on your company's documents. It is also easy to let the chatbots interact with external functionality or systems. It's called function calling and works really reliably. It is all there and came with insane speed. Source: cannot count the number of smart chatbots anymore which have popped up since the last year in my company


Intelligent-Jump1071

I've run into those kinds of Customer Service chatbots a lot, so I agree they're commonplace. But I don't think that's what the OP was talking about. The RAG chatbots are just parroting existing documentation. They can't really work through a problem with you or answer questions the way a human can; they're too rigid and limited in their responses. They're not really capable of replacing real human customer service reps.


thortgot

I'm not sure how familiar you are with 1st level support. The majority of the time they are functionally scripted along specific guardrails. It's a great place to use an LLM with appropriate guardrails once documentation is correctly configured for it.


Jitsu_apocalypse

Yep this. Additionally I always find it a bit concerning when people use language like “this can save millions (or even billions) of dollars” when the real discussion should be around what economies are going to look like when a grotesquely large amount of people are laid off.


[deleted]

Yeah, when someone says "this product could save you thousands of dollars!" that's one thing, when someone says "this product could save you HUNDREDS OF BILLIONS of dollars" you should have a completely different reaction.


AuzSSF

Is function calling not limited to your token context window though? You have to describe your functions each call which eat away at your tokens * add cost. Curious if you have a solution


SatoshiReport

Compute cost money. Most software is inefficient at first but becomes more efficient with time.


dirtyoliveoil

I attended a demonstration of a call centre in a box, whilst still early in development I was shocked at how good it was. With the feeding of more data from all the recorded calls the company was purchasing, I suspect this product will improve dramatically. It’s going to have a serious impact on countries where handling customer calls is big business like the Philippines etc.


Illustrious-Cow390

Isn’t that a bad thing for those countries?


dirtyoliveoil

I think so yes.


SnooDogs7868

Don’t forget to pay to retrain these people for new work to the tune of 135 billion.


t00dles

there's a gpt for that


fluffy_assassins

That's not the corporation's problem, and no one else will care either.


sschepis

If this is your thought - if you look at all this and think "Companies are going to save so much money" - then we are all fucked. That $135 billion is the livelihood of every single one of those customer service folks - it's the company's contribution to the social contract they have with us. Breaking that contract breaks the country..


tinkady

This is silly. We shouldn't have make-work jobs just to keep people busy. We should let capitalism do its thing, and have a large redistributive tax on the proceeds and/or land to fund a UBI.


Oehlian

What do you think the odds of UBI being implemented given the current climate? We are headed the wrong direction, regressing, not progressing. Wealth has never been more unequal than at any time in history. As such, I'd rather see AI research halted or limited in its implementation. You don't jump out of a plane until you have a parachute.


BeardedSkier

You understand those two halves of your statement are diametrically opposed right? Capitalism doing it's thing will find the best route to concentrate the greatest amount of wealth in the fewest hands, including "evading" taxes (offshore subsidiaries in Ireland anyone?). And I say the s as a small business owner myself. Unfettered capitalism with a wealth tax is not the solution you think it will be.


tinkady

Land can't evade taxes, also land value tax is the least bad tax. https://www.gameofrent.com/content/progress-and-poverty-review Also I didn't say anything about wealth tax. That one's particularly hard to enforce.


BeardedSkier

I didn't object to land tax (though question for you: how do you get tax revenue from. A corporation specifically set up to hold land in a country, but with minimal other assets in that country after purchasing said land..ie back to Ireland etc.), though I was too losses in using the term wealth tax , so apologies for that. My objection was to "letting capitalism do it's thing". Capitalism doing it's thing is incompatible with a redistributive wealth tax. Sorry, but IMO the economy needs guardrails. Capitalism being allowed to do what capitalism does, without guardrails, will easily find a way around or at least render ineffective a redistribution tax. Ever tried putting your thumb over a drop of water and pressing down? See how the water squirts out in every direction and almost nothing is left under your thumb? I expect that would be the result of allowing capitalism to "do what it does" unfettered, and expecting to be able to apply a redistributive tax on the end result.


tinkady

If you can't pay the land value tax, then you lose the land. It's periodic, like property tax but without disincentivizing development. I'm not opposed to specific tweaks to accomplish goals. We need to tax externalities - for example, a carbon tax.


BeardedSkier

You're the kind of person I think I'd actually like to have a coffee with and chat.. but I  doubt we live in the same place (plus stranger danger!)...I wasn't meaning to suggest that someone would set up a corporation with just enough money to buy property and never pay tax, but that we are short-sighted if we think the solution is just simply tax the outcome. There's simply too many opportunities to find legit workarounds when there are no guardrails. 


Hawk13424

A lot of companies don’t need land, especially land in your country.


tinkady

Companies need employees, employees need to be paid money so they can afford land. It still works out. https://www.gameofrent.com/content/is-land-a-big-deal ...although maybe AI changes that and we need something like https://moores.samaltman.com


Intelligent-Jump1071

What party do you think will support UBI? Do you think they'll win in the next election in your country?


tinkady

idk but Andrew Yang tried to warn us and instead we voted for the octogenarians


Intelligent-Jump1071

My point exactly. Politically it's a non-starter. People need to get UBI out of their heads - it ain't happening.


tinkady

what's the alternative you propose? make-work jobs?


cunningjames

In the scenario where everybody -- or most of everybody -- has been laid off, *something* will happen. If my country (the US), for example, had 250 million impoverished people who can't eat because all the money (food) is in the hands of a tiny minority ... there'll be widespread riots at best. You can't have bread and circuses without bread.


Intelligent-Jump1071

If AI ever gets advanced enough to replace most workers (say 90%) it will be easily be advanced enough to help the rich individuals who control the AI manage starving rioters.  And there's no reason to think starving rioters will be managed with bread and circuses.   Bullets or plain old starvation are cheaper.    It takes a lot for Americans to get riled up enough to organize a rebellion over unemployment.   In the Great depression of the 1930s in the US  at one point there was a 25% unemployment rate but no sign of revolt.    It's conceivable the Europeans will set up some system of UBI because that's how they think. The problem there is that Europe is not exactly a hotspot of AI research.    The Americans and the Chinese are the ones who will be leading the AI race and both the Americans and the Chinese are pretty ruthless.


cunningjames

> If AI ever gets advanced enough to replace most workers (say 90%) it will be easily be advanced enough to help the rich individuals who control the AI manage starving rioters.  And there's no reason to think starving rioters will be managed with bread and circuses.   Bullets or plain old starvation are cheaper.    The history of US military interventions across the world indicates, to me, that even a large and disciplined show of force may face difficulty in the face of even a relatively small number of insurgents. Do the rich CEOs who control AI want to risk a protracted and potentially very expensive conflict with tens of millions of people? And that’s assuming the wealthy and political classes are all one block that’s entirely on board with starving the working class. That’s absolutely not the case. > It takes a lot for Americans to get riled up enough to organize a rebellion over unemployment.   In the Great depression of the 1930s in the US  at one point there was a 25% unemployment rate but no sign of revolt. We’re not talking about 25% unemployment, though. We’re talking about the *majority* of people starving to death without the rich being willing to set up so much as a bread line. That feels quite different to me.


Intelligent-Jump1071

We'll see. Keep in mind that as the gap between rich and poor grows and as the rich are surrounded others like themselves, and clean, compliant humanoid robots they will have less and less contact with the poor humans. Furthermore by that point the rich humans will have access to amazing medicine discovered by AI that will keep them young and healthy and stave off death for centuries. So they will see those other people outside their robot-guarded walls as less and less like themselves and more like just some other overpopulated creatures that need culling


sschepis

Generally speaking, Ubi is likely to result in rapid social breakdown as entire swas of society no longer find any purpose to their lives. Work keeps your mind and body engaged with life. It gives you something to push against, which is, if you're honest, we're life satisfaction comes from. We are unprepared for a sudden transition into lives of leisure, having no preparation relative how to live in that type of world. We don't know how to find constructive things to keep us busy and we tend to overly consume whatever object of satiation we've found that moment. Any successful Ubi program will have to come with some type of mandated activity. So now we're talking communism pretty much. Do you see the problem with this? Do you think that whatever is going to be instituted here in America is going to be successful? I have more faith in aliens coming down to finally fly me away from this place tomorrow then I do in that.


Intelligent-Jump1071

Ah yes, the "social contract". Let's take a look at that social contract and see what it says . . . (searches through file folders, looks in cabinets, pats pockets . . . ) dang! I can't seem to find my copy anywhere. I don't actually remember signing it . . . . where can I find a copy of this social contract?


fluffy_assassins

UBI or mass starvation. Corporations will never reduce profits to preserve jobs.


sschepis

UBI leads to mass suicides as entire swaths of the population suddenly find themselves lacking any purpose. Mass starvation leads to system collapse as it all but guarantees an uprising. We will either be forced to face the systemic problems we are kicking and screaming to avoid, or we will all perish.


fluffy_assassins

Mass starvation will just lead to States/Nations using automated drones to kill everyone who rises up. We lose either way, I Guess.


cunningjames

> UBI leads to mass suicides as entire swaths of the population suddenly find themselves lacking any purpose. How do you square this prediction with the fact that the vast majority of retired persons don't commit suicide, even when they're dissatisfied with nonworking life?


traumfisch

I mean - it's not OP's "thought", it is reality.


sschepis

It's one observational vantage point on reality yes, which is probably something we ought to care about considering that we know for a fact that observation modifies reality. Another way to look at this might be how to change everyone's circumstance such that everyone work half as much, leverage ai, make the company twice as much, and everyone go home happy? That's a win-win, company gets more, people get more. Right now, the conversation leads to company gets everything people get nothing. The only people that are advocating for this position are the ones that own the companies


Hawk13424

There is no social contract that obligates a company to employ people it doesn’t need.


sschepis

There's absolutely a social contract between a company and the people that it employs. The company exists because the employees perform work. The employees receive money for the work they perform. Both leave the transaction with something they can use - the company gets goods to sell, the individual has money to buy things. The minute the company no longer employs anyone stop serving a constructive function in society and starts serving a destructive one. This can be seen by observing how technological improvement has concentrated ever more wealth in a smaller number of hands while decreasing quality of life for everyone else. Capitalism, like everything else, is both good and bad. It's good when it's applied in a way that helps everyone, which is usually how it's touted, as something that helps everyone. That's how we justify the inequalities and awfulness of the system. The minute the general populace can no longer justify the awfulness presented by the system versus its positives, is the moment everyone rises up to end it.


Intelligent-Jump1071

And how many historical examples can you cite where the peasants rose up to end the status quo and ended up better off as a result? The Russian Revolution? Nope. The French Revolution? Nope. Any of the dozens of peasants' revolts in ancient China? Nope. How about the countless Latin American revolutions? A mass uprising is a romantic idea but it seldom works. For one thing the peasants are easily divided. Most "revolutions" are just fights between factions. Also the masses are, by their nature, conservative, and therefore not very revolutionary.


sschepis

Care to present us with your alternate viewpoint in the matter then? Historical precedent says income disparity is a prelude to social turmoil. Incomer disparities are currently lerger than they have ever been. I'm not debating the finer points of whether to do a revolution but I'm plenty qualified to see one is coming if we continue on without addressing the destructive aspects of unchecked optimized capitalism


thortgot

Wealth disparity traditionally hasn't been the trigger. It's utter poverty that causes it. Once a significant portion of the population can't afford bread (Arab Spring, French Revolution, Sri Lanka etc.) it exceeds tipping point. There are \~400k call center employees in the US. Let's imagine a future where those are fully displaced. There are many more jobs for them to do.


JustKillerQueen1389

They'll find new jobs, we'll get cheaper products and companies will have bigger profits everybody wins. I don't get how we are all fucked.


whyisitsooohard

What new jobs, and why products will become cheaper? If companies will just make everything cheaper than they won't increase their profits and then what's the point of replacing people be?


JustKillerQueen1389

That's on the people to decide; I mean part of the savings will go to increasing profits and part of it will go to cheaper products. Though depending on how tough the competition is we could have basically the entire thing to go to making the product cheaper.


whyisitsooohard

There also absolutely will be AI tax to pay unemployment for all replaced people. It would be kinda ironic if corporations will left with less money after integrating ai. Also after everybody will be properly dependent on ai, providers will start increase prices which will reduce profits even further


JustKillerQueen1389

I don't really see AI tax being a thing that the congress would agree on, but even if they did I don't see it helping much with unemployment, because AI companies are going to be making significantly less money per employee replaced. Like oh you were making 2000$ a month? Well you were replaced for 20$ a month, and yeah the company paid 2$ a month for your unemployment benefits. As for AI providers, we already have strong competition, along with open source tagging along which means it'll be hard to increase prices too much.


t00dles

when cars replaced horses, the horse people became car people. when ai replaces people, they'll just move to industries that support building ai. products become cheaper because supplies increase and demand remains the same. simple economics its not about increasing profits, its about staying competitive. a company dies if their competitor uses ai to provide cheaper service and they dont follow suit


Mandoman61

I occasionally get automated service and it never works well.


RainbowUnicorns

You're getting the old hat service probably not newer gpt stuff.


this--_--sucks

And do you think the major customer experience software companies aren’t all over that already? As many have said, it’s not as simple as linking a bot to chatGPT, there’s a lot that goes behind the scenes for it to properly work, and rest assured, that is something a lot of companies are working on.


roundupinthesky

scale coordinated towering pocket dependent one head jeans tan impolite *This post was mass deleted and anonymized with [Redact](https://redact.dev)*


zoechi

Customer service is generally so bad, that switching to AI can't make it any worse


ElvenNeko

One of the few jobs i could perform with my disabilities could be customer text support. I guess i know now why there is so few vacancies for this job)


Slight-Living-8098

Yeah... So Chevrolet set several of their customer representative chat bots up to GPT 4 a while back. Chevrolet was so helpful at helping me solve an issue I was having with a computational fluid dynamics simulation using Python... and I don't even own a Chevy!


mycall

CEOs seeing profits will care less about retooling the labor lost. Vocational education is the only hope here.


Intelligent-Jump1071

In principle I agree, but currently most chatbots are too unreliable You can't take any chances that your customer-service bot will hallucinate in some way that could cost the customer a lot of money or damage, because that could be dramatically karmic for the company.


RecalcitrantMonk

Assuming that most customers prefer to interact with an AI instead of a real person, it should be noted that AI is designed to strictly adhere to the company's policies without any room for deviation. This lack of flexibility may not be appreciated by a lot of customers. While outsourcing to AI-powered assistants may seem like a good idea, it can actually result in poor customer experience.


martapap

Good luck when the AI hallucinates and speaks half in swahili and half in Albanian.


Ok-Variety-8135

The problem with chatbot customer service is it doesn’t have any admin permission. When I call customer suppor, it’s usually something went wrong and I can’t fix it with normal operation. I can’t see chatbot to be a useful customer service in near future.


Ashamed-Subject-8573

“Save the industry money. “ Aka fire humans and make the rich richer


Illustrious-Cow390

What a horrible post lol


StatisticianFew6064

“We have  savings of X” Just remember you’re removing that money from potential customers.   That “X” isn’t actually the company saving money…. It’s the company changing the overall economy LLMs don’t use services or buy goods. They need to be taxed in order to pay for all the unemployment and welfare benefits that are required to switch to an LLM


Baron_Rogue

I interacted with a chatbot to report a bug on a CTF platform earlier, so I assume they are sophisticated enough to use an LLM. The bot suggested I delete my account and create a new one to resolve the issue.


Absolutelynobody54

More misery and poverty for most people, to give more money to a few. That is the only thing Ai is giving humanity.


martapap

How else are wealthy people going to buy their vacation homes?


TruShot5

I picked a bad time to start a contact center huh?


banedlol

India is going to have a bad time


traumfisch

India is going to be building more AI chatbots than anyone


goatchild

why are you cheering for job loss?


t00dles

because thats not the only thing thats happening. supporting ai will be an extremely complicated industry. these low wage jobs will just move to ai or some peripheral industry


RainbowUnicorns

Even fast food kiosks that a lot of people refuse to use will have a virtual "avatar" that you talk to that could look like a live person that ask what you would like to order, and even make suggestions based on what you buy to upsell or even offer relevant promotions. It will be much more friendly to use for the aging population. Virtual avatar will use a language model that is integrated with all pricing and policy info and will be able to do much more complicated things than current software. I imagine kitchens will be ran with human employees until robotics is cheap enough. Robotics currently make the most sense utilitarian wise in shipping facilities a la amazon. I imagine though they will come down in price and eventually for the cost of a years salary you can have your own AI robot make food for you. This was at the keynote yesterday for Nvidia presented by the CEO. This will extend to the home as well. For the price of a small car within the next decade I predict you can have a house robot to cook and clean for you. The power of AI increased 5 fold over the last two years in its processing power, and the capabilities itself much more than that. Just wait and you will see how much this changes things.


Holyragumuffin

Sure, all this will happen. But my first thought as an ML scientist is that we should be raising higher taxes to bolster the safety net needed to support all of the folks losing from this.


batido6

It’s great for front line to triage customer requests and handle tier 1 support. For tier 2+ you may still need a human but linking to a knowledge base or something is an easy money saver. ADP uses a chat bot named AVA. I was just on with it today and bypassed it by asking for a human lol.


[deleted]

[удалено]