How to Hack a $1.3T Market

Forget concrete. The new foundation for real estate success is digital, and Pacaso leads the way. Their tech unlocks a $1.3T real estate market. They've already earned $110M+ in gross profits in their operating history and reserved the Nasdaq ticker PCSO.

Researchers say an AI-powered transcription tool used in hospitals invents things no one ever said

GARANCE BURKE and HILKE SCHELLMANN
October 26, 2024

SAN FRANCISCO (AP) -- Tech behemoth OpenAI has touted its artificial intelligence-powered transcription tool Whisper as having near "human level robustness and accuracy."

But Whisper has a major flaw: It is prone to making up chunks of text or even entire sentences, according to interviews with more than a dozen software engineers, developers and academic researchers. Those experts said some of the invented text -- known in the industry as hallucinations -- can include racial commentary, violent rhetoric and even imagined medical treatments.

Experts said that such fabrications are problematic because Whisper is being used in a slew of industries worldwide to translate and transcribe interviews, generate text in popular consumer technologies and create subtitles for videos.

More concerning, they said, is a rush by medical centers to utilize Whisper-based tools to transcribe patients' consultations with doctors, despite OpenAI' s warnings that the tool should not be used in "high-risk domains."

The full extent of the problem is difficult to discern, but researchers and engineers said they frequently have come across Whisper's hallucinations in their work. A University of Michigan researcher conducting a study of public meetings, for example, said he found hallucinations in eight out of every 10 audio transcriptions he inspected, before he started trying to improve the model.

A machine learning engineer said he initially discovered hallucinations in about half of the over 100 hours of Whisper transcriptions he analyzed. A third developer said he found hallucinations in nearly every one of the 26,000 transcripts he created with Whisper.

The problems persist even in well-recorded, short audio samples. A recent study by computer scientists uncovered 187 hallucinations in over 13,000 clear audio snippets they examined.

That trend would lead to tens of thousands of faulty transcriptions over millions of recordings, researchers said.

Such mistakes could have "really grave consequences," particularly in hospital settings, said Alondra Nelson, who led the White House Office of Science and Technology Policy for the Biden administration until last year.

"Nobody wants a misdiagnosis," said Nelson, a professor at the Institute for Advanced Study in Princeton, New Jersey. "There should be a higher bar."

Whisper also is used to create closed captioning for the Deaf and hard of hearing -- a population at particular risk for faulty transcriptions. That's because the Deaf and hard of hearing have no way of identifying fabrications are "hidden amongst all this other text," said Christian Vogler, who is deaf and directs Gallaudet University's Technology Access Program.

OpenAI urged to address problem

The prevalence of such hallucinations has led experts, advocates and former OpenAI employees to call for the federal government to consider AI regulations. At minimum, they said, OpenAI needs to address the flaw.

"This seems solvable if the company is willing to prioritize it," said William Saunders, a San Francisco-based research engineer who quit OpenAI in February over concerns with the company's direction. "It's problematic if you put this out there and people are overconfident about what it can do and integrate it into all these other systems."

An OpenAI spokesperson said the company continually studies how to reduce hallucinations and appreciated the researchers' findings, adding that OpenAI incorporates feedback in model updates.

While most developers assume that transcription tools misspell words or make other errors, engineers and researchers said they had never seen another AI-powered transcription tool hallucinate as much as Whisper.

Whisper hallucinations

The tool is integrated into some versions of OpenAI's flagship chatbot ChatGPT, and is a built-in offering in Oracle and Microsoft's cloud computing platforms, which service thousands of companies worldwide. It is also used to transcribe and translate text into multiple languages.

In the last month alone, one recent version of Whisper was downloaded over 4.2 million times from open-source AI platform HuggingFace. Sanchit Gandhi, a machine-learning engineer there, said Whisper is the most popular open-source speech recognition model and is built into everything from call centers to voice assistants.

Professors Allison Koenecke of Cornell University and Mona Sloane of the University of Virginia examined thousands of short snippets they obtained from TalkBank, a research repository hosted at Carnegie Mellon University. They determined that nearly 40% of the hallucinations were harmful or concerning because the speaker could be misinterpreted or misrepresented.

In an example they uncovered, a speaker said, "He, the boy, was going to, I'm not sure exactly, take the umbrella."

But the transcription software added: "He took a big piece of a cross, a teeny, small piece ... I'm sure he didn't have a terror knife so he killed a number of people."

A speaker in another recording described "two other girls and one lady." Whisper invented extra commentary on race, adding "two other girls and one lady, um, which were Black."

In a third transcription, Whisper invented a non-existent medication called "hyperactivated antibiotics."

Researchers aren't certain why Whisper and similar tools hallucinate, but software developers said the fabrications tend to occur amid pauses, background sounds or music playing.

OpenAI recommended in its online disclosures against using Whisper in "decision-making contexts, where flaws in accuracy can lead to pronounced flaws in outcomes."

Transcribing doctor appointments

That warning hasn't stopped hospitals or medical centers from using speech-to-text models, including Whisper, to transcribe what's said during doctor's visits to free up medical providers to spend less time on note-taking or report writing.

Over 30,000 clinicians and 40 health systems, including the Mankato Clinic in Minnesota and Children's Hospital Los Angeles, have started using a Whisper-based tool built by Nabla, which has offices in France and the U.S.

That tool was fine tuned on medical language to transcribe and summarize patients' interactions, said Nabla's chief technology officer Martin Raison.

Company officials said they are aware that Whisper can hallucinate and are mitigating the problem.

It's impossible to compare Nabla's AI-generated transcript to the original recording because Nabla's tool erases the original audio for "data safety reasons," Raison said.

Nabla said the tool has been used to transcribe an estimated 7 million medical visits.

Saunders, the former OpenAI engineer, said erasing the original audio could be worrisome if transcripts aren't double checked or clinicians can't access the recording to verify they are correct.

"You can't catch errors if you take away the ground truth," he said.

Nabla said that no model is perfect, and that theirs currently requires medical providers to quickly edit and approve transcribed notes, but that could change.

Privacy concerns

Because patient meetings with their doctors are confidential, it is hard to know how AI-generated transcripts are affecting them.

A California state lawmaker, Rebecca Bauer-Kahan, said she took one of her children to the doctor earlier this year, and refused to sign a form the health network provided that sought her permission to share the consultation audio with vendors that included Microsoft Azure, the cloud computing system run by OpenAI's largest investor. Bauer-Kahan didn't want such intimate medical conversations being shared with tech companies, she said.

"The release was very specific that for-profit companies would have the right to have this," said Bauer-Kahan, a Democrat who represents part of the San Francisco suburbs in the state Assembly. "I was like 'absolutely not.'"

John Muir Health spokesman Ben Drew said the health system complies with state and federal privacy laws.

___

Schellmann reported from New York.

___

This story was produced in partnership with the Pulitzer Center's AI Accountability Network, which also partially supported the academic Whisper study.

___

The Associated Press receives financial assistance from the Omidyar Network to support coverage of artificial intelligence and its impact on society. AP is solely responsible for all content. Find AP's standards for working with philanthropies, a list of supporters and funded coverage areas at AP.org.

___

The Associated Press and OpenAI have a licensing and technology agreement allowing OpenAI access to part of the AP's text archives.

Continue Reading...

Popular

Trump keeps saying the GOP mega bill will eliminate taxes on Social Security. It does not

WASHINGTON (AP) — President keeps saying that Republicans' mega legislation will eliminate taxes on federal benefits.

Cathie Wood's Alpha Surge: $250 Million Circle Windfall, Big AMD Buys

Cathie Wood, founder and CEO of ARK Invest, has recently been on a hot streak of successful and high-profile trades through her flagship ARK ETFs. 

Is Elon's Empire Crumbling? - Ad

Jeff Brown - the legend who called Tesla and Nvidia early - says Elon is about to launch a $25T AI revolution. This isn't another chatbot. It's real-world AI that could 14X the impact of ChatGPT. But after July 23rd, it may be too late.

Marjorie Taylor Greene: 'Muslim Takeover' Threatens American Women

Rep. Marjorie Taylor Greene (R-Ga.) has expressed her fears regarding what she believes to be the most significant threat to the freedom of American women.

Mystery Seller Behind $105 Million Plot Next to Jeff Bezos Estate Finally Revealed

A plot of land neighboring Jeff Bezos' property in Miami was sold for a staggering $105 million. The undeveloped lot was sold by German investor Mikhail Peleg.

The Unlikely Duo Rethinking Wearable Technology - Ad

Why did a surgeon and Snap exec team up? They saw the missing ingredient in the $84T wearables market: behavior. So they built Emteq and invented smart glasses sensors to track real-time expressions, unlocking insights into why we do things. With a $15M term sheet already secured, Emteq is what Big Tech's missing.

BigBear.ai (BBAI) Stock Soars 21%: What's Going On?

Shares of BigBear.ai are trading sharply higher Tuesday afternoon. The rally comes without any new company-specific announcements.

A Historic Gold Announcement Is About to Rock Wall Street - Ad

The greatest investor of all time is about to validate what Garrett Goggin has been saying for months: Gold is entering a once-in-a-generation mania. Front-running Buffett has never been more urgent - and four tiny miners could be your ticket to 100X gains.

Sig Sauer, faced with lawsuits over a popular pistol, gets protection in New Hampshire

CONCORD, N.H. (AP) — Faced with mounting lawsuits over a popular pistol, New Hampshire-based Sig Sauer asked for — and got — protection in the form of a new state law that makes it harder to .

How Senate Republicans want to change the tax breaks in Trump's big bill

WASHINGTON (AP) — House and Senate Republicans are taking slightly different approaches when it comes to the tax cuts that lawmakers are looking to include in their massive

Buffett's Next Big Investment... Is It This Gold Play? - Ad

Buffett's $325B cash pile is losing value fast - and one gold miner could be his next move. It trades at a 43% discount to its free cash flow, and fits his criteria to a T. With gold set to soar and valuations stretched, now's your chance to front-run Buffett before August 15.

Netflix's Squid Game Finale Takes Over Seoul With Massive Parade, Star-Studded Fan Event

Netflix celebrated the finale of Squid Game with a parade and fan event in Seoul, featuring characters and emotional reflections from cast.

Trump's Tariffs, Boeing's Blowout, Tesla's Robotaxi, BYD's European Triumph And GM's Massive Recall: This Week In Mobility

This week's business and finance highlights include Japan's rejection of Trump's auto tariffs, Boeing's scrutiny over a mid-air incident, Tesla's Robotaxi launch, BYD's dominance in Europe, and GM's massive vehicle recall.

The Tesla Shock Nobody Sees Coming - Ad

While headlines scream "Tesla is doomed"...Jeff Brown has uncovered a revolutionary AI breakthrough buried inside Tesla's labs. One that is helping AI escape from our computer screens and manifest itself here in the real world all while creating a 25,000% growth market explosion starting as early as July 23rd.

Quantum Stocks Surge And Stumble: Here's What Happened This Week

Quantum Computing announced a $200 million private placement, Microsoft's CEO said quantum computing could take scientific advancements to the next level and more from the week.

Back This Medical AI Tech - Ad

You don't often find healthcare tech company valued at $4M. HeartSciences is an exception. With $75M invested, including R&D, clinical trials, and product development, their patented AI software is approaching FDA submission, a potential major inflection point.

Russian Spy Chief And CIA Director Ratcliffe Agree To 'Call Any Time' Amid Tensions: Report

Russian spy chief Sergei Naryshkin and the Director of the Central Intelligence Agency (CIA), John Ratcliffe, have reportedly agreed to have direct communication at any time.

FDA Submission Expected This Summer - Invest Before It Happens - Ad

This company is making heart disease easier to detect with AI. Their tech has officially been designated a "Breakthrough" by the FDA. You now have an opportunity to get in at a potential inflection point.

Nations are meeting to drum up trillions to combat poverty — but the US isn't going

UNITED NATIONS (AP) — Many of the world’s nations are gathering starting Monday in Spain for a high-level conference to tackle the and try to drum up trillions of dollars needed to close it. The United States, previously a major contributor, pulled its participation, so finding funding will be tough.

This Coin Could Surge Like Bitcoin Did Back in 2013... - Ad

A new coin is emerging in the crypto world. And investing in it now could end up like Bitcoin or Ethereum during their first bull runs.

Iran's Top Diplomat To UN Says Uranium Enrichment To Continue: An 'Inalienable Right'

Iran reaffirms commitment to nuclear program despite US strikes. Ambassador cites Non-Proliferation Treaty as justification. Impact remains debated.

The Missing Ingredient for Big Tech's $3T Recipe? - Ad

Big Tech doesn't build everything; they buy what's next. And only Emteq's tech offers real behavior insights for smart glasses, key to unlocking the $3T XR market. Their patented sensors track live facial muscle movements. With health-focused smart glasses in demand, only Emteq has this tech.

Trump ramps up attacks on the Federal Reserve but Powell sticks to 'wait and see' stance

WASHINGTON (AP) — Federal Reserve Chair Jerome Powell on Tuesday stuck to that the central bank will keep its key rate on hold while it waits to see how President Donald Trump's effect the economy, despite the from the White House, which wants lower borrowing costs.

OpenAI Says Tokens Are Not Equity, No Partnership With Robinhood: Musk Says AI Company's Equity 'Is Fake'

OpenAI says the "OpenAI tokens" offered by Robinhood are not OpenAI equity and clarified that it has no partnership or involvement with Robinhood.

How to Hack a $1.3T Market - Ad

Forget concrete. The new foundation for real estate success is digital, and Pacaso leads the way. Their tech unlocks a $1.3T real estate market. They've already earned $110M+ in gross profits in their operating history and reserved the Nasdaq ticker PCSO.

Trump Says Israel Ready For 60 Day Ceasefire In Gaza, Hopes Hamas Takes The Deal Because It Won't 'Get Better'

Trump proposed 60-day ceasefire in Gaza, which Israel has agreed to. Qatar and Egypt will deliver the proposal to Hamas. Trump meets Netanyahu next week.

Is Elon's Empire Crumbling? - Ad

Jeff Brown - the legend who called Tesla and Nvidia early - says Elon is about to launch a $25T AI revolution. This isn't another chatbot. It's real-world AI that could 14X the impact of ChatGPT. But after July 23rd, it may be too late.

PHOTO ESSAY: In rural Alaska, collecting driftwood saves time, money and brings jobs

GALENA, Alaska (AP) — As ice breaks on the Yukon River, Jake Pogrebinsky looks at the water from shore, searching for a large log floating downstream.

The Unlikely Duo Rethinking Wearable Technology - Ad

Why did a surgeon and Snap exec team up? They saw the missing ingredient in the $84T wearables market: behavior. So they built Emteq and invented smart glasses sensors to track real-time expressions, unlocking insights into why we do things. With a $15M term sheet already secured, Emteq is what Big Tech's missing.

Robinhood Just Made Crypto Simpler — And Way More Powerful

Robinhood Markets (NASDAQ:HOOD) has made a significant push into crypto, unveiling a suite of inno

FDA vaccine official restricted COVID vaccine approvals against the advice of agency staff

WASHINGTON (AP) — The working under Health Secretary recently of two COVID-19 vaccines, disregarding recommendations from government scientists, according to federal documents released Wednesday.

A Historic Gold Announcement Is About to Rock Wall Street - Ad

The greatest investor of all time is about to validate what Garrett Goggin has been saying for months: Gold is entering a once-in-a-generation mania. Front-running Buffett has never been more urgent - and four tiny miners could be your ticket to 100X gains.

Senate Defends Trump's War Powers, Rejects Push For Congressional Approval For Further Action Against Iran

Senate rejects proposal to limit Trump's military powers against Iran. Vote largely split along party lines with only one Dem voting against.

Trending Now

Information, charts or examples are for illustration and educational purposes only and not for individualized investment management This message contains commercial elements, such as advertising. We only send these offers to those who have opted in to our newsletter. Past performance is not indicative of future results. For these reasons we strongly suggest trading in a DEMO/Simulated account. The information provided by us is for educational and informational purposes only. We make no representations or warranties concerning the products, practices or procedures of any company or entity mentioned or recommended and have not determined if the statements and opinions of the advertiser are accurate, correct or truthful. If you use, act upon or make decisions in reliance on information contained or any external source linked within it, you do so at your own peril and agree to hold us, our officers, directors, shareholders, affiliates and agents without fault.

Copyright trendadvisor.net
Privacy Policy | Terms of Service