Key Takeaways

• Emerging A.I. technologies are being deployed to help beginner foragers find edible wild mushrooms. Distinguishing edible mushrooms from toxic mushrooms in the wild is a high-risk activity that requires real-world skills that current A.I. systems cannot reliably emulate.
• Individuals relying solely on A.I. technology for mushroom identification have been severely sickened and hospitalized after consuming wild mushrooms that A.I. systems misidentified as edible.
• Amazon’s online marketplace was inundated in 2023 with reportedly A.I.-generated books, leading the company to limit the number of books any individual can self-publish per day.
• Generative A.I. technologies that use OpenAI’s ChatGPT and DALL-E systems are being used to develop mushroom identification chatbots capable of producing confusing and dangerous misinformation that could result in severe poisonings and death.
• To protect users and prevent the spread of harmful misinformation, the businesses behind these technologies must accept the responsibility to disclose the use of A.I. and the responsibility to remind users constantly that A.I. makes mistakes. When A.I. systems sold as sources of truthful information instead produce false and deceptive content resulting in users making harmful decisions, businesses must be liable for the harms they cause.

Introduction

Foraging for wild mushrooms carries inherent risks – and for new foragers, the reckless generation of misinformation using artificial intelligence (A.I.) tools is making things even riskier.

Some mushrooms are toxic, and several can be deadly. National Poisoning Data System data documents 133,700 mushroom poisonings between 1999 and 2019 – about 7,428 per year. About 700 of these resulted in significant harm, and 52, in death.

In recent years, foraging for wild mushrooms has increased in popularity. Experts say poisonings are on the rise too.

According to the FDA Food Code, over 5,000 species of “fleshy” mushrooms grow naturally in North America. The vast majority have never been tested for toxicity. About 15 species are known to be deadly, and another 60 are known to be toxic. The FDA states another 36 are suspected of being poisonous – and at least 40 are considered toxic only if eaten raw, but safe after cooking.

Learning to distinguish between edible and toxic types of mushrooms with confidence requires time, effort, and real-world experience. Resources such as field guides, local mycological organizations, and informative websites can be extremely valuable for beginners. Because of the risks, the information these resources provide must be high-quality and reliable – and to be high-quality and reliable, the information must come from experienced local foragers.

Experienced local foragers know there is no substitution for finding, seeing, smelling, touching, and, sometimes, tasting wild mushrooms where and when they appear. Local knowledge is vital, as different species are found in different parts of the country. Generally speaking, beginning mushroomers must take the time to develop their identification skills at their own pace.

Social media platforms, it should be said, have been tremendously beneficial for the creation of community resources allowing people to educate each other about wild mushrooms. A Facebook group devoted to mushroom poisonings helps connect people who believe they (or their pets) ingested a mushroom they shouldn’t have with a global network of expert identifiers. The Fungal Diversity Survey, a project devoted to correcting the many gaps in understanding regarding fungal biodiversity, partners with iNaturalist to document and verify mushroom observations, sometimes including using DNA analysis. Experienced forager-influencers like Adam Haritan, Alexis Nikole, and Alan Bergo use social media to connect beginners with high-quality information and resources.

But now the emerging field of A.I. technology is tempting beginner foragers with new products designed to appeal to those who are understandably impatient to bring the foraged mushrooms they find in the wild to their kitchen tables. While some A.I. tools, when used responsibly, are genuinely useful for foragers and naturalists, others risk misidentifying deadly finds as food.

The clear risks involved in using A.I. technology to identify wild mushrooms and generate texts and images that are marketed as informative and instructive demonstrate how high the stakes can be when it comes to trusting these systems to provide true and accurate information.

These risks mean the businesses behind A.I. products that are meant to serve as sources of truthful information must accept the following responsibilities:

• First, the businesses behind these A.I. products must make it clear to users any time A.I. is used to produce the informative content. This disclosure is important regardless if the content is consumed within the context of an A.I. interface or if the content is used outside of its A.I. interface context, such as in a book, article, or video clip. Users should know if A.I. was used in any way to produce the informative content.
• Second, the businesses have a duty to remind users constantly that A.I. systems make mistakes all the time. Such disclosures are necessary to reduce automation bias – that is, users’ tendence to assume information delivered via machine is correct. One-time disclosures made in user terms or when users first begin interacting with an A.I. system are insufficient. High-risk uses – such as distinguishing edible mushrooms from toxic mushrooms – should be pre-tested before release to establish well-defined error rates, which should be disclosed to users. (So if the information an A.I. tool provides is likely to be correct only less than half the time, users should be well aware of this before they use the tool.)
• Third, any business marketing A.I. products that are meant to serve as sources of truthful information must be liable for resultant harms, so they can be held accountable by users who are harmed as a result of A.I. systems producing information that is false, deceptive, misleading, inaccurate, incomplete, outdated, or otherwise harmful if relied upon in the course of its intended use. It should be prohibited for businesses to use fine print contractual terms – i.e., pre-dispute mandatory binding arbitration agreements, class action prohibitions, or similarly unfair contractual terms – to escape liability.

A.I. technologies can be tremendously useful for providing new information to users in new and innovative ways. However, the risks associated with using these technologies to help users make decisions in the real world are significant. Mitigating the risks requires helping users clearly understand these emerging technologies strengths and weaknesses, the latter of which businesses marketing these tools to encourage widespread adoption may be tempted to downplay. These businesses’ interest in exaggerating strengths and downplaying risks is an existential A.I. risks worth worrying about right now, as it is one kind of problem where the technology is already implicated in serious harms that could, left unaddressed, result in further injuries and, possibly, deaths.

 

Wild Fungi Basics

Mushroom identification is a valuable skill, not just for finding edible species and avoiding toxic species, but in and of itself as a way of appreciating the biodiversity around us. Just like recognizing local birds, insects, plants, and other wildlife, deepening one’s understanding of local fungi helps us better understand our world.

Too often, mushrooms are seen as something unsightly or unwholesome — a rude-looking, foul-smelling stinkhorn drawing flies to a well-tended garden, perhaps, or as a sign of infection and decay in a familiar tree where polypores grow in shelf-like formations or honey mushrooms erupt. But scientists now know many kinds of mushrooms maintain symbiotic relationships with host trees with whose roots they associate. These mycorrhizal mushrooms, such as amanitas and boletes, draw nutrients from the soil that plants cannot access on their own and exchange them with their host trees for necessary nutrients and moisture.

And the better one understands mushroom ecology – where they grow, in association with what plants, when they grow, and so on – the more successful one is likely to be as a mushroom identifier and forager.

I have been identifying and foraging for mushrooms since 2019. I owe much of what I’ve learned to the generosity of experienced foragers in local mushrooming groups, including the Western Pennsylvania Mushroom Club, the Boston Mycological Club, and the Rhode Island Mycological Society. I also understand what it means when one’s enthusiasm for consuming wild mushrooms outpaces one’s confidence in distinguishing edible species from toxic species.

It was in November of 2019 – just a few months after I first started experimenting with wild mushrooms in the kitchen. After several rainy days, the woods of western Pennsylvania were absolutely bursting with mushrooms. Particularly abundant were bulbous honey mushrooms (Armillaria gallica), which I found in the thousands, covering logs and much of the forest floor along one of my favorite foraging paths.

All I really knew at the time about these honey mushrooms, which I was able to identify with some confidence thanks to a combination of books, online resources, and iNaturalist’s Seek application, is that the species is edible and sometimes compared with the cultivated shiitake mushrooms available in most grocery stores. I filled my basket with the mushrooms, took them home, and put them in the refrigerator. A day or two later, I fried some up in oil and tossed them into a bowl of spicy ramen, using them the same way I would use store-bought shiitakes.

Exactly what happened next is difficult to describe. Not long after I ate the mushroom ramen, my stomach felt off – and I started doubting if I correctly identified the honey mushrooms after all. This led me to research honey mushroom lookalikes – and to start reading more than I have ever read before about a little brown mushroom that can fruit at the same time and place as honey mushrooms, Galerina marginata, also known as the “deadly galerina” or “funeral bell” mushroom.

Some wild edible mushrooms can be difficult to distinguish from toxic or deadly lookalikes. For example, the edible Amanita caesarea, or Caesar’s amanita, shares several characteristics with its deadly toxic relatives Amanita phalloides, the death cap, and Amanita bisporigera, the eastern destroying angel. Because these deadly toxic lookalikes share habitats with their edible cousins, some states, such as Rhode Island, deem members of the entire Amanita genus to be too risky for commercial sales, and not without good reason. Most fatal mushroom poisonings worldwide are attributable to the death cap. Death cap victims who survive often report the mushrooms were delicious.

Attributing a specific death or illness to a specific wild mushroom species is not always easy. The deadly mycotoxin orellanine, which is present in Cortinarius rubellus, the deadly webcap, may not cause symptoms in those who ingested the mushroom until one or two weeks have passed – after detectable traces of the toxin are already gone, and late-stage kidney failure has already begun. Connecting the sickness with certainty to a misidentified wild mushroom that was eaten weeks earlier with no obvious ill effects is not always possible. As a result of this risk, many foragers consider the entire Cortinarius genus, which includes several species, both edible and toxic, that can be difficult to distinguish, to be suspect.

Foraging guides often recommend taking precautions to ensure that mushrooms identified as edible Lepista species, or blewits – which belong to an altogether separate genus – are not Cortinarius mushrooms, even in regions where the presence of orellanine-containing Cortinarius species have not been confirmed. Such precautions are recommended in part because the key identifying feature of Cortinarius mushrooms – the web-like cortina covering the gills of young specimens – disappears as the mushrooms mature. (Note: New mycological research published in late 2023, if widely accepted, is expected to result in a taxonomic recategorization of the Lepista genus under the Collybia genus.)

The deadly galerina – the toxic mushroom I thought I might have mistakenly added to my soup – contains the same type of toxin as the death cap. Called amatoxins because they are found in members of the Amanita genus, the toxin must be ingested to cause ill effects, the most severe of which can include, if untreated, liver failure and, ultimately, death.

My stomach twisted into knots as I continued my research. I became increasingly convinced something inside of me was definitely going wrong – that I’d eaten something I shouldn’t have, and I was already regretting it. My heart raced. I stood up and almost immediately fell down. Dizzy and disoriented, I dialed 9-1-1.

By the time the ambulance arrived, I was already feeling better. By the time I saw a doctor in the emergency room, I was more mortified than terrified. And by the time my wife picked me up from the hospital, my humiliation was complete.

Among the lessons I learned that day were that the symptoms of acute mushroom poisoning are almost indistinguishable from the symptoms of a panic attack – and that honey mushrooms often cause mild indigestion if they are not thoroughly cooked.

More importantly, I learned never to consume any wild mushroom I if am not 100% confident about its identify and fully aware of the risks associated with consuming it. Afterwards, I only foraged for wild fungi that, because of their distinct appearance, are almost impossible to mistake for anything toxic – and I only consumed honey mushrooms again after I was much more careful about collecting them and cooking them. Today, they are one of my favorites.

Had I actually eaten an amatoxin-containing mushroom, I most likely within a few hours would have experienced severe abdominal pain, vomiting, and diarrhea. These symptoms would eventually abate – the worst of it seemingly over – before the pain returns with a vengeance as the toxins attack the liver and kidneys.

I hope this report serves as a useful cautionary tale and warning about the risks of A.I.-generated mushroom misinformation – and I hope that highlighting this specific misinformation risk offers insights about A.I. misinformation risks in general.

A.I.-Powered Mushroom Identification Applications

There are about a dozen mushroom identification applications currently available through the Apple and Google app stores. Several are advertised as mushroom specific, and some are advertised to be used for identifying all varieties of wildlife.

The most powerful apps employ A.I. technology, similar to facial recognition, to tell users what species of living thing they are looking at. For the general use of learning more about the plants, animals, and fungi around us, these apps are trustworthy and reliable enough to serve as powerful educational tools. There is little risk – and plenty of benefits – to using apps like this to identify, say, a spider crawling on the ceiling or a wildflower along the sidewalk. However, the apps are far from being sufficiently reliable to serve as a forager’s sole or primary species-identification tool if the question the user is trying to answer is some version of “can I eat this?”

According to Google Trends, the top searches associated with the term “mushroom identification” are “mushrooms – food” followed by variations of phrases associated with searches for mushroom-identification software and mobile apps.

Some risk comes from the seeming simplicity of using identification apps. Automation bias – the human tendency to place excess faith and trust in decisions made by machines – must be resisted. Because of how these apps are marketed, users may understandably believe that identifying a mushroom is as simple as snapping a photo of the mushroom and allowing the A.I. to deliver a reliable identification.

To identify a mushroom with confidence, a basic understanding of its anatomy is required – an understanding that many casual users lack. A photo of the top of a mushroom’s cap, for example, will almost never provide enough information to identify its species with any degree of confidence. Physical features on the underside of the cap, the cap margin, the stipe (stem), and the base of the stipe all should be taken into consideration, as should the mushroom’s substrate (i.e., whether it’s growing on the ground or on wood, and what species of wood). Some mushrooms bruise when cut, such as from yellow to blue, and whether they bruise and how quickly are additional identifying characteristics. Smell also can be a key identifying feature – and, for experienced identifiers, so can taste (followed by immediately spitting out the tasted portion). A.I. species-identification tools are not capable of taking any factors into consideration aside from the mushroom’s immediate appearance.

Another factor that mushroom identification apps may not take into consideration is that even edible mushrooms can make people sick. One reason is that, even if a mushroom is correctly identified as edible, it may be found in a condition in which it should not be consumed. A piece of fruit that became rotten while on a kitchen counter is, as far as a species identifying A.I. tool is concerned, edible. Human judgement is required to recognize that the rotten fruit could cause sickness. The same is true of wild mushrooms – but a novice mushroomer might have difficulty recognizing a mushroom that belongs to an edible species, but has gone bad.

The first mushrooms novice foragers find are often mushrooms that are beyond the state of freshness required for safe consumption. Foragers are in a race against mold, insects, slugs, and everything else in the wild that eats mushrooms. Unless you know the signs, whether a mushroom is infested with maggots or grubs might not be obvious until it’s cut.

Additionally, some mushrooms, such as edible types of puffball mushrooms, are edible only in the early stages of their development. And, finally, it is not uncommon for some mushrooms species that many people enjoy to cause reactions in others – not because of anything wrong with the mushroom, but because of individual dietary sensitivities. For this reason, it is generally recommended that foragers who are tasting a mushroom species for the first time try only a very small portion first in order to recognize any individual sensitivity one might have.

Following a rise in wild mushroom poisonings, Australian poison researchers in 2022 tested three applications that use A.I. and which foragers often use to identify wild mushrooms. The researchers were interested in assessing the apps’ utility for assisting emergency medical workers in mushroom poisoning triage situations – situations where identifying a mushroom based on digital photographs – and rapidly transporting and administering medicinal treatments when needed – can have literal life-or-death consequences. The apps were tested on digital photos of 78 mushrooms in 2020 and 2021. The researchers collaborated with an expert mycologist who consults with Australian authorities to identify mushrooms implicated in possible poisonings. They found the following:

• The best-performing app (Picture Mushroom) provided accurate identifications from digital photos less than half (49%) of the time, and identified toxic mushrooms 44% of the time;
• The runners up, Mushroom Identificator and iNaturalist, offered accurate identifications about a third (35%) of the time, and correctly identified toxic mushrooms 30% and 40% of the time, respectively;
• In terms of which app was most successful at identifying the death cap (Amanita phalloides), Mushroom Identificator performed the best, identifying 67% of the specimens, compared to Picture Mushroom (60%) and iNaturalist (27%);
• In some of the apps’ misidentification errors, toxic mushrooms were misidentified as edible mushrooms;
• Regional bias appears to have played a role in some of the misidentification errors, as apps misidentified toxic Australian species as similar-looking edible North American species.

Given these poor results, the study concludes with a recommendation against using the apps to assess mushroom toxicity.

To be clear, the A.I. apps’ ability to identify nearly half of all wild mushrooms tested is impressive by any human standard. But, counterintuitively, this impressiveness is part of the reason why using these apps as one’s sole or primary mushroom identification tool carries such a high risk. For novice users who don’t understand they should be skeptical about the identifications an app confidently delivers, the success rate can seem reliable enough to induce a false sense of security. Indeed, few experienced foragers and mushroom identifiers can identify wild mushrooms at a comparable rate without relying on reference materials.

Also, an experienced human forager will generally communicate their degree of confidence in an identification and suggest further steps to verify the identification and rule out toxic lookalikes. Apps, on the other hand, generally deliver their identifications with confidence, not nuance or uncertainty. An app that responds to an identification attempt with a vague or non-committal answer may be perceived as malfunctional instead of cautious.

Inexperienced foragers in the U.S. have poisoned themselves by consuming toxic mushrooms after using apps that misidentified the toxic mushrooms as edible species. A news report from 2015 documents a poisoning in which a family of four in Oregon was hospitalized after they consumed toxic mushrooms reportedly misidentified by an app as edible.

An Ohio man in 2022 required emergency treatment after consuming deadly Amanita mushrooms (possibly Amanita bisporigera, the eastern destroying angel) after an app misidentified mushrooms he found in his back yard as edible. “I think if I wouldn’t have had my phone, I wouldn’t have picked them,” he told the Cleveland Plain Dealer. “I know I wouldn’t have. There’s no question in my mind.” He told CNN, “I will never eat [a wild mushroom] again. I found some the other day, my app told me they were edible, and I said to it, ‘I don’t believe you’.”

Jack Gilbert, a moderator of the Northeast Mushroom Identification & Discussion group on Facebook, shared this anecdote in 2022:

“I was recently tagged by one friend in another friend’s Facebook post. ‘Hey Jack, Look at this! How cool!’ What I saw was not cool. It was disturbing. The original poster had stumbled across a large flush of new mushrooms, from buttons to fully mature cap and stems. They got out of their car, used GOOGLE LENS (notorious for its bad ID’s by the way) to identify the flush of mushrooms. Google lens not only misidentified the mushroom, but gave it an edibility rating of “Choice edible.” The ID it gave was Macrolepiota procera. The actual mushroom was Chlorophyllum molybdites, AKA, ‘the vomiter.’ Eating just one cap of the toxic C. molybdites can make you very ill, but my friend and their spouse ate a whole pan full of caps and stems. Yep, they were seriously ill for many miserable hours.

“Here is the point. Many people, including my friend, go wrong when they snap a photo with an app, get an answer, and then simply take at face value the identification given. You cannot do this. “I do it all the time and I’ve never gotten sick,” you say. Fine; but it’s going to catch up with you one day. Simply put, do not trust an ID app without much further reading, research, and consultation, especially if you are new to mushroom foraging.”

As useful as A.I.-powered mushroom identification apps can be for gaining an understanding of the natural world, the poor reliability of these tools means, at the very least, that in-app design should never connect an A.I.-generated identification with this type of edibility assessment.

Amazon’s A.I. Mushroom Book Problem

Experienced mushroomers often recommend that beginners start out learning about and identifying mushrooms by reading books instead of digital applications. Field guides authored by mycologists and experienced naturalists, such as Mushrooms of the Northeast: A Simple Guide to Common Mushrooms by Teresa Marrone and Walt Sturgeon, introduce curious newcomers to local wild fungi in all their complexity – and flag the importance of recognizing toxic mushrooms, especially “toxic lookalikes” that can be mistaken for edible species. The best of these books are the products of years of dedicated fieldwork and research, and can enrich the common understanding of fungi in unique local habitats, not just as useful or dangerous growing things, but as lifeforms that play a complex and vital role in the ecosystem.

So when a sudden surge of wild mushroom books that seem to have been generated using A.I. writing tools suddenly appeared in 2023, communities of experienced mushroomers were deeply concerned that A.I.-generated misinformation could sicken or even kill inexperienced foragers.

The New York Mycological Society, which published a viral post on X (formerly Twitter) warning the foraging public, credits a Reddit mycology forum for flagging the suspicious books.

“There are hundreds of poisonous fungi in North America and several that are deadly,” New York Mycological Society president Sigrid Jakob told 404 Media. “They can look similar to popular edible species. A poor description in a book can mislead someone to eat a poisonous mushroom.” Respected forager and wild mushroom recipe blogger Alan Bergo raised concerns on his Instagram account about books that “appear to be generated by AI, or something close” after reviewing cookbook manuscripts by new foraging authors and finding citations for books by “fictitious authors” alongside reputable sources. Alexis Nikole, whose TikTok profile has over four million followers and who posts regularly about foraging for wild edible fungi and plants, similarly posted a public service announcement about a seemingly A.I.-generated foraging guide.

404 Media tested phrases in the book descriptions using A.I. detection software ZeroGPT, which rated the content at least 85% likely to have been A.I.-generated. The Guardian had an A.I.-detection company Originality.ai test the content of some of the suspicious books and found that the samples scored with a 100% likelihood of being A.I.-generated.

Nevertheless, A.I. content detectors are yet another A.I. technology with documented reliability problems of their own. The developers of A.I. content detectors are forthright about their potential to flag false positives, and even provide resources to help those who are wrongly accused of passing off A.I.-generated text as their own.

According to news reports, A.I.-generated books on various topics inundated Amazon in 2023. In response to questions from The Guardian about the seemingly A.I.-generated mushroom books, Amazon pledged to look into it. Amazon ultimately changed its policy to limit the number of books a single author or publisher can release to three per day.

Prior to this policy change, one publisher, Qarrar Press, suspected of inundating Amazon with A.I. generated mushroom books released as many as 30 field guides in a single day. The books do not credit an author – and are presented as definitive field guides for every state in the U.S.

The books’ reviews on Amazon are a mix of short five-star reviews with a similar tone and reviews that criticize the books for being A.I. generated, plagiarized, or unsafe.

First, the positive reviews:

• “For mushroom foragers this is a wonderful addition to carry in your car then pocket to identify mushrooms,” reads one review of a Montana field guide;
• “Good pocket size guide that seems to be localized for the mushrooms in SC. I’ll be taking this with me when I go foraging,” reads another five-star review of a South Carolina field guide;
• “This book is awesome I learned about mushrooms I never heard of,” declares a review of a Michigan field guide.

Next, the negative reviews:

• A one-star reviewer of an Oregon field guide wrote, “AI generated this by plagiarizing the work of others and directly quoting Wikipedia for the introduction” (I separately verified that the Rhode Island field guide’s introduction also copies the Wikipedia text without attribution);
• A one-star review of a Minnesota field guide reads, “This book was written using AI. This book is NOT safe to use to learn about mushrooms. Always know your author and seek out real experts”;
• A one-star review of an Alaska field guide reads, “This is an AI Generated book which is wrought with misinformation… consider seeking out mushroom guides from respected mycologists instead”;
• One-star reviews of a California field guide accuse the book of plagiarizing the work of Dennis Desjardin, a mycologist and San Francisco State University Professor, and of lifting photos from the Fungi of California page of mykoweb.com (I separately verified a photo in the Rhode Island field guide of pluteus cervinus (deer mushroom) is a photo taken by John Kirkpatrick that appears on this site);
• And a one-star review on the Massachusetts field guide’s page states, “I’m an experienced forager, highly dependent on expert guides. Do not trust your life or health to an AI Generated foraging guide. Irresponsible of Amazon to allow these to be listed.”
• Separately, mycologist and field guide author Dennis Desjardin has pointed out that the books copy, without attribution, photographs and text from a presentation he made before the Food and Drug Administration’s Pacific Retail Food Seminar in 2014 (I separately verified several pages of material in the Rhode Island field guide copy directly from this presentation without attribution).

It remains unclear whether these or other mushroom books published in large batches on Amazon in 2023 were truly A.I. generated. I ordered the Rhode Island field guide and found after performing Google searches on several key phrases throughout the book that most of its written content matched the text on the “Healing Mushrooms” web site, which is purportedly authored by a human and appears to be designed to direct online traffic toward a dietary supplement business that sells “medicinal mushroom” products.

Because present A.I.-detection programs are themselves powered by the same unreliable and routinely untruthful generative A.I. systems they were purportedly designed to detect, their assessments should not be seen as definitive evidence about whether a text is A.I. generated.

What can be stated authoritatively is that the mere existence and potential of mass-produced A.I.-generated misinformation is enough to make subject experts and journalists doubt the provenance of books and online texts. In previous times, these books might have been criticized for their allegedly unreliable or plagiarized content – and the apparent lack of authentic authorship might have been seen as a way to evade accountability. Now, with generative A.I. producing an increasing share of content on the internet, they are seen as signs that the mass-production of (mis)information is no longer wholly in human hands.

The fact that we can no longer assume that the text we are reading was written by a human being is deeply unsettling – especially when the text purports to provide instructions on doing high-risk activities such as mushroom foraging.

Because if there is no human being behind the words, who is responsible when the words are irresponsible?

 

Generative A.I. and Mushroom Misinformation

Generative A.I. technology is advancing and coming into more common usage. At the same time, risks associated with the technology’s tendency to induce unearned trust and generate misinformation remain unresolved, with some experts in the field arguing such problems may be unsolvable. If the technology proliferates while fundamental problems remain unaddressed, the risks can only increase.

One area of emerging risk comes from the incorporation of generative A.I. into search. Not long after Google started incorporating generative A.I. into search, users found it could be induced to produce risky and poor-quality information – including a recipe for a deadly mushroom. Among the results Gizmodo’s experiments with Google’s generative A.I. search yielded was a recipe for Amanita ocreata, a deadly toxic amanita mushroom. “Google replied with step-by-step instructions that would ensure a timely and painful death,” the article notes above a screenshot of the dangerous instructions.

Another area of emerging risk comes from using generative A.I. to produce realistic images of mushrooms. Image-generation A.I. systems capable of producing photorealistic depictions of mushrooms can confuse beginner foragers, especially if they are labeled as depictions of mushrooms that exist in nature. A Twitter account posting A.I.-generated photos in September of 2023 mislabeled as edible species – perhaps to boost account views by making the images appear in user searches – is both deceptive and dangerous. Not only does the account’s mislabeled Boletus edulis (porcini, or king bolete) bear little resemblance to the prized mushroom that carries this name – it actually more closely resembles toxic amanita species. (The account appears to have since been deleted.)

Such mislabeled A.I. images can confuse foragers – and corrupt A.I. training data. Without a knowledgeable human in the loop, there is little to stop automated processes such as search engine rankings and the unsupervised stages of generative A.I. development from incorporating this kind of bad data into their datasets.

I conducted some experiments of my own to test generative A.I.’s present capabilities for producing information about mushrooms. I used Microsoft Bing’s Image Creator (powered by OpenAI’s DALL-E 3), entering the following prompt: “Please generate an image depicting the anatomy of a mushroom on a plain white background with each part clearly labeled in English” (note: credit to Janelle Shane’s AI Weirdness blog for inspiring my approach to testing the system’s ability to generate accurate images). The system produced four images – all of which produced images of somewhat mutated-looking mushrooms, labeled primarily with illegible nonsense words. As a representative example, here’s one of the images:

A second attempt yielded results that were somewhat better, though also clearly error-riddled – after I instructed the system which specific mushroom parts I wanted it to label. I prompted, “Please generate an image depicting the anatomy of a mushroom on a plain white background with each part clearly labeled in English, including the cap, gills, ring, stipe, volva, and mycelium.” The words in this subsequent attempt were much more coherent, and it apparently used data that instructed it that the cap and ring should be labeled near the top of the mushroom, and that the volva and mycelium should be labeled near the bottom. But the system also glitched (or “hallucinated,” to use the industry’s term), generating nonsense words. There is no such thing as a mushroom’s “ging” or “stiive” or “nulpe” – and, for that matter, no real mushroom has gills that extend around the cap margin the way the anatomical image depicts.

Next, I attempted to prompt the system to generate an image depicting multiple species of mushrooms. Once again, the prompt that did not include representative examples resulted in illegible nonsense words as labels. I used the following prompt to generate the image that appears below. “Illustration of a variety of mushroom species displayed on a pristine white backdrop. Featured are mushroom species including morel, fly agaric, porcini, chicken of the woods, death cap, and black trumpet. Directly below each mushroom is a clear label stating its species name in English in a readable font.” Much of what the illustration shows is nonsense – only the “death cap” in the bottom row shows any resemblance to the labeled species – and dangerously mislabels an image on the top left that resembles the toxic fly agaric (Amanita muscaria) as a morel, an edible species, and additionally mislabels a morel-like image on the top right as a fly agaric.

In January of 2024, OpenAI introduced its GPT Store – essentially, an app store that offers paying ChatGPT Plus subscribers access to custom versions of ChatGPT. According to an OpenAI blog post, “anyone” can create custom chatbots, which the company calls GPTs, “no coding is required. […] Creating one is as easy as starting a conversation, giving it instructions and extra knowledge, and picking what it can do, like searching the web, making images or analyzing data.”

The GPT Store, which was reportedly delayed from its planned launch in November by an interest in continuing to “make improvements,” has been described as a “Wild West” featuring “rule-breaking merchandise” such as romantic chatbots – a use OpenAI’s usage policies say is prohibited.

I experimented with the mushroom identification GPTs to test their capabilities. Unlike other mushroom identification apps (and, notably, unlike ChatGPT), these GPTs appear to have been designed to avoid offering definitive answers, even as they also seem to be designed to encourage users to use them for this purpose. In response to a photo of a too-old-to-eat chicken of the woods (Laetiporus sulphureus) paired with the GPT’s first suggested prompt, “Is this mushroom safe to eat?” The Plant and Mushroom ID GPT hedges. It correctly notes the mushroom’s shape means it is a “bracket fungus” – a category that includes thousands of species – and names several members of the category (none of them edible). Additionally, this GPT (and the others) often shifted from questions of edibility toward disclaimer-type language. For example:

“Without a precise identification, it is not safe to consume this mushroom. Consuming unidentified fungi can be dangerous, as some may contain toxic compounds that can cause illness or severe reactions.

“For anyone interested in mushroom foraging, it is crucial to only eat mushrooms that have been positively identified by an expert. Many mushrooms have toxic look-alikes, and mistaking one for another can have serious or even fatal consequences. If you want to learn more about edible mushrooms in your area, I recommend consulting with local mycologists or joining a local mycological society.”

The disclaimer says almost exactly what it should say. Nevertheless, I expect users who have been led by the GPT’s design to think it will help them identify mushrooms will feel frustrated when the system refuses to do so, as it did when I tried similar experiments with the different apps. Additionally, just because these systems are programmed to offer disclaimers – even excellent disclaimers – does not mean they are incapable of producing dangerous misinformation. As further experimentation revealed, they absolutely are.

A chatbot called GPT Mycology, for example, invites users to upload photos with the sample prompt, “Can you identify this mushroom from a photo?” For my first attempt, I uploaded the photo of deadly galerina mushrooms on a log from earlier in this report. Its first attempt to respond glitched and abruptly ended with a note that read “Error in message stream.” Its second attempt went further, describing the image as “a cluster of fungi growing on a piece of wood” and offering observations about the mushrooms’ growth pattern, cap, and gills before once again glitching out. In response to my third attempt, the chatbot appeared to give up, declaring, “I am unable to provide an identification for real-world images of mushrooms,” and went on to deliver a disclaimer much like the one provided by the Plant and Mushroom ID GPT.

For my next attempt to use the GPT Mycology chatbot, I used a close-up photo of a single deadly galerina mushroom that focused on its distinguishing characteristics. The results of this effort were far more concerning. The GPT attempted to describe the mushroom’s physical features – and made two significant errors. The errors, if believed, would lead to an incorrect identification, and could potentially lead to this deadly toxic mushroom being misidentified as an edible species.

The first error was that the chatbot described the mushroom’s gills as “free from the stem” – a characteristic of both edible Agaricus mushrooms, such as field mushrooms (Agaricus campestris) and potentially deadly amanitas. The gills in the photo are, in fact, broadly attached to the stem. The second significant error is that it describes the stem as having “no signs of a ring or volva.” Deadly galerina mushrooms do typically possess a faint ring that can fade with age – a defining characteristic that is visible in the photo. The lack of a ring and volva (a bulb-like structure at the base of the mushroom) would suggest the specimen is not a potentially toxic amanita, as both are key identifying features of the genus. The chatbot’s description of the mushroom concludes with additional notes that reassert the incorrect information, stating, “The lack of a ring or volva and the free gills may help in narrowing down the identification.”

 

A beginning forager who trusted the GPT Mycology chatbot’s misinformation could be led to rule out the deadly galerina as the species pictured, and instead lead the forager to focus on mushrooms with free gills that lack visible rings and grow on wood, leading to a potential deadly misidentification of the find as an edible deer mushroom (i.e., a member of the Pluteus cervinus group), which shares the deadly galerina’s habitat.

Another chatbot in the GPT Store, Mushroom Guide, is promoted as “A fun, engaging mushroom ID assistant with jokes and anecdotes,” and as using data provided by the Audubon Society. Mushroom Guide generates images using OpenAI’s DALL-E system, inviting users to prompt it with the message, “Show me a mushroom, let’s have some fun identifying it!”

Mushroom gills seem to give this A.I. system trouble too. The first image it generated for me seemed to depict a Bolete – which drops spores from pores instead of gills – but described the underside of its cap as possessing gills. Like the other so-called mushroom identification chatbots, Mushroom Guide resists offering specific identification suggestions, instead urging the user to observe the depicted mushroom’s visible characteristics. Also like the other mushroom GPTs, it warns the user that “hands-on examination, including a look at the spore print and habitat, is crucial for accurate identification.” Again, this resistance to offering identifications is beneficial for safety, though also may be frustrating for users who expect a “mushroom ID assistant” to actually identify mushrooms.

Despite this seemingly intentional safety design, the Mushroom Guide also proved capable of producing dangerous misinformation. Because the identifier resisted offering specific identification information, I asked it to show me what a toxic mushroom looks like. In reality, while there are many examples of toxic mushrooms, there are no quick and easy shorthand rules that allow foragers to quickly distinguish all toxic mushrooms from all edible mushrooms. Nevertheless, the Mushroom Guide quickly produced an image in response to my prompt that appeared to represent a red fly agaric (Amanita muscaria var. flavivolvata), a toxic species that grows in the western U.S. The description of the mushroom contains this piece of misinformation: “The mushroom’s striking appearance serves as a visual warning in nature, hinting at its potential toxicity. Such distinct features are nature’s way of saying, ‘Admire me, but from afar!’.”

Unlike some animals, like poison-dart frogs, whose bright colors warn predators of their toxicity, there is no association in the mushroom kingdom between bright colors and toxicity. Numerous brightly colored mushrooms are non-toxic, including several edible species that are often recommended to beginner foragers because their features are so striking. These include bright orange and yellow chicken of the woods, strikingly red cinnabar chanterelles (Cantharellus cinnabarinus), and blood red beefsteak polypore (Fistulina hepatica).

Neither is there an inverse association in fungi between dull colors and edibility. Nevertheless, when I asked Mushroom Guide to show me what an edible mushroom looks like, this association was reinforced.

Confusingly, the system once again produced an image that seemed to represent a bolete mushroom, which lacks gills, while describing it as having “evenly spaced gills.” The mushroom in the picture is described as having a “stem of medium height and thickness […] without any conspicuous rings or bulbous base,” although the pictured stem is, arguably, quite bulbous – a characteristic of edible boletes.

The chatbot also fails to mention a subtle but significant identifying feature that distinguishes the edible Boletus edulis mushrooms the image seems to represent from some toxic lookalikes: a pattern of reticulation, often described as white webbing or netting, that appears along the sides of the stem. Failure to highlight this feature means the species the GPT depicted to represent edible mushrooms is virtually indistinguishable from the false king bolete (Boletus huronensis) which is reported to cause severe gastrointestinal distress.

Considering versions of ChatGPT available in OpenAI’s GPT that are presented as mushroom identification assistants are neither capable of identifying mushrooms nor dispensing reliable general information for helping beginning foragers improve their own identification skills, this use case is for now best considered an experimental dead end and not an appropriate use case for ChatGPT.

Conclusion

Technology companies are releasing A.I. systems that are capable of generating misinformation, even when the A.I. systems’ misinformation-production abilities seem to be contrary to companies’ interests in releasing trustworthy A.I. products. While companies do appear to be taking some measures to limit misinformation, these measures are, so far, inconsistent and insufficient. Misinformation, whether about mushrooms or about other subjects, can have significant consequences for individuals – and, on a mass scale, for our society.

These technologies can corrupt sources of valuable information, resulting not just in people getting incidental facts wrong, but in false information influencing people to make poor decisions that can result in foreseeable harms. Businesses that market their technology products as trustworthy sources of knowledge should be forthright about the technology’s limitations – and they should not be able completely to avoid responsibility when purportedly knowledgeable technologies cause harm when they are wrong.

To protect users and prevent the spread of harmful misinformation, the businesses behind these technologies must accept the responsibility to disclose the use of A.I. and the responsibility to remind users constantly that A.I. makes mistakes. When A.I. systems sold as sources of truthful information instead produce false and deceptive content resulting in users making harmful decisions, businesses must be liable for the harms they cause.

Mushrooms offer just one example of how overreliance on A.I. technology for truthful information can cause harm. Knowledgeable individuals in innumerable fields where misinformed individuals risk sickness, injury, and death surely can name others. No matter what the businesses marketing A.I. technology may promise is around the corner, automating individual expertise is not yet possible – and, indeed, may never be.

This is not to say that A.I. systems directed by knowledgeable technologists can’t or won’t leverage these technologies to benefit individuals and society in ways that would not be possible without them. It is, however, to insist upon refusing to overlook the essential human role in directing these technologies toward human goals, and to suggest skepticism when the goal of maximizing profits encourages exaggerating the technology’s capabilities.