Episode 28: LLMs Are Not Human Subjects, March 4 2024
Episode 28: LLMs Are Not Human Subjects, March 4 2024  
Podcast: Mystery AI Hype Theater 3000
Published On: Wed Mar 13 2024
Description: Alex and Emily put on their social scientist hats and take on the churn of research papers suggesting that LLMs could be used to replace human labor in social science research -- or even human subjects. Why these writings are essentially calls to fabricate data.References:PNAS: ChatGPT outperforms crowd workers for text-annotation tasksBeware the Hype: ChatGPT Didn't Replace Human Data AnnotatorsChatGPT Can Replace the Underpaid Workers Who Train AI, Researchers SayPolitical Analysis: Out of One, Many: Using Language Models to Simulate Human SamplesBehavioral Research Methods: Can large language models help augment English psycholinguistic datasets?Information Systems Journal: Editorial: The ethics of using generative AI for qualitative data analysisFresh AI Hell:Advertising vs. reality, synthetic Willy Wonka editionhttps://x.com/AlsikkanTV/status/1762235022851948668?s=20https://twitter.com/CultureCrave/status/1762739767471714379https://twitter.com/xriskology/status/1762891492476006491?t=bNQ1AQlju36tQYxnm8BPVQ&s=19A news outlet used an LLM to generate a story...and it falsely quoted EmilyAI Invents Quote From Real Person in Article by Bihar News Site: A Wake-Up Call?Trump supporters target Black voters with faked AI imagesSeeking Reliable Election Information? Don’t Trust AICheck out future streams on Twitch. Meanwhile, send us any AI Hell you see. Our merch store is now live on the DAIR website! Find our book, The AI Con, here. Subscribe to our newsletter via Buttondown. Follow us! Emily Bluesky: emilymbender.bsky.social Mastodon: dair-community.social/@EmilyMBender Alex Bluesky: alexhanna.bsky.social Mastodon: dair-community.social/@alex Twitter: @alexhanna Music by Toby Menon.Artwork by Naomi Pleasure-Park. Production by Ozzy Llinas Goodman.