USC Information Sciences Institute
USC Information Sciences Institute
  • 485
  • 142 403
Can LLMs Generate Novel Research Ideas?
Date Presented: 11/14/2024
Speaker: Chenglei Si, Stanford University
Visit links below to subscribe and for details on upcoming seminars:
www.isi.edu/isi-seminar-series/
www.isi.edu/events/
Abstract: Recent advancements in large language models (LLMs) have sparked excitement about their potential to accelerate scientific discovery, particularly in automating research. However, no prior evaluations have demonstrated that LLMs can generate novel, expert-level ideas. In this talk, we present a human study that evaluates research idea generation, controlling for confounders, and conduct the first direct comparison between LLMs and expert NLP researchers. By recruiting over 100 NLP researchers to generate ideas and review them blindly, we find that LLM-generated ideas are judged as more novel than human-generated ones, though slightly weaker in feasibility. Our study highlights critical challenges in building and evaluating research agents, such as LLMs’ self-evaluation failures and lack of diversity in generation. We also discuss limitations of the current work and outline future directions to address them.
Speaker's Bio: Chenglei Si is a second-year PhD student at Stanford NLP where he is co-advised by Tatsu Hashimoto and Diyi Yang. His research lies at the intersection of NLP and HCI. These days, he is most interested in how LLMs can transform scientific research, and new forms of human-AI co-intelligence. Towards this end, he splits his time between working with LLMs and running human studies.
Переглядів: 14

Відео

OATH-Frames: Characterizing Online Attitudes Towards Homelessness with LLM Assistants
Переглядів 2121 годину тому
Date Presented: 11/7/2024 Speaker: Jaspreet Ranjit, USC Visit links below to subscribe and for details on upcoming seminars: www.isi.edu/isi-seminar-series/ www.isi.edu/events/ Abstract: Public attitudes towards key societal issues, expressed on online media, are of immense value in policy and reform efforts, yet challenging to understand at scale. We study one such social issue: homelessness i...
InterIntent: Investigating Social Intelligence of LLMs via Intention Understanding in a Game Context
Переглядів 3914 днів тому
Date Presented: 10/31/24 Speaker: Ziyi Liu, USC Visit links below to subscribe and for details on upcoming seminars: www.isi.edu/isi-seminar-series/ www.isi.edu/events/ Abstract: Large language models (LLMs) have demonstrated the potential to mimic human social intelligence. However, most studies focus on simplistic and static self-report or performance-based tests, which limits the depth and v...
Mission: Impossible Language Models
Переглядів 10521 день тому
Date Presented: 10/24/24 Speaker: Julie Kallini, Stanford University Visit links below to subscribe and for details on upcoming seminars: www.isi.edu/isi-seminar-series/ www.isi.edu/events/ Abstract: Chomsky and others have very directly claimed that large language models (LLMs) are equally capable of learning languages that are possible and impossible for humans to learn. However, there is ver...
Why Are Human Laws So Difficult For AI to Follow?
Переглядів 13028 днів тому
Date Presented: 10/18/2024 Speaker: John Licato, USF Visit links below to subscribe and for details on upcoming seminars: www.isi.edu/isi-seminar-series/ www.isi.edu/events/ Abstract: Although it is now incredibly easy to create and deploy a chatbot for almost any application, powered by highly capable LLMs, even the best systems still tend to perform poorly when they need to interpret and reas...
“AI needs to have a trustworthiness index and sources monitoring”
Переглядів 9 тис.Місяць тому
Adam Russell, Director of the AI Division at ISI, is on a journey to help build a better world with AI. Amandeep Singh, a senior research engineer at ISI, shares why he believes artificial intelligence should be used in moderation. He explains what his work in knowledge graphs accomplishes and how it can be used to power AI assistants. He describes the premise of one of his favorite recent game...
Modeling American Sign Language via Linguistic Knowledge Infusion (Includes ASL Interpretation)
Переглядів 137Місяць тому
Date Presented: 9/26/2024 Speaker: Lee Kezar, USC Visit links below to subscribe and for details on upcoming seminars: www.isi.edu/isi-seminar-series/ www.isi.edu/events/ Abstract: As language technologies rapidly gain popularity and utility, many of the 70 million deaf and hard-of-hearing people who prefer a sign language are left behind. While NLP research into American Sign Language (ASL) is...
“AI is not smarter than us, but it will try harder than us, for better or worse”
Переглядів 42Місяць тому
Adam Russell, Director of the AI Division at ISI, is on a journey to help build a better world with AI. Karl Jacob, co-founder and CEO of LoanSnap, discusses how his experiences at USC and ISI led him to where he is today. He shares the skill he gained outside of the classroom that has helped him through his career as a “serial entrepreneur.” He explains why the amount of power artificial intel...
Computational Epidemiology During the COVID-19 Pandemic
Переглядів 34Місяць тому
Date Presented: 9/18/2024 Speaker: Professor Maimuna Majumder, Harvard Medical School and Boston Children’s Hospital Abstract: In this talk, Professor Majumder will discuss her recent work in computational epidemiology (comp epi): an interdisciplinary field at the intersection of machine learning, digital data, and public health. With a focus on the COVID-19 pandemic, topics covered will includ...
How Far Are We from Achieving AI Automation in the Digital World?
Переглядів 2892 місяці тому
Date Presented: 9/13/2024 Speaker: Shuyan Zhou, Duke University Abstract: For years, my dream has been to create autonomous AI agents capable of carrying out tedious procedural tasks (e.g., arranging conference travel), allowing me to focus on more creative and exciting tasks. Modern AI models, especially large language models (LLMs) like ChatGPT, have suddenly brought us much closer to achievi...
"I worry about AI researchers not knowing enough about social sciences"
Переглядів 762 місяці тому
Adam Russell, Director of the AI Division at ISI, is on a journey to help build a better world with AI. Michael Pazzani, a principal scientist at ISI, discusses what surprises him about the way LLMs can explain ideas. He explains how studying psychology and his past work in “personalization for the wireless web” affect his experiences interacting with technology today. He explores the subject o...
Composable Interventions for Language Models
Переглядів 593 місяці тому
Date Presented: 8/9/2024 Speaker: Arinbjörn Kolbeinsson Abstract: Test-time interventions for language models can enhance factual accuracy, mitigate harmful outputs, and improve model efficiency without costly retraining. But despite a flood of new methods, different types of interventions are largely developing independently. In practice, multiple interventions must be applied sequentially to ...
“Today you have to do AI combined with some other field to be relevant”
Переглядів 603 місяці тому
Adam Russell, Director of the AI Division at ISI, is on a journey to help build a better world with AI. Jose-Luis Ambite, a principal scientist at ISI, discusses a recent AI social media trend that shocked him. He describes the benefits of federated learning for privacy, and what problems in the world he thinks it might solve. He shares the short story that influenced his view on AI and why oth...
Nexa AI - Functional Tokens for On-device Multimodal Models
Переглядів 1814 місяці тому
Date Presented: 7/12/2024 Speakers: Alex Chen Zach Li Abstract: Tokenizing corpora into semantic tokens has proven effective for large language models. However, this approach encounters challenges when applied to function calls, leading to inaccuracies and hallucinations. To address this issue, we have pioneered a new training methodology using functional tokens, transforming complex function c...
“If you study AI you need to be curious and not pay attention to the noise”
Переглядів 734 місяці тому
Adam Russell, Director of the AI Division at ISI, is on a journey to help build a better world with AI. Pei Zhou, a research assistant at ISI, shares why good communication is so important in his field. He describes a challenge some AI models face and how improved self-awareness could help solve it. He explains how students can approach research to find their passion, which he believes is impor...
On Robustness and Generative Modeling
Переглядів 2144 місяці тому
On Robustness and Generative Modeling
“AI could be an advisor to someone who does not have a support system”
Переглядів 454 місяці тому
“AI could be an advisor to someone who does not have a support system”
"There is no end goal for AI ethics, there will always be something new to mitigate"
Переглядів 715 місяців тому
"There is no end goal for AI ethics, there will always be something new to mitigate"
Human-AI Interaction: From Supporting Surgical Training to Inspecting Social Bias in LLMs
Переглядів 585 місяців тому
Human-AI Interaction: From Supporting Surgical Training to Inspecting Social Bias in LLMs
“I worry that AI will be used to manipulate large groups of people”
Переглядів 1025 місяців тому
“I worry that AI will be used to manipulate large groups of people”
AI for Fostering Constructive Online Conversations
Переглядів 525 місяців тому
AI for Fostering Constructive Online Conversations
"AIs will give you answers even if they don’t know"
Переглядів 546 місяців тому
"AIs will give you answers even if they don’t know"
Causal Inference to Inform Curation Practices in Online Platforms
Переглядів 366 місяців тому
Causal Inference to Inform Curation Practices in Online Platforms
Event Extraction for Epidemic Prediction
Переглядів 656 місяців тому
Event Extraction for Epidemic Prediction
Understanding LLMs through their Generative Behavior, Successes and Shortcomings
Переглядів 1116 місяців тому
Understanding LLMs through their Generative Behavior, Successes and Shortcomings
“We need to determine who is responsible when AI makes a mistake”
Переглядів 456 місяців тому
“We need to determine who is responsible when AI makes a mistake”
Staring into the Abyss and Eating Glass: Presentation by Rajiv Maheswaran
Переглядів 766 місяців тому
Staring into the Abyss and Eating Glass: Presentation by Rajiv Maheswaran
How to Steal ChatGPT’s Embedding Size, and Other Low-rank Logit Tricks
Переглядів 1456 місяців тому
How to Steal ChatGPT’s Embedding Size, and Other Low-rank Logit Tricks
AI in Structural Biology: Roots of Success
Переглядів 1076 місяців тому
AI in Structural Biology: Roots of Success
DeLLMa: A Framework for Decision Making Under Uncertainty with Large Language Models
Переглядів 2276 місяців тому
DeLLMa: A Framework for Decision Making Under Uncertainty with Large Language Models

КОМЕНТАРІ

  • @YeuMai-cl5ts
    @YeuMai-cl5ts 56 хвилин тому

    ❤❤❤❤❤❤❤❤

  • @YeuMai-cl5ts
    @YeuMai-cl5ts 57 хвилин тому

    Rff

  • @LencyLeaño
    @LencyLeaño День тому

    1:44

  • @HakMap-g3y
    @HakMap-g3y День тому

  • @HưngTuấn-n9c
    @HưngTuấn-n9c 2 дні тому

  • @karenlind5833
    @karenlind5833 5 днів тому

    I'm a targeted individual under my will.

  • @potsti7777
    @potsti7777 24 дні тому

    This is over my head. 🥴🤔

  • @HazelJames-q4m
    @HazelJames-q4m 28 днів тому

    Electronic harassment 20 plus month's in fergus county. Targeted by jeff. His obsessed with torchering me. Ive been under attack more aggressive everyday that goes on. Tonight is apparently my last night possibly alive. I shouldve never been targeted. Ex girlfriend i assume.

  • @margaretfahey8090
    @margaretfahey8090 Місяць тому

    MICROWAVE RADIATION WARFARE. In 2007, the Register reported that the pentagon was developing a parallel to planet earth with billions of individual nodes to reflect every man, woman and child on earth. This is called the sentient world simulation and it is is a neural network with massive computing power. It went live in 2007. Every node on the sentient world simulation is given an identifier which is called an Avator. Elon Musk has stated that we are all collectively programming artificial intelligence. Google and all the humans that connect to it are one giant cybernetic collective, where we are plugged in as nodes on a network. It is a human machine interface which dials into all the other machines, computers and humans and also hooked into global positioning satellites and this is known as the internet of things. They are embodying all data and all knowledge into a working model. The cabal wish to be able to predict and control future events with this system. Those who work for what is known as "The Blue Brain Project" are building a model for the brain where they are bringing all the information that they have ever accumulated on the brain and upload it into this environment and organise it. Then they will develop algorithms that connect all of this data. They will be able to understand how the brain builds a version of the world and then projects this version of the world outwards. They may be trying to recreate a collective consciousness. They can now plug us into the internet against our wills because they have injected most of us with nano technology which has formed networks inside our own brains which can be connected to the internet. I am a targeted individual which means I have been implanted with both pre-programmed and programmable, injectable, self-assembling technology. Unknown remotely located criminals both collect and transmit data to that self-assembling technology which is inside both my body and brain. This results in me being forced to hear voice messages and to see artificially generated imagery and to feel sensations inside my body and on the surface of my skin in the form of pain, pulsing energy, other sensations which feels like human touch, attempts at remote sexual stimulation even though I am an old lady and non-sexual, external takeover of a selection of my muscles, brain to brain interface where some of the capabilities of my brain become suppressed and unknown remotely located criminal neuro operatives then take over those brain capabilities and act through me by speaking their words out of my mouth or laughing through my mouth or grimacing while using my facial muscles. These as yet unknown criminal neuro operatives force me to experience all of the above experiences and much more, too numerous to mention. I write a short paragraph about these unwanted experiences almost every day and I have been doing so for a many years and I then post this information online under my own name and address and eircode. In today's post I wish to inform my readers that one of the abilities of those unknown criminals who remotely manipulate my body and brain is to strongly stimulate the nano technology in a part of my body so that I then by that means become hyper aware of that body part such as my right breast or similar. I don't like this being done to me at all. I am a fairly old lady who is non-sexual and this is quite annoying. I am only writing about it because the entire subject matter of being injected with self-assembling technology and then having that technology remotely stimulated by unknown remotely located criminals is a new topic for many people throughout the entire world. It is not being dealt with by the main stream media because they themselves have injectable technology inside their own brains and their brains are under external remote control. I don't know what they believe at this stage but I do know that they are no longer able to think independently and logically and they haven't done so for the past several decades. Another aspect of being remotely stimulated and manipulated which seriously angers me is the constant remotely generated forced imagry that is being continually transmitted into my brain. Some of it is sexually degrading material and it does not come from my thought processes. I am very upset by it. This morning those unknown criminals who transmit messages into my brain were talking among themselves while I was being forced to hear their conversation inside my head. One of the ask the other if I had ever stolen money or anything. The other one responded that I had never stolen anything ever. Almost immediately a video of a bundle of money was transmitted inside my head and in that video my hand was pulling a note from the middle of that bundle of money. I don't know who created that video and transmitted it to me and I don't know if whoever created it was trying to imply to others that the video in question was taken from my memory banks but I can assure the reader that it did not come from my memory banks and instead it was falsefied imagery. This is simply one example of falsefied imagery which is constantly being transmitted to me in order to upset me and to misrepresent me to others who may be remote neural monitoring my thought processes from other locations. As being a victim of remote neural monitoring combined with remote neural manipulation is a relatively new topic it is sometimes difficult to write about with total clarity. I speak and write the absolute truth at all times. Those who imply that I and others are mentally disordered rather than being remote neural manipulated are holding back the whole process of solving this widespread and ever growing crime of non-consensual remote neural manipulation. Those who continue to imply that we are simply mentally disordered are doing the whole world an extreme disfavour and they are dragging us all down into enslavement. Do not allow them to get away with their lies. This is not a medical matter. It is a matter for the department of defence. The Department of Defence in each country must defend everyone against being remotely tortured by criminals who use digital signals to remotely torture us. My name is Gretta Fahey and my postal address is Newbrook, Claremorris, Co. Mayo, Eircode F12 Y560, Republic of Ireland. My landline home telephone number is 094 9360901. Today's date is 11th October, 2024.

  • @brianlarocca4390
    @brianlarocca4390 Місяць тому

    Too bad audio is poor quality.

  • @billallen275
    @billallen275 2 місяці тому

    The first thing I see is that the IPCC has erased the warm period that ended about 1,000 years ago.

  • @Sandheip
    @Sandheip 2 місяці тому

    Important discussion on human-agent collaboration in AI by Shiwali Mohan. Insightful perspectives on robust collaboration and AI systems. #AI #HumanAgentCollaboration #Innovation #Research

  • @AlgoNudger
    @AlgoNudger 3 місяці тому

    Thanks.

  • @martinelangton3934
    @martinelangton3934 3 місяці тому

    where are the bodies aera Network... ? the stolen biofields?

  • @aathethird5814
    @aathethird5814 4 місяці тому

    C.S.P. C.C.J

  • @enterthekraken
    @enterthekraken 4 місяці тому

    One correction: you mentioned that the difference between ATC and SAM radar is that one wants to help you land but the other doesn’t want to help you land. That’s wrong. They both want to help you land, it’s just that one wants to do it really quickly.

  • @talisikid1618
    @talisikid1618 4 місяці тому

    Actually, the ipcc is a propaganda clearinghouse. Not reputable at all.

  • @dylapoo
    @dylapoo 5 місяців тому

    The questions remind me of early grad school when people ask questions just to demonstrate they can be smart too like - chill out

  • @martinelangton3934
    @martinelangton3934 5 місяців тому

    suppotting democide under lie cover

  • @Whiteandpolite21
    @Whiteandpolite21 5 місяців тому

    Are you using an AI voice🔦

  • @AlgoNudger
    @AlgoNudger 6 місяців тому

    Stochastic Parrot's convo?

  • @NuLiForm
    @NuLiForm 6 місяців тому

    The AI is not the only one who is having difficulty comprehending the world around it..humans, as a whole, have become Terrible spellers. What i see here is not an issue of semantics..it is an issue of spelling. Realistically...Tycoon?..does not require a capital T..unless it was plural. Tycoons, can begin a sentence. The word tycoon is Singular & can not begin a sentence...thus, giving it capitalization is incorrect spelling. How can AI be expected to learn correctly if the code and/or words it is given are misspelled? It will become confused, just as we see here..and ask you if you meant something else. Common sense. Better give AI the initiative & tools to spell check & correct all input....& hook it up with dictionaries. Make them Older versions, as the newer ones actually have typos. It's Shameful & Shocking really, how humans have become so illiterate. In the age of spell check?, where all you have to do is Use it....this is inexcusable.

  • @aBigBadWolf
    @aBigBadWolf 6 місяців тому

    terrible audio

  • @AlgoNudger
    @AlgoNudger 7 місяців тому

    Thanks.

  • @SOSmysteries
    @SOSmysteries 7 місяців тому

    Really enjoyed the host and the guest. Thank you. Please keep going.

  • @jerrymdanowski9825
    @jerrymdanowski9825 7 місяців тому

    They like noise 4/Some Reason...Responce ?

  • @mbsweeney1906
    @mbsweeney1906 8 місяців тому

    Electronic Warfare on Citizens - Targetted Individuals

  • @BlockMonsta
    @BlockMonsta 9 місяців тому

    If we had satellites phones would still work

  • @rickacton7540
    @rickacton7540 9 місяців тому

    these ppl just introduce new buzzwords for the same issues

  • @Baka_Komuso
    @Baka_Komuso 9 місяців тому

    Hire an audio visual production engineer . USC should be embarrassed by such poor quality of production. Terrible.

  • @noor-jumanakagalwala7053
    @noor-jumanakagalwala7053 9 місяців тому

    You are awesome!

  • @jaepark9133
    @jaepark9133 10 місяців тому

    Great prof and a researcher!

  • @cleetx3546
    @cleetx3546 11 місяців тому

    I'm not convinced that "nobody" looks at that data. I believe every data item is taken in by AI to learn even the simplest things like human maneurisms

    • @Ruthyism-
      @Ruthyism- 9 місяців тому

      I think you’re 100% correct.

    • @enterthekraken
      @enterthekraken 4 місяці тому

      I don’t deal in EW but there are cases where you don’t want to use all the data: if 99% of your data is empty sky and you want to learn about the other 1%, sometimes you want to rebalance your dataset so the algorithm can learn it better.

  • @blogintonblakley2708
    @blogintonblakley2708 11 місяців тому

    None of you understand how sick and twisted this all is, huh? "Oh but it's necessary, there are greedy bad people out there." No, they aren't out there. {points} They are right there. Right where you are. All of this "necessary" stuff arises from how civilization itself is organized. Layers of authority. That's why war is necessary, because the people who run the world want to run all of it. They do it for the greater greed. But you lot... the enablers. You do it for lesser greed. For that little bit of extra status and privilege you get from those in search of the greater greed. None of you are Elon or Jeff... you just enable them so you can have nice stuff... but not the best. {shrugs}

    • @cleetx3546
      @cleetx3546 11 місяців тому

      Oh thank goodness someone else sees it! Thanks for making this comment.

    • @synapses53
      @synapses53 Місяць тому

      True . So true . 👍

  • @dosorioa84
    @dosorioa84 11 місяців тому

    Brilliant presentation!

  • @mssaxonUCSB
    @mssaxonUCSB Рік тому

    Thank you USC ISI for hosting me and uploading this recording, and thanks for watching! Some minor errata I wanted to correct: - Slides shown from 37:00-40:30 should all be titled "Analyzing behavior of models by switching the prompt midway through a run" rather than "switching the seed" - However, the top half of 39:09-40:30 shows fixed seed, no promptswap while bottom uses promptswap Feel free to reach out if you have any questions!

  • @LCG-f8w
    @LCG-f8w Рік тому

    I don't understand what the h_i is, in the paper she said it's the activation of the transformer layer, but there's no such term in the original transformer paper, anyone knows what is that? the paper said h_i is the concatenation of all layers' activation in step i, I'm confused.

  • @屁屁-g1u
    @屁屁-g1u Рік тому

    姚哥真帅👍

  • @josephbannister1264
    @josephbannister1264 Рік тому

    You actually make doing research at ISI sound fun. Congrats on creating a great career for yourself!

  • @cassidycash8805
    @cassidycash8805 Рік тому

    'Promo SM'

  • @bassemtossoun
    @bassemtossoun Рік тому

    Does anyone know how I can watch the full documentary? Thanks!

  • @mosesmccabe8983
    @mosesmccabe8983 Рік тому

    I used MOSIS in my undergrad at Chico State to design a SAR ADC Chip. fun but challenging project. Anyway, who can a non college student download the software?

  • @infonyourmark
    @infonyourmark Рік тому

    No more questions during the talk

  • @infonyourmark
    @infonyourmark Рік тому

    👾🤖

  • @mikeg1444
    @mikeg1444 Рік тому

    Hello, do you have the presentation slides available?

  • @kevon217
    @kevon217 Рік тому

    Fascinating approach! I really want to try this. Seems very powerful and customizable. Since this presentation was 2 years ago (2021), are there any current (2023) resources/developments you would suggest to take a look at?

  • @user-wr4yl7tx3w
    @user-wr4yl7tx3w Рік тому

    Isn’t group basically a generalization of individuals perceived on some structure. Like stereotypes. Why not see every individual as unique with different life experiences. Surely if we are to fall back on categorizing, then the homeless are the most shunned by all groups, where there’s far less social stigma to treating them like cockroach Metamorphosis.

    • @ammardotw
      @ammardotw Рік тому

      I couldn't agree more, Y! Thank you for your comment.

  • @Infinite.pause_abilities
    @Infinite.pause_abilities Рік тому

    At 18 minutes… mind.

  • @lingli1987
    @lingli1987 Рік тому

    where can I see the slides

  • @trumanerichson6297
    @trumanerichson6297 Рік тому

    💐 P𝐫O𝕞O𝓢m