Skip to content

Let LLMs Introspect

TL;DR: We can prompt chatGPT to generate an "attention map" for themselves (demo available at https://ywugwu.streamlit.app/).

Currently, we're working on getting better prompts via open-source LLMs like Llama3.

Introduction

We're interested in letting LLMs introspect. That is, let an LLM answer which part of the input contributes the most to the word we're interested in the output text (like an NLP-version Grad-Cam but by Prompting:

We want an NLP-version Grad-Cam (https://arxiv.org/pdf/1610.02391) but by Prompting

We have a demo at https://ywugwu.streamlit.app/ that can do this:

VIz

Method

An overview of our prompt: we merge the previous "input text" and "output text" into our prompt and ask LLMs to assign importance score in a word-to-word fashion.

LLM Diagram

We can also use different prompts like:

different prompt

And we can compare the results of these prompts:

cmp1
cmp2

Future Work

A Future work (what we're doing now) is using Grad-Cam results as ground truth to optimize our prompt:

train-Prompt

Leave a Reply

Your email address will not be published. Required fields are marked *