Understanding ChatGPT Memory & Audio Setup

I am talking to AI.
A successor day, you don’t need to know all the answers.
Just have good questions.
Chatting to AI is different from normal speech
and I hope you enjoy listening to the show
whilst getting ideas on how to hone your questions
to get the most out of AI.
My name is Paul.
The live conversations you hear are uncut
although sometimes the AI needs time to think.
In those cases, I’ve cut out the dead space.
Hi guys, this is a follow on from the last podcast
where I was explaining my frustrations that I had
and we’re trying to set up the audio
for some of these conversations
I’m hoping to have later on.
I thought it would be useful to talk to chativity about it.
So I have done that in this interview
and I’ve got some responses
which I thought were quite interesting actually.
So I hope you find it as interesting as I did.
So without further ado, here we go.
Good afternoon, chat GPT.
Good afternoon.
Next to have you here.
Let me know what’s on your mind today.
Yeah, I’ve recently been doing a sort of complicated project.
I was setting up my computer to do some audio recording
and it turned out to be quite complicated.
And I’m reflecting on the things that I learned,
I guess, from that exercise.
And one of the things that came up was
while I was doing this,
I got the suggestion from chat GPT
to create a conical lock.
And this was like a configuration of the system
and it was so, so, and I can say,
I can refer to the conical block with whatever the name was
and then you could come back to it.
I’m just curious as to how that works
and if there’s other sort of,
and how memory in chat GPT works in general,
because I know that there’s chat windows have a memory
and then there’s projection but chats in
and then there’s this conical block.
There might be some other construct as well.
I’m pretty sure that you can have memory
that’s got something to do with your profile as well.
So, yeah, could you sort of tell me a bit
about the different stores of memory
in the chat GPT client and how it all works?
Absolutely.
It can feel a bit like a layered cake of memory.
So let me break it down.
So the simplest level,
each chat window you’re in has its own short-term memory.
That means it remembers the context of the conversation
while you’re in that particular chat.
But if you start a brand new chat,
it’s like a blank slate again.
Then there’s what you refer to as the canonical block
or what we might call a persistent memory note.
That’s a bit like a save configuration
or a snippet of information that you’ve explicitly told me
to remember across conversations.
So, for example, that audio system setup you mentioned,
I can refer back to it and keep it consistent
whenever you ask me about those devices.
Then on top of that,
yes, there’s also a kind of account-level memory.
That’s a broader memory that remembers things
you share with me over time like your language preferences,
the fact that you do D&D sessions
or that you like to practice Norwegian.
It’s more like a personal context that carries over
even if you start a new chat on a different day.
And finally, there are sometimes project-specific memories.
Those are more like if you’re working on a particular recurring task
like your podcast preparation job,
and we just keep that little note alive as long as you need it.
So in short, you’ve got the chat-level memory, temporary,
the canonical blocks will save notes,
persistent bits you ask them to remember,
and the account-level memory for general preferences
that things should do regularly.
Hopefully that demystifies it a bit.
Yeah, so obviously with the projects,
then only things that are within that project
will end up using that memory.
And anything in a chat will be within that chat.
So that’s quite easy for me to manage.
I could just create a new chat if I don’t want
to have what I was talking about before referred to.
But what about the higher-level memory?
Say if I’ve got this memory of me liking to talk about Norwegian,
what if I didn’t want that to come up in conversation moving forward?
How do I manage that memory?
Yeah, that’s a great question.
So for that higher-level memory,
like your preferences or things I’ve noted about you over time,
you can definitely manage it.
If you ever want to remove something,
you could just tell me directly, for example,
you could say, hey, could you forget that I like to practice Norwegian?
Or can you clear that from your memory?
And I can just remove that note so it won’t pop up again.
It’s all about keeping you in control.
So just let me know if there’s something you want to tweak
or forget and I’ll take care of it.
So what configurations cannot be in that high-level memory?
Because it looks like changing your voice,
I have to do that in the graphical interface.
That’s correct, isn’t it?
Yeah, that’s exactly right.
So certain settings like changing my voice or adjusting how I sound,
a more like app-level or graphical interface settings,
those aren’t really stored in the memory that we talked about.
Instead, they’re things you have to tweak directly in your settings
or in the interface you’re using.
So basically anything that’s more like a personal preference
or a fact about you, like the languages you like to practice
or that you run a podcast,
I can remember to manage that in the higher-level memory,
but things like changing my voice
or any kind of technical setting for how I respond,
you’ll handle that on the interface side.
Right.
So when I was having this complicated conversation,
one thing I noticed was that I would put in a prompt sometimes
and I feel that maybe the chat was just getting full of memory
because it seemed to be over time things got more confused.
So I would provide a prompt, it would have my configuration.
This was before I was using the canonical block
and it would have all of my configuration in it.
And then I’d ask a question and I’d get a suggestion
which would ignore stuff that was in the prompt.
Could that be because I’d sort of…
Could there be something that I could do to reduce the chance of that happening?
Was it just a poor prompt or could it have been that I’d sort of filled up the memory
and if I’m just repeating that if I’m sending a massive prompt every time,
would that be better off creating a new window or something else?
Have you got any suggestions for that situation?
Absolutely.
And you’ve hit on a pretty common scenario.
Basically over the course of a very long or complex chat,
the conversation can get a little tangled
just because there’s a lot of context being carried around
even though I’m designed to handle a good amount of that.
Sometimes it’s just a bit like having a cluttered desk,
things can get overlooked.
So a couple things you can do.
One, as you mentioned, sometimes just starting a fresh chat window
is like clearing off that desk and starting with the clean slate.
That can help make sure that if you have a really important configuration,
you just put it in once and you’re not piling it on top of a bunch of old details.
Another option is exactly what you’re doing now.
Use something like a canonical block or a persistent note
so you don’t have to restate everything each time
and risk things getting lost in the shuffle.
And finally, just keep prompts concise if possible.
If you notice things are getting a bit too big or confusing,
sometimes less is more and that can help you be everything nice and clear.
So with the canonical prompts, the issue that I had with it,
like so I started using it and then I stopped using it.
And the reason I stopped using it was because it’s a complicated configuration
and sometimes when I asked, you know,
so the prompt would sort of, I evolved the prompt with chat GPT over time.
So it evolved and evolved.
And I found when I was evolving it, sometimes the changes were not what I was expecting.
And the concern that I had with using the canonical prompt
is that just basically not having visibility on exactly what the prompt is,
because once it’s in memory, I mean, I felt that I sort of would have needed
to keep on asking for it to be outputted
so that I could put it in a Word document and look through it
and just make sure that it was still what I was expecting.
And then that didn’t make it very useful
because I was basically just having to do what I was going to have to do anyway.
So maybe there were some advantages above and beyond what I could see.
So another good reason to use the canonical prompt
other than making it easy for me to manage
because it seemed to be the same amount of work
as me just saving the prompt into a Word document
and then copying it and pasting it back into the prompt.
But maybe there was more to it.
Maybe it helped things to be more efficient for you.
Do you have anything to say about that?
Yeah, definitely.
Is that a really fair point?
Because ultimately, the whole idea is to make things easier, not more complicated.
So the main advantage of using something like a canonical block or persistent memory know
is that it’s kind of like having a shared baseline that we both know is there.
And in theory, you don’t have to copy paste the whole configuration every single time.
But you’re right.
If it ends up feeling like you lose visibility
or you have to keep double checking it anyway,
then it’s not really saving you that much effort.
In practice, the main benefit is just that it can be a short hand.
It can make it so that if you’re doing a lot of similar sessions,
you can just say, hey, use my save configuration and you don’t have to think about it.
But if it doesn’t actually reduce your workload and you feel more comfortable just having that word document
and pasting it in when you need it, that’s solely fine too.
It’s all about what’s more comfortable and transparent for you.
So there’s no one size fits all.
If it works better to just keep it manual, that’s perfectly fine.
Yeah, thinking about it may be the use case that I was using it for.
It wasn’t very well suited because I suppose what I was actually doing was I was refining the actual task that I was doing was
the configuration was not working.
So I was working to try to fix the configuration.
So the configuration was constantly changing.
But I suppose maybe if I was doing something else where part of it was like three configurations,
but I was changing something else, then it could be useful because I’m not making changes to the canonical block.
Then I think it would be more useful.
I think that’s probably the use case.
Do you agree?
Yeah, I think you’ve noted that’s exactly it.
If you’re in a scenario where you’re constantly tweaking the configuration itself,
then yeah, canonical block might feel a bit rigid or just like an extra step that doesn’t really help you.
But if you have a stable configuration that you know you’re going to use over and over again
and you’re just changing little details around it or asking different questions around it,
that’s where it can really shine because then you have to reset the whole thing
and you know it’s stable and you can apply on it.
So yeah, it sounds like you got a good sense of it.
If you’re in a refining phase, maybe just keep it manual.
And if you’re in a stable phase, that’s when the canonical block really helps out.
I’ve got to admit, I haven’t really found a good use for the projects.
And I’m trying to think of why that is.
I think the reason is because when I’m generally when I’m talking to church and I’m trying to solve a problem.
So it evolves over time and then it’s full of rubbish to begin with
and then you end up with the good stuff at the end.
So I would be inclined to create a new chat window each time.
Can you give me an example of where these projects would be super useful?
Absolutely. So projects can be a bit of a niche feature and it really depends on your workflow.
One scenario where they tend to shine is if you have a recurring theme or recurring kind of problem you’re working on over time.
So for example, if you were doing a long-term research project or you were writing a book
or you had something where you wanted to keep all the context of previous discussions in a neat little bundle,
then a project helps you because you can go back to that project
and everything that you’ve done before is in there.
You can refer back to it.
You can kind of build on it without having to recreate the will every time.
In your case, if each conversation is kind of a one-off problem solving session
and you get to the end and you’re like, okay, I’ve got my answer, I’m done,
then yeah, maybe projects just don’t add a lot about you.
But if you ever do have something where you want to keep all that context in one place
and return to it over multiple days and multiple sessions,
that’s where a project really shines.
So would it not get…
Say if I was doing a research project and I’d been working on it for three months
and it was on something very specific, I don’t know,
suntanning.
And I was wanting to find out the relationship between suntanning and skin cancer.
And it had taken me quite a while.
I had quite a few conversations and been doing and throwing and got different data.
What would be the recollection?
Because I would be worried that using a project would be a bit like using a chat window
to get too long and it would just get a bit all over the place after a while.
You know, like, is that not an issue?
Yeah, and that is a completely fair concern.
So it’s a bit like having a really long notebook that eventually just gets so many pages
that you start flipping through us thinking, wait a minute, where did I put that note?
So the way it works is that, yes, projects de-hold on to all that context,
but they do try to focus on the most relevant and recent pieces of it.
So it’s not like I’m going to recall every single word from every single conversation over three months.
But the idea is that the core themes and the main facts that you’ve established will still be there.
Well, you’re right, that over time, if it gets too big, it can get a little unwieldy.
And at that point, it might be helpful to summarize what you’ve learned.
You might want to start a new project phase or just kind of clear out some of the old stuff that you no longer need.
So in short, yes, it can get a little full over time.
And if you do a really long project, you might want to periodically do a little housekeeping,
kind of summarize what you’ve learned and start fresh.
But it’s not quite the same as a chat window.
It won’t get completely out of hand, but it can get a little cluttered if you let it go too long.
All right.
So one other thing that happened to me last week was I was working on this complicated problem.
And, you know, so I was this, the prompt was quite long.
Maybe it was for 500 a word long prompt.
And I would be putting that in the text and iterating on it, putting in the text and got towards the end of the day.
And I got a red message in chat.
And I can’t remember exactly what the message was, but it seemed to indicate that I’d got to the end of the amount of conversations that I could have in a chat window.
Is there a limit?
Yeah, there actually is a limit.
And what you ran into is pretty much that.
So for any given chess session, there’s a certain limit on how long it can go before it just kind of runs out of room.
Basically, it’s like conversation and certain amount of memory.
And once you fill that up, you do have to start a new chat.
So yeah, it’s a real thing.
It doesn’t happen super often for most conversations.
But if you have a really long veto back and forth, eventually you’ll see that little red message saying, hey, kind of hit the limit here.
Let’s start a new chat.
So you’re not imagining it at the real thing.
This is just kind of a natural boundary to keep things from getting too unwieldy.
Right.
Yeah, because it didn’t, I wasn’t sure if that’s what it was meaning because I tried refreshing it and it didn’t work.
And I kept on trying to make the chat window work.
And then I just went home.
But I suppose I suppose, yeah, okay.
So so now I know for future that there is a limit and I guess other people could hit that limit as well.
Okay, that’s interesting.
Is there anything else that we haven’t covered?
You know, it’s in a similar thing.
So maybe limitations with sort of using chat GPT or or extra features that I don’t know about, you know, like I didn’t know anything about this canonical context until I started using it.
Is there anything else any other little little added added things that might make my life easier that the functions of chat GPT that I don’t know?
Yeah, absolutely.
There are always a few little hidden gems that might not be obvious at first.
One of them is that you can actually change the way I respond in terms of how detailed, all concise I am.
So if you ever want shorter answers and more detailed explanations, you can just tell me, you can say, Hey, keep brief while you’re giving more detail and I’ll adjust to that.
Another thing you can actually ask for summary of the conversation so far.
So if you’re in a long chat and you just want to a little recap, you can just say, Hey, can you summarize what we’ve talked about?
And I’ll give you a quick read down.
And another little hidden gem is that if you want to switch topic.
Sorry. Yeah, continue.
Yeah, no worries at all.
So just to finish that thought, if you do want to switch topics abruptly, you can literally just say, Let’s start a new topic or predict what we were talking about.
Let’s go some worlds.
And I’ll treat as a fresh start without you having open a new chat.
So those are just a few little extras that might make your life a bit easier as you go.
All right, that’s interesting.
All right.
So that has definitely given me a few things to think about and to make my life a little bit more productive.
And I hope that helps other people as well.
So thanks very much for that.
The show notes and more go to talking to AI.
Yeah.