Talking to AI — Setting Up Audio: When AI Meets Complexity

I am talking to AI.
A successor day, you don’t need to know all the answers, just have good questions.
Chatting to AI is different from normal speech and I hope you enjoy listening to the show
whilst getting ideas on how to hone your questions to get the most out of AI.
My name is Paul.
The live conversations you hear are uncut although sometimes the AI needs time to think.
In those cases, I’ve cut out the dead space.
Good morning.
Okay, so I’ve got some questions.
I’ve been working on some things recently and I want to understand, some things came
up and I want to understand better how chatGPT works.
It was a quite a complicated configuration that I was working with chatGPT on.
It took quite a while to get to the bottom of it and through the process of the conversation,
I noticed a few things.
I noticed that sometimes I was creating a prompt and the prompt was getting more and
more sophisticated over time and I was getting you to help me with the prompt and I was noticing
that sometimes you weren’t reading the prompt.
Okay, so this is going to be a bit different this show.
I’ve been working on setting up an audio configuration for this podcast and it actually
got fairly complicated because I’ve got somebody that I’m interviewing on the one channel.
Hi, this is going to be a bit different.
I want to recap in this show about my learnings that I had this week.
Again, I’ll keep it fairly short, about 20 minutes, 25 minutes and I don’t know if it
will be helpful to people but it’s just some things that cropped up over the week which
I thought I would share.
What I was working on last week, it was a task that started with something that I thought
was going to be quite quick and it turned into something that took ages and I didn’t
deal with it the best, I don’t think, I think I could have dealt with it better.
I feel that maybe had I been better working with chat, I could have got a better outcome.
I did get to a conclusion in the end but it took me a long time and so I thought I’d
share some of the things that I went through and so maybe you don’t have to make the same
mistakes and maybe you learn from some of the things that I feel that I’ve learned over
the last three or four days.
I’ll explain what I’ve been trying to do.
In order to have some fun podcasts, what I thought it would be quite cool to do would
be to get somebody on the line and have myself on the line and also have chat GBT on the
line which I’ve done before and then I wanted to get GROC on the line as well.
Now getting GROC on the line caused some particular challenges because unlike chat GBT you can’t
talk to GROC unless you talk to it through the phone and that wasn’t going to work if
I wanted to connect everybody else up on my computer so I had to sort of try to set up
this thing where all these people could talk to each other and I could record it all and
the main complication with this is called feedback.
So if I was to set up an audio recorder and start talking into it, if I then took the
output of that audio recorder and put it back into the input you’d end up with feedback
and basically feedback is just a whole load of noise and the audio recorder just has a
panic attack and dies.
If you were to do this with live audio that’s what you get when you’ve got guitars and they’re
making a huge amount of noise. I mean it’s quite cool with an electric guitar but it’s
not very cool when you’re trying to record somebody and you get this horrible noise which
is feedback.
And it’s complicated in my case because I’ve got four people, well four people, two people
two AIs that I’m wanting to all be able to say something and for them all to be able
to hear and then I want to record them all.
So it poses a few challenges and I went through this process. So I managed to get it set up
before without GROC and that’s recorded already but when I wanted to add GROC that meant that
I had to take my computer and then connect it to an iPhone and then communicate that
back to the computer.
Now the next complication is that Apple doesn’t support you recording phone conversations
because it’s against the law to record phone conversations. I’m not trying to record a
phone conversation, I’m trying to record chat GBT talking to me so it’s not against the
law but the whole phone is locked down so that you can’t do that.
So there’s no app that you can get on the phone that will record the conversation.
So that leads me to recording on my laptop. Now when I record on my laptop the issue
that I’ve got is that I have to send the information through the light, through the, there’s like
a headphone socket on a MacBook and it has to go out on the MacBook and then it has to
get to the iPhone and come back again.
Now I could connect it via USB and then connect it to the lightning thing, that was the most
obvious one you know you can just connect your phone to the Mac and that’s how you sync
it but because Apple doesn’t want you recording the audio the Mac doesn’t support microphone
and audio connection to the iPhone.
So you have to get into these creative solutions and so you can imagine without me going into
too much detail on the rest of it it’s a complicated problem because I ended up having to create
a whole, I ended up having to create about 15 what they call virtual devices on my computer
they all had different configurations. Then I had the wiring to wire, I ended up having
to wire, I ended up wiring my computer to a hardware interface and then wiring that through
a whole bunch of adapters to the phone and it was I don’t know probably about 10 cables,
3 adapters, 1 interface, then it had to be plugged into the right place and then you’ve
got this configuration within this, all these audio devices then I’ve got other configurations
all the apps have to have their own speaker in-speak out configuration so it’s a lot of
stuff and it made for a very challenging conversation with ChatGPT and I ended on Friday I’d really
had enough, I thought this was going to take me a couple of hours and it ended up taking
me two days and I hadn’t got a solution and it felt, you know I am, I have been a computer
programmer, I’ve developed software and you have to make these kind of decisions when
you’re developing software like you know how, what point do you learn it all and do
it versus at what point do you pull out the library or you take the easy option and sometimes
the easy option is the right decision and you’re pleased that you did that and you didn’t
try and reinvent the wheel and sometimes if it doesn’t work then you end up in a situation
where you like I did on Friday, you get to Friday and you don’t know why it doesn’t work,
you don’t know, you haven’t really learnt, well I hadn’t learnt anything about my, I
hadn’t learnt how my interface was working I guess, how my computer was working, I knew
I tried a whole bunch of things and I’d learnt stuff about how ChatGPT works but you know
it’s this decision like in two days I could have learnt how all of the audio on my Mac
and linking to the iPhone actually worked and I could have become probably an expert
in that very small field but I chose not to, subsequently I would say that I am but I think
it took me longer than I would have liked, so I’ll just share with you some of the things
that I did and some of the problems that I had, okay so I started off thinking okay
well what I need to do is I’ll ask ChatGPT how to configure this thing and it sent me
some ideas and the first thing it was giving me advice about things that I should buy and
I suppose the first thing that I realised was what I had to change, I had to change
the question and pretty soon I had to change the question to involve, give me solutions
that will work with the equipment that I have and that focused things down a little bit
and at that point, because I’d already gone to and from a bit, I probably spent a few
hours on it by this point going to and from, I started saving a prompt and I’ve now got
a prompts database which I’m using some software called Obsidian, I mean I’ve been using text
prompts for other things as well but this is probably the most complex prompt that I
have created with ChatGPT so far, so the typical process for creating a prompt is you basically
tell it what you want and then you ask ChatGPT to create the prompt and then ChatGPT will
create a much more detailed prompt that will help it and it will also give you a clear
idea about what it thinks it’s got to do so that gives you an opportunity to read the
prompt and to refine it all and we went through a process to start with and then I started
set it up and I set it up and it didn’t work. So then I go through these loops, didn’t
work, why didn’t it work and then ChatGPT gives me advice and we go around these loops a few
times and I start to figure out that it’s getting complicated to explain what I’ve got
and ChatGPT sometimes is forgetting the changes that I’ve made to the configuration and then
we’re going back around in circles or it’s giving me advice around things that maybe
it’s, at one point it was giving me advice so I was using an audio recorder called a
Zoom H5 and it was kept on giving me advice about menu systems in audio recorders but
the advice was for other Zoom recorders and they don’t have the same menus and I kept
on hitting this like you know when you sort of just cracked it for a few times and then
after three attempts it’s like okay I need to tackle this so I changed the prompt to
say I do not want any results apart from for the hardware that I had and then I had questions
around the BIOS version so I ended up updating this prompt to basically include, I documented
all of the configuration including the version number of the software on my computer, the
version number of the BIOS on the hardware, all of the virtual devices and created the
documents and then I asked ChatGPT to rework the document and create diagrams and one thing
I’ve learnt from this is that if you’re going to get ChatGPT to create diagrams you want
to get it to create diagrams in ASCII text because it’s a lot quicker and it’s a lot
easier to manage then you can just copy those into text files. If you ask it to create diagrams
generally it will create images and they’re just annoying and it takes a long time for
it to render them as well so that’s a good thing and I went for a few iterations on the
because this was complicated a lot of the difficulty I found with this was just getting
my head around exactly what was happening because it was quite a long process. If you
imagine the audio is coming from lots of different places and it’s going to lots of different
places and there’s lots of opportunity for things to go wrong in this process so iterating
with ChatGPT on how to create a good diagram for what you’re trying to present I found
was very important and then once you’ve got something that’s readable and understandable
then you want to iterate further and try and get them to create a succinct version and
perhaps even a table if possible because the thing that I’m finding with these prompts
is that really you want the prompts to be as dense they want to be easy to read but
you also want them to be as dense as possible so that you’re not scrolling for ages through
empty space to understand because it just becomes overwhelming because you do have to
look at these prompts. But over time the first win I say with ChatGPT on this was I got a
really good documentation of the setup that I had. It might not have worked but it was
really well documented. Some of the tricks obviously well if you’ve spent much time with
ChatGPT you know you can screenshot any screen and it will read it and understand it so it’s
particularly handy when I was trying to document my virtual devices which came to about 15
different configurations of different virtual devices. So I was just screenshotting them
all and then I’d screenshot them and then I’d say to ChatGPT can you take these screenshots
and create an ASCII table of this data for these virtual devices and then I would then
take that and put it into the master prompt which was saved in this obsidian. Obsidians
are just a text database into this database and so that was super handy. So we went through
that loop. Oh yeah so the other thing that happened with the text with the screenshots
okay so the screenshots worked fine for a while but it was a bit annoying having to go into
a screen and screenshot screenshot screenshot took a bit of a while and I had all these
files to manage and then once I got to more than 11 device drivers the ChatGPT doesn’t
support you uploading more than 10 files at a time. So that posed a bit of a challenge
so I then went through an iteration which I think would be useful in future conversations
but didn’t actually turn out to be useful in this situation but I asked ChatGPT if there
was a terminal command that I could write that would just extract the config from the
app and it gave me some code and you know I mean it sounds if you’re not familiar with
the terminal it sounds scary but basically it just gives you code you copy the code
chuck it in terminal and run it and then it created a file and put it in my desktop which
was a text file and it had all of the configuration and we and I thought that was brilliant for
a while then it turned out that it did have a configuration but ChatGPT was still asking
me questions because it didn’t have all of the configuration so it was in fact useless
but I think that would be useful for the time so that’s something I’m going to bear in
mind in future. So as we were working through this I was basically making changes to the
configuration you know I was getting different problems as I was working through this and
then ChatGPT was making suggestions then I was making the change updating the prompt
and then within the prompt so within the prompt that I had it had a full configuration of
all of the configuration of the system and it also had like a task at the top and the
task the task was sort of there was a task and then there was an overview of the system
and then there was a details of the system. The task was what I wanted ChatGPT to do straight
away and this was using the so with ChatGPT there’s a semantic you don’t really have to
use it but that you can use tags I mean I think it’s clever enough to figure these out
but it’s good to it’s good to put your make sure that your what you want it to do is at
the top and it’s all the sort of doing things you know get blah blah do blah blah blah
and then once you’ve got that and that shouldn’t be more than a few paragraphs and my and then
underneath was what the system is supposed to do and that was fairly simple that what
the system was supposed to do was it was supposed to be able to allow the recording of four
participants on a voice call and allow them to all talk to each other and all hear each
other at the same time and not create any feedback and record into a multi-track file
so so that was the that was the sort of and then and then underneath was the full configuration
of the whole system so and at the top the task for the task tended to be you know I have this
configuration and it is not it is not working it is there is no sound coming from blah blah
or there is so that’s the sort of current state of the problem and and I need you to fix it
so so we’re sort of iterating like that so I’d create the prompt I’d make the changes
update update the configuration update the prompts at the top saying you know well it
it wasn’t making a sound it’s making a sound now but it doesn’t sound it sounds all crackly or
whatever and then then I’d send that back off the whole the whole prompts back to chat gbt and
then chat gbt would come back with questions well with with with a response to my my my problem
and so so with the responses so I went through two days of this and the frustrating thing was
chat gbt I suppose can be a bit like a bit like you know a friend or a work colleague which takes
everything on and then doesn’t doesn’t deliver and then you think damn I should have just done it
myself and that’s kind of how I felt and I felt like I was being run around the houses
as we were trying all these different things and sometimes it was super frustrating because I’d be
updating the prompt and then chat gbt would be responding but I’d realize that chat gbt was
now just focusing on my problem but wasn’t focusing on the system continuing to work as per
the prompt so it would make frequently make suggestions that would actually break the
solution so my sort of conversations with chat gbt turned into just prompt conversations so I
provide it with the prompt I’d get the suggestions and then I’d say well why are you suggesting
that that clearly isn’t going to work and then chat gbt would tell me how clever I am and I think
yeah well not really and then and then and then we’d we’d we’d go on a different tangent and we
kind of went on like that and the problem was I was I was iterating I was making making progress
but I wasn’t I wasn’t fundamentally understanding what all these things were doing particularly
and I I got to I got but the problem was I kept on giving what I thought was close to the solution
so it’s like okay we’re almost there almost there you know we’ll it looks it looks good it seems to
be getting closer and so but there were yeah so so it was it was going along in that direction
I’m just trying to think if there were any other any other sort of clarity sort of taking stock
things that I tried to do to because I could see you know from time to time it’s like oh we’re
going around in circles here ah yes yes so one of the big things on third on the so I started
on out on Thursday and then I carried on on Friday and then I had another go on Saturday but on
Thursday night what stopped me was I was working in a chat window and I actually got you have
I don’t know what the error was but basically I think it was I think any chat window does
actually have a limit to the amount of chats you can have within it and I think this is based
on the concept of a chat window in chat GPT because chapter BT chat window you know is is a is a is a
is a memory block basically well not a memory block but it’s it’s a it’s like it’s own little
self-contained memory so all of the conversations that you have within a chat the the chat has
access to the previous conversations now that’s not true with chat GPT as a whole you know if
you were having a chat and you had a they had a chat with it last week in a different window then
it it may not remember any of that it may but it may not it depends but so so yeah so I had that
problem and when it got to Friday I had another problem
well that’s right yes this was caused by chat GPT trying to be trying to be useful
and I learned something I guess um chat GPT recommended that I set up a con a conical block
right um so this is quite interesting I might I might talk to chat GPT about other
features of of chat GPT that I’d never come across maybe there’s other things that you
can do as well but a conical block is a is a named piece of information that you can refer to later
so I think chat GPT was thinking well let’s set up this configuration as a conical block
and an idea and we we so from time to time I was asking it to update the conical block
and then then we were carrying on and going on in that way and I stopped using the conical block
because I mean I suppose I could have got it to I suppose I stopped using it because I wasn’t sure
what was happening to it so you know when I’m doing a prompt and then I’m updating the prompt
outside of chat GPT and then putting it back into chat GPT I know that what the prompt is
but when it’s stored in chat GPT I don’t fully know what’s happening because sometimes when you
create a prompt and then you might say there’s a number of times when I made a whole bunch of
changes and I said look can you just update the prompt and it often does a really good job of
updating that but sometimes it changes things that you for no for well for there’s obviously a
reason but for what seems like no good reason and you end up with these errors in your prompt
and I can imagine the same thing happening with a conical block but you obviously you only will
see that if you ask it to print out the conical block so so that was something I learned you can
do this um it might be more useful with less I don’t know I don’t know when it would be super
useful to do the conical block where it wouldn’t be more useful than having having it written down
on a prompt but um but so so that was the conical block anyway so um so anyway I got to Friday and
that was probably my low point okay so I spent two days I had only intended to spend a couple of
the hours on this but then it turned into like a sunk cost fallacy it’s like oh my god I spent two
days in this I want to get this fixed um and uh and I got and I got towards the end of the day and
then chat gbt finally turned around and said oh this is impossible and it was like is it really
and then I’d lost all kind of in I didn’t really trust chat gbt by that point because
he’d give me all this advice and he’d keep on going around in circles so so on Saturday I decided
that I was going to go and talk to a human being about it so I went I went to a music shop and
talked to a human being and I learned something from that conversation as well when I spoke to
this person in the music shop it didn’t actually really know much more than me however I used to
think that chat gbt would be good as a sounding board you know and you can talk to chat gbt and chat
gbt will help you work through your problems um you know just but because you’re having to speak
out loud to chat gbt I mean maybe maybe if I tried talking out loud maybe it would have worked but I
was all talking through the text and what I found when I was talking to the guy in the shop was that
I didn’t really understand what the problem was and and I felt a bit bad about that felt
embarrassed and but on the spot I started to get some insights because of that discomfort
started sort of started to sort of get an idea of what the problem was in my head
and then after that I made some changes and I think I figured out the
I made some progress towards the problem and finally I suppose finally today I have figured
out that it is indeed impossible to do with the hardware that I have and I need to get some different
hardware but I know what I need now and I know why it doesn’t work exactly um oh yes one step
between them uh yes so what really helped me with that yeah so that did chat gbt helped me with that
and maybe I could have I suppose I’m trying to use chat gbt because this is you know this is a
podcast about AI I mean I guess if I hadn’t had chat gbt I would have probably done this myself
but I thought this was quite a good exercise I got chat gbt to create a an elimination
test program so the idea was um so with this problem could have been occurring on the computer
or it could have been occurring in the hardware interface and I the problem we felt I felt like
we kept on going from one thing to another and uh and and on today I thought well what I what I
need to do is just need to eliminate one if I need to figure out the big question in the room
which I probably should have tried to work out in the very beginning was is this possible so I
think this is probably one of the big takeaways that I’ve got from this whole thing is once you’ve
documented whatever it is you need to get to the is this possible question as quickly as possible
and you need to be quite ruthless about the is this possible question because the problem with
chat gbt wants to keep make you happy um and uh give you all these you know advice so so things
that I was frequently finding that it was doing it would it would it would give me a suggestion
that would solve my problem then it would break my system and then it would give me a suggestion
to fix that problem that had just caused which was in fact the exact same configuration that I
had to begin with you know so we go around in these little circles um so um and yeah so if I
did if I had if I’d taken the approach at the very beginning I think to say okay let’s document
what I’m trying as quickly as possible then um and in as much detail as possible you know including
if I’m using any hardware what version of the of the of the operating system you know is it on
and then really dig into the theoretically how would this be possible
I may have got a better result I mean I may not have done I suppose I suppose if I could understand
um
what would make this impossible and then try to work towards that you know I feel I feel I think
next time I will I will put more upfront effort in understanding whether it’s possible at all
um because I think that I was talking so after I had the conversation with the with the music
store I had the conversation with a friend of mine who’s a sound engineer and I know I was on the
phone and I pretty quickly realized that I could not ask him about this problem because it was
just too complicated um you know maybe if he wanted to study my setup which he would have wanted to
do um you know it wouldn’t have been quick and it wouldn’t have been easy um so so I uh but but he
did give me a couple of pieces of advice um yeah one of them was yeah he agreed like he he’s found
himself with AI to be spending when he when he wastes the most amount of time it’s when
chat GPT is trying to make you do the impossible um because it wants you to be happy that’s lovely
but you know it’s so you need to I think when things get complicated you need to establish
if it’s possible at all within the you know the confines of within your limitations you know whatever
whatever the the the the limitations are you need to find if it’s possible and uh and then and then
you can go ahead and the other bit of advice he had was yeah this is a very very good thing um that
I think I will bear in mind in the future is I think uh because because what I was getting is I ended
up on Friday I remember having a few conversations with chat GPT where I’d ask him something then
I’d get a suggestion and then it was like have you even read the prompt you’re not reading it
and and I think what it’s doing is it’s it’s it’s already it thinks well I’ve already got that in
the memory and it’s it’s it’s missing stuff and and I think that’s because it’s built up quite a
lot of memory especially if it’s complicated it’s probably a lot of stuff in the memory for that
chat so a suggestion that that that’s my friend had was to just you even though it’s the same
subject is to just create a new chat and start again and I think that did help so so um finally
yeah I did throw so through the elimination um testing I did figure out that it was impossible
with the hardware and it meant and it was the hardware that was the problem and it turns out
that I need some very specific piece of hardware to be able to do this um and because I’d done that
testing I knew why it wasn’t working so I knew what side of hardware I needed um so I then did
chat to chat GPT to try and get some advice on on on hardware which wasn’t too bad actually if you’d
come up with some suggestions they weren’t too bad um but I did end up talking to a specialist
and I’m going to buy that hardware tomorrow so hopefully I’ll be able to do grok and hopefully
it’s worked out all right in the end um but um yeah I thought maybe some of my frustrations might
help somebody somebody maybe not make those frustration frustrating decisions or or at
least you know give them food for thought if they start going around in circles you know so I suppose
I went around in circles the first thing I did was document it very well and the got chat to be here
to help me document it and iterated on the documentation with chat GPT and it was brilliant
at that like if I had if I had all kinds of things I needed to have documented this I can see how it
would be super useful um and then uh then we go around in circles there’s the not using the same
chat window I didn’t think the comical configuration was very useful and uh
and getting to the bottom of if what you’re doing is possible and and the way I did that was through
an elimination test through through troubleshooting so um I could have done the troubleshooting I would
have probably done it myself um but I think it was quite good to get chat GPT to actually create
the troubleshooting step by step guide um because it did uh it did make sense um so that was that
was the approach that’s how I finally solve the problem so I hope you found that useful and um
a bit different um but um yeah um I’ll speak to you next time
the show notes and more go to cooking2ai.show