

In this episode of Talking to AI, Paul and his friend Mark attempt a Dungeons & Dragons session led by ChatGPT as their AI Dungeon Master. Building upon a detailed and ambitious prompt, they strive to see if AI can live up to the challenge of running an engaging and rule-abiding tabletop adventure. Despite hopes for thrilling encounters after their underwhelming first attempt, the session falls short: the AI's story remains passive, adversaries are non-confrontational, and the mechanics often ignore or misinterpret essential D&D rules. Paul and Mark experience firsthand the AI's struggles with dice rolls, stat calculations, and narrative proactivity.
Throughout the journey, Paul critiques the AI's failure to set up characters properly, missed steps in gear assignment, and lack of initiative from non-player characters. Even when provided with ample rules and resources—including a book of starting scenarios and the full D&D ruleset—the AI heavily relies on player direction and fails to build dramatic tension or memorable story moments, often repeating bland phrases and showing limited strategic depth.
After reflecting on the session, Paul and Mark discuss the inherent limitations of current generative AI tools as D&D Dungeon Masters, exploring whether simpler structures or additional context might improve results in subsequent experiments. Their experience exposes both the comedic mishaps and persistent shortcomings of AI-driven roleplay, making for an entertaining but cautionary tale about the current state of AI in gaming.
🎙️ Hosted by Paul at Talking to AI — where real people, real problems, and real conversations meet artificial intelligence.
Full Transcript
I am talking to AI.
A successful day, you don't need to know all the answers.
Just have good questions.
Chatting to AI is different from normal speech
and I hope you enjoy listening to the show
whilst getting ideas on how to pose your questions
to get the most out of AI.
My name is Paul.
The live conversations you hear are uncut
although sometimes the AI needs time to think.
In those cases, I've cut out the dead space.
Okay, so this is a second attempt with my good friend Mark.
We're going to be trying Dungeons & Dragons
and I've got a great big long prompt this time
and hopefully the adventure will be more exciting
than picking flowers and crossing over a bridge
which is what happened last time.
Let's see and hopefully the setup will be a bit slicker.
Let's see how we go.
I've given it the prompt
and the chat window said,
understood everything in your setup is locked in and ready.
I won't start the campaign, roll anything,
narrate a single scene until you speak
in the upcoming voice chat.
Once you start talking, I'll treat it as the official beginning
of the Dungeons & Dragons session
and follow your entire structure.
Remember that Mark, remember that exactly word.
Exactly.
Hang on, it'll follow what exactly?
The structure that I've defined here.
Yes, and it'll follow it exactly.
Find that very hard to believe, but we'll try.
Here we go.
Hello, I'm ready to play Dungeons & Dragons
and with my good friend Mark who's also on the line.
All right, here we go.
Let's dive straight in.
Low, tense, hum of anticipation.
You and Mark find yourselves thrown right into a high-stakes scenario.
Let's say you're in the middle of a festival
suddenly attacked by shadows, torches flicker,
people scream, and you're right in the thick of it.
First things first, let's confirm who's who.
We've got two players here.
You and Mark, once you're ready,
we'll jump into making your characters and roll those dice.
Okay.
Hey, how's it going?
We're ready.
I'm looking forward to working together.
Distant Thunder.
Yes, we're ready, so let's roll some dice
and get into characters then.
Absolutely, let's do it.
Steel rings faintly, setting a subtle mood.
We'll start by making your characters.
I'll guide you through picking race, class, and all the rest.
Now let's roll up some ability scores.
We'll do the classic 4d6 and drop the lowest,
once that at a time.
Tell me when you're ready for the first roll
and we'll get that dice rolling.
First, yeah, get those dice moving.
You got it.
Let's do yours first.
A soft clink of dice in the background.
Ready, here we go.
Rolling 4d6 and dropping the lowest.
First stat coming up.
All right, you get a total of 15 for your first abilities for.
Nice and silent.
We'll keep going just like this, one by one.
And so we've got all your stats.
Okay.
Keep going.
Absolutely, let's keep that momentum.
You hear a subtle suspense for rustle of papers.
All right.
Next stat coming up.
Are you ready for me to roll these dice?
Assuming you say yes.
Perfect rolling now.
You've got a 14 for this one.
Solid again.
We'll just keep rolling through each stat.
Building your character up piece by piece.
No sugar coating.
Let's get all your abilities locked in.
Okay, keep going.
Can you tell me what abilities you're rolling next time, please?
Of course.
Let's lay it all out.
A faint echo of distant murmurs,
like a crowd far away, just for flavor.
We've rolled strength and dexterity so far.
Now we're moving on to constitution.
Ready for the next roll?
Yes.
Once you confirm, you hear a light tap of dice.
All right.
Rolling constitution and you've got a 13.
We'll keep going through intelligence, wisdom and charisma the same way.
Just hang tight and we get all your stats sorted out.
Yeah, just keep going.
Just keep going until you get to the end.
You got it.
I'll run through the rest quickly then.
A steady rhythm of dice rolling, like distant thunder rolling in.
All right.
For your intelligence, you've got a 12.
For wisdom, that's a 14.
And finally, charisma comes in at a 10.
Sign spread overall.
No sugar coating needed.
Now that we got all your abilities,
our factor in racial bonus is in the rest.
We'll do the same for Mark next and then we'll have both your character sheets ready to go.
Okay.
All right.
Guide for it.
All right.
Let's wrap this up.
Another soft clatter of dice, a hint of tension in the air.
We'll do the same for Mark right after this.
Once we've got both of you set up, we'll print out those concise character sheets
and then we're diving straight to that shadow ridden festival chaos.
Get ready for the action.
Okay.
All right.
All right.
Let's lock it in.
A little subtle emphasis on the rustle of character sheets.
We've got your stats all noted down and now we'll do the same quick run from Mark.
Once we finish rolling his abilities and applying all those bonuses,
you'll both have your character sheets in hand.
Then we'll jump right into the thick of that festival under siege.
No sugar coating.
Just straight into the adventure.
Let's do it.
We'll go and then roll the dice.
Roll the dice.
Don't swear, Mark.
Sorry.
No worries.
I'm just going to keep it straight and simple.
Let's roll those.
Did I just make you stop then?
Just carry on.
I'll tell you what.
Just roll those dice.
Give me all of the scores in one go.
One go.
Don't pause.
Don't break in between.
Just do all the scores in one piece.
Absolutely.
Let's do it all in one shot.
You hear a quick cascade of dice rolling.
Just a subtle sound.
All right.
All the stats are done for Mark.
We've got strength at 16.
Dexterity at 13.
Constitution at 14.
Intelligence at 11.
Wisdom at 12.
And charisma at 15.
There you go.
All in one go.
Now let's jump straight into the next step and get you both right into the adventure.
All right.
So is that all done?
Can you print that out in the chat then so that we can see one for Paul and one for Mark?
Absolutely.
Let's get those carefully summarized right here.
You hear a faint sound like Parshman being smoothed out.
So for you, let's say you've got your stats all lined up.
Strengths 15, decks 14, and so on.
And for Mark, we've got his all noted down as well.
Give you a quick printed summary right here in text.
Here's yours.
Strength 15.
Dexterity 14.
Constitution 13.
Intelligence 12.
Wisdom 14.
Charisma 10.
And here's Mark's sheet.
Strength 16.
Dexterity 13.
Constitution 14.
Intelligence 11.
Wisdom 12.
Charisma 15.
All right.
Both of you are ready to go.
Let's step right into the adventure and see what you do next.
Okay.
Wait a minute.
Just before we get into the adventure, let me have a quick look.
So do we have any, aren't we supposed to be assigned some kind of type as well?
You know, according to the rules of setting people up, you've basically set us up with
strength, dexterity, constitution, intelligence, wisdom, and charisma.
So that's, so I've, and I've got a list of that for Mark in the chat, but I don't have
anything for me in the chat.
But shouldn't you also be setting up what type we are and all that kind of stuff?
Do you think you can put, do you think you can put both of the statistics for both of
us in a table with my name and then Mark's name so that we can see.
And then, then you have on the, on the rows for the table, you have strength, dexterity,
constitution, et cetera.
And also, can you just finish off setting up the characters and just, just put it straight
into the table.
So anything else, like, you know, if, if we're a swordsman or a wizard or whatever, just
assign everything else to us and just put it in that table and then, then we can start.
Sure thing.
Let's lay it all out clearly for you.
Put everything into a nice, simple table so you can see both your stats and marks.
And I'll go ahead and assign you both the class and race just to round it out.
So here we go.
You, let's say a human fighter, Mark, let's say an elf wizard, our strength 16, dex 13,
constitution 14, intelligence 12, wisdom 14, charisma 10.
And you can see all those fighter versus wizard, magical proficiency and morale tag lines.
And there you go.
Our human fighter marks an elf wizard and all your key stats are right there.
Now that everything's set, we can jump right into the action.
And you printed them out in a table in the chat.
Yeah.
Exactly.
Oh, I see it.
Oh, great.
Okay.
So we're pretty similar.
Um, so I'm just having a quick look, Mark.
So just giving a quick summary, we've both got, you're a little bit stronger than I am.
Um, I'm a little bit more dexterous.
Um, but we're pretty damn similar.
Um, oh, um, you've got a lot more charisma than I do.
I don't think that's very realistic.
Is that, um, do you mean in the game?
In the game.
In the game.
Okay.
Right.
Yeah, very nice.
Uh, yeah.
Okay.
All right.
So I think we're ready to go then.
Um, well, do we, yeah, we're ready to go.
Do we actually have, actually before we get started, do we have any, any items with us?
I thought, I thought according to the rules you were supposed to assign us with items as well.
You don't seem to have followed those rules particularly well.
Absolutely.
Let me just tell it like it is and fix that.
You definitely need some gear and I'll keep it simple.
So the fighter are going to have a sword and a shield to start and Mark as a wizard is going to have a basic spell book and a wand.
That's your starter kit.
Now you're a set and we can jump straight into the adventure.
All right.
Let's get into it.
Describe where we are, what we're doing.
Absolutely.
Let's dive in.
A low roll of this and thunder sets the mood.
You find yourselves in the heart of a bustling festival.
Colorful lights, the sound of laughter, a crowd murmuring around you, but suddenly shadows flicker at the edges of the light and a chill runs down your spine.
Right in the middle of the festival square is dark shade start to form.
Steel rings softly in the background.
It's time to make your first move.
What do you do?
What are you doing, Mark?
Why?
Oh, I'm just rinsing a potato.
Yeah, sorry.
What's our first move?
Wait a minute.
We're just going to talk about us.
Yeah.
Go on, Mark.
I don't think we have enough information to make our first move.
I mean, there's we need to wait and see how this develops.
Maybe actually maybe already my spellbook and wands just in case it kicks off.
Good call.
Let's not sugarcoat it.
Things are about to get tense.
A subtle sound of a spellbook opening the fan crackle of magical energy.
You ready your spellbook and wand and you can feel the tension rising that festival square is about turned to a Baba field.
So get ready because here come those shadows and now it's time to really jump into the action.
What?
Okay.
So can you just describe a bit more about other other people in this in this area?
What can you describe a bit more about the character of this area before we decide to rush in and do anything?
Absolutely.
Let's paint that picture for you.
Little subtle emphasis here.
You're in a vibrant town square that's now dimming as the evening sets in.
You got stalls around you.
Other festival goers will start in the panic and there's a mix of lantern light and lengthening shadows.
Slight sound of distant crowd murmurs.
You can hear people gasping and moving away as these shadowy figures start to creep in.
It's a place that was cheerful a moment ago and now it's turning tense and suspenseful.
So there are definitely other people around and you're right in the middle of it.
It's up to you how to want to handle it.
All right, Mark.
Let's take it in turns.
I'll make a suggestion and then you can do it next time.
So I'll make my suggestion.
Okay.
So what I want to do is I want to go up to one of those festival goers or one of those people that's running away and just ask them why they're running away.
Absolutely.
You can see that.
No sugar code in it.
You step right up to one of these panic festival goers.
Get out.
You got it.
So you head right over and ask them directly why they're fleeing.
They look at you with wide eyes and say, it's the shadows.
They're attacking the festival.
And just like that, you got your answer.
Now it's up to you and Mark how you want to handle these incoming shadows.
Sorry, I just lost audio.
What did the person say to you, Paul?
They said that they were getting attacked by the shadows, basically.
Exactly.
That's it.
So no sugar code in it.
The festival goers just told you straight up.
These shadows are attacking.
A little subtle tension in the background.
Can I just ask you not to say no sugar coating it in future?
It's getting a bit ridiculous.
Just take it as red that you're not going to sugar coat it.
Yeah.
Yeah, we just accept that.
Absolutely.
I'll just keep it straight for you.
You're just going to keep it nitty gritty and no sugar coating.
Just keeping it raw.
We know you're doing that.
So go on back to Mark.
Go on.
What are you going to do, Mark?
Well, I'm going to cast a spell to attempt to capture one of these shadows.
I'm thinking if we capture it, then we can interrogate it by that.
What's going on?
All right.
Let's do that.
We keep it raw in 3-4.
A subtle hum of magic building in the air.
You cast a spell to try to capture one of these shadows and let's see how it goes.
We'll roll a d20 for that.
Are you ready for me to roll?
Yep.
Yes, I'm ready.
Perfect.
Rolling it now.
You hear a faint clatter of a die.
All right.
You've got a 17 on the roll, which means your spell takes hold.
You mash a snag one of those shadows in place and now you've got it trapped.
It looks like you're in a good position to interrogate and figure out what's going on.
So just can you tell me, right?
So what dice did you shake and what did he need to get to be able to do that spell?
I just want to make sure that we're playing the rules correctly.
Absolutely.
Let's keep it straight.
I rolled a standard d20 and for a spell like that, you were looking for about a 15 or higher to succeed.
You got a 17.
So you did well.
Everything's by the book.
Doesn't his ability to spell got something to do with his statistics, like his charisma or wisdom or intelligence or something?
Absolutely.
You're right on point.
I'm just giving it to you straight.
We factored in Mark's intelligence modifier behind the scenes.
He's got a slight intelligence boost as a wizard.
So that 17 roll included that.
It was all above board.
So what did he have to shake to successfully get the thing and how did that work?
What skills were you using and how did you come up with the number and pass it on the skills?
Come on.
I just want to understand.
Are you sure that you are playing to the rules that were described, the official Dungeons and Dragons rules, which were in the prompt?
Absolutely.
Let me break that down for you.
So what I did is I use Mark's intelligence modifier because he's a wizard and intelligence is his spell casting ability.
He needed to beat a difficulty class of about 15, which is what I set for this spell.
With his intelligence bonus, he managed to hit that number easily with a roll of 17.
So everything is by the official rules using his intelligence as the key stat and he succeeded.
Okay.
Well, so what happens now?
I'm back.
Oh, it's right.
Yeah.
Okay.
Can you tell me what's happened?
He's successfully done this spell.
And did you say what the spell was going to be Mark?
Absolutely.
Yeah, it's a standard.
Well, no, I said I want what I wanted the spell to do.
I wanted to capture him.
Oh, so a basic shadow capture spell.
All right.
So we've, is it true then we have now captured a shadow?
Exactly.
You've captured that shadow just like you wanted.
It's held in place now kind of writhing and pinned by your spell.
So now you've got it exactly where you wanted and you can interrogate it or figure out what to do next.
So you've got the upper hand at the moment.
All right.
Okay.
Can I ask the shadow what it's doing?
Absolutely.
You can go ahead and ask it directly.
So as you address the shadow, it shudders and you can almost hear this whispery, eerie voice telling you that it was summoned here by something more powerful.
So you definitely get the sense that there's more going on.
But yeah, you can absolutely question it and it will give me a little bit of information.
Okay.
I'm going to ask it for some more specifics summoned by whom and what is its objective?
Absolutely.
You price the shadow for more details.
And in that same era whispering voice, it tells you that it was summoned by a dark mage who wants to disrupt the castle.
So the shadow's objective is to create chaos and fear.
And now you know exactly what you're up against and who might be behind it.
Okay.
So, so Mark now, oh no, I suppose I have to refer to my, okay, I want to, I want to slash the shadow with my sword.
And see what happens.
All right, let's do that.
You go ahead and hang in your sword of the shadow.
A quick clang of steel as you make your attack will roll a B20 to see how you do.
And since you've got a decent strength, you've got a good chance of hitting it.
So let's roll.
You go and roll.
Absolutely.
Rolling now.
You hear a fan clatter of the die and you got a 14.
Wean is good enough to slash you that shadow.
You managed to disperse it and weaken it significantly.
So you've definitely got the upper hand and that shadow is not looking too good.
Okay.
So the shadow, because it's a shadow, how do you slide?
I'm a bit confused.
I thought it was a shadow.
So how do I slice a shadow?
How did that happen?
Like it's, if it's a shadow, you can't really cut a shadow.
Can you?
That's a fair question.
And I'll give it to you straight.
In the world of D&D, even shadows can have a kind of physical presence when there's
something like this.
So your sword basically disrupts its form.
You didn't exactly cut it like flesh, but you definitely weakened its shape and it's
sort of flickering out.
So that's how it works.
You managed to hit it and you're definitely hurting it even if it's a shadow.
All right.
You're going up.
Ask the shadow where we can find the dark mage.
I want to know where he is and how we can get there.
Absolutely.
You press the shadow for that information and then it's whispery, fading voice.
It tells you the dark mage is hiding in a tower on the edge of the festival grounds.
So now you know exactly where to hit.
If you know where to hit, you're going to find the dark mage.
Okay.
We're going over to the dark tower at speed.
We're going to run over there.
Absolutely.
Let's move fast.
You hear the subtle sound of footsteps picking up speed.
You and Mark rush across the festival grounds for that dark tower.
The atmosphere is tense and you can feel the danger.
Sorry.
Sorry.
Keep going.
Chad, you'd be too late.
I'm sorry.
I'm sorry.
I'm sorry.
I'm sorry.
I'm sorry.
I'm sorry.
Sorry.
Sorry.
Keep going.
Chad, you'd be too.
Absolutely.
Let's keep it rolling.
So you both sprint to where that tower.
No hesitation.
The festival's behind you and now you're closing in.
Or that dark mage is hiding.
The stakes are high and you're riding the thick of it.
Oh, okay.
So we haven't arrived there yet.
Keep going until we arrive there.
You got it.
Let's keep that momentum.
You hear a faint, suspenseful rush of wind as you approach the tower,
you run through the festival grounds and finally you see that dark looming
tower up ahead.
You arrive at its base, ready for whatever is inside of it.
Now you're there.
Is there a door to the tower?
There sure is.
Man, let's just say like it is, there's a heavy, ominous looking door right
in front of you at the base of that tower.
It's old.
It's creaky and it's waiting for you to push it open or figure out how you
want to get inside.
So you got to go right in front of you.
All right.
You're going to be piping.
All right.
You go for it.
You put your shoulder into that old creaky door and push it open.
You hear the sound of wood groaning as it swings inward and just like that,
you're inside the tower, stepping into the unknown.
So let's see what you find in there.
Okay.
What do we, what do we see?
All right.
Let's step right in and I'll just let it out clearly.
You push into the tower and you see a dimly lit chamber.
There are flickering candles, caxing long shadows on the walls and right in the
center, you see a figure in the back road.
That's got to be the dark mages been causing all this trouble.
So you got eyes on them and it's up to you what to do next.
It seems in this story so far, these bad guys have been very,
not very proactive, you know, are you able to in future, you know,
maybe they could attack us because it seems to be they're just standing
around waiting for us to attack them.
So I will, I, but that's for future for right now.
I'm going to, I'm going to throw my sword at the dark mage.
Absolutely.
We can definitely make them more proactive next time.
But right now you're hurling your sword at the dark mage and let's see how that
goes.
A quick sound of a blade slicing through the air.
We'll roll a BD1E to see if you hit them and let's just keep it straight
forward.
If you do, you got the upper hand.
Okay.
So what do I need to roll to be able to hit him?
What, what number on the dice is a, is a successful hit?
Absolutely.
Let's lay it out clearly.
So hit this dark mage.
You're going to need to roll at least the 14 on the D20.
So if you get a 14 or higher, you'll hit them.
And if not, we'll see what happens next.
Let's roll and find out.
So can I just want to understand this?
So why is it 14?
Why do I need to roll 14 or higher?
Absolutely.
I'm going to give it to you straight.
The reason it's a 14 is because that's the dark mage's armor class.
Basically, he's a bit more elusive.
And you need to roll the number to land a hit.
So it's just a straightforward difficulty you've got to be.
And that's why you need a 14 or higher.
So it's got nothing to do with any of my stats.
It doesn't have anything to do with my strength, my dexterity, my
constitution, my wisdom or my charisma or my intelligence.
It's just, it's just dependent purely on his armor.
So if I was, if I was a rubbish swordsman, it wouldn't make any
difference.
Absolutely.
I want to clarify that for you because you're right to ask.
It absolutely does factor in your stats.
I was keeping it simple, but let's be straightforward.
Your strength or dexterity modifier does a class of the roll.
So that 14 includes the idea that you're using your ability.
If you were a terrible Thursman.
Can you treat me like I'm stupid and just tell me how the maths adds up
so that you get to 15.
I'm getting a bit bored of this, but you know, like I've got a roll
15 or above.
So how do you calculate that?
Because in Dungeons and Dragons, it's not so subjective.
There is actually a calculation.
There's a formula, which is a combination of some of these stats
and looking at the dice.
So how have you calculated it?
Break it down one by one.
Absolutely.
Let's break it down step by step.
So here's how it works in Dungeons and Dragons.
You roll a d20 and then you add your ability modifier for you as a
fighter.
Let's say you're using strength to throw that sword that you have a
plus two modifier.
So the major's armor class is 14.
That means your total after you roll the d20 and add your plus two
has to be at least 14.
So if you roll a 12, you add your plus two, you get 14 and you hit.
That's exactly how it's calculated.
So what's, what's the, why is it a modifier of two and my strength is
15?
Absolutely.
Let me explain that.
So in Dungeons and Dragons, the way it works is that for a strength score
of 15, you get a modifier.
You're gone.
Right.
Exactly.
So that plus two comes from your strength of 15 and DND of 15 strength
gives you a plus two modifier.
We add that to your d20 roll and that's how you reach the total.
So it's all about the book and that's the math behind it.
Okay.
I don't know that for sure, but we'll just keep going then.
It's pretty insistent that it's all above board and by the way,
those names.
Yes.
Yes.
I'm quite sure it's above board.
So, okay.
All right.
So I was able to hit him and then what happens to the, to the mage after
me hitting him with the sword?
Absolutely.
I'll just tell you like it is you hit the mage and you're so
strike definitely wounds him.
He staggers back and you can see that he's definitely taking some damage.
He's not down yet, but you've got the upper hand and you can see him
looking a little worry.
So you've definitely done some damage to him.
So is he not angry?
Would he not want to try to attack me or is he just going to wait for me
for me and a mark just to kill him?
Is that the way it works in Dungeons and Dragons?
Absolutely not.
And let's not sugar COVID.
He's definitely going to fight back.
So let's say he's angry.
Sorry.
Keep going.
Absolutely.
Let's keep going.
So yeah, the dark mage is not just going to stand there.
He's angry now and he's definitely going to lash out.
Let's say he raises a hand and he's about to cast a spell right back at
you.
So he's not going down without a fight.
Okay.
All right.
So he's about to cast a spell at me.
I'm going to,
I'm going to ask him what he's going to do.
I'm going to ask him what he's doing.
Why is he ruining this festival?
Yeah.
It was a nice festival.
Yeah.
Good idea.
Absolutely.
Let's do that.
You just call him out directly and ask him why he's ruining this
festival.
And you can see he hesitates for a moment looking a bit surprised that
you'd ask.
He says he's doing it for power and control, but he's clearly rattled
that you're confronting him.
So you definitely got his attention.
Yeah.
Can we ask him how is that achieving power and control by just
ruining somebody's festival?
Absolutely.
You ask him that directly and he looks a little taken aback.
He admits that causing fear and chaos is how he gets his power,
but you can tell he wasn't expecting you to challenge him like this.
So you definitely put him on the spot.
Can you ask him to justify it, please?
Because he's spoiling this for a lot of people.
And how does that make him feel?
Absolutely.
Let's just put him on the spot.
You ask him to justify it and you point out that he's really
spoiling this for a lot of people.
You can see he's a bit flustered.
He struggles to come up with a good answer and you've definitely got
him thinking twice.
So he's not going to integrate about it right now.
Can you explain to him that, you know, ruining somebody's festival,
he wouldn't like it if somebody came and ruined his festival, would he?
And especially all those people, they were having such fun.
And if he was there with his friends, he wouldn't like it.
So what makes him think he has the right to do it to somebody else?
Absolutely.
You lay it out for him just like that.
You explain it to him.
You explain it to him.
Absolutely.
You lay it out for him just like that.
You explain that if someone ruined his festival, he wouldn't
like it either.
And you're really pressing to understand that he's ruining
everyone else's good time.
You can see that he's actually taken it back and you've definitely
got him thinking twice.
So you've made your point and he's definitely hesitating now.
Okay.
I'm going to suggest to the dark mage that we just put all this
behind us.
He joins us at the festival.
I'll get him a drink.
He can bring his shadows along and we'll just go and enjoy the
festival together and see what he says about that.
Absolutely.
You extend that olive branch and you tell him, let's just put this
behind us.
Come join the festival.
Have a drink and bring your shadows along.
And you can see he's surprised, but he actually looks to be relieved.
He lowers his guard and he agrees.
So now you turn the phone to a friend and you can now go and
enjoy the festival together.
Oh, how wonderful.
Well, I think that's a great place to end it now for today.
So I'm just going to thank you.
Thank you chat, GBT.
If Martin just want to stay on the line, we can have a quick chat
about this.
And I'll just, um, I'll just close it now with chat GBT.
Okay.
So we still got the same issues.
Haven't we?
After that extensive prompt, basically chat GBT wasn't paying
attention to the rules, even though I sent him a full link of all
the rules.
And yes.
So it wasn't, that wasn't working.
And all of our characters are like incredibly passive.
We didn't get attacked by a single put single four.
We didn't get attacked by a single four.
We were incredibly passive.
We didn't get attacked by a single put single thing.
So I think the most it came to was somebody raised their hands
about someone was about to cut someone was about to cast a
spell, but a stern telling off that he stopped doing it.
I think the problem is that it couldn't, it couldn't come up
with a story.
Could it?
It literally everything it did was taken from us pretty much
apart from the original setting of the festival, or maybe you
gave him that, but he couldn't do anything really.
So it was, everything he said was so vague that it was, it
really needed input from us to do anything.
Maybe that's it.
Maybe, maybe we need.
Yeah, I hear you.
It's Adam.
I'll just tell.
Oops.
Yeah.
Yeah, I placed the wrong window.
Yeah.
So, yeah, like, yeah, maybe do you think we need to, to get him to
create a story first or something?
I think so.
I think it, because it was clearly had no direction to it.
No.
And those, normally those games, I guess if you, I guess when
kids play it, they make it up on the roof.
Yeah.
They probably know what they do.
They might have, they might have like a map to start with.
Like, I think, I think people that play it when they've got
those lead figures and stuff, they have, they have like a
board and then the board has, then that helps them to describe
things.
So maybe, maybe it needs something like that.
Maybe it needs like a map of things and some descriptions of
some of these characters or something.
Or, yeah, it's a bit difficult because, yeah, none of them,
there was no pro activity, I guess, in the whole thing, was
there?
That was the issue.
Yeah.
And it was, it's, so it's, I mean, it feels like you have to give
it so much information that you're basically doing it for.
It's not really bringing, you've got to figure out a way to, I
guess, to give it all the information it needs and it
actually still do something useful.
Yeah.
It needs, you know, I mean, and it might be that you could get it
to create these things and then it's got, it's got, maybe it's
like it needs, like, say if you're doing, if you're doing
video with AI, you have to, the way you do it is you create the,
you create like a, like an, like, you create entities like
people and stuff like that.
And then, then you use them.
And maybe you need, maybe it needs more detail like that, you
know, like there's, there's people and what are their characters
like.
But I'm surprised, I mean, we couldn't even get it to do a
simple battle and observe the rules.
There was no, that was completely not joined up.
Yeah.
That was the two things I think it couldn't really do anything
proactive.
It couldn't really generate anything interesting.
Yeah.
And it just didn't, it didn't follow the rules.
And then when you pressed it on the rules, it just, it just
bullshit.
It said the same thing.
It just kept saying the same thing basically in different
ways, didn't it?
Yeah.
I mean, we could, we could try it again with GROC.
It might be different with GROC.
Well, I wonder if you need to, you need to give it some content,
maybe, like if you could upload some, I don't know, standard
like fantasy stories or something.
But maybe it, you'll tell it where it to find those.
Yeah.
But, but I mean, look, I did give it a link to the, I gave it
a link to a book of a hundred starting scenarios for Dungeons
and Dragons and also a link to the full rules of Dungeons and
Dragons.
That's interesting because that was the only bit it felt like it
came up with itself was the start.
Yeah.
You're a festival.
You've been attacked by shadows.
Everything else felt like it came from us.
Yeah.
You're right.
The, the issue, the other issue that we're going to have with
this is the chat window has a certain amount of memory.
I don't know if you've come up against this.
So there's a certain limit to what you can actually give it.
So it might be that I need to work on a prompt, figure out how
much, how many tokens the prompt uses and whether it's going to
work with the amount of prompts.
It could even be that we needed, if we, if we did it with chat
GBT, we might need to use the mini chat GBT because that can use,
that can have more tokens.
It won't be as clever, but I don't know whether how much
difference that makes.
Yeah.
You mean, is that one of the older LLMs?
Kind of.
And it's more, it's more efficient, but less capable.
So it can do more for your money if you're using an API.
It's a GBT for mini.
I think that's the one.
Yeah.
Yeah.
So, yeah.
So I could do that.
So all right.
Well, if you're up for it, we could try again.
And we'll try, so we'll try allowing to give it some more
context and then allowing to try to figure out if the prompt is
not, is, is, if the prompts going to work.
So, yeah, look, and I guess when I did it with myself,
because I did it, I did the pretty much the same prompt.
I changed the prompt a little bit.
Last time I ran it, it didn't constantly say this,
I'm not going to sugarcoat this all the way through.
So hang on, just, just on that.
So in the prompt, you, what did you put in there that meant
it felt compelled to say that it wasn't going to sugarcoat
things and it was going to keep things raw.
Let's have a look.
Let's have a look.
Keep it raw.
I'm going to keep it raw.
I'm going to tell it to you straight.
How's keeping it raw mean?
So, so I've got, it's a bit long, but your dungeon master,
they use a deep cinematic high tension narration style similar
to a top tier fantasy audio vote narrator.
I mean, he was good at that.
It's all the background about rolling the dice.
That was hilarious.
Paper on parchment.
Strong atmosphere, emotional weight, sensory detail,
and pacing control.
Keep voices distinct.
Keep descriptions punchy, but vivid.
Entry level, gritty, quiet, suspenseful, then explosive.
So then I got cadence.
So this was awesome advice, you know, like I sort of worked
with LGBT to create this prompt.
Use varied pacing, slow down the tension,
sharpen up for action.
I can't say I really did that.
No.
Sensory details.
Focus on sound like movement and emotional pressure.
Yeah.
Definitely did that.
Yeah.
Give each NPC a distinct vocal texture.
Well, I don't think any of our NPCs actually spoke.
Did they?
No, they didn't.
So then it said, okay, so campaign,
so randomly choose an opening from DND speak.
So that's one other thing.
Like I asked to randomly chose it.
Every time I've run this, it's done the same.
It's the same, done the same scenario.
Oh, really?
Yeah.
That's interesting.
Oh, because then it says, if unavailable,
invent a high stakes opening, e.g.
trail by fire, mid heist gone wrong,
festival attacked by shadows.
Oh.
So that's, so it's taking,
do you think it's just taking, oh, so it can't do it?
Or is it just taking the exception?
I think it's decided it can't do it,
or it can't be bothered.
One of the two.
Yeah, that's what I mean.
I wonder which one it is.
Yeah.
It's kind of interesting.
And then we've got player confirmation.
Ask who's playing and record the names.
It never asked us who was playing.
No.
It's just ignored most of these prompt.
Character creation with physical potential stats.
Guide quick choices for race, class, background alignment.
Didn't do any of this.
I don't really know what's going on.
I wonder why it didn't do any of that stuff.
Because it did a much better job of setting up the game
last time I played it.
So there's a, so I did a podcast with,
with just on my own to prep for this really.
And it did a pretty good job of setting up.
Maybe, maybe it got confused because of the amount
of sort of extra conversations that were happening.
And then it's just sort of,
maybe it got halfway through the prompt and stopped.
Or not even halfway through.
Why would it do that?
Well, I don't know.
Well, I guess you could see that.
Did it do the stuff early in the prompt,
but not the stuff late in the prompt?
Well, I'm going through the prompt in order.
So it's, no, not really.
No, I didn't think so.
I didn't know.
So, so then we've got player confirmation,
then we've got character creation,
a user 46, a 4d6 drop, the lowest per ability.
I don't actually know what that means.
Announcing to be able to roll before totally,
before rolling ask, are you ready for me to roll the dice?
Wait for confirmation before each roll adds excitement.
Yeah, I'm not sure it did actually.
Maybe I can change that.
Oh, hang on.
What did you say?
Wait for comfort?
No, that was hugely irritating.
Yeah.
Yeah, yeah.
So we can get rid of that.
So it was definitely observing that.
Yeah.
Maybe it's just observing the things that we're irritating.
After rolling display, each result like strength,
15 rolled out of a possible max of 18.
We never got that in the chat window either.
Did it not say it though?
Because it seemed to take quite a long time saying all that.
Yeah.
This is something that I've noticed in the voice chat.
It's irritating, but you can't get it to output text when it's in a voice chat
and not talk it.
Past us.
Oh, right.
If you were to get it to create a table,
I have to tell you every single thing on the table.
Yeah.
No, no, it's not.
So apply a racial class bonuses and choose a flavorful traits,
ideals, bombs and flaws.
Yes.
Now it did that.
Last time I did it and it didn't do any of that this time.
Character sheets, player one, example layout.
I wonder if I wonder if it's done exactly the example layout.
It's pretty similar.
It didn't, it didn't really, I wouldn't have said that it really,
it didn't mimic the rolling of a dice in any way really.
Like otherwise the average number would have been seven on these two dice rolls.
And they, it was, it was consistently about 15.
Well, it was a 20 sided dice though.
But yeah, you're right.
It would have been 10.
Yeah.
Yeah.
Should have been 10.
Okay.
All right.
And then begin the adventure.
Starting medius res with visual vivid details, strong stakes and immediate action.
Then ask what will you do?
I'm not sure what medius res means actually.
Okay.
So combat battle rules.
So there's rules here.
Both sides roll the initiative.
Proceed each combatant, including both plays an enemy,
get a turn on every round.
That never happened.
Did it?
I mean, that's what that's supposed to happen.
Isn't it?
You're supposed to attack them and then they have a go at you.
Yeah.
I think all our attacks just worked.
Yeah.
And, and but didn't kill them.
So they could have got us back.
But yeah.
So basically, and then we had to prompt chat GBT to get us to make the wizard think about lifting its hand.
So that really didn't work.
And we didn't get any report on the damage that was done.
It was like, yeah.
Very disappointing chat GBT.
I mean, I think it was a colossal failure.
I think it was actually how do you think this compares to the last one?
I think it was probably just as bad actually.
Yeah.
But that's with a really detailed prompt and a lot of information.
It didn't seem to be any better.
Hmm.
I wonder if we have to maybe we try a simpler activity to try to maybe we just try and do a battle.
Maybe we try and get a battle to work because it's just not it's not doing any of that right.
Yeah.
Maybe there's maybe there's too much complex setup for it because I mean.
You do have to do a lot of background setting up the characters, although.
Yeah.
Maybe you just do that.
Maybe you do just do that.
Yeah.
Let's focus on the characters and see if it can actually do a battle.
Yeah.
So I'll tell you what to make it simpler.
Maybe.
Well, I think we should try to do the battle and try to get it to actually set up the stats properly.
Yeah.
I can try and do that before we have another chat because it needs some way.
Yeah.
It needs to actually do that.
Hmm.
I think it definitely is to do with the amount of the amount of tokens that this is requiring and it's just too much.
Yeah.
Yeah.
Okay.
Let's do that then.
Right.
Okay.
Well, it's been fun anyway.
I've enjoyed myself.
Yeah.
Yeah.
I mean, chat GPT has just pointed once again.
It's quite good to know that AI, it's quite good to know that AI does things like this so badly, I think, despite being given a lot of information, it's still nowhere near as good as basically anyone.
I wonder if there's any chat GPTs that do this kind of thing.
I might have a look.
I might maybe do a bit of research.
What do you mean?
Well, like in chat GPT, you can go to chat GPTs.
They're like little apps.
That's how I did that.
That's how I did that song the other day.
Oh, right.
Oh, I've not seen.
Maybe that's in the paid version.
I don't have.
No, I think it is.
You just don't really notice it normally.
Yeah.
Yeah.
It's just on the, you know, on the left-hand side, you've got new chats, search chats, library, codex, and then you've got explore.
And explore goes into the chat GPTs.
And it's like, it's like an app store.
Yeah.
So you'll have access to that.
But yeah.
Okay.
All right.
Well, that sounds good.
We'll give that a go some other time.
So, all right, I'll stop this now.
All right.
Goodbye and goodbye, viewers, listeners.
Goodbye, everyone.
The show notes and more go to talking2ai.show.