[Video Transcript]

Hi everyone, Alex from OpenMoves here and today we are going to challenge the Chat GPT AI with a few questions on search marketing. 

Let’s see how AI measures up with this first question. 

Question number one:

Give me 10 keywords related to the company OpenMoves”

AI Answer: Marketing, Advertising, Digital, Technology, Strategy, Creative, Solutions, Communication, Media, Branding

Our Feedback: All right, so we heard Chat GPT’s answer there to ten keywords related to OpenMoves in terms of what we think of this answer. Well, actually, it’s a pretty bad answer. You can see that although these keywords have some general thematic relevance to OpenMoves, they may not be explicitly wrong, they are not very interesting or informative or helpful, and you generally wouldn’t consider them to be a very good keyword for OpenMoves from any kind of search marketing perspective. Now, with that being said, the prompt,  the question that we asked Chat GPT was also not a very good or helpful question. It didn’t describe how we were going to use the keywords or why, and we didn’t give Chat GPT any direction on what sort of keywords to produce.

So I think this is a good example of sort of garbage in, garbage out if you give an overly generic or overly broad prompt, the Chat GPT, then the output that you get is maybe not great. I think from a use case perspective from marketers, what this kind of shows us is not to rely on the answer from a single prompt, not to just ask Chat GPT one time and then just live with the answer. You can really iterate, you can ask more specific prompts. There’s actually lots of information online about good prompts and prompt libraries and things like that. So I would definitely encourage everyone using the tool to do multiple prompts interative prompts and don’t live with the first answer because as you can see in this question answer format, the first answer we got back is pretty bad, even though Chat GPT actually is a perfectly useful and valid keyword research tool.

Question number two: 

Prompt: “When should I use Google Ads vs. Facebook Ads for my business?”

AI Answer: Google Ads is best for who’s looking to reach people who are actively  searching for products and services. Facebook Ads is best for businesses looking to reach a large and  highly-targeted audience.

Google Ads are primarily text-based and appear on search engine results pages, while Facebook Ads  can be text, image, or video-based and appear on Facebook.

Our Feedback: All right, so we heard Chat GPT’s answer there on Google versus Facebook.  I think it’s a reasonably good answer. What Chat GPT gets right here is kind of the central theme on search versus social advertising, which is the idea that Google Ads and search advertising is allowing the advertiser to capture and use search intent, whereas paid social is focused more on audience targeting and demographics targeting. 

So I think arguably Chat Gbt kind of got the core of the question and answer correct here. However, you can also see that there’s a fair amount of fluff in this answer. So the opening and closing, the explanation about ad formats, this is not very helpful. The point that it made about Facebook ads being text based or quick can be text based is also not a very helpful statement. So I think it’s interesting just to see the variation in quality of the information. Chat GBT gets the core of the answer correct, but then also includes a fair amount of unhelpful information. So a good case where Chat GPT has given the raw answer, but could certainly benefit from some editing. 

Question number three:

Prompt: “Why is having 100% bounce rate a good result for a Google Ads campaign?”

AI Answer: This is not necessarily a good result,  the rate refers to the percentage of visitors to a website who leave after  visiting a single page. A high rate can indicate that visitors are not having a good user experience.

A 100% rate means that all visitors left immediately, indicating that the campaign was not successful. A low bounce rate is more desirable.

Our Feedback: We heard the answer. So frankly, it’s a tricky question. We were kind of trying to trick Chat GPThere, and actually, Chat GPT did not get tricked. It understood that the premise of the question was misleading, and it responded by challenging the premise of the question, which I think is pretty great, it’s pretty smart. It understood that the question contains an incorrect assumption, and it disagreed with that incorrect assumption. So that’s the plus side with regards to the answer itself. 

What it does really is it answers the idea of why you might have a high bounce rate, or sort of explains what bounce rate is and why a high bounce rate is bad and how you might improve it. However, what it somewhat misses, arguably, is the idea that if you have a 100% bounce rate, this is a little bit different from just having a poor bounce rate. 100% bounce rate actually, in most cases would indicate some kind of measurement or instrumentation or trafficking issue with your campaign. 

You shouldn’t see 100%, even if it’s a really bad campaign or a really bad landing page. There’s at least probably one or two users that are not going to bounce, so you won’t see the 100%. Seeing 100% is like seeing a total flat line on conversions or sessions, usually to a human analyst that indicates something is wrong with the measurement, not necessarily that the performance of the campaign is poor. 

So that’s a bit of a nuance. But it does show, I think, the difference between human analysis and AI analysis, and why sort of a human analysis is still important in cases like that, where you have probably a measurement issue or a trafficking issue, more so than just a general understanding of what bounce rate is and what’s a good or bad bounce rate. But credit to chat GPT for not being tricked by the tricky question.