Can we Really Guarantee the Quality of AI Generated Ideas?

TL;DR

Our idea generator uses “signals” or real-time world events, news stories or interesting creative announcements to feed a large language model, trained to think like creative strategists. Users can input their brand or campaign info and the tool will start to generate ideas for creative exploration. We wanted to know if these idea starters were worthwhile so we blind tested our AI generated ideas against human ideas created with the same information. In our survey of over 500 people, we found that people could not tell the difference between AI or human ideas in this format. They were also just as likely to prefer an AI idea as they were to prefer a human one.

Accelerating teams

Our focus at Rehab is to uncover ways in which AI can be deployed safely and ethically to enhance human effort, not replace it. So it might seem that an idea generator is dangerous territory. In actual fact, our system is designed to support the brainstorming and ideation part of the process and help creatives and strategists move from this initial stage to further development fast.

We use AI here as we know its strengths and weaknesses. Using carefully designed prompts and our own blend of data inputs we can get our model to behave like a strategist does and generate ideas. But we’re talking about a paragraph of text with an interesting area to explore, based on real-time information coming into the system. The AI cannot create a fully fledged campaign creative treatment taking into account all factors like budget, audience, market share, competitor propositions etc., yet. So this becomes a useful tool to fast track creative ideation and bring real-time information to bear fast, tasks the creative or strategists would have to do manually without this tool.

But if the tool is generating ideas, however embryonic, we still need to know that they are worth something. Because if they are suspect to creatives and they don’t resonate with audiences, using this tool is just more time wasted.

Testing the ideas

We took 10 ideas generated by our tool and 10 generated through a traditional strategic process. The project was real though all names were changed to protect the identities of those involved ;). It concerned creating opportunities for a sports brand to engage with audiences and reward fans. We ensured each idea was roughly the same length and provided respondents with two questions for each idea, in this order:

1. Which idea do you prefer?

2. Which idea do you think was generated by AI?

Our audience preferred AI ideas 54% of the time

Our audience failed to identify the AI idea 59% of the time

In simple terms, the general public was just as likely to prefer an AI idea to a human idea, choices were being made in terms of simple preference and the AI ideas held their ground.

Most importantly, identifying an AI idea was no better than guesswork for our audience.

Right Place, Right Time

A huge contributing factor in this test was Rehab’s overall approach to the tool in the first instance. Identifying where in the creative process to deploy this type of tool. If the audience had been faced with the creative outputs or a fully formulated idea built entirely by AI thinking, the results would have been very different. Just take a look at what happened when we asked AI to visualize some of the most Iconic Cannes Lions campaigns in history.

AI-generated ideas will not be winning Gold Lions or Grand Prix any time soon, but they can be used at the right point in the creative process to:

• Accelerate thinking, weaving in live information from relevant sectors and industries

• Eliminate bad territories much faster

• Create areas for exploration that could lead to an eventual idea

• Iterate quickly on a concept to provide more food for thought

• Do what would take a team of creatives and strategists a good few brainstorming sessions in seconds flat

Can you tell the difference?

Data is super important to understand validity in relatively untested areas of tech and creativity. We’ve taken these initial steps to ensure that when we talk to teams looking for AI-assisted idea generation, we know that what we are providing is worthwhile, but we always want more data!

We’d love to show you the types of ideas the system is generating and get your feedback on their quality and what types of ideas work best for you.

Hit us up, on social or drop us a line here to book in a demo of our platform and share your feedback.