Copilot Lied to Me? Real Talk on AI in Power Platform

Intro

Copilot lied to me. Is ChatGPT any better?

One of the key tools used by successful developers has long been a good search engine and the ability to assemble a quality search phrase with the right keywords. Sometimes equally as useful is the ability to scan the search results and zero in on the highest quality results to get our answer.

With AI taking over everything, this method is shifting from the ability to assemble good search phrases to the ability to engineer the best AI prompt.

The Setup

I've been doing lots of Power Platform design and development lately. I had some pretty specific questions come up. I did not have a definitive answer for these, so I asked Copilot. Some of the answers I got wasted my time. Read on if you don't want to waste yours.

My First Scenario

I had a question about Power Platform licensing that I wanted to sort out. The scenario: my end users are accessing a Model Driven App with a per app license. One of those users reported an error when trying to run a flow that was showing up in the Flows menu. The question is: can those users run a flow that is triggered from the flows menu in the MDA?

I started with the prompt below and fed it to Copilot for the answer.

• Does a power apps per app license allow you to run a flow from the flows menu in the command bar for selected items in a view?

This ended up being a dead end. Copilot's response was somewhat confusing but on the surface it says yes. This answer also matches a general understanding that if a flow functions in support of an app, the users of the app can run the flow under their Power Apps license.

I still didn't know why my users could not run the flow so I asked the same question to ChatGPT and Google, where I finally got the right answer... from Google.

The answer to the question...

In a nutshell, the Flows menu is intended to allow users to create their own flows and "attach" them to a specific table / recordset in the Model Driven App by using the "For a selected item" Dataverse trigger in Power Automate. What this really means is each user is connecting to Dataverse with a flow that's actually "independent" of the Model Driven App, thereby requiring them to have a Power Automate license.

Then, I had a Second Question Arise from the Same Project...

Here's the setup... We had a reporting need to apply criteria that would retrieve records from one table based on when each of those records was related to another table in Dataverse. This relationship was managed with a many-to-many relationship or hidden intersect table, for those that know. Since these tables are kind of  hidden, it's not that easy to get at with a System View. So the question was, can we use the standard CreatedOn or ModifiedOn columns, that exist on every Dataverse table, as a way to select the rows we want for our report?

Here are the prompts that I gave to Copilot...

• Do intersect tables in Dataverse have a CreatedOn column?
• Do many-to-many relationships in Dataverse have a CreatedOn value?

Wow, that seemed super promising. Now the only thing left to do was create the FetchXML query to pull this together. But, I still couldn't find what I was looking for. So I dug even deeper and figured out that I could use the Dynamics oData API to pull out the records from the intersect table and actually look at them. Low and behold there were no audit columns 🙁

I wonder what ChatGPT would have told me? Pretty definitive. It said the same thing.

So then I just asked Google and got what I needed.

Clearly the responses are different, and at best a bit ambiguous on the Copilot side. Could just be luck of the draw, but I spent an hour or two going on an assumption that my intersect table was auditable as a standalone table before I found out otherwise. 🙁

The Moral of the Story

You can't trust an AI Chatbot to save time all the time. Always double-check the response with other sources. Take as much pride in creating good AI prompts as you do in crafting your search terms. Good old fashioned investigative ability and problem solving skills are still winning.

Conclusion

Copilot is virtually everywhere now and is meant to be helpful. However, it's always a good idea to cross-reference your questions / research with other sources, search engines, or AI platforms.

Friendly reminder... If you're thinking Copilot will take your job, or even do your job for you... it's still not a replacement for an experienced human with domain knowledge and relevant skills