LLMs, Workfront & Challenging Conversations

Question

Hi Workfront Community,Crowd-surfing for ideas on how to constructively guide some conversations we’ve been having internally at my org, hoping others dealing with the same challenge might have perspectives to share. With the push to adopt LLM tools, we’re getting requests from users that have exported Workfront data and used Copilot/Cursor to create reports/dashboards that they want built in Workfront.  The Problem(s):Suggested metrics/KPIs calculated based on completely fabricated/hallucinated data 	Suggested metrics/KPIs that sound good on paper, but are inedible word-salads  	Suggested metrics/KPIs that ARE reasonable, but rely on analytical capabilities and/or functionality that Workfront reporting tools simply don’t have. 	Suggested metrics/KPIs that rely on functionality Workfront DOES have, but we don’t have implemented, licensed or don’t use in the way necessary to create the calculations	Suggested visualizations that either won’t actually work, or again aren’t supported by anything Workfront has natively implemented The crux of the problem is really that the LLMs are fabricating such pretty, executive-friendly, brilliant-sounding suggestions that they’re getting shown/approved by senior leadership before anyone asks if they can actually be implemented.Then I'm the bad guy explaining that just because Copilot claims we can calculate the air-speed velocity of a laden Dober-doodle by subtracting the Baseline Issue ‘s Commit Date from the Custom Form’s Rate Card doesn’t mean I’m going to even try it. I’m trying to figure out constructive ways to have this conversation, both with the requestors who may not have the Workfront experience to understand WHY Copilot’s gibberish doesn’t work, and with our leadership to help guide their expectations as well. Anyone else working through this sort of discussion and have helpful ideas?

Lyndsy-Denk · Answer

What a doozy, but I feel like this is coming for all of us. I think we all battle the occasional request for this type of stuff, but now gen AI is, as you said, hallucinating the possibility, which makes people feel more hopeful that it can happen.

I think you have an answer for each of these scenarios. The hard part is convincing people who aren’t admins to understand that just because Copilot said it, doesn’t mean it’s real. Our grandparnets might say, “If Copilot told you to jump off a bridge, would you do it?”

As an extension to #3 and #5: Workfront does reporting, not analytics. And if they want analytics, then we need to attach an analytics tool to Workfront (preferably with an analyst involved because most admins aren’t also analysts). In fact, you might consult with an analyst in your company to help you craft a better response. I suspect those people are constantly pushing back.

For #1, #2, and #4: Things I’ve said before:

“If you can give me the data, I can see what we can get out of Workfront.”
“We don’t currently collect that data because each person works differently. If you want the data, we will need to adjust ways of working.”
“Without getting too nerdy, that comparison of data means jumping too far across the data. It’s apples and oranges.

To be honest, most of my stakeholders walk away thinking I’m just being stubborn. It wasn’t until we had a consultant come into analyzer our system and ways of working and he said what I said, did they believe me.

Solidarity!

Sign up

Login with SSO

Login to the community

Login with SSO

Scanning file for viruses.

This file cannot be downloaded