Expand my Community achievements bar.

Latest Community Ideas Review is Out: Discover What’s New and What to Expect!

System Admins - Significant Custom Form Problem

Avatar

Level 10

System Administrators, I'm not sure if any of you have experienced this behavior, but I'll just leave the text for my ticket here in case you have. It is a major concern because historically, custom form modifications were instantly applied to the system. It doesn't appear to be happening that way anymore.

--------------

During a routine change to a calculated field on a project custom form, I updated the calculation as a system administrator and tested the change on a project and it was a success. However, a different user with manage rights refreshed the project, and the calculated field failed. When the other user hovered over the ? tooltip, we noticed that the calculated field did not reflect the update that had been made as a system administrator. The user logged out, cleared cache/cookies, but was still not seeing the change 30 minutes later. I logged in on behalf of the user and am still seeing the old calculation, clearly indicating it isn't replicating for all users.

How is this possible? Why aren't updates to custom fields replicating for all users as they always had before? This seems to be related to load balancing for performance, but is unacceptable as it creates a significant risk of errors in our systems and data if we don't know when changes are available to the end users. This is not the first time I've witnessed this in the past month, but I fear it is creating many data/workflow problems that we have yet to uncover.

Topics

Topics help categorize Community content and increase your ability to discover relevant content.

3 Replies

Avatar

Level 10

FWIW: I've seen similar behavior but assumed it was my impatience.

In my case, I added a calculated field to a user account, told it to update the calculation. Found a couple days later it missed a few people, applied the recalc again, and all was well.

WF has always had a similar issue with Recalculate Finance where individual users have to kick a project to see changes, or sometimes a SysAdmin has to do it because it won't stick if a user does it.

Not sure why sometimes a system-wide change of a calculation (even system-level or behind-the-scenes) requires a "second hit" to make it stick.

I've never chased these with Support since they are transient and impossible to prove once resolved. (We did chase the Recalc Finance one for a bit but…never got a good answer…).

Avatar

Level 9

I've not seen precisely the issue as described, but we did chase down a particular bizarre behavior with Custom Forms that overlaps pretty significantly. In my instance, the initial issue was reported to me as people having submitted a particular Custom Form without filling in a required field. When I reviewed the data on the Custom Form, the reports were seemingly correct, the 'required field' functionality hadn't worked properly (or so it seemed at first). In speaking with the people completing the forms, they were absolutely insistent that they had completed that particular field, and the issue was wide-spread enough to not simply be one person working too quickly. We did isolate the issue to one specific field on the form though, and that proved the issue.

Through a combination of screenshots, logging in as myself and logging 'in as' the affected users, we realized that while they all seemed to be seeing the same form - they weren't seeing the same VERSION of the form. The text label displaying for the affected field had been updated to clarify it, a seemingly innocuous change. In version A of the form, the field was called "Date and Time", and in version B of the form, the field was called "Date and Time - CST".

If the end-user's browser had a cached copy of version A, then they completed the field 'Date and Time', passed the required field validation, and were able to successfully save the form. However, once the data reached the database, no field called 'Date and Time' existed - and the information input vanished. Thus the user viewing Version B perceived that the required data hadn't been properly completed, even though the submitted knew they had completed it.

I suspect that your custom form modifications ARE being instantly applied - the end user's browser simply isn't refreshing the version of the form they're being served. Or in Kevin's example, the timeline IS being recalculated as expected - it's just not being served to the user correctly. I don't know anywhere near enough about web-dev to know if this is a problem Workfront even CAN solve, or that's a function of browser caching behavior that's outside their locus of control.

For our environment, if we make any key changes like that, we send out a note with instructions on refreshing their browser cache so they see the updated information. This is one that's nearly impossible for an admin to test, since our browsers will "see" the updated form even if an end-user doesn't. (Seriously, I thought I was losing my mind, the screenshots were the only thing that let me untangle it.) In a larger environment, I might have my IT team set-up a remote machine for me to use for this kind of testing so I could truly be in a different environment.

Having written all of that, I re-read the initial writeup and I see you're noting that when you log-in as the user you're seeing the form differently than when you're in your own account, which is slightly different than my scenario. At that point, I'd have to look at how complex that formula is, and how many instances of it exist in your implementation. I'd expect a complex calculation that exists thousands of times in an implementation to take longer to recalculate across all the projects than a simple one that exists on only a few projects. If I had to test it, I'd run a report showing the result of that calculation across all projects and save it offline. Then make my change, and run that report again. Can you identify a pattern of projects changed/not changed etc? Do they all appear updated in the report, but not when you log in as those users etc?

Not sure what I'd expect to find, but that would be my initial troubleshooting in your shoes.

Katherine

Avatar

Level 10
Hi Katherine, Thank you for the thorough response and a summary of your use case. In a response to my ticket, I expressed my concerns over data loss and you’ve just validated those concerns. It isn’t possible that this is cached in the browser because, as you noticed, I was logged in on behalf of the user on my device in another browser (IE) that had its cache/cookies cleared and that hadn’t even accessed that form. The version of the form that was being served up to my admin user was not the same form being served up to the other user. I also had the other user on a screen share and she wasn’t seeing the updated version of the form/field, regardless of logout/login, browser cache/cookie clearing. The only conclusion I can draw is load balancing has been activated for performance reasons and isn’t performing instantaneously as it should. While load balancing is typically safe for web servers, it’s risky business with databases and for this reason. I once configured a six-node web cluster (3 apache web servers, 3 mysql database servers) so I’ve been down this road. I’ve raised the “data loss” alarm n my ticket and to our CSC so I’m looking forward to hearing what the findings are. Having to schedule production changes to Workfront forms will reduce our agility significantly if we must wait for changes to propagate for all users. Narayan