Handling large number of custom data - best practice

[LukaszMatyszewski]

Hi All,

Our current project form manages approximately 200 custom fields of various types, many of which are supported by logic. Over time, we've realized that we need to limit the complexity to maintain smooth performance. I'm curious about how others handle larger and more complex sets of custom data, as we aim to expand our solution. My initial idea is to divide the main form into several forms based on data use cases. Is this the right approach? What are the pros and cons of doing this? Does anyone have other suggestions?

[skyehansen]

I think some of the unspoken problems with maintaining a larger data set like this, is that it's difficult to tell if a particular field is being used or not. When I look back over my history, here's what I think my instances have struggled with.

1) Display logic errors: if your display logic contains a mistake, and your users aren't experienced enough to spot it, a whole set of fields might not be getting filled in.

2) Field options retirement: if your display logic is based on someone choosing a particular option, and this option gets retired, a whole set of fields might not be getting filled in.

3) With over 200 custom fields to fill in, unless they're all required fields, it's difficult to tell if any one field isn't being used.

All of this results in a form being "cluttered" (fields everyone is avoiding filling in, leading to reports which are largely empty, and more fields every day being added into the mess).

While determining whether you want a unified form or separated forms is going to be helpful, I'd first make sure you're practicing good data hygiene and governance. There's an opportunity to utilize Fusion scenarios to help you keep count, but if this isn't possible, I'd recommend you reach out to Doug den Hoed who has a similar hygiene utility.

[Doug_Den_Hoed__AtAppStore]

Thanks for the tag, @skyehansen. 

@LukaszMatyszewski (and @RachelP), what Skye alluded to was our Health Checker solution, which lets you lets you systematically analyze your Workfront instance using key Business Rules to identify, assess, and then deactivate or purge data clutter. Happy to chat further via doug.denhoed@atappstore.com.

Regards,

Doug

[RachelP]

Same here!  Almost the same situation too, except we only have one or two custom forms currently in production.  But I'm working on a new one, that is going to be a lot more detailed, a lot longer, and some of it will need logic.  I have been wondering, would it be best to make several short custom forms instead of one longer main form? perhaps breaking up the main form by sections, or whatever makes sense, but would creating a series of shorter forms be smarter than creating this main custom form? 

If i end up with one form that's long, or complicated, with a variety of field types, calculated fields, logic, etc. I'm worried about screwing up everyone's projects later by making any changes... 

Also, if we want some fields to auto-populate based on information we already have in workfront.  For example, we'll need a field I think called "Project Title" but we want it to be populated by our workfront project name... things like that.  we may even want to pull information from an external source, would this even be possible?  And would even be a good idea at all?  My initial response is that this is a Very Bad Idea (but it's a request from a higher-up) considering an external data source could become unavailable or change structure or whatever in the future, how much of a disaster might that be?

[RandyRoberts · Accepted solution]

I prefer to have a single "Project" form with lots of display logic that will show or hide items depending on some choices "what type of project is it?". For me, this makes more sense than having people have to remember to add several different forms. Anytime a human has to make a choice, it's inviting a mistake.

Having said all that, a complicated form is not without it's drawbacks, maintenance being the preeminent one.

I have a few rules regarding form creation:

1. Don't forget that every custom field is an attribute added to the database. The more fields you have, the more complex and unwieldy your database is.

2. Scrutinise every field request.

3. Avoid multi-part questions; "yes, no, other. Then a second field for the "other".

4. Field names:

  • Don't duplicate built-in field names.

  • Field names should be as short as possible.

  • If it's a multi-use field, name it vaguely like "Total Amount". If it's specific use, name it concisely like "Total Days Off".

5. If it's multi-selection, use checkboxes. If single choice only, use Radio Buttons.

6. Use a "Deprecated Fields" section that is hidden from all but admins. This way you can hide deprecated fields without deleting them from historical objects.

7. Always check to see if you already have a field that can be used instead of creating a new one.

That's all I can think of off the top of my head.

As far as external sources go, that's sometimes a maintenance drain. You have to keep up with all the changes and updates on the source database. For some sources it's easy but for some, like Facebook, they change everything all the time with no notice and it constantly breaks your apps and field mapping. I think you have to consider the source when deciding.

[amybillmayer]

Hi Randy, your response in this string is super helpful. I'm wondering if you could expand a bit on #3 - Avoid multi-part questions; "yes, no, other. Then a second field for the "other".

We have a TON of these "other" fields in our intake form.

1. What is your reasoning behind not using them?

2. What would be your recommendation to avoid them?

Our typical scenario is offering selections from a dropdown with "Other" as the last selection - with, of course, a second field to fill in.

Thank you!!

[RandyRoberts]

Hi, I'm glad my post helped, it was a culmination of many years of building forms. Regarding the "other" field, if you have a field with bunch of choices and an "other" choice that triggers another field you now have double the fields you probably need. If you have 10 multiple choice fields and each one has an "other" option, you now have 20 fields! I would usually include a single field per form called "Other Instructions" or something like that where the user could list all the info from ALL the "other" fields in one place. Then just replace the "Other" text in the dropdown to "Listed in Other Instructions Field". Then instead of having 20 fields, you only have 11.

Another option would be to just make it an open text field right from the start.

Anytime you report on one field that triggers an "other" field, you have to filter and report on a two-field combo, and since there is no field validation in Workfront, you can't control what's in the "other" field making it a serious challenge to filter and report on.

The request that always makes me shake my head is when someone asks for a field called something like "Direct Reports" with a yes or no checkbox which triggers a "List Direct Reports" field if "yes". When all you really need is a text field called "Direct Reports" where you list your reports. If it's empty, you know they don't have any negating the need to the previous field.