So this is tricky. In order to in-line edit a custom field, the custom form has to be attached. In order to attach a custom form, you need manage access. Soooo, if you don't want to increase their access, you would have to pre-emptively go in and add all the custom forms that might need to be edited...