You are looking for a report type called 'Journal Entry'. That will have the old value/new value pairings for common fields like task name, as well as any custom fields your instance has elected to track when their custom forms were built.
If you aren't careful with your filters, it can return an absolutely overwhelming amount of data, so filter carefully to a day or two while you figure out which columns you need on the report. I'd suggest starting with the base columns and adding 'Old Text Value' and 'New Text Value' at minimum.
This filter should get you started also. You want the 'Task ID is not blank' part or you'll get all the records where the user changed the value of a Project name or Issue name too.