I think you're talking about the "Activity was Logged" filter which is the SFDC filter and determines if there are any tasks or events at the activity level for a person or not. "Activity was Logged" filter only sees the "Tasks" and "Events" that exist in Salesforce. Anything else that shows up in the SFDC Lead/Contact's activity history is not seen by that filter unless it is also a task or event in Salesforce.
I don't think you should use that filter to determine if the person is engaged/active or not. You should rather be using lead activity filters - e.g., "Filled Out Form", "Visited Webpage", etc. in your SL to filter out people who're engaging with your marketing assets. Following is the SL configuration that we used in the past, FYR -
Basically, you'd want to include the filters for identifying any person activities logged as a result of their action v.s. the activity that was logged as a result of your marketing efforts, you'd not want to include Was Sent Email as that's not a good example of something that prospect engagement, it merely shows that you sent out an email. One more thing, since you'd be creating a SL that references the lead activity log, you should also be cognizant of Marketo's data retention policies for high volume and normal activities listed on this page here.
Please let us know if you have any questions.