If I were to upload someone's email to Salesforce which is tied to Marketo and that person were to visit our site, would I be able to track them even if they have never filled out a form, opened an email, etc.? Or would their activity be anonymous?

It is anonymous. 
A lead needs to do one of the following to be tracked:

• Click a link in a Marketo email to a page with the munchkin code
• Fill out a marketo form
• Use Marketo's SOAP or Munchkin API (for developers) to associate an anonymous lead with a known record.

If someone is a record in our database and they have not done any of the above actions, we cannot track their online behaviour.
 

In the interest of completeness: they don't have to fill out a form per se: if you pass the right information in the URL, you can create a form and send it on their behalf without them ever seeing it (like so).  Though that does still count as a form post.

But the point remains that the page must have some way of identifying the user. If you can get them to click a link that has ?email_address=dustinb@example.com in the URL -- or you could lightly encode the email address and then decode it on the page if you wanted -- then you can make them "known."

Or you can pass the SFDC ID and then use a combination of the SFDC and Marketo APIs to make them known.

Otherwise I hope it makes sense that they must be anonymous, just like an organic visit to your site from anywhere.

[OK, again just for full completeness, if you happened to have some other web tracking system that had already tracked them somehow, and you could load that code via your website as well, you could extract the information and then pass it to Marketo.]