Hi
Please have a look at this forum post discussing similar use case.
Link 1:-http://help-forums.adobe.com/content/adobeforums/en/experience-manager-forum/adobe-experience-manage...
// How to decline image upload when the size of the image is more than the expected?
Write a custom Sling Servlet that uploads DAM Assets. In the Sling Servlet - use the Asset Manager API. You have full control and if the asset is larger than you want, you can prevent it from going into the DAM. See this article on how to use Asset Manager API:
https://helpx.adobe.com/experience-manager/using/uploading-files-aem1.html
Read all the comments.
Link 2:- http://help-forums.adobe.com/content/adobeforums/en/experience-manager-forum/adobe-experience-manage...
// Restrict file uploads by size & type in DAM upload dialog
I hope this will help you.
~kautuk
Kautuk Sahni