No. User's who have at least read access to a file have the ability to view and download it. You can monitor download activity from the file's Activity page or through Data Feeds. 

For more information, see:

• Activity page
• Data Feeds