Magento product and category images are stored in the following directory.
Media > Catalog > Products for product images.
media > Catalog > Category for category images.
Media directory is the storage of the Magento media files – images out of the box, generated thumbnails, uploaded products images. It is also used as a container for importing images through the mass import/export tools.
As far as we can see when deleting an image from Magento Admin, the physical file do not delete. You may have to manually delete from the FTP.
To read more about eCommerce Websites and Shopping Cart Solutions click here.