Magento product and category images are stored in the following directory.
Media > Catalog > Products for product images.
media > Catalog > Category for category images.
Media directory is the storage of the Magento media files – images out of the box, generated thumbnails, uploaded products images. It is also used as a container for importing images through the mass import/export tools.
As far as we can see when deleting an image from Magento Admin, the physical file do not delete. You may have to manually delete from the FTP.