There’s a library cleanup routine which will delete any media items not linked to a layout. So simply delete the layouts you no longer require, and then run the cleanup.
If it’s not enabled, you can enable it in the CMS settings. You’ll then see the button at the top right of the Library page.
I’d recommend taking a backup (database and CMS library) prior to running it so you can roll back if it does something you weren’t expecting.