That's..the whole point of the move deleter. In RBY, HM moves were impossible to remove once you taught them to a Pokemon, so the move deleter was added in GSC in order to remedy that issue.
:p
It would've been much better if HMs could naturally be deleted (which isn't an unpopular opinion afaik), but that wouldn't have been possible in Gen I either without serious repercussions, since HMs could be deposited in the PC (due to the structure of the Gen I bag); if it were possible and someone deleted an HM move while the disk is in the PC, they could've ended up getting stuck and unable to progress or backtrack, forcing them to restart.
Edit: Also, see
here for the PS subforum.