If you want to remove the results of the installer (the drivers), do not just delete the EXE.