I think that without workers the player plays more, it gives a twist to the classic idle formula, it makes the game more unique, I think you should keep it like that but there may be people who prefer playing an idle game that's more idle