I really like the idea! Having to pay attention to both minigames.
The only thing I would change is how abruptly the game ends when you bump into the sword.
I would add a health bar to the hand, like the skeleton has for failed questions. Maybe lose a finger each time you touch the sword.
The art is superb! I love the characters!