You don't need to remove the broken piece of dipstick. If it's stuck in the tube, use something long and thin to push it the rest of the way in; it'll fall harmlessly to the bottom of the oil pan and stay there. Usually they don't get stuck in the tube, they fall into the pan on their own.
Then you bend the top of the tube inward towards the block a little (so the new dipstick won't get smacked by the crank, which is what breaks them) and replace the stick.
_________________
一期一会
Too many people who were born on third base actually believe they've hit a triple.