Rishi Sunak is facing a third by-election after a close ally of Boris Johnson said he is stepping down from the Commons with "immediate effect".