Stanley Fischer said Friday it was too soon to tell whether Britain's vote to leave the European Union had changed the U.S. economic outlook.