Controlling shareholder Sumner Redstone removed five of Viacom's directors, from the company's board, according to sources familiar with the situation.