Your charisma can also be as low as you want if you avoid as many charisma checks as possible.