Every time this drawback is taken, 20 more Endbringers will be made and 3 more will become active. So taking this all 4 times would leave you with 85 inactive Endbringers and 15 currently active ones, for a total of 100 to contend with.
This also speeds the rate at which Endbringers attack by an amount commensurate to the number you add.