I once placed some old cylinder seals in a container of silicon fluid; a month later, they appeared identical, which is not surprising given that silicon is a relatively inert substance.
I believe that the reason why people report swollen old seals after filling with silicon is because their systems still contain glycol brake fluid (bleed nipples tend to be higher up, and it is impossible to remove all fluid by simply bleeding). Silicon fluid is less dense than both glycol and water, (and there will be some residue of both in the system). Silicon fluid will repel any moisture, so a glycol mixture with a high water content will be concentrated at the bottom of the cylinders, causing the damage. In a glycol system, water is absorbed and dispersed throughout the system, resulting in a considerably less concentrated solution.