I've spent a good amount of time searching for a documented limit and could not find one.
Through testing it seems like the cap is around 30,000 vertices for a single feature.
I've also seen complex polygon featureclasses when you have more than 10,000 features have rendering issues. (I didnt hunt it down, but maybe it was a combination of many features and many vertices together as a problem).
Well we cant "fix" this issue (the KML is correct per the spec), we added a warning in ArcGIS 10.1 when creating KML, that if you have more than 30,000 vertices, not all KML may draw correctly. This way you will have a heads up and a couple of ideas on modifying your data.
However please note, ArcGIS Explorer seems to have no problem drawing 30,000+ vertices features.