Well, I checked again once, and Registering as Versioned, should not make those options Grayed Out.
As I suggested earlier, try the below:
Is the data in that Feature Dataset consumed in any services? Maybe a Lock is preventing?
If you make sure that there are no connections to the Geodatabase and then try, does it help?