Hi Kat,
Yes, you need to be in ArcMap with data added.
The Conflict Dialog box (it's called "Conflicts") is in the ArcMap Versioing toolbar. Once you have reconciled one version with its parent, if there's conflict, then you shall be able to see the conflicts window.
Regards,
Ben