Hi Scott,
This is not currently possible via the Survey123 website settings for a survey or via the form settings available in Connect. Currently only the time frame that the survey is open and closed for can be set via the Survey123 website settings: Share survey—Survey123 for ArcGIS | Documentation.
If this feature is important for your workflows and projects, I would suggest you raise it as an idea on the Survey123 ArcGIS Ideas page, and I would encourage you to submit an enhancement request via Esri Support. Our Support team will assign an official enhancement number for your records. Similar requests from other customers can then be attached to the same enhancement request, which helps us assess demand for the enhancement and prioritize it accordingly.
Regards,
Phil.