Hi Jessie,
Do you have this application shared publicly? I just set up a filter with the following settings:
When I use the Filter widget it does appear to sort those values alphabetically:
Do these settings match with what you have configured?
Thanks,
Katie