The xml file was there. The extension for the shapefile metadata is .shp.xml. The Windows Explorer took the first extension part and ignored the second.
I left the Filter field blank and ran the task. It imported everything in the folder, including prj, dbf and etc.
I then changed the Filter to **.shp.xml and ran the task. This time it worked correctly and imported only the metadata file. So the Filter for importing shapefiles should be **.shp.xml
Thanks a lot for your help!