Hello Evan / Greg -
I would remove the line specifying the attribute field names from the CSV file you are writing into the folder the 'Watch a folder for new .csv files' inbound connector is watching ... better to include only the actual event data in the file and leave out the attribute field names. |
In a reply to Greg, Evan cross-references an older thread, watching a folder with csv files, in which I state: "the first two lines of the CSV file are reserved for a comma separated list of field name and a comma separated list of field data types - your actual event data needs to start with the 3rd line of the text file." But then in this thread I suggest the opposite, to remove specification of attribute field names from the CSV file.
So which is it, you're asking. Should a CSV file include a two-line header with field names and types, or shouldn't it?
My apologies for the confusion. In the older thread I must have been thinking of when a CSV file is used as the source of event enrichment, not when the CSV file is used as the source of the event data.
- An enrichment file would need the header rows to specify field names and field types.
- You would not want a CSV input file to include anything more than the (comma) delimited data values.
Evan is correct, above, when he suggests that the OS level user responsible for running the GeoEvent Windows service is the account which needs permission to access a registered folder. I don't remember if it was at 10.2.2 or a later release that we added visual indication of the apparent permissions on a registered folder to the Site > GeoEvent > Data Stores page in Manager. That particular enhancement might help clear up potential issues with whether or not an input connector will be able to retrieve data from a registered folder.
Also, Greg points out that when registering the system folder, the user-interface's default is to place a value 'input' in the 'Input Directory' parameter. I always remove this value and leave the 'Input Directory' parameter unspecified. The default would only work if you wanted your 'Watch a Folder for New CSV Files' input to look for a sub-directory named input beneath a registered system path C:\GeoEvent\csv (for example).
When working with file-based input and output (and enrichment) I usually register separate folders:
- GeoEvent_Input -- C:\GeoEvent\input
- GeoEvent_Output -- C:\GeoEvent\output
- GeoEvent_Enrich -- C:\GeoEvent\enrichment
I don't configure my 'Watch a Folder for New CSV Files' to expect an 'input' sub-directory, and I don't allow it to 'Include Subfolders'.
Finally, please keep in mind that the GeoEvent Extension generally supports RESTful data streams. The inputs capable of watching a system folder for files have some significant limitations and are generally intended to prove that real-time analytics you have designed in a GeoEvent Service behave as you intend. Moving toward production we expect real-time data feeds to arrive via HTTP/POST or as replies to queries you make on an external server's URL.
Best Regards -
RJ