Create New Field w/ Field Calculator (Regular Expression) between :

4294
17
07-28-2016 10:52 AM
Mtclimber03
Occasional Contributor

Hey guys,

I'm not sure if using the field calculator (regular expression) is the function in GeoEvents I should be using to accomplish what I need so let me try and explain. I have geoevents recieving a JSON REST endpoint that contains data that looks like/similar to the below.

{

"record" : {

  "ID" : "18",

  "PARENT_RECORD_ID" : "0",

  "PARENT_PAGE_ID" : "0",

  "PARENT_ELEMENT_ID" : "0",

  "CREATED_DATE" : "2016-06-29 17:43:32",

  "CREATED_BY" : "username",

  "CREATED_LOCATION" : "36.863681:42.992592:557.165466:65.000000:18.446595:-1.000000:-1.000000:1467211407.825839",

  "CREATED_DEVICE_ID" : "917f86ddc676fa0ef39f910c6460838d4451f0b8",

  "MODIFIED_DATE" : "2016-06-29 17:43:34",

  "MODIFIED_BY" : "username",

  "MODIFIED_LOCATION" : "36.863681:42.992592:557.165466:65.000000:18.446595:-1.000000:-1.000000:1467211407.825839",

  "MODIFIED_DEVICE_ID" : "917f86ddc676fa0ef39f910c6460838d4451f0b8",

  "SERVER_MODIFIED_DATE" : "2016-06-29 17:43:38",

  "project_number" : "8675309j",

  "project_name" : "name of project",

  "project_nickname" : "nickname",

  "date_of_report" : "2016-06-29",

  "data_type" : "completed_activity",

  "admin_div_1" : "some_governorate",

  "admin_div_2" : "Any_Town",

  "admin_div_3" : "Any_Loc",

  "admin_div_4" : "Any_where",

  "a99999_benev_funding" : 204,

  "a99999_benev_funding_male" : 0,

  "a99999_benev_funding_female" : 204,

  "a91602" : 85,

  "a91602_male" : 28,

  "a91602_female" : 57,

  "a99999_children_counseling" : 16,

  "a99999_children_counseling_male" : 6,

  "a99999_children_counseling_female" : 10,

  "a99999_hhheads_seminars" : 0,

  "a99999_hhheads_seminars_male" : 0,

  "a99999_hhheads_seminars_female" : 0,

  "a99999_cohesion_activities" : 47,

  "a99999_cohesion_activities_male" : 18,

  "a99999_cohesion_activities_female" : 29,

  "a90107" : 31,

  "a90107_male" : 15,

  "a90107_female" : 16,

  "a90309" : 97,

  "a90309_male" : 39,

  "a90309_female" : 58,

  "a90308" : 12,

  "a90308_male" : 6,

  "a90308_female" : 6,

  "a99999_leaders_trained" : 0,

  "a99999_leaders_trained_male" : 0,

  "a99999_leaders_trained_female" : 0,

  "a99999_cfs" : 40,

  "a99999_cfs_male" : 20,

  "a99999_cfs_female" : 20,

  "a50200_health_services" : 366,

  "a50200_health_services_male" : 128,

  "a50200_health_services_female" : 238,

  "a50200_dental" : 0,

  "a50200_dental_male" : 0,

  "a50200_dental_female" : 0,

  "a50200_obgyn" : 76,

  "a50200_obgyn_male" : 0,

  "a50200_obgyn_female" : 76,

  "a99999_emergency" : 60,

  "a99999_emergency_male" : 30,

  "a99999_emergency_female" : 30,

  "data_submission_status" : "validated",

  "assign_to" : "username"

  },

From here I need to use the record.CREATED_LOCATION coordinates to generate two new fields for latitude and longitude that can be used for the geometry. The 1st two sections of the record.CREATED_LOCATION are the x:y. Can some one point me in the right direction for what needs to be done to achieve this? Or alternatively let me know if there is another place in the geoevents manager I should be setting this up? So far I have my JSON feed successfully passing records into a feature service, just not mapping the points.

0 Kudos
17 Replies
AlexanderBrown5
Occasional Contributor II

Hi Heather,

I am sorry to hear that you are having trouble.  Can you please paste some screenshots of:

1. GeoEvent definitions (input and output)

2. Services (including any processor details like your field mapper).

Feel free to include any other details.  Hopefully we can get you a resolution rather quickly!

~Alex

0 Kudos
AllWeatherHeather
Occasional Contributor

Hi Alex!

I have great news! It turns out I was missing the XML Object Name parameter on my input, now I'm receiving great GeoEvents. The next issue I'm encountering is using the regular expressions. This is the field I am getting for "Coords":

-67.9301166666667,47.1880166666667,149

Using your instructions, I've been able pull out the first number but I'm having trouble getting the "47.188..." because of the 149 on the end.

I'm on the hunt for regular expression resources. Any ideas?

Also, there is another, more complex field that I will need to parse next (at least I know I'll need speed), can this be done with the same methodology?

7/9/2017 3:49:48 PM<br />
Lat/Lng: 47.188017, -67.930117<br />
Speed: 0.0 kph<br />
Altitude: 149.0 m<br />
Heading: 0&deg;<br />

Thanks,

Heather

0 Kudos
AlexanderBrown5
Occasional Contributor II

Heather,

That is great news!  For the "Coords", assuming that is exactly how that field is being ingested:

Pattern: (?<=,)[^,]*(?=,)

You can definitely parse through the more complicated field that has speed, altitude, etc.  

Can you post a sample xml with the more complicated field included?  

~Alex

0 Kudos
AllWeatherHeather
Occasional Contributor

Alex,

Thanks for the pattern, it worked great! I still need to add the next step to create geometry and I'll work on that over the next couple of days. In the meantime, here is a sample of the xml for one placemark. I'm hoping to extract the speed and the time. We are in the Atlantic time zone so I'm guessing the time inside the "CDATA" tag is the best one for us to use.

Thanks!

<Placemark>
<name>CH Unit # 5162 Cab # 71</name>
<address>Main Street, New Brunswick, Canada</address>
<description>
<![CDATA[
20/07/2017 10:37:01<br /> Lat/Lng: 46.174067, -65.886867<br /> Speed: 0.0 kph<br /> Altitude: 11.0 m<br /> Heading: 314&deg;<br />
]]>
</description>
<TimeStamp>
<when>2017-07-20T09:37:01-04:00</when>
</TimeStamp>
<styleUrl>#asset18131</styleUrl>
<Point>
<coordinates>-65.8868666666667,46.1740666666667,11</coordinates>
</Point>
</Placemark>
0 Kudos
AllWeatherHeather
Occasional Contributor

Hi Alex,

I added the piece for calculating the geometry using your recommended syntax and I'm not getting any errors in the log that I can see. However, now that it is hooked up to the feature service update output tool only the attributes are getting updated and not the position. Is there another way I can test to see if any geometry is getting generated?


Thanks,

Heather

0 Kudos
AllWeatherHeather
Occasional Contributor

I found the answer to the geometry question. The missing piece is the spatialReference when calculating the geometry field. This was my final expression:

   '{' + ' "x":'+ Longitude + ',' + ' "y":'+ Latitude + ',' + ' "spatialReference":{"wkid":4326} }'

 

AlexanderBrown5
Occasional Contributor II

Heather,

Fantastic! I am very glad you figured it out.  In general, I like writing all my outputs to a TCP socket (see my suggestion above https://community.esri.com/external-link.jspa?url=https%3A%2F%2Fwww.arcgis.com%2Fhome%2Fitem.html%3F... ) is a good way to troubleshoot your outputs.

For parsing out your additional fields from this string (if matches exactly):

<![CDATA[20 / 07 / 2017 10: 37: 01 < br / > Lat / Lng: 46.174067,-65.886867 < br / > Speed: 0.6 kph < br / > Altitude: 11.0 m < br / > Heading: 314 & deg; < br / >]] >

  • Speed
    • Pattern:  (?<=Speed: )[^*]*(?= kph)

  • Altitude
    • Pattern: (?<=Altitude: )[^*]*(?= m)
  • Heading
    • Pattern: (?<=Heading: )[^*]*(?= &)

Here is my TCP socket output, you can see Speed (0.9), Altitude (11) and Heading (314) written at the end of my output GeoEvent Definition:

These regular expressions parse through that string okay.  You might need to investigate the data coming in to ensure the trailing end of those expressions are general enough to catch all specific permutations.  Either way, that should put you down the path of figuring out what exactly you need (if the string changes).

~Alex

AllWeatherHeather
Occasional Contributor

This worked great Alex, thanks for all your help!