Latest Contributions by ChrisM

I generated a "total viewshed" of a DEM using a series of random observer points. The resulting viewshed attribute table lists the values as the number of observer observer points visible from a count of pixels with that value. So, if you click on a cell, it will show you how many observer points can see the pixel. However, the attribute table, of course, summarizes this, so every row with a value of, say, 6 is listed with the number of pixels with that value (see 400). This is fine, but I see a weird issue I can't figure out how to fix, especially when I use a lot of observer points. I see that there are multiple rows with the same count but different numbers of observer points in the value column. In other words, rather than one row in which 500 pixels can be seen from 1 observer location, it will have a series of rows in which, say, 500 pixels can be seen by 1, 501 can be seen by 1, 503 seen by 1, and so on. I assume this is happening because given the large number of points distributed across the landscape these are separate individual cells that can be seen by separate groups of observer points. So the 500 pixels that can be seen by 1 are different pixels than the 501 that can be seen by 1. My confusion is trying to understand the organization of the data to use in other applications. For example, for larger count values I can easily say something like "20,000 pixels are viewable by x number of sites" since there is only one row. But if I want to say the same thing for 1 site, I have several options. I find this is a challenge also in talking about any cumulative frequencies ... View more

I posted this over in Geoprocessing the other day and thought I'd seek input here   I've been doing a lot of comparative research on creating viewsheds and comparing the data, not just in ArcGIS but other platforms and general theory. I keep running into some issues. Let me explain. I have 10 sites on hilltops that I want to assess intervisibility. The line of site analysis is not great in this case. So, I just calculated viewsheds for each site, converted the visible raster data to polygons and determined which sites fall into each other's viewsheds. Fairly simple, though not without some issues and simplifying assumptions. I can also, of course dependent on the size of the base DEM and other parameters, determine what the average viewshed area is for each site. However, I want to do a couple of additional things. I'd like to be able to compare the overall average viewshed or visibility of the sites with what the average visibility would be throughout the area, regardless of physical location. That is, I'd like to assess quantitatively if the hilltop locations have over all better visibility than other locations. First, I used the raster calculator and added all the site viewsheds together to create their total viewshed. This obviously wasn't necessary since both the viewshed tool and the visibility tool produce the same resulting raster data product. Nevertheless, that's what I did. To get the comparative dataset, I generated 1000 random points in the study area, focusing a tad more in the middle to reduce edge effects. In this case, I can't take the time to generate individual viewsheds for each point, so I just did the visibility analysis. As I said, I could have done that with the hilltop sites, but then I would not have been able to really assess what each hilltop sites individual viewshed areas were. Anyway, doing this produces another raster. You can clearly see that the hilltop locations are very visible in the random set, but that they have an over all lower visibility. However, making meaningful comparisons is hard. I looked at the source statistics for both rasters, and you can see that the mean for the hilltop total viewshed raster is much higher (assuming that the source statistics are recording visible pixels, which I guess they would have). But how do I compare these statistically? That's my over all question, but here are some embedded questions that keep popping up: 1) To compare these two rasters, would I need to normalize them to put them in the same scale? In this case, would I need to divide the rasters in the raster calculator by the number of sites used (10 and 1000) (or would I normalize by the number of pixels (I doubt this since they are the same in both cases). 2) Although the viewsheds show how many many pixels (Count) are visible by how many sites (value) it does not show which sites are visible and it does not show what the area of viewshed is for each site. Now, I can do this with more steps with the 10 hilltop sites, as I discussed above. Conversely I can use the Observer Points tool (or the observer function in visibility) to produce a table that lists the observer points that are visible and the number of pixels visible. I could sum up all the times each site is listed as 1 and add up the counts for them to get the differences in their viewsheds in terms of pixels. This really isn't necessary since, as I said, I can do that more easily by either recording the total visible pixels in their individual viewsheds or convert the visible portions of the individual viewsheds to polygons and calculate area. However, and here's an issue, I can't do that with the random points. Observer point tool has a 16 point limit and, as I said, calculating individual viewsheds for each 1000 points would take too long. I played around with extracting the underlying raster data and appending it to the point data files for each group, but I realized that that does not give me what I want at all, and I'm not really sure what that data is saying anyway. In otherwords, the rasters show visibility in terms of the location and number of pixels viewable by a particular point and give an average visibility. They don't show what the sizes or average sizes of the individual viewsheds are for the points. So, the rasters help address one question (average regional visibility of all hilltop sites against average regional visibility in general as approximated by a series of 1000 points) but they don't address another question (the average size of the viewshed against the average size of the viewshed of the random sample). 3) If I am stuck just comparing the hilltop total viewshed against the random point viewshed, what kind of tool can I use to test for significant differences? I can't see much in arcmap. I thought about just generating new point files for each raster, with a point for every pixel and with the raster data as the point attributes. Then, at least, I could have a data file that I could more easily calculate means, etc. and conduct some tests. But, as I said, the source statistics for the rasters are there. But how can I compare them meaningfully? Anyway, I'm sorry that this is so long. I suspect a number of other issues and problems might arise or be obvious in my post. ... View more

I've been doing a lot of comparative research on creating viewsheds and comparing the data, not just in ArcGIS but other platforms and general theory. I keep running into some issues. Let me explain. I have 10 sites on hilltops that I want to assess intervisibility. The line of site analysis is not great in this case. So, I just calculated viewsheds for each site, converted the visible raster data to polygons and determined which sites fall into each other's viewsheds. Fairly simple, though not without some issues and simplifying assumptions. I can also, of course dependent on the size of the base DEM and other parameters, determine what the average viewshed area is for each site. However, I want to do a couple of additional things. I'd like to be able to compare the overall average viewshed or visibility of the sites with what the average visibility would be throughout the area, regardless of physical location. That is, I'd like to assess quantitatively if the hilltop locations have over all better visibility than other locations. First, I used the raster calculator and added all the site viewsheds together to create their total viewshed. This obviously wasn't necessary since both the viewshed tool and the visibility tool produce the same resulting raster data product. Nevertheless, that's what I did. To get the comparative dataset, I generated 1000 random points in the study area, focusing a tad more in the middle to reduce edge effects. In this case, I can't take the time to generate individual viewsheds for each point, so I just did the visibility analysis. As I said, I could have done that with the hilltop sites, but then I would not have been able to really assess what each hilltop sites individual viewshed areas were. Anyway, doing this produces another raster. You can clearly see that the hilltop locations are very visible in the random set, but that they have an over all lower visibility. However, making meaningful comparisons is hard. I looked at the source statistics for both rasters, and you can see that the mean for the hilltop total viewshed raster is much higher (assuming that the source statistics are recording visible pixels, which I guess they would have). But how do I compare these statistically? That's my over all question, but here are some embedded questions that keep popping up: 1) To compare these two rasters, would I need to normalize them to put them in the same scale? In this case, would I need to divide the rasters in the raster calculator by the number of sites used (10 and 1000) (or would I normalize by the number of pixels (I doubt this since they are the same in both cases). 2) Although the viewsheds show how many many pixels (Count) are visible by how many sites (value) it does not show which sites are visible and it does not show what the area of viewshed is for each site. Now, I can do this with more steps with the 10 hilltop sites, as I discussed above. Conversely I can use the Observer Points tool (or the observer function in visibility) to produce a table that lists the observer points that are visible and the number of pixels visible. I could sum up all the times each site is listed as 1 and add up the counts for them to get the differences in their viewsheds in terms of pixels. This really isn't necessary since, as I said, I can do that more easily by either recording the total visible pixels in their individual viewsheds or convert the visible portions of the individual viewsheds to polygons and calculate area. However, and here's an issue, I can't do that with the random points. Observer point tool has a 16 point limit and, as I said, calculating individual viewsheds for each 1000 points would take too long. I played around with extracting the underlying raster data and appending it to the point data files for each group, but I realized that that does not give me what I want at all, and I'm not really sure what that data is saying anyway. In otherwords, the rasters show visibility in terms of the location and number of pixels viewable by a particular point and give an average visibility. They don't show what the sizes or average sizes of the individual viewsheds are for the points. So, the rasters help address one question (average regional visibility of all hilltop sites against average regional visibility in general as approximated by a series of 1000 points) but they don't address another question (the average size of the viewshed against the average size of the viewshed of the random sample). 3) If I am stuck just comparing the hilltop total viewshed against the random point viewshed, what kind of tool can I use to test for significant differences? I can't see much in arcmap. I thought about just generating new point files for each raster, with a point for every pixel and with the raster data as the point attributes. Then, at least, I could have a data file that I could more easily calculate means, etc. and conduct some tests. But, as I said, the source statistics for the rasters are there. But how can I compare them meaningfully? Anyway, I'm sorry that this is so long. I suspect a number of other issues and problems might arise or be obvious in my post. ... View more

I can't recall ever having trouble creating feature classes from xy tables. Usually, I have a table in excel and I save it as either csv or txt and follow the steps in ArcCatalog. I have a bunch of x,y,z data points I recently converted from an arbitrary grid to UTM. Looks fine in some programs like Global Mapper, but I'm having a hell of a time in ArcGIS/Catalog. When it works, only about 4 points show up (when there are almost 2000). Also, most of the time arccatalog is not reading the z value and giving me this option in the dropdown. Sometimes it does and then when I try again it doesnt. I checked the properties and the z values are set either as text or long integer, where as x and y as double. I can't seem to change the data types, though. I basically just want a shapefile with points with the x, y, and z data. Any suggestions? I've tried formatting cells in excel to numbers before saving as a csv or txt with no change. However, whenever I do try to open the file as an excel worksheet rather than the csv or txt I get an error saying something like, can't connect with data base, etc. Suggestions? I've never encountered this issue before. ... View more