adding massive amount of points features to map

[Krystianp]

hello,

I've got problem, I'm creating map based application, and now I have to add to map about 100 000 graphic points, from coordinates. I was trying in many ways to do it, but always process of adding points takes about 7-10 secounds, or my web browser just hang on.

is it even possible to add this amount of data to map without having problems with performance?
thanks for any advices!:)

[nicogis]

you could think something how cluster http://help.arcgis.com/en/webapi/javascript/arcgis/help/jssamples/layers_point_clustering.html
but depends from your workflow. Can you give further details how you add your points?

[Krystianp]

now I'm testing many options to add points. I get points coordinates from an array, then to avoid hang up a web browser I'm spliting data for parts (1 part = 1000 points) and then tryin to add via API function. My code looks like:

points is an array with coordinates in format [[x1,y1],[x2,y2].....[x100000,y100000]];

var layerSelectedBoreholes = new esri.layers.GraphicsLayer({displayOnPan: false});

var length = points.length;
var tmparray = [];
var i = 0;
 while (i < length){
  tmparray.push(points);
  i++;
  if (i % 1000 == 0){
   pointsPart = getMultiPointFeature(tmparray);
   var gra = new esri.Graphic(pointsPart);
   layerSelectedBoreholes.add(gra);
   console.log(i);
   tmparray = [];
  }
 }
function getMultiPointFeature(points){
 var multiPoint = {"geometry":{"points":points,"spatialReference":2180},
      "symbol":{
         "style":"STYLE_CIRCLE",
         "size":3,
         "color":[0,0,0,255],
         "type":"esriSMS",
         "outline":{
            "style":"STYLE_SOLID",
            "color":[0,0,0,255],
            "width":0,
            "type":"esriSLS"
           }        
      }
 
      };
 return multiPoint;
}

thanks for advice with Point Clustering I have to check it.

[derekswingley1 · Accepted solution]

I've got problem, I'm creating map based application, and now I have to add to map about 100 000 graphic points, from coordinates. I was trying in many ways to do it, but always process of adding points takes about 7-10 secounds, or my web browser just hang on.

is it even possible to add this amount of data to map without having problems with performance?
thanks for any advices!:)

This is a bad idea. Even if performance weren't an issue, interacting with 100k is a poor user experience. Clustering is one option, but will likely still be slow. A heat map and/or scale dependency (only show your points when the map is at a large scale where only a few hundred, or maybe thousand points would be displayed) would probably be a better way to display your data. We also have a help topic on displaying lots of point data:  http://help.arcgis.com/en/webapi/javascript/arcgis/help/jshelp/limits_for_graphics.htm

[DavidWillis]

I have a similar problem, in that I have just over 5000 points to plot on a map of the world - I am clustering them, though (using the code from the point clustering example in the JavaScript API).  My issue is that it works just fine, up until 4600 points - any more, and the map simply doesn't draw.  Any ideas as to why that might be?

[StephenLead]

it works just fine, up until 4600 points - any more, and the map simply doesn't draw.  Any ideas as to why that might be?

Can you post a link to your site, or the code?

How are you determining that the problem occurs after exactly 4,600 points? What happens if you use a definition query on the layer to request fewer points? What happens when the map fails - do you see an error message, blank screen, etc?

Cheers,
Steve

[DavidWillis]

Steve,
  Here's my javascript listing (it's not on the production site yet):

dojo.require("dijit.layout.BorderContainer");
            dojo.require("dijit.layout.ContentPane");
            dojo.require("esri.map");
            dojo.require("esri.dijit.Popup");
            dojo.require("extras.ClusterLayer");
           
      
            var map, clusterlayer;

            function init() {
                var popupOptions = {
                    "markerSymbol": new esri.symbol.SimpleMarkerSymbol("circle", 20, null, new dojo.Color([0, 0, 0, 0.25])),
                    "marginLeft": "20",
                    "marginTop": "20"
                };
                var popup = new esri.dijit.Popup(popupOptions, dojo.create("div"));
                map = new esri.Map("map", {infoWindow: popup});
                var basemap = new esri.layers.ArcGISTiledMapServiceLayer("http://services.arcgisonline.com/ArcGIS/rest/services/Ocean_Basemap/MapServer");
                map.addLayer(basemap);
               
                dojo.connect(map, 'onLoad', function(theMap) {
                    //resize the map when the browser resizes
                    dojo.connect(dijit.byId('map', 'resize', map,map.resize));
                    dojo.style(dojo.query("a.action.zoomTo")[0], "display", "none");
                    var markers = <%=getmarkers%>;
                    addClusters(markers);
                   
                });
            }

   function readData(){
   
          var m = <%=getmarkers%>;
          //getmarkers is a json file with location and description data
         
         }
         
            function addClusters(resp) {
                var markerInfo = {};
                var wgs = new esri.SpatialReference({ "wkid": 4326 });
                //console.log("Add marker info");
                markerInfo.data = dojo.map(resp, function(p) {
                    var latlng = new esri.geometry.Point(parseFloat(p.lat), parseFloat(p.lng), wgs);
                    var webMercator = esri.geometry.geographicToWebMercator(latlng);
                    var attributes = {
                        "Victim": p.Victim,
                        "Description": p.Description,
                        "Date": p.Date
                    };
                return { "x": webMercator.x, "y": webMercator.y, "attributes": attributes };
                });

                // popupTemplate to work with attributes specific to this dataset
                var popupTemplate = esri.dijit.PopupTemplate({
                    "title": "", "fieldInfos": [
                        { "fieldName": "Victim", visible: true },
                        { "fieldName": "Description", visible: true },
                        { "fieldName": "Date", visible: true }
                    ]

                });

                // cluster layer that uses OpenLayers style clustering
                clusterLayer = new extras.ClusterLayer({
                    "data": markerInfo.data,
                    "distance": 100,
                    "id": "clusters",
                    "labelColor": "#fff",
                    "labelOffset": 10,
                    "resolution": map.extent.getWidth() / map.width,
                    "singleColor": "#888",
                    "singleTemplate": popupTemplate
                });
                var defaultSym = new esri.symbol.SimpleMarkerSymbol().setSize(4);
                var renderer = new esri.renderer.ClassBreaksRenderer(defaultSym, "clusterCount");
                var blue = new esri.symbol.PictureMarkerSymbol("http://static.arcgis.com/images/Symbols/Shapes/BluePin1LargeB.png", 32, 32).setOffset(0, 15);
                var green = new esri.symbol.PictureMarkerSymbol("http://static.arcgis.com/images/Symbols/Shapes/GreenPin1LargeB.png", 64, 64).setOffset(0, 15);
                var red = new esri.symbol.PictureMarkerSymbol("http://static.arcgis.com/images/Symbols/Shapes/RedPin1LargeB.png", 72, 72).setOffset(0, 15);
                renderer.addBreak(0, 2, blue);
                renderer.addBreak(2, 201, green);
                renderer.addBreak(200, 10001, red);

    clusterLayer.setRenderer(renderer);
                map.addLayer(clusterLayer);

                // close the info window when the map is clicked
                dojo.connect(map, "onClick", cleanUp);
                // close the info window when esc is pressed
                dojo.connect(map, "onKeyDown", function(e) {
                    if ( e.keyCode == 27 ) {
                        cleanUp();
                    }
                });
            }

            function cleanUp() {
                map.infoWindow.hide();
                clusterLayer.clearSingles();
            }

            function error(err) {
                console.log("something failed: ", err);
            }

            // show cluster extents
            function showExtents() {
                var extents = new esri.layers.GraphicsLayer();
                var sym = new esri.symbol.SimpleFillSymbol().setColor(new dojo.Color([205,193,197,0.5]));

                dojo.forEach(clusterLayer._clusters, function(c) {
                    var e = c.attributes.extent;
                    extents.add(new esri.Graphic(new esri.geometry.Extent(e[0], e[1], e[2], e[3]), sym));
                }, this);
                map.addLayer(extents, 0);
            }
           
   dojo.ready(init);

As for the 4600 cutoff, I'm using a query from our database to populate the data - using a time range I can control the number of points rendered.  The page works fine with 4583 points.  When the query returns 4634, I get a blank screen, so the cutoff is somewhere right around 4600 - just not sure why...

Thanks!

Dave

[DavidWillis]

Aha!  Found the problem - actually had nothing to do with the code (at least not directly).  Someone slipped a non-printing character into one of the database entries, back in 1999 - that was causing my json parser to terminate unexpectedly, and so the map never got any data to populate...

One I added a filter to take out the extraneous characters, works perfectly!

Dave

[StephenLead]

Glad you got it sorted.

A tip for these forums is to surround your code in code tags - see the # button at the top right of the editor. This prevents the editor from removing white spaces and indentation and makes your code more readable.

Cheers,
Steve

[gabrielvazquez]

now I'm testing many options to add points. I get points coordinates from an array, then to avoid hang up a web browser I'm spliting data for parts (1 part = 1000 points) and then tryin to add via API function. My code looks like:

points is an array with coordinates in format [[x1,y1],[x2,y2].....[x100000,y100000]];

I am trying to do something very similar, however not with as many points. I'm fairly new to Javascript and the ESRI JS API, so I'm struggling to replicate your example. I can build a multiple points feature when I define the individual points, like

var mp = {"geometry": { "points": [[long, lat], [long, lat]], rest of code} } however when I try to pass an array with lat/long into this feature it doesnt seem to work properly.

I think my issue has to do with how im building the array, and possibly where I'm placing the array within the code. If possible can you share all of your code, or at least more detail pertaining to how/where your defining you array. Right now, I'm simply trying to build a array holding two locations as a test and the map loads, however it doesn't seem to recognize the array.