Stabilizing Offline Synchronization for C# Application

[StephanMagal]

Hi,

We have been using ArcGis for our online WPF application since several months now and after some tweaks to fix the problems we faced we now have a functional online cartography tool inside our application.

We developped recently an offline mode for the same WPF application allowing users to use the application without being connected and synchronize their data with our servers once they're back on the connected mode.

To do so, we followed the normal 2 step process :
   - The first step is downloading the local arcgis db, when online the user select the area he wish to download on the map we then use the GenerateGeodatabaseAsync function to retrieve this portion.
   - The second step is once the user has done his modifications in the offline mode, we synchronize his local arcgis db using the SyncGeodatabaseAsync function with the one on our server.

In most cases it works fine. But we frequently run into various problems that seem to appear all along this process.

- Sometimes the local arcgis db file get corrupted and can no longer be synchronized, we are forced to delete it manually on the user machine and then redownload it as in the first step, losing all the modifications the user had made before.
- Sometimes the setting of the result of the TaskCompletionSource to the GeoDatabaseStatusInfo throw an InvalidOperationException on the SyncGeodatabaseAsync task when the user tries to synchronize.

After a lot of digging, we could not find the reason(s) of these bugs nor could we reproduce them consistently.
Some elements that could help :
- Users are often on very bad/slow network that may have micro-cuts.
- We use a dedicated database & server for the ArcGis services and data, our application business database and services are on a different server.

Could you point us where to look to fix thoses recurring problems and stabilize the ArcGis synchronization ?

[AndyWright]

Hi Stephen,

We are experiencing the same type of issues, so I'm very interested to see if Esri has any feedback for you on this one.  We have a very difficult time replicating this stuff as well ...

[JenniferNery]

Hi Stephan and Andy,

I am sorry to hear that you're experiencing issues with syncing your offline edits.

While I'm not exactly sure what's causing geodatabase corruption yet, I know that InvalidOperationException("No data to upload") may occur if the SyncGeodatabaseParameters.SyncDirection is Upload only and there were no changes found in the geodatabase.

Before you sync, can you check sync direction and if edits exist on the geodatabase?

 var hasChanges = geodatabase.FeatureTables.Any(t => t.HasEdits);‍

How do you know that geodatabase is corrupted? At what point or which error message indicates this? If you suspect that geodatabase is corrupted, are you able to do open the geodatabase and add the tables to the map or do any operations on them? For example the following code:

var geodatabase = await Geodatabase.OpenAsync(geodatabasePath);
foreach (var table in geodatabase.FeatureTables)
{
    var features = await table.QueryAsync(new QueryFilter() { WhereClause = "1=1" });
    MyMapView.Map.Layers.Add(new FeatureLayer(table));
}‍‍‍‍‍‍

Do either of you plan to upgrade to v100? In our Update 1, we'll have OfflineMapTask and OfflineMapSyncTask that may hopefully make this workflow simpler.

We appreciate any other info or call stack you can provide so we can reproduce this and make sure that future versions of the API does not have it.

Thanks.

[xiaoguangyan]

Hi, Jennifer

Could you plz check my latest(singleinstance) asked question? Thanks

[StephanMagal]

Hi Jennifer,

- We are doing a bidirectional synchronization without checking if any edits have been done on any side.

- We cannot say for sure exactly when the geodatabase get corrupted. On synchronization we get the error message "Error creating feature layer : the database disk image is malformed", we also frequently get the error message "Error creating feature layer : Item not found in the database. (Table, domain, relationship class, ect.) : gpkg_contents". When we encounter this error, the geodatabse can no longer be used or openned by the application.

- I'm not sure on which version we actually run. I've checked in the keycodes files on the server and it seems that we run on the  v101

Thanks for your response.

[AndyWright]

Jennifer,

We are doing bidirectional syncs, but we are definitely checking for edits before we do an upload of any kind.  We know our geodatabases are corrupted because our SQLite viewer software DBBrowser doesn't recognize them as valid SQLite databases and can't open them. Our application is currently using the 100.0 API. 

[JenniferNery]

Thank you both for the additional information. The InvalidOperationException I mentioned previously was for v10.2.x of the API. I'll try to see if I can get a reproducer for v100. Meanwhile, I was wondering if either of you could give Update 1 a try to see if the sync issues are resolved. I'm curious what may be raising InvalidOperationException - if it may be what leads to geodatabase being corrupted. Does it come with additional error message? Is the feature service endpoint token-secured?

I am not sure if this will help at all with recovering data from the geodatabase (if it's already corrupted) but there is this tool you can use to convert runtime Geodatabase into FileGeodatabase, perhaps it'll help save whatever offline data they had before the sync failed. http://desktop.arcgis.com/en/arcmap/latest/tools/conversion-toolbox/copy-runtime-geodatabase-to-file.... or maybe create a copy of geodatabase before sync.