Group, split and reclass using numpy, python and arcpy

[DanPatterson]

You have a gdb table and you need to group data, sort it, split it and reclass it.

Doing it outside is often the easiest.

This demonstrates the powerful arcpy.da functions of NumPyArrayToTable and TableToNumPyArray

I have a data set with a field containing unsorted values containing 1, 2 or 3.

I need to group the 1's, 2's and 3's together, then break those groups into sequential bunches of 12.

Follow the code.

The imports....

import numpy as np
from numpy.lib.recfunctions import unstructured_to_structured as uts
from numpy.lib.recfunctions import structured_to_unstructured as stu
import arcpy

Load the table into a structured array

tbl = # full path to the gdb table or featureclass table in the gdb
arr_ = arcpy.da.TableToNumPyArray(tbl, ['ID_num', 'Val_'])
arr = stu(arr_)

Do the work  (I will elaborate soon ...

idx_srted = np.lexsort((arr[:, 0], arr[:, 1]))  # sort the array/field
srted = arr[idx_srted]  # sort the array
idx_split = np.nonzero(srted[:-1, 1] - srted[1:, 1])[0] + 1 # split indices
split_arrs = np.array_split(srted, idx_split)  # split the sub arrays
#
# now group into bunches of 12
groups = [np.arange(0, len(i), 12) for i in split_arrs]
cnt = 1  # default counter
out = []
groups = [np.arange(0, len(i), 12) for i in split_arrs]
cnt = 1  # default counter
out = []
for i, sub in enumerate(split_arrs):  # now do the work
    bits = np.array_split(sub, groups[i])  # split each subgroup
    for bit in bits:
        if bit.size > 0:
            zz = np.repeat(cnt, len(bit))[:, None]  # repeat the counter
            comb = np.concatenate((bit, zz), axis=1)  # magic
            out.append(comb)
            cnt += 1
    cnt = 1

Now send it back to Pro, make the table.

arr2 = np.concatenate(out, axis=0)
dt2 = np.dtype([('ID_num', 'i8'), ('Val_', 'i8'), ('New_key', 'i8')])
arr2 = uts(arr2, dtype=dt2)
tbl2 = r"C:\Arc_projects\Test_project\Test_pro28.gdb\tbl_2"
arcpy.da.NumPyArrayToTable(arr2, tbl2)

Open the table and compare the before and after.

Have fun with NumPy and Arcpy