Community

loop through headers in csv file and make shapefiles based on dates in the header

622
1
09-09-2022 02:31 PM
VasudhaChaturvedi
by
New Contributor II
‎09-09-2022 02:31 PM

I am unable to figure a way out how to create a shapefile after looping through the csv files and matching the headers (dates in format Oct11/2018). Each shapefile should have all the fields with the date that matches the header. In this way I want to create multiple shapefiles for each single date and with the date as the name of the shapefile.

 

import pandas as pd
import numpy as np
import glob
import csv
import arcpy,os
from arcpy import env

arcpy.env.workspace = "D:/datasets/regression_interpolation_rasters/gst_airtemp/excels_shapefiles"
path = "D:/datasets/regression_interpolation_rasters/gst_airtemp/excels_shapefiles"
#arcpy.env.overwriteOutput = True
scratchGDB = arcpy.env.scratchGDB
csvlist = glob.glob(path + "/*.csv")

headers_dict = dict()
for filename in all_csv:
    #print(filename)##15
    df = pd.read_csv(filename, index_col= None, header=0, parse_dates = True, usecols=lambda x: x not in ["Stat_name","Name", "Altitude", "Lat", "Long"])
    #print(df)
    #print(type(df))
    headers_dict[filename] = df.columns
    #print(type(headers_dict))

for csvfile in csvlist:
    #print(csvfile)##24
    headers = pd.read_csv(csvfile, index_col= None, header=0, parse_dates = True, usecols=lambda x: x not in ["Stat_name","Name", "Altitude", "Lat", "Long"])
    #print(headers)
    #print(type(headers))

    
    headers_dict = headers
    arcpy.TableToTable_conversion(csvfile, scratchGDB, "CSV")
    outLayer = "date"
    #print(table)
    #print(type(table))
    spatialreference = arcpy.SpatialReference(32632)
    arcpy.MakeXYEventLayer_management(os.path.join(scratchGDB, "CSV"), "Long","Lat",outLayer,spatialreference,"Altitude")
    arcpy.FeatureClassToShapefile_conversion(outLayer, path) 
    arcpy.MakeFeatureLayer_management(outLayer, "outLayer_lyr") 
    arcpy.SelectLayerByAttribute_management("outLayer_lyr", "NEW_SELECTION")

 

Here is one example using an image

This is the csv file converted to shapefile with all the dates

VasudhaChaturvedi_1-1663147185972.png

 

Converted shapefile I want

VasudhaChaturvedi_4-1663148447389.png

 

VasudhaChaturvedi_3-1663147592285.png

 

 

 

Tags (4)
1 Reply
RogerDunnGIS
by
Occasional Contributor II
‎10-06-2022 08:49 AM

I have a few questions

  1. You use a variable called all_csv but I don't see it defined.  Perhaps it's defined above where you copied and pasted?
  2. Your usecols expression appears to be filtering out the columns that you need.  For instance, I can see that you need the Long and Lat fields, but the DataFrame creation appears to contain fields that do not include Long and Lat
  3. A sample CSV would be nice, even if it contained a header row and two features

 

0 Kudos