By The Querying Intern


2019-02-05 22:08:07 8 Comments

I am having a strange problem with the ArcPy Table to Table function. For some reason, when I use the function to create a CSV file in a folder that exists outside of my python project folder, several columns in the CSV file have their headers cut short. For example, I have a column titled "PASSING_400_MICROM" that gets truncated to "PASSING_400_MICR'.

Thus far, I have discovered the following about the problem:

  1. The truncation of the same column headers appear to occur in any folder, both on my shared network and local machine, that isn't the python project folder. All the headers are truncated to 16 characters in length.
  2. No truncation occurs if I place the CSV within my project file. (However, I can't do this for the final code, as people on other computers will use my code)

I've done a lot of searching, and I still don't understand why this is happening and how to fix it. I am relatively new to Python and ArcGIS, so I don't even know if I'm searching for the correct things.

What might be causing this problem, and how I can fix it?

Here is an example of the function I am using:

arcpy.TableToTable_conversion(in_rows=DatabaseLocation + '\\Table',
                                  out_path=CSVLocation,
                                  out_name='Table.csv',
                                  where_clause=whereclause)

The same thing happens when I run the tool from it's dialog.

2 comments

@user2856 2019-02-05 23:14:10

As a workaround, you could write out the csv file yourself:

import os, csv
import arcpy


def table_to_csv(table, csv_file, fields, where_clause=None):
    """ Example to export a table or feature class attribute table to CSV"""

    with open(csv_file, 'wb') as csv_file_obj, arcpy.da.SearchCursor(table, fields, where_clause=where_clause) as rows:
        csv_writer = csv.writer(csv_file_obj)
        csv_writer.writerow(rows.fields)
        for row in rows:
            csv_writer.writerow(row)

# Rest of your code etc...

# arcpy.TableToTable_conversion(in_rows=DatabaseLocation + '\\Table',
#                                   out_path=CSVLocation,
#                                   out_name='Table.csv',
#                                   where_clause=whereclause)

table_to_csv(os.path.join(DatabaseLocation, 'Table'), 
             os.path.join(CSVLocation,'Table.csv'), 
             ['field1', 'field2'],
             whereclause)

@Kirk Kuykendall 2019-02-06 01:23:32

+1 If you have any dates, you might need to write a schema.ini too.

@The Querying Intern 2019-02-06 16:04:58

Thank you for your suggestion! I tried to run the code however, and I kept getting this error: TypeError: SearchCursor() got multiple values for keyword argument 'where_clause' . Thoughts? I looked it up, and in some forums it had something to do with not putting "self" as an argument, but I don't know if that applies in this function.

@user2856 2019-02-07 22:47:46

@TheQueryingIntern I can't reproduce that. If you're using this code exactly as posted (i.e as a standalone function, not part of a class method - self is only relevant for class methods) it's not possible to get that exception. The only way to get a similar exception (TypeError: table_to_csv() got multiple values for keyword argument 'where_clause' note "table_to_csv()" not "SearchCursor") is to call it with TWO where_clause arguments, one as a positional argument and one a keyword argument, i.e table_to_csv(table, csvfile, ["field"], "ID=123", where_clause="ID=456") etc...

@Hornbydd 2019-02-05 22:34:33

I just tried this in 10.6.1 running on Windows 10. I added a test field called "PASSING_400_MICROM" and set all values to be "YES". When I went to tun the Table to Table tool and set the output to be a CSV file it truncated the field name to 11 characters. I tried editing the field map and it set it back to 11 characters.

My gut feeling is that this tool is imposing some sort of field name limitation, like it or not.

At this point I would suggest you head over to the ESRI Code Sharing website, the first place of call for user created tools, be it from the community at large or an ESRI development team. I have used the Excel and CSV Conversion Toolbox when needing to convert to CSV because of the flakiness of the Table to Table tool.

Related Questions

Sponsored Content

0 Answered Questions

1 Answered Questions

3 Answered Questions

1 Answered Questions

[SOLVED] Grib to CSV use lat lon

0 Answered Questions

Joining csv table and shapefile via ArcPy?

2 Answered Questions

2 Answered Questions

[SOLVED] How to export list of feature classes in multi gdb's to a csv file?

  • 2016-04-25 14:11:45
  • joseph_k
  • 256 View
  • 1 Score
  • 2 Answer
  • Tags:   arcpy arcgis-10.3

1 Answered Questions

[SOLVED] QGIS - Import CSV column names as numbers

  • 2015-10-22 11:13:23
  • coding_bird
  • 130 View
  • 4 Score
  • 1 Answer
  • Tags:   qgis csv

2 Answered Questions

[SOLVED] Error adding coordinates in CSV files

Sponsored Content