By Lyubomyr Shaydariv


2011-05-14 13:53:39 8 Comments

I'm developing a part of an application that's responsible for exporting some data into CSV files. The application always uses UTF-8 because of its multilingual nature at all levels. But opening such CSV files (containing e.g. diacritics, cyrillic letters, Greek letters) in Excel does not achieve the expected results showing something like Г„/Г¤, Г–/Г¶. And I don't know how to force Excel understand that the open CSV file is encoded in UTF-8. I also tried specifying UTF-8 BOM EF BB BF, but Excel ignores that.

Is there any workaround?

P.S. Which tools may potentially behave like Excel does?


UPDATE

I have to say that I've confused the community with the formulation of the question. When I was asking this question, I asked for a way of opening a UTF-8 CSV file in Excel without any problems for a user, in a fluent and transparent way. However, I used a wrong formulation asking for doing it automatically. That is very confusing and it clashes with VBA macro automation. There are two answers for this questions that I appreciate the most: the very first answer by Alex https://stackoverflow.com/a/6002338/166589, and I've accepted this answer; and the second one by Mark https://stackoverflow.com/a/6488070/166589 that have appeared a little later. From the usability point of view, Excel seemed to have lack of a good user-friendly UTF-8 CSV support, so I consider both answers are correct, and I have accepted Alex's answer first because it really stated that Excel was not able to do that transparently. That is what I confused with automatically here. Mark's answer promotes a more complicated way for more advanced users to achieve the expected result. Both answers are great, but Alex's one fits my not clearly specified question a little better.


UPDATE 2

Five months later after the last edit, I've noticed that Alex's answer has disappeared for some reason. I really hope it wasn't a technical issue and I hope there is no more discussion on which answer is greater now. So I'm accepting Mark's answer as the best one.

25 comments

@e-zinc 2012-01-26 10:11:43

We have used this workaround:

  1. Convert CSV to UTF-16 LE
  2. Insert BOM at beginning of file
  3. Use tab as field separator

@Husky 2013-05-06 11:32:16

Thanks! I tried all the other answers in this thread, but converting to UTF-8 simply didn't work. When i tried UTF-16 with BOM it worked instantly.

@Gere 2016-07-24 17:38:54

This solution works in Excel for Mac OS

@Elmue 2016-07-27 22:36:58

This does NOT work in Office 2007.

@multidynamic 2016-09-13 23:29:33

For Excel for Mac 2011 I was successful with UTF-16 little endian csv files

@user1073075 2016-10-31 10:04:16

Thanks, this solution is excellent for giving my users a unicode csv that they can open in Excel.

@Christopher Adi Pascual 2018-09-20 08:15:40

You can convert .csv file to UTF-8 with BOM via Notepad++:

  1. Open the file in Notepad++.
  2. Go to menu EncodingConvert to UTF-8.
  3. Go to menu FileSave.
  4. Close Notepad++.
  5. Open the file in Excel .

Worked in Microsoft Excel 2013 (15.0.5093.1000) MSO (15.0.5101.1000) 64-bit from Microsoft Office Professional Plus 2013 on Windows 8.1 with locale for non-Unicode programs set to "German (Germany)".

@pakalbekim 2018-12-20 20:05:35

Did what you said, didn't work at all. Converting/encoding, setting character sets, nothing worked in my case. Import file to google sheets, download as csv, worked flawlessly. Your welcome from 2045

@pakalbekim 2018-12-20 20:06:50

I tried everything I could find on this thread and similar, nothing worked fully. However, importing to google sheets and simply downloading as csv worked like a charm. Try it out if you come to my frustration point.

@StriplingWarrior 2012-07-09 16:40:03

The UTF-8 Byte-order marker will clue Excel 2007+ in to the fact that you're using UTF-8. (See this SO post).

In case anybody is having the same issues I was, .NET's UTF8 encoding class does not output a byte-order marker in a GetBytes() call. You need to use streams (or use a workaround) to get the BOM to output.

@Lyubomyr Shaydariv 2012-07-09 16:49:49

Thanks for the post link (I didn't know about the UTF-16 case).

@vy32 2014-11-26 15:59:42

Inserting a UTF-8 BOM seems to be the way to go. After that it's transparent.

@bgusach 2015-11-26 09:26:04

Thank you very much. This is way better than trying to guess what encoding excel fancies.

@Sébastien 2016-06-06 08:02:52

I use Notepad++ to easily convert the .csv from UTF-8 to UTF-8 with BOM

@user369142 2016-11-14 17:23:15

Yes - best solution. Lots of confusing posts around setting headers in http responses. This solves the problem. Same can be seen when opening the file in notepad and saving right back down using the UTF-8 option. Also adds the byte-order marker.

@user369142 2016-11-14 17:23:56

Solution from the post referenced: private byte[] AddEncodingPreamble(byte[] byteArray, Encoding encoding) { return encoding.GetPreamble().Concat(byteArray).ToArray(); }

@Elmue 2016-11-25 14:41:41

You did not read the question! Lyubomyr Shaydariv wrote: " I also tried specifying UTF-8 BOM EF BB BF, but Excel ignores that."

@StriplingWarrior 2016-11-26 21:57:22

@Elmue: I figured maybe (like me) he thought he was saving it with a byte-order marker, but wasn't because some library didn't work how he thought it did. I found it worthwhile to add this answer, and evidently a number of other people have found it useful too. At any rate, it seems like a better answer than saying not to use CSV, or instructing users how to open the file in some bizarre way.

@Elmue 2016-12-01 13:25:13

No. The cause that it did not work for him is not that he was unable to save a simple 3 bytes to a file. The cause that it does not work is that at least Excel 2007 IGNORES the BOM completely.

@StriplingWarrior 2016-12-01 16:04:17

@Elmue: I believe Excel 2007 SP3 solves that issue.

@Elmue 2016-12-01 22:53:43

And how do you know that your users have installed this SP? I prefer to write code that works on all computers. Do you also hate these stupid webpages which say at the bottom: "This webpage is optimized for Firefox". It is a shame if a programmer is not able to write code that runs on all browsers. And it is a shame if a programmer tells his users that they have to download and install a huge update to use their software. And what if your user does not have admin privileges? I reapeat once more: Your answer is NO solution for the problem!

@StriplingWarrior 2016-12-02 00:41:00

@Elmue: I'll grant that it's an imperfect solution. If there were any way to automatically make this work for older versions of Excel, I'd prefer them. But recognizing that there is no solution to the problem, and at least 69 people have found this answer to be helpful so far, I'm going to leave it here. Thank you for your observations.

@srgstm 2017-03-25 04:14:49

I use plain old Notepad to easily convert .csv from UTF-8 to UTF-8 with BOM. Just open and save.

@Thomas Andrews 2017-03-27 18:23:30

The Mac version of Excel seems to still ignore the BOM. (Excel for Mac 2011.)

@marsze 2018-12-12 07:50:59

.NET does output a BOM if you employ the streams correctly, see here: stackoverflow.com/questions/5266069/…

@StriplingWarrior 2018-12-12 17:15:28

@marsze: Thanks for pointing that out. There is another answer on the link that I posted which uses streams. I guess the lesson is that GetBytes() was never intended to be used to produce an entire file's contents by itself: we're supposed to use streams and let them call into GetPreamble() and GetBytes() and such.

@marsze 2018-12-13 09:14:45

@StriplingWarrior Very true, a BOM belongs only at the start of streams. What I like about that other answer (except that it's basically just another way of writing it: string.ToBytes(encoding) instead of encoding.ToBytes(string)) is that you can pass new UTF8Encoding(true) or new UTF8Encoding(false) for controlling the output of a BOM or not.

@Elmue 2016-11-25 14:31:10

It is incredible that there are so many answers but none answers the question:

"When I was asking this question, I asked for a way of opening a UTF-8 CSV file in Excel without any problems for a user,..."

The answer marked as the accepted answer with 200+ up-votes is useless for me because I don't want to give my users a manual how to configure Excel. Apart from that: this manual will apply to one Excel version but other Excel versions have different menus and configuration dialogs. You would need a manual for each Excel version.

So the question is how to make Excel show UTF8 data with a simple double click?

Well at least in Excel 2007 this is not possible if you use CSV files because the UTF8 BOM is ignored and you will see only garbage. This is already part of the question of Lyubomyr Shaydariv:

"I also tried specifying UTF-8 BOM EF BB BF, but Excel ignores that."

I make the same experience: Writing russian or greek data into a UTF8 CSV file with BOM results in garbage in Excel:

Content of UTF8 CSV file:

Colum1;Column2
Val1;Val2
Авиабилет;Tλληνικ

Result in Excel 2007:

CSV UTF8 Excel

A solution is to not use CSV at all. This format is implemented so stupidly by Microsoft that it depends on the region settings in control panel if comma or semicolon is used as separator. So the same CSV file may open correctly on one computer but on anther computer not. "CSV" means "Comma Separated Values" but for example on a german Windows by default semicolon must be used as separator while comma does not work. (Here it should be named SSV = Semicolon Separated Values) CSV files cannot be interchanged between different language versions of Windows. This is an additional problem to the UTF-8 problem.

Excel exists since decades. It is a shame that Microsoft was not able to implement such a basic thing as CSV import in all these years.


However, if you put the same values into a HTML file and save that file as UTF8 file with BOM with the file extension XLS you will get the correct result.

Content of UTF8 XLS file:

<table>
<tr><td>Colum1</td><td>Column2</td></tr>
<tr><td>Val1</td><td>Val2</td></tr>
<tr><td>Авиабилет</td><td>Tλληνικ</td></tr>
</table>

Result in Excel 2007:

UTF8 HTML Excel

You can even use colors in HTML which Excel will show correctly.

<style>
.Head { background-color:gray; color:white; }
.Red  { color:red; }
</style>
<table border=1>
<tr><td class=Head>Colum1</td><td class=Head>Column2</td></tr>
<tr><td>Val1</td><td>Val2</td></tr>
<tr><td class=Red>Авиабилет</td><td class=Red>Tλληνικ</td></tr>
</table>

Result in Excel 2007:

UTF8 HTML Excel

In this case only the table itself has a black border and lines. If you want ALL cells to display gridlines this is also possible in HTML:

<html xmlns:x="urn:schemas-microsoft-com:office:excel">
    <head>
        <meta http-equiv="content-type" content="text/plain; charset=UTF-8"/>
        <xml>
            <x:ExcelWorkbook>
                <x:ExcelWorksheets>
                    <x:ExcelWorksheet>
                        <x:Name>MySuperSheet</x:Name>
                        <x:WorksheetOptions>
                            <x:DisplayGridlines/>
                        </x:WorksheetOptions>
                    </x:ExcelWorksheet>
                </x:ExcelWorksheets>
            </x:ExcelWorkbook>
        </xml>
    </head>
    <body>
        <table>
            <tr><td>Colum1</td><td>Column2</td></tr>
            <tr><td>Val1</td><td>Val2</td></tr>
            <tr><td>Авиабилет</td><td>Tλληνικ</td></tr>
        </table>
    </body>
</html>

This code even allows to specify the name of the worksheet (here "MySuperSheet")

Result in Excel 2007:

enter image description here

@Lyubomyr Shaydariv 2016-11-25 14:45:06

And it's also incredible that people still answer it. :) However, I would argue that CSV is dead: CSV is a deadly simple plain streaming format, and, when I was on that project, we used it really heavily for components interaction too. I would never use HTML format because of its too-representive nature, and I couldn't ever make the users of that project export data to HTML just to make it Excel-friendly. Excel sucked at UTF-8 CSV, and I have no clue what's changed 5 years since then. But that comma-, semi-colon- and tab-delimited files is true madness, I guess.

@Elmue 2016-11-25 14:58:48

It is not incredible that people still answer your question because Google brings them here: All with the same question: How do I get stupid Excel to do what I want? Enter 3 words into Google: "excel csv utf8" and your question is the FIRST result world wide. You wrote a bestseller!

@Elmue 2016-11-25 15:09:41

By the way: Why did you mark an answer as accepted answer which does not answer what you wanted to know ? You wrote: "When I was asking this question, I asked for a way of opening a UTF-8 CSV file in Excel without any problems for a user,"

@Lyubomyr Shaydariv 2016-11-25 16:08:20

Just googled for "excel csv utf8", and I didn't know the question is at the top. :) Regarding accepting the answer: I marked the answer as the best one just because it really answered how it could be done in Excel. Maybe I was too strict when I said "without any problems". I think that import wizards are fine for me being a programmer, but it's shame that Excel required (or still requires) to do import rather than just open.

@StriplingWarrior 2016-12-02 00:40:50

Since you've taken the time to point out how none of the other answers here cause Excel to open a CSV file correctly, it's worth mentioning that this answer doesn't do that either. The originally-accepted answer (which was deleted by a moderator) recommended exporting the data to an Excel file directly, or exporting an HTML file and allowing users to open the file in Excel. Either of those solutions are less hacky than saving an HTML file with an XLS file extension, but no answer has been proposed that actually answers the question.

@Elmue 2016-12-02 13:48:50

@Warrior: "..but no answer has been proposed that actually answers the question." This is not correct. I HAVE answered that question: At least with Excel 2007 it is NOT POSSIBLE. This is the definite answer. If Excel ignores the UTF8 BOM and interprets data as ANSI, there is no way to import for example greek or russian text into Excel via CSV by a simple double click on the file (which was the question: "without any problems for a user"). What I propose is a working solution that functions with ALL Excel versions with a double click and does not require additional steps in the menus of Excel.

@Cyril Duchon-Doris 2017-01-03 12:03:03

Hell, like I have a choice to export as something else than CSV when it's the only button I can see in my interface... I agree that it is quite rubbish though, but we have to deal with it.

@Terence Chow 2017-12-15 15:22:18

Is there anyway to preserve the original excel cell lines instead of showing a blank page?

@Elmue 2017-12-15 17:42:48

I updated my answer with an additional example to show all gridlines.

@Jeremy 2018-05-16 21:07:47

Not only does this not answer the question, it's completely wrong to state CSV is "legacy" and "dead". Just because you doesn't see the use for it, does not mean it's dead. Also, you clearly don't understand it's history or how widely used it is today. This is better than XML (due to smaller packet sizes) and the issue here is how Microsoft simply does not follow a standard for the UTF8 format. The correct answer would be to use open office and not Microsoft as Microsoft has never been good at encoding in the first place.

@Elmue 2018-05-17 21:55:07

You are partly correct. I updated my answer. It is not the fault of CSV itself. It is the fault of Microsoft. But the quesion was about Microsoft Excel so you surely do not answer the original question if you recommend to use Open Office (which has other bugs which may even be worse). You surely will not write to the recipients of your emails: "Please install Open Office to see the attached CSV file correctly". There must be better solutions. And my answer is a far better solution than asking people to install Open Office just to open an CSV file.

@praaveen 2018-08-15 12:27:24

hi i'm using ruby on rails for csv generation. In our application we plan to go for the multi language(I18n) and we faced an issue while viewing I18n content in the CSV file of windows excel.

Was fine with Linux (Ubuntu) and mac.

We identified that windows excel need to be imported the data again to view the actual data. While import we will get more options to choose character set.

But this can’t be educated for each and every user, so solution we looking for is to open just by double click.

Then we identified the way of showing data by open mode and bom in windows excel with the help of aghuddleston gist. Added at reference.

Example I18n content

In Mac and Linux

Swedish : Förnamn English : First name

In Windows

Swedish : Förnamn English : First name

def user_information_report(report_file_path, user_id)
    user = User.find(user_id)
    I18n.locale = user.current_lang
    open_mode = "w+:UTF-16LE:UTF-8"
    bom = "\xEF\xBB\xBF"
    body user, open_mode, bom
  end

def headers
    headers = [
        "ID", "SDN ID",
        I18n.t('sys_first_name'), I18n.t('sys_last_name'), I18n.t('sys_dob'),
        I18n.t('sys_gender'), I18n.t('sys_email'), I18n.t('sys_address'),
        I18n.t('sys_city'), I18n.t('sys_state'), I18n.t('sys_zip'),
        I18n.t('sys_phone_number')
    ]
  end

def body tenant, open_mode, bom
    File.open(report_file_path, open_mode) do |f|
      csv_file = CSV.generate(col_sep: "\t") do |csv|
        csv << headers
        tenant.patients.find_each(batch_size: 10) do |patient|
          csv <<  [
              patient.id, patient.patientid,
              patient.first_name, patient.last_name, "#{patient.dob}",
              "#{translate_gender(patient.gender)}", patient.email, "#{patient.address_1.to_s} #{patient.address_2.to_s}",
              "#{patient.city}", "#{patient.state}",  "#{patient.zip}",
              "#{patient.phone_number}"
          ]
        end
      end
      f.write bom
      f.write(csv_file)
    end
  end

Important things to note here is open mode and bom

open_mode = "w+:UTF-16LE:UTF-8"

bom = "\xEF\xBB\xBF"

Before writing the CSV insert BOM

f.write bom

f.write(csv_file)

Windows and Mac

File can be opened directly by double clicking.

Linux (ubuntu)

While opening a file ask for the separator options -> choose “TAB” enter image description here

@yoyo 2018-08-14 21:21:00

I am generating csv files from a simple C# application and had the same problem. My solution was to ensure the file is written with UTF8 encoding, like so:

// Use UTF8 encoding so that Excel is ok with accents and such.
using (StreamWriter writer = new StreamWriter(path, false, Encoding.UTF8))
{
    SaveCSV(writer);
}

I originally had the following code, with which accents look fine in Notepad++ but were getting mangled in Excel:

using (StreamWriter writer = new StreamWriter(path))
{
    SaveCSV(writer);
}

Your mileage may vary - I'm using .NET 4 and Excel from Office 365.

@Nestor Urquiza 2017-12-08 17:40:49

As I posted on http://thinkinginsoftware.blogspot.com/2017/12/correctly-generate-csv-that-excel-can.html:

Tell the software developer in charge of generating the CSV to correct it. As a quick workaround you can use gsed to insert the UTF-8 BOM at the beginning of the string:

gsed -i '1s/^\(\xef\xbb\xbf\)\?/\xef\xbb\xbf/' file.csv

This command inserts the UTF-4 BOM if not present. Therefore it is an idempotent command. Now you should be able to double click the file and open it in Excel.

@RandomGuy 2018-02-09 12:07:49

Thanks this fixed it. I have just converted the csv ';' separated to UTF-BOM in Notepad++ and know excels opens it correctly.

@EFC 2018-07-26 14:49:09

Exactly what was needed! Here is a pointer to how to implement this in PHP stackoverflow.com/questions/25686191/…

@Flavien 2018-10-12 13:23:35

This is the correct solution.

@cobp 2019-05-06 12:20:03

This is the workaround for excel which could not recognize utf chareacters in csv files. So this correction to fix excel issue.

@Dan 2017-11-06 02:40:36

  1. Download & install LibreOffice Calc
  2. Open the csv file of your choice in LibreOffice Calc
  3. Thank the heavens that an import text wizard shows up...
  4. ...select your delimiter and character encoding options
  5. Select the resulting data in Calc and copy paste to Excel

@CannedMoose 2017-11-01 17:10:03

This is not accurately addressing the question but since i stumbled across this and the above solutions didn't work for me or had requirements i couldn't meet, here is another way to add the BOM when you have access to vim:

vim -e -s +"set bomb|set encoding=utf-8|wq" filename.csv

@Christoph 2017-04-08 19:26:20

A truly amazing list of answers, but since one pretty good one is still missing, I'll mention it here: open the csv file with google sheets and save it back to your local computer as an excel file.

In contrast to Microsoft, Google has managed to support UTF-8 csv files so it just works to open the file there. And the export to excel format also just works. So even though this may not be the preferred solution for all, it is pretty fail safe and the number of clicks is not as high as it may sound, especially when you're already logged into google anyway.

@TheCharlatan 2016-12-01 13:03:53

Yes, this is possible. As previously noted by multiple users, there seems to be a problem with excel reading the correct Byte Order Mark when the file is encoded in UTF-8. With UTF-16 it does not seem to have a problem, so it is endemic to UTF-8. The solution I use for this is adding the BOM, TWICE. For this I execute the following sed command twice:

sed -I '1s/^/\xef\xbb\xbf/' *.csv

, where the wildcard can be replaced with any file name. However, this leads to a mutation of the sep= at the beginning of the .csv file. The .csv file will then open normally in excel, but with an extra row with "sep=" in the first cell. The "sep=" can also be removed in the source .csv itself, but when opening the file with VBA the delimiter should be specified:

Workbooks.Open(name, Format:=6, Delimiter:=";", Local:=True)

Format 6 is the .csv format. Set Local to true, in case there are dates in the file. If Local is not set to true the dates will be Americanized, which in some cases will corrupt the .csv format.

@Cloudranger 2016-05-10 10:16:02

If you want to make it fully automatic, one click, or to load automatically into Excel from say a web page, but can't generate proper Excel files, then I would suggest looking at SYLK format as an alternative. OK it is not as simple as CSV but it is text based and very easy to implement and it supports UTF-8 with no issues.

I wrote a PHP class that receives the data and outputs a SYLK file which will open directly in Excel by just clicking the file (or will auto-launch Excel if you write the file to a web page with the correct mime type. You can even add formatting (like bold, format numbers in particular ways etc) and change column sizes, or auto size columns to the text in the columns and all in all the code is probably not more than about 100 lines.

It is dead easy to reverse engineer SYLK by creating a simple spreadsheet and saving as SYLK and then reading it with a text editor. The first block are headers and standard number formats that you will recognise (which you just regurgitate in every file you create), then the data is simply an X/Y coordinate and a value.

@Found-it-finally 2013-06-12 19:48:13

This is an old question but comes up in the search at top. I found after a lot of efforts that adding BOM characters at the beginning of csv file helps.

I have briefed it here: https://sites.google.com/site/ritechtips/home/the-multi-line-fields-csv-file-and-excel-import---ha

@Lyubomyr Shaydariv 2013-06-12 20:03:11

Thanks for the reply. Hm, as I mentioned in the question, EF BB BF seems not to be working. Did you try any other BOM?

@Ixtlilton 2013-10-22 11:02:01

Had the same problems with PHP-generated CSV files. Excel ignored the BOM when the Separator was defined via "sep=,\n" at the beginning of the content (but of course after the BOM).

So adding a BOM ("\xEF\xBB\xBF") at the beginning of the content and setting the semicolon as separator via fputcsv($fh, $data_array, ";"); does the trick.

@Wagner Bertolini Junior 2015-10-13 14:38:25

Just for help users interested on opening the file on Excel that achieve this thread like me.

I have used the wizard below and it worked fine for me, importing an UTF-8 file. Not transparent, but useful if you already have the file.

  1. Open Microsoft Excel 2007.
  2. Click on the Data menu bar option.
  3. Click on the From Text icon.
  4. Navigate to the location of the file that you want to import. Click on the filename and then click on the Import button. The Text Import Wizard - Step 1 or 3 window will now appear on the screen.
  5. Choose the file type that best describes your data - Delimited or Fixed Width.
  6. Choose 65001: Unicode (UTF-8) from the drop-down list that appears next to File origin.
  7. Click on the Next button to display the Text Import Wizard - Step 2 or 3 window.
  8. Place a checkmark next to the delimiter that was used in the file you wish to import into Microsoft Excel 2007. The Data preview window will show you how your data will appear based on the delimiter that you chose.
  9. Click on the Next button to display the Text Import Wizard - Step 3 of 3.
  10. Choose the appropriate data format for each column of data that you want to import. You also have the option to not import one or more columns of data if you want.
  11. Click on the Finish button to finish importing your data into Microsoft Excel 2007.

Source: https://www.itg.ias.edu/content/how-import-csv-file-uses-utf-8-character-encoding-0

@Aki 2014-04-29 13:39:49

Old question but heck, the simplest solution is:

  1. Open CSV in Notepad
  2. Save As -> select the right encoding
  3. Open the new file

@Lyubomyr Shaydariv 2014-04-29 14:57:07

Which Excel version do you use?

@Aki 2014-04-29 15:18:54

Excel 2010. /////

@Auberon Vacher 2014-07-23 12:50:06

Brilliant, simply works and solves the problem (at least for me)

@Veverke 2015-05-12 16:18:10

Nope, because the problem is to have Excel open the .csv file as a csv file. I mean, I have a process where I generate csv files. The user then would only be required to open the files. Here he/she is required to do more than that.

@Mark Ransom 2015-09-17 16:14:12

@Veverke if you generate files on a regular basis this isn't the best solution. But if it's a one-off, this answer is perfect. It works because Notepad will put the BOM at the beginning of a UTF-8 or UTF-16 file.

@user2925795 2019-03-28 16:45:44

Works like a charm. ((:

@zax 2014-02-28 21:18:08

Yes it is possible. When writing the stream creating the csv, the first thing to do is this:

myStream.Write(Encoding.UTF8.GetPreamble(), 0, Encoding.UTF8.GetPreamble().Length)

@Elmue 2016-11-25 14:37:52

You did not read the question! Lyubomyr Shaydariv wrote: " I also tried specifying UTF-8 BOM EF BB BF, but Excel ignores that."

@zax 2016-11-28 04:29:55

This is working code. If you write out UTF-8 files from .Net applications with this lead, Excel will recognize the file as UTF8.

@Elmue 2016-12-01 13:26:40

May be in YOUR Excel it works. But at least in Excel 2007 the BOM is ignored completely. So if you intend to write a code that works on all Excel versions your code is not usefull.

@Nobilis 2013-10-18 09:49:47

This is my working solution:

vbFILEOPEN = "your_utf8_file.csv"
Workbooks.OpenText Filename:=vbFILEOPEN, DataType:=xlDelimited, Semicolon:=True, Local:=True, Origin:=65001

The key is Origin:=65001

@Steve Jessop 2013-07-09 09:18:26

I have had the same issue in the past (how to produce files that Excel can read, and other tools can also read). I was using TSV rather than CSV, but the same problem with encodings came up.

I failed to find any way to get Excel to recognize UTF-8 automatically, and I was not willing/able to inflict on the consumers of the files complicated instructions how to open them. So I encoded them as UTF-16le (with a BOM) instead of UTF-8. Twice the size, but Excel can recognize the encoding. And they compress well, so the size rarely (but sadly not never) matters.

@Lyubomyr Shaydariv 2013-07-09 20:04:32

It's an old question, but I'm happy that people still answer it. Thank you. :)

@Steve Jessop 2013-07-12 08:09:01

@LyubomyrShaydariv: since I answered this question, a colleague told me that the latest Excel does identify UTF-8 CSV files provided they have an initial BOM. So in a few years, when everyone in the world (or anyway, everyone we deliver the files to) is on that version of Excel or better, I might be able to change my encoding :-)

@Veverke 2015-05-12 16:22:46

Strange, for me it does not work. Turns the characters into chinese symbols... (they are supposed to be hebrew)

@Henno 2016-04-07 09:18:48

I had chinese characters in Excel when I saved in "UTF-16 BE with BOM". When I tried "UTF-16 LE with BOM" it worked! I used Sublime Text for testing.

@sshturma 2013-04-26 07:52:21

The bug with ignored BOM seems to be fixed for Excel 2013. I had same problem with Cyrillic letters, but adding BOM character \uFEFF did help.

@Galdo 2013-09-27 19:47:04

Since I added \uFEFF at the beginning of my CSV file (generated in Java), Excel is able to open them correctly! Thx

@Abdullah 2013-12-31 07:09:04

that's solved my issue too , in php it looks like this: $utf8_with_bom = chr(239) . chr(187) . chr(191) . $csvText;

@nathan hayfield 2014-10-15 18:58:35

@updola thanks so much for this comment, saved me hours

@tounaobun 2014-12-05 14:41:43

It works for me in Windows Microsoft Excel 2010

@Nelson 2015-09-16 07:35:40

UTF-8, by definition, does not use nor should use the BOM characters. The way Excel reads BOM screws up the advantage of UTF-8 over Unicode, which is backwards compatibility with ASCII. Adding the BOM will make Excel work, but break other proper UTF-8/ASCII file reads.

@Mark Ransom 2015-09-17 16:09:28

@Nelson the BOM shouldn't be necessary, but it is. Microsoft values backwards compatibility and Windows programs have always assumed a file would be encoded with a MBCS character set. Changing that assumption would break too many legacy systems. It would be nice if you could specify UTF-8 as your MBCS encoding, but they've never gotten that to work consistently.

@Nelson 2015-09-18 01:43:09

@MarkRansom but that's not UTF-8 specification at all. By definition, UTF-8 is suppose to be compatible with standard ASCII at the binary level, which means no characters above 127. Everything else under is suppose to be straight up "standard" with no special characters. What Microsoft should be doing is DEFAULT UTF-8 if there are no BOM, not default to their MS craptastic encoding.

@Mark Ransom 2015-09-18 01:54:33

@Nelson no, what Microsoft "should" be doing is making sure that things that used to work, continue to work. And that's exactly what they're doing. I know the spec says it isn't necessary, and in a perfect world it wouldn't be. But that's not the world we live in.

@Nelson 2015-09-18 02:09:33

@MarkRansom Microsoft does an atrocious job of keeping things working. Just look at this thorough report on the Excel inconsistencies in dealing with BOM: wiki.scn.sap.com/wiki/display/ABAP/…

@Panagiotis Kanavos 2016-01-04 15:55:45

@Nelson Wikipedia seems to think that UTF-8 does have a BOM The UTF-8 representation of the BOM is the byte sequence 0xEF,0xBB,0xBF. Anyway, CSV is an imprecise format. Moreover, the link you provided shows limited knowledge of non-US cultures - ; is the list separators for several European languages because , is the decimal separator and can't be used as a list separator. If one wants fidelity, one should use the XLSX format - it's just a compressed XML with a very well defined schema.

@Alveoli 2016-06-13 16:37:23

This should be the new accepted answer. Works like a dream!

@Nakilon 2017-01-26 15:31:40

\FE\FF makes Mac OS Excel 2016 the whole file as Japanese hieroglyphs.

@Dherik 2018-12-06 20:03:40

Works for me too. The BOM character needs to be at the beginning of the file.

@Rex 2013-03-25 13:41:02

This is an old question but I've just encountered had a similar problem and the solution may help others:

Had the same issue where writing out CSV text data to a file, then opening the resulting .csv in Excel shifts all the text into a single column. After having a read of the above answers I tried the following, which seems to sort the problem out.

Apply an encoding of UTF-8 when you create your StreamWriter. That's it.

Example:

using (StreamWriter output = new StreamWriter(outputFileName, false, Encoding.UTF8, 2 << 22)) {
   /* ... do stuff .... */
   output.Close();
}

@Rex 2016-08-10 10:58:38

@elmue care to elaborate a little? Surely outputting CSV using the correct encoding to start with ensures there's no compatibility issues with Excel later in the workflow?

@Elmue 2016-08-17 02:38:52

The code is wrong because you do not need output.Close() if you have a using() statement. Apart from that Excel CSV inport is very primitive. I would not use it at all. If you want to import in Excel use a HTML table instead and open it in Excel.

@Rex 2016-11-23 09:55:06

Thanks for clarifying @Elmue - unfortunately I've run into a number of scenarios (including files across network drives) where a close is required explicitly as well as the disposal. Not found a reason why as I think I remember seeing a call to Close() in the Dispose stack but there we go. You're also incorrect in your statement about CSV import being primitive as it does not need the inefficient HTML approach you've suggested. In fact creating additional programmatic steps to take plain-text data, transform into HTML then pull into Excel seems counter intuitive

@Rex 2016-11-23 09:58:41

continued... However, different scenarios have different requirements and my example correctly shows how to specify the encoding as the OP requested

@Elmue 2016-11-25 13:42:31

Please read the original question again: "I also tried specifying UTF-8 BOM EF BB BF, but Excel ignores that." I tested the same and come to the same result: Excel does NOT recognize UTf8 BOM. Try it! Write a CSV file and put greek or russian charcters into it. Then open that in Excel and you will get garbarge. So your answer does not solve anything.

@Elmue 2016-11-25 13:51:03

CSV is primitive because it depends on your settings in control panel if comma or semicolon is used as separator which is the most stupid design. A file with comma is detected correctly on some computers but on other computers not. If the data itself contains commas you must quote them. And UTF8 does NOT work. All these problems do not appear loading a HTML table into Excel.

@Elmue 2016-11-25 13:52:15

The programmatic steps to create an HTML table are ridiculous. Save the following into a file with the extension XLS and save it as UTF8 with BOM and open it in Excel: <table><tr><td>Colum1</td><td>Column2</td></tr> <tr><td>Авиабилет</td><td>Tλληνικ</td></tr></table>

@Rex 2016-11-30 14:24:18

I have read the OP - I've supplied a way to create the CSVs via Excel that solve the problem to start with, and also open forcing the specific encoding the OP has asked for. Seems to work just fine with Excel 2013 and .NET 4 so I'm guessing there's a version difference in the Office type libraries between our experiments? The HTML approach is inefficient and I'm still surprised you're trying to extol it's virtues - what you're describing as plaint text anomalies is down to the encoding. Try importing as plain text in a simple using {} and save as a new file forcing the encoding. It works.

@Elmue 2016-12-01 13:20:28

As I already said your approach does not work in Excel 2007 and as you never know which Excel versions your users use, it is not a generic solution.

@Rex 2018-02-13 20:22:05

Seemed to work fine across 2007 and 2013 so not sure where you're going wrong

@Elmue 2018-02-13 23:16:56

An UTF8 BOM definitely does NOT work with Excel 2007. It is ignored.

@alc 2012-03-19 15:02:40

Simple vba macro for opening utf-8 text and csv files

Sub OpenTextFile()

   filetoopen = Application.GetOpenFilename("Text Files (*.txt;*.csv), *.txt;*.csv")
   If filetoopen = Null Or filetoopen = Empty Then Exit Sub

   Workbooks.OpenText Filename:=filetoopen, _
   Origin:=65001, DataType:=xlDelimited, Comma:=True

End Sub

Origin:=65001 is UTF-8. Comma:True for .csv files distributed in colums

Save it in Personal.xlsb to have it always available. Personalise excel toolbar adding a macro call button and open files from there. You can add more formating to the macro, like column autofit , alignment,etc.

@Lyubomyr Shaydariv 2012-03-20 05:42:14

Thanks. Unfortunately, this is the suggestion for really advanced Excel users, and what I'm aiming for, asking this question, is making Excel do it itself automatically.

@Sharunas Bielskis 2016-12-07 15:57:35

I tried to use this code snippet but it doesn't work for me. I use Excel 2013.

@Bragabondio 2011-12-06 20:59:19

First save the Excel spreadsheet as Unicode text. Open the TXT file using Internet explorer and click "Save as" TXT Encoding - choose the appropriate encoding, i.e. for Win Cyrillic 1251

@Mark 2011-06-27 02:19:21

Alex is correct, but as you have to export to csv, you can give the users this advice when opening the csv files:

  1. Save the exported file as a csv
  2. Open Excel
  3. Import the data using Data-->Import External Data --> Import Data
  4. Select the file type of "csv" and browse to your file
  5. In the import wizard change the File_Origin to "65001 UTF" (or choose correct language character identifier)
  6. Change the Delimiter to comma
  7. Select where to import to and Finish

This way the special characters should show correctly.

@Lyubomyr Shaydariv 2011-06-27 09:01:03

Thank you for the answer, Mark. It also makes me think that no one cares to make CSV a bit smarter than it is... I guess you agree that in most cases BOMs would not affect the CSV import, or the importer module could even recognize such marks. But no one cares, unfortunately.

@Ted 2013-01-10 20:21:36

Thx! This was the way to go =)

@GhostInTheSecureShell 2013-06-28 13:36:52

You can then save your imported CSV file as an XLSX and Excel seems to maintain support for some of the more unusual characters. BTW I'm using Excel 2007.

@Kerridge0 2013-10-05 15:29:50

Didn't work for multi-line fields for me, had to use Notepad++ first and "encode -> encode in UTF-8" then save and it worked.

@Max Williams 2014-01-23 13:24:02

This is how to open a utf8 csv in excel. But i thought the question regarded making Excel open utf8 files as utf8 automatically without having to be told?

@Pacerier 2015-05-05 04:25:22

@Mark, How can we get it to work just by normal "double-click"? You can't expect users to do any importing whatsoever

@Muneeb Mirza 2015-08-19 10:04:48

Though the answer sure helped but Its 2015 and it is still like that. Hate to manually do it like that. There should be a way to automate it.

@Rasmus Bidstrup 2015-09-11 14:44:35

oh thanks! this helps so much

@user3640967 2015-10-08 10:03:58

This worked perfectly... until I tried to resave the CSV file and now all of the special characters are messed up again. I know it wasn't part of the question, but did anyone have any success trying to save these files??? Seems like yet another simple operation that the almighty Excel can't even get half right :'(

@Sun 2016-05-12 22:12:17

I would use Microsoft Access to create your Text Import Specification where you can set charset, delimiters, exclude columns, etc. This gets saved to a spec that is applied every time you read the CSV file. Then you reference the linked Access table instead of CSV.

@Siamak 2018-10-27 03:11:56

It didn't work. Excel opened it as a txt file. It didn't recognized double quotations.

Related Questions

Sponsored Content

24 Answered Questions

[SOLVED] Dealing with commas in a CSV file

  • 2009-04-20 18:52:33
  • Bob The Janitor
  • 489083 View
  • 440 Score
  • 24 Answer
  • Tags:   csv

36 Answered Questions

[SOLVED] Excel to CSV with UTF8 encoding

17 Answered Questions

[SOLVED] Save PL/pgSQL output from PostgreSQL to a CSV file

42 Answered Questions

[SOLVED] How to create Excel (.XLS and .XLSX) file in C# without installing Ms Office?

  • 2008-09-29 22:30:28
  • mistrmark
  • 997859 View
  • 1732 Score
  • 42 Answer
  • Tags:   c# .net excel file-io

34 Answered Questions

[SOLVED] Stop Excel from automatically converting certain text values to dates

  • 2008-10-02 23:30:43
  • user16324
  • 439009 View
  • 474 Score
  • 34 Answer
  • Tags:   excel csv import

0 Answered Questions

30 Answered Questions

[SOLVED] How can I output a UTF-8 CSV in PHP that Excel will read properly?

1 Answered Questions

[SOLVED] Opening CSV with UTF-8 BOM via Excel

  • 2013-11-28 22:12:45
  • me1111
  • 5628 View
  • 5 Score
  • 1 Answer
  • Tags:   java excel csv utf-8

0 Answered Questions

0 Answered Questions

how to convert Excel file to CSV and prevent UTF-8 encoding

Sponsored Content