By Matt


2008-09-23 19:23:48 8 Comments

How do I copy a file in Python?

I couldn't find anything under os.

16 comments

@Algebra 2018-01-16 03:25:13

Firstly, I made an exhaustive cheatsheet of shutil methods for your reference.

shutil_methods =
{'copy':['shutil.copyfileobj',
          'shutil.copyfile',
          'shutil.copymode',
          'shutil.copystat',
          'shutil.copy',
          'shutil.copy2',
          'shutil.copytree',],
 'move':['shutil.rmtree',
         'shutil.move',],
 'exception': ['exception shutil.SameFileError',
                 'exception shutil.Error'],
 'others':['shutil.disk_usage',
             'shutil.chown',
             'shutil.which',
             'shutil.ignore_patterns',]
}

Secondly, explain methods of copy in exmaples:

  1. shutil.copyfileobj(fsrc, fdst[, length]) manipulate opened objects
In [3]: src = '~/Documents/Head+First+SQL.pdf'
In [4]: dst = '~/desktop'
In [5]: shutil.copyfileobj(src, dst)
AttributeError: 'str' object has no attribute 'read'
#copy the file object
In [7]: with open(src, 'rb') as f1,open(os.path.join(dst,'test.pdf'), 'wb') as f2:
    ...:      shutil.copyfileobj(f1, f2)
In [8]: os.stat(os.path.join(dst,'test.pdf'))
Out[8]: os.stat_result(st_mode=33188, st_ino=8598319475, st_dev=16777220, st_nlink=1, st_uid=501, st_gid=20, st_size=13507926, st_atime=1516067347, st_mtime=1516067335, st_ctime=1516067345)
  1. shutil.copyfile(src, dst, *, follow_symlinks=True) Copy and rename
In [9]: shutil.copyfile(src, dst)
IsADirectoryError: [Errno 21] Is a directory: ~/desktop'
#so dst should be a filename instead of a directory name
  1. shutil.copy() Copy without preseving the metadata
In [10]: shutil.copy(src, dst)
Out[10]: ~/desktop/Head+First+SQL.pdf'
#check their metadata
In [25]: os.stat(src)
Out[25]: os.stat_result(st_mode=33188, st_ino=597749, st_dev=16777220, st_nlink=1, st_uid=501, st_gid=20, st_size=13507926, st_atime=1516066425, st_mtime=1493698739, st_ctime=1514871215)
In [26]: os.stat(os.path.join(dst, 'Head+First+SQL.pdf'))
Out[26]: os.stat_result(st_mode=33188, st_ino=8598313736, st_dev=16777220, st_nlink=1, st_uid=501, st_gid=20, st_size=13507926, st_atime=1516066427, st_mtime=1516066425, st_ctime=1516066425)
# st_atime,st_mtime,st_ctime changed
  1. shutil.copy2() Copy with preseving the metadata
In [30]: shutil.copy2(src, dst)
Out[30]: ~/desktop/Head+First+SQL.pdf'
In [31]: os.stat(src)
Out[31]: os.stat_result(st_mode=33188, st_ino=597749, st_dev=16777220, st_nlink=1, st_uid=501, st_gid=20, st_size=13507926, st_atime=1516067055, st_mtime=1493698739, st_ctime=1514871215)
In [32]: os.stat(os.path.join(dst, 'Head+First+SQL.pdf'))
Out[32]: os.stat_result(st_mode=33188, st_ino=8598313736, st_dev=16777220, st_nlink=1, st_uid=501, st_gid=20, st_size=13507926, st_atime=1516067063, st_mtime=1493698739, st_ctime=1516067055)
# Preseved st_mtime
  1. shutil.copytree()

Recursively copy an entire directory tree rooted at src, returning the destination directory

@Savai Maheshwari 2019-06-10 10:05:55

Python provides in-built functions for easily copying files using the Operating System Shell utilities.

Following command is used to Copy File

shutil.copy(src,dst)

Following command is used to Copy File with MetaData Information

shutil.copystat(src,dst)

@yellow01 2017-08-15 13:46:42

For small files and using only python built-ins, you can use the following one-liner:

with open(source, 'r') as src, open(dest, 'w') as dst: dst.write(src.read())

As @maxschlepzig mentioned in the comments below, this is not optimal way for applications where the file is too large or when memory is critical, thus Swati's answer should be preferred.

@maxschlepzig 2017-10-02 07:11:47

This reads the complete source file into memory before writing it back. Thus, this unnecessarily wastes memory for all but the smallest file copy operations.

@soundstripe 2017-11-17 01:39:52

Is that true? I think .read() and .write() are buffered by default (at least for CPython).

@maxschlepzig 2019-04-29 11:00:56

@soundstripe, Of course this is true. The fact that the file object returned by open() does buffered IO, by default doesn't help you here, because read() is specified as: 'If n is negative or omitted, read until EOF.' That means that the read() returns the complete file content as a string.

@yellow01 2019-04-29 18:36:08

@maxschlepzig I get your point and I admit I wasn't aware of it. The reason I provided this answer was in case someone wanted to do a simple file copy using only built-ins, without needing to import a module for it. Of course memory optimization should not be a concern if you want this option. Anyways thank you for clearing that out. I updated the answer accordingly.

@S471 2019-03-23 00:46:19

open(destination, 'wb').write(open(source, 'rb').read())

Open the source file in read mode, and write to destination file in write mode.

@Raúl Salinas-Monteagudo 2019-03-29 11:06:57

The idea is nice and the code is beautiful, but a proper copy() function can do more things, such as copying attributes (+x bit), or for example deleting the already-copied bytes in case a disk-full condition is found.

@connectyourcharger 2019-03-30 22:19:25

All answers need explanation, even if it is one sentence. No explanation sets bad precedent and is not helpful in understanding the program. What if a complete Python noob came along and saw this, wanted to use it, but couldn't because they don't understand it? You want to be helpful to all in your answers.

@luckydonald 2019-04-22 22:37:33

Isn't that missing the .close() on all of those open(...)s?

@S471 2019-04-23 21:19:20

No need of .close(), as we are NOT STORING the file pointer object anywhere(neither for the src file nor for the destination file).

@Marcel Waldvogel 2019-04-29 03:17:21

AFAIK, it is undefined when the files are actually closed, @SundeepBorra. Using with (as in the example above) is recommended and not more complicated. Using read() on a raw file reads the entire file into memory, which may be too big. Use a standard function like from shutil so that you and whoever else is involved in the code does not need to worry about special cases. docs.python.org/3/library/io.html#io.BufferedReader

@maxschlepzig 2019-04-29 11:04:38

Same suboptimal memory-wasting approach as yellow01's answer.

@Marc 2019-04-25 14:09:14

As of Python 3.5 you can do the following for small files (ie: text files, small jpegs):

from pathlib import Path

source = Path('../path/to/my/file.txt')
destination = Path('../path/where/i/want/to/store/it.txt')
destination.write_bytes(source.read_bytes())

write_bytes will overwrite whatever was at the destination's location

@Marcel Waldvogel 2019-04-29 03:21:10

And then someone uses the code (accidentally or purposefully) on a large file… Using functions from shutil handles all the special cases for you and gives you peace of mind.

@Jean-François Fabre 2019-04-29 18:16:27

at least it doesn't repeat the same solutions over and over again.

@Swati 2008-09-23 19:25:35

shutil has many methods you can use. One of which is:

from shutil import copyfile

copyfile(src, dst)

Copy the contents of the file named src to a file named dst. The destination location must be writable; otherwise, an IOError exception will be raised. If dst already exists, it will be replaced. Special files such as character or block devices and pipes cannot be copied with this function. src and dst are path names given as strings.

@Matt 2008-09-23 19:47:01

What is the difference between copy and copyfile?

@Owen 2008-09-23 19:51:07

in copy(src, dst) the dst can be a directory.

@Kevin Horn 2009-10-19 20:50:27

Note that not all metadata will be copied, depending on your platform.

@Waterbyte 2018-10-22 11:53:51

Note that it is not an atomic operation. Take care using it in a threaded application.

@zwep 2018-11-27 10:00:06

Note that it cant handle abbreviations like ~, but it can deal with relative paths

@Jack M 2018-12-25 14:34:37

@Owen Indeed it can, but the directory has to already exist. By the way, as long as the target directory already exists, dst can either end or not end with a trailing slash, it doesn't matter.

@Alexei Martianov 2019-01-03 05:37:11

To create all intermediate-level destination directories you could use os.makedirs() before copying, details here stackoverflow.com/questions/12842997/…

@jezrael 2015-05-20 20:01:48

┌──────────────────┬───────────────┬──────────────────┬──────────────┬───────────┐
│     Function     │Copies metadata│Copies permissions│Can use buffer│Dest dir OK│
├──────────────────┼───────────────┼──────────────────┼──────────────┼───────────┤
│shutil.copy       │      No       │        Yes       │    No        │    Yes    │
│shutil.copyfile   │      No       │        No        │    No        │    No     │
│shutil.copy2      │      Yes      │        Yes       │    No        │    Yes    │
│shutil.copyfileobj│      No       │        No        │    Yes       │    No     │
└──────────────────┴───────────────┴──────────────────┴──────────────┴───────────┘

@Conchylicultor 2016-10-10 18:47:58

Could also add that copy and copy2 accept directory as destination contrary to copyfile which require to specify the complete name.

@Conchylicultor 2017-02-01 21:48:44

@jezrael Extracted from the doc: dst must be the complete target file name; look at shutil.copy() for a copy that accepts a target directory path. (here: docs.python.org/3/library/shutil.html#shutil.copyfile)

@One Man Monkey Squad 2019-01-16 13:51:32

Now this is just bad, bad API design.

@Katie 2019-02-07 22:00:55

What does the "Dest dir OK" column indicate? Does it mean it doesn't check if the destination directory is OK before copying?

@FabioSpaghetti 2019-02-08 14:29:51

According to your table the copyfile is totally useless !! while in the previous answer with 2115 votes, it's the first suggested !! and also this answer has got votes !! it's confusing !!!!

@Nathaniel Jones 2019-04-08 00:13:55

@KatieS "Dest dir OK" indicates whether it's okay to specify a directory as the destination, as opposed to a file. From the docs on shutil.copy(src, dst, *, follow_symlinks=True) (also applies to copy2): "If dst specifies a directory, the file will be copied into dst using the base filename from src."

@Nathaniel Jones 2019-04-08 00:28:47

@FabioSpaghetti On the contrary: according to the table, copyfile only provides a basic set of functionality. For extra features, use one of the other three. The table organizes what is otherwise a somewhat disorganized API.

@secsilm 2019-08-30 09:43:02

According to the newest doc, copy2 may not keep meta info. Warning Even the higher-level file copying functions (shutil.copy(), shutil.copy2()) cannot copy all file metadata.

@maxschlepzig 2017-07-09 11:50:12

You can use one of the copy functions from the shutil package:

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Function              preserves     supports          accepts     copies other
                      permissions   directory dest.   file obj    metadata  
――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
shutil.copy              ✔             ✔                 ☐           ☐
shutil.copy2             ✔             ✔                 ☐           ✔
shutil.copyfile          ☐             ☐                 ☐           ☐
shutil.copyfileobj       ☐             ☐                 ✔           ☐
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Example:

import shutil
shutil.copy('/etc/hostname', '/var/tmp/testhostname')

@lightalchemist 2018-12-02 07:40:25

Just curious, how did you generate that table?

@maxschlepzig 2018-12-02 17:02:38

@lightalchemist I just used vim as a scratchpad, copied the used unicode symbols from a wikipedia table and copied the result into the stackoverflow editor for final polishing.

@pi. 2008-09-24 07:21:12

Copying a file is a relatively straightforward operation as shown by the examples below, but you should instead use the shutil stdlib module for that.

def copyfileobj_example(source, dest, buffer_size=1024*1024):
    """      
    Copy a file from source to dest. source and dest
    must be file-like objects, i.e. any object with a read or
    write method, like for example StringIO.
    """
    while True:
        copy_buffer = source.read(buffer_size)
        if not copy_buffer:
            break
        dest.write(copy_buffer)

If you want to copy by filename you could do something like this:

def copyfile_example(source, dest):
    # Beware, this example does not handle any edge cases!
    with open(source, 'rb') as src, open(dest, 'wb') as dst:
        copyfileobj_example(src, dst)

@pi. 2009-03-31 15:20:03

I noticed a while ago that the module is called shutil (singular) and not shutils (plural), and indeed it is in Python 2.3. Nevertheless I leave this function here as an example.

@LaC 2012-01-16 18:01:40

Copying a file's contents is a straightforward operation. Copying the file with its metadata is anything but straightforward, even more so if you want to be cross-platform.

@pi. 2012-01-17 15:08:42

True. Looking at the shutil docs, the copyfile function also won't copy metadata.

@Haroldo_OK 2014-08-19 13:14:47

BTW, if you want to copy the contents between two file-like objects, there's shutil.copyfileobj(fsrc, fdst)

@ErlVolton 2014-10-28 18:48:48

Yes, I'm not sure why you wouldn't just copy the source of shutil.copyfileobj. Also, you don't have any try, finally to handle closing the files after exceptions. I would say however, that your function shouldn't be responsible for opening and closing the files at all. That should go in a wrapper function, like how shutil.copyfile wraps shutil.copyfileobj.

@user1016274 2015-05-28 14:56:00

The above code should specify dest to be writeable: open(dest, 'wb')

@Marcel Waldvogel 2019-04-29 03:31:30

@pi., for copying metadata, we have shutil.copy() and shutil.copy2(). stackoverflow.com/a/44996087/2445204

@kmario23 2018-01-22 03:05:32

In Python, you can copy the files using


import os
import shutil
import subprocess

1) Copying files using shutil module

shutil.copyfile signature

shutil.copyfile(src_file, dest_file, *, follow_symlinks=True)

# example    
shutil.copyfile('source.txt', 'destination.txt')

shutil.copy signature

shutil.copy(src_file, dest_file, *, follow_symlinks=True)

# example
shutil.copy('source.txt', 'destination.txt')

shutil.copy2 signature

shutil.copy2(src_file, dest_file, *, follow_symlinks=True)

# example
shutil.copy2('source.txt', 'destination.txt')  

shutil.copyfileobj signature

shutil.copyfileobj(src_file_object, dest_file_object[, length])

# example
file_src = 'source.txt'  
f_src = open(file_src, 'rb')

file_dest = 'destination.txt'  
f_dest = open(file_dest, 'wb')

shutil.copyfileobj(f_src, f_dest)  

2) Copying files using os module

os.popen signature

os.popen(cmd[, mode[, bufsize]])

# example
# In Unix/Linux
os.popen('cp source.txt destination.txt') 

# In Windows
os.popen('copy source.txt destination.txt')

os.system signature

os.system(command)


# In Linux/Unix
os.system('cp source.txt destination.txt')  

# In Windows
os.system('copy source.txt destination.txt')

3) Copying files using subprocess module

subprocess.call signature

subprocess.call(args, *, stdin=None, stdout=None, stderr=None, shell=False)

# example (WARNING: setting `shell=True` might be a security-risk)
# In Linux/Unix
status = subprocess.call('cp source.txt destination.txt', shell=True) 

# In Windows
status = subprocess.call('copy source.txt destination.txt', shell=True)

subprocess.check_output signature

subprocess.check_output(args, *, stdin=None, stderr=None, shell=False, universal_newlines=False)

# example (WARNING: setting `shell=True` might be a security-risk)
# In Linux/Unix
status = subprocess.check_output('cp source.txt destination.txt', shell=True)

# In Windows
status = subprocess.check_output('copy source.txt destination.txt', shell=True)

@Marcel Waldvogel 2019-04-29 03:03:26

Using single-string commands is bad coding style (flexibility, reliability and security), instead use ['copy', sourcefile, destfile] syntax wherever possible, especially if the parameters are from user input.

@maxschlepzig 2019-04-29 11:08:39

Why do you list so many bad alternatives to the shutil copy functions?

@Jean-François Fabre 2019-04-29 17:19:34

shutil is built-in, no need to provide non-portable alternatives. The answer could be actually improved by removing the system dependent solutions, and after that removal, this answer is just a copy of the existing answers / a copy of the documentation.

@Jean-François Fabre 2019-04-29 18:12:11

os.popen is deprecated for a while now. and check_output doesn't return the status but the output (which is empty in the case of copy/cp)

@unmounted 2008-09-23 19:29:41

copy2(src,dst) is often more useful than copyfile(src,dst) because:

  • it allows dst to be a directory (instead of the complete target filename), in which case the basename of src is used for creating the new file;
  • it preserves the original modification and access info (mtime and atime) in the file metadata (however, this comes with a slight overhead).

Here is a short example:

import shutil
shutil.copy2('/src/dir/file.ext', '/dst/dir/newname.ext') # complete target filename given
shutil.copy2('/src/file.ext', '/dst/dir') # target filename is /dst/dir/file.ext

@Vijay 2014-05-07 08:31:38

I am trying to randomly copy 100k files from 1 million files. copyfile is considerably faster than copy2

@Zak 2015-08-19 18:56:38

am I correct to assume that shutil.copy2('/dir/file.ext', '/new/dir/') (with slash after the target path) will remove the ambiguity over whether to copy to a new file called "dir" or to put the file into a directory of that name?

@Sheljohn 2017-03-13 17:57:30

@Vijay I believe this overhead is due to copying the metadata.

@Sheljohn 2017-03-13 18:14:31

@Zak There is no ambiguity if /new/dir is an existing directory, see @MatthewAlpert's comment.

@martonbognar 2018-06-07 12:00:25

@Zak You are correct, adding a slash to the end removes the ambiguity. If /new/dir/ does not exist, Python will throw an IsADirectoryError, otherwise it copies the file to /new/dir/ under the original name.

@deepdive 2016-04-04 07:08:24

from subprocess import call
call("cp -p <file> <file>", shell=True)

@Kevin Meier 2016-09-13 10:02:06

This depends on the platform, so i would not use is.

@buhtz 2017-04-02 07:57:45

Such a call is unsecure. Please refere to the subproces docu about it.

@Corey Goldberg 2017-06-12 14:05:24

this is not portable, and unnecessary since you can just use shutil.

@Baris Demiray 2017-07-07 09:29:27

Hmm why Python, then?

@MilkyWay90 2018-11-09 15:51:05

Maybe detect the operating system before starting (whether it's DOS or Unix, because those are the two most used)

@ytpillai 2015-05-25 05:11:47

For large files, what I did was read the file line by line and read each line into an array. Then, once the array reached a certain size, append it to a new file.

for line in open("file.txt", "r"):
    list.append(line)
    if len(list) == 1000000: 
        output.writelines(list)
        del list[:]

@owns 2015-06-12 17:30:16

this seems a little redundant since the writer should handle buffering. for l in open('file.txt','r'): output.write(l) should work find; just setup the output stream buffer to your needs. or you can go by the bytes by looping over a try with output.write(read(n)); output.flush() where n is the number of bytes you'd like to write at a time. both of these also don't have an condition to check which is a bonus.

@ytpillai 2015-06-13 18:42:08

Yes, but I thought that maybe this could be easier to understand because it copies entire lines rather than parts of them (in case we don't know how many bytes each line is).

@owns 2015-06-15 17:47:45

Very true. Coding for teaching and coding for efficiency are very different.

@ytpillai 2016-11-30 03:33:10

@owns To add to this question a year later, writelines() has shown slightly better performance over write() since we don't waste time consistently opening a new filestream, and instead write new lines as one large bytefeed.

@owns 2017-05-03 00:24:46

looking at the source - writelines calls write, hg.python.org/cpython/file/c6880edaf6f3/Modules/_io/bytesio.‌​c. Also, the file stream is already open, so write wouldn't need to reopen it every time.

@maxschlepzig 2019-04-29 19:04:40

This is awful. It does unnecessary work for no good reason. It doesn't work for arbitrary files. The copy isn't byte-identical if the input has unusual line endings on systems like Windows. Why do you think that this might be easier to understand than a call to a copy function in shutil? Even when ignoring shutil, a simple block read/write loop (using unbuffered IO) is straight forward, would be efficient and would make much more sense than this, and thus is surely easier to teach and understand.

@mark 2014-12-19 23:18:02

You could use os.system('cp nameoffilegeneratedbyprogram /otherdirectory/')

or as I did it,

os.system('cp '+ rawfile + ' rawdata.dat')

where rawfile is the name that I had generated inside the program.

This is a Linux only solution

@Corey Goldberg 2017-06-12 14:05:17

this is not portable, and unnecessary since you can just use shutil.

@maxschlepzig 2017-07-09 11:52:44

Even when shutil is not available - subprocess.run() (without shell=True!) is the better alternative to os.system().

@Hiadore 2019-03-12 09:07:45

shutil is more portable

@Marcel Waldvogel 2019-04-29 03:09:48

subprocess.run() as suggested by @maxschlepzig is a big step forward, when calling external programs. For flexibility and security however, use the ['cp', rawfile, 'rawdata.dat'] form of passing the command line. (However, for copying, shutil and friends are recommended over calling an external program.)

@Jean-François Fabre 2019-04-29 17:21:58

try that with filenames with spaces in it.

@Airsource Ltd 2008-09-23 19:27:23

Use the shutil module.

copyfile(src, dst)

Copy the contents of the file named src to a file named dst. The destination location must be writable; otherwise, an IOError exception will be raised. If dst already exists, it will be replaced. Special files such as character or block devices and pipes cannot be copied with this function. src and dst are path names given as strings.

Take a look at filesys for all the file and directory handling functions available in standard Python modules.

@Noam Manos 2011-03-15 10:11:21

Directory and File copy example - From Tim Golden's Python Stuff:

http://timgolden.me.uk/python/win32_how_do_i/copy-a-file.html

import os
import shutil
import tempfile

filename1 = tempfile.mktemp (".txt")
open (filename1, "w").close ()
filename2 = filename1 + ".copy"
print filename1, "=>", filename2

shutil.copy (filename1, filename2)

if os.path.isfile (filename2): print "Success"

dirname1 = tempfile.mktemp (".dir")
os.mkdir (dirname1)
dirname2 = dirname1 + ".copy"
print dirname1, "=>", dirname2

shutil.copytree (dirname1, dirname2)

if os.path.isdir (dirname2): print "Success"

Related Questions

Sponsored Content

11 Answered Questions

[SOLVED] What is the difference between the 'COPY' and 'ADD' commands in a Dockerfile?

  • 2014-07-25 14:31:20
  • Steve
  • 581582 View
  • 1918 Score
  • 11 Answer
  • Tags:   docker dockerfile

60 Answered Questions

[SOLVED] Calling an external command in Python

28 Answered Questions

[SOLVED] How to read a file line-by-line into a list?

15 Answered Questions

[SOLVED] How to clone or copy a list?

12 Answered Questions

[SOLVED] How can I make a time delay in Python?

22 Answered Questions

[SOLVED] How do I list all files of a directory?

  • 2010-07-08 19:31:22
  • duhhunjonn
  • 3527135 View
  • 3474 Score
  • 22 Answer
  • Tags:   python directory

25 Answered Questions

[SOLVED] How can I safely create a nested directory?

37 Answered Questions

[SOLVED] How do I check whether a file exists without exceptions?

57 Answered Questions

[SOLVED] How do I include a JavaScript file in another JavaScript file?

11 Answered Questions

[SOLVED] How to copy a folder from remote to local using scp?

Sponsored Content