By Parand


2008-11-07 18:56:45 8 Comments

What is the most elegant way to check if the directory a file is going to be written to exists, and if not, create the directory using Python? Here is what I tried:

import os

file_path = "/my/directory/filename.txt"
directory = os.path.dirname(file_path)

try:
    os.stat(directory)
except:
    os.mkdir(directory)       

f = file(filename)

Somehow, I missed os.path.exists (thanks kanja, Blair, and Douglas). This is what I have now:

def ensure_dir(file_path):
    directory = os.path.dirname(file_path)
    if not os.path.exists(directory):
        os.makedirs(directory)

Is there a flag for "open", that makes this happen automatically?

25 comments

@Hussam Kurd 2019-05-19 00:32:23

You have to set the full path before creating the directory:

import os,sys,inspect
import pathlib

currentdir = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
your_folder = currentdir + "/" + "your_folder"

if not os.path.exists(your_folder):
   pathlib.Path(your_folder).mkdir(parents=True, exist_ok=True)

This works for me and hopefully, it will works for you as well

@Geoff Paul Bremner 2018-09-11 18:23:57

Why not use subprocess module if running on a machine that supports command mkdir with -p option ? Works on python 2.7 and python 3.6

from subprocess import call
call(['mkdir', '-p', 'path1/path2/path3'])

Should do the trick on most systems.

In situations where portability doesn't matter (ex, using docker) the solution is a clean 2 lines. You also don't have to add logic to check if directories exist or not. Finally, it is safe to re-run without any side effects

If you need error handling:

from subprocess import check_call
try:
    check_call(['mkdir', '-p', 'path1/path2/path3'])
except:
    handle...

@Manivannan Murugavel 2018-04-16 07:30:10

Use this command check and create dir

 if not os.path.isdir(test_img_dir):
     os.mkdir(test_img_dir)

@tashuhka 2016-03-29 15:50:53

For a one-liner solution, you can use IPython.utils.path.ensure_dir_exists():

from IPython.utils.path import ensure_dir_exists
ensure_dir_exists(dir)

From the documentation: Ensure that a directory exists. If it doesn’t exist, try to create it and protect against a race condition if another process is doing the same.

@jkdev 2016-10-05 03:33:40

New IPython documentation available here.

@Acumenus 2016-10-21 04:06:29

The IPython module is absolutely not guaranteed to be present. It is natively present on my Mac, but not on any of my Linux installs of Python. Basically, it is not one of the modules listed in the Python Module Index.

@tashuhka 2016-10-21 08:44:00

Sure. In order to install the package, just run the usual pip install ipython or include the dependency in your requirements.txt or pom.xml. Documentation: ipython.org/install.html

@Douglas Mayle 2008-11-07 19:01:25

Check os.makedirs: (It makes sure the complete path exists.)
To handle the fact the directory might exist, catch OSError. (If exist_ok is False (the default), an OSError is raised if the target directory already exists.)

import os
try:
    os.makedirs('./path/to/somewhere')
except OSError:
    pass

@Blair Conrad 2008-11-07 19:09:55

with the try/except, you will mask errors in directory creation, in the case when the directory didn't exist but for some reason you can't make it

@Acumenus 2013-01-16 17:33:50

OSError will be raised here if the path is an existing file or directory. I have posted an answer to address this.

@Chris Johnson 2015-10-14 21:31:51

This is halfway there. You do need to check the sub-error condition of OSError before deciding to ignore it. See stackoverflow.com/a/5032238/763269.

@Blair Conrad 2008-11-07 19:06:07

I see two answers with good qualities, each with a small flaw, so I will give my take on it:

Try os.path.exists, and consider os.makedirs for the creation.

import os
if not os.path.exists(directory):
    os.makedirs(directory)

As noted in comments and elsewhere, there's a race condition – if the directory is created between the os.path.exists and the os.makedirs calls, the os.makedirs will fail with an OSError. Unfortunately, blanket-catching OSError and continuing is not foolproof, as it will ignore a failure to create the directory due to other factors, such as insufficient permissions, full disk, etc.

One option would be to trap the OSError and examine the embedded error code (see Is there a cross-platform way of getting information from Python’s OSError):

import os, errno

try:
    os.makedirs(directory)
except OSError as e:
    if e.errno != errno.EEXIST:
        raise

Alternatively, there could be a second os.path.exists, but suppose another created the directory after the first check, then removed it before the second one – we could still be fooled.

Depending on the application, the danger of concurrent operations may be more or less than the danger posed by other factors such as file permissions. The developer would have to know more about the particular application being developed and its expected environment before choosing an implementation.

Modern versions of Python improve this code quite a bit, both by exposing FileExistsError (in 3.3+)...

try:
    os.makedirs("path/to/directory")
except FileExistsError:
    # directory already exists
    pass

...and by allowing a keyword argument to os.makedirs called exist_ok (in 3.2+).

os.makedirs("path/to/directory", exist_ok=True)  # succeeds even if directory exists.

@Blair Conrad 2008-11-07 20:35:41

The race condition is a good point, but the approach in stackoverflow.com/questions/273192/#273208, will mask a failure to create the directory. Don't feel bad for voting down - you don't like the answer. It's what votes are for.

@Andrew 2011-11-28 19:10:39

Remember that os.path.exists() isn't free. If the normal case is that the directory will be there, then the case where it isn't should be handled as an exception. In other words, try to open and write to your file, catch the OSError exception and, based on errno, do your makedir() and re-try or re-raise. This creates duplication of code unless you wrap the writing in a local method.

@Acumenus 2013-02-14 17:32:59

os.path.exists also returns True for a file. I have posted an answer to address this.

@Bobble 2013-05-20 06:50:40

As commenters to other answers here have noted, the exists_ok parameter to os.makedirs() can be used to cover how prior existence of the path is handled, since Python 3.2.

@drevicko 2013-07-06 06:41:48

os.mkdirs() can create unintended folders if a path separator is accidentally left out, the current folder is not as expected, a path element contains the path separator. If you use os.mkdir() these bugs will raise an exception, alerting you to their existence.

@Chris Krycho 2014-11-04 01:28:27

It might be useful to add the pathlib.Path::is_dir and pathlib.Path::mkdir functions to this answer. The pathlib module, added on a provisional basis to the standard library in 3.4, provides an object-oriented, system-agnostic approach to path handling. This includes handling for path separators (via overloading /). The package is available for installation via pip to earlier versions as well.

@Purrell 2015-02-27 00:48:31

too bad this is marked as correct and upvoted to 1000. You can, and should catch errno.EEXIST

@user3547021 2019-08-30 15:58:25

you can check if os.path.exists returns false after os.makedirs fails, and if the path does not exists, it is clear that it has not been created in the meanwhile, so it is a privilege issue

@hiro protagonist 2016-12-14 16:06:09

Starting from Python 3.5, pathlib.Path.mkdir has an exist_ok flag:

from pathlib import Path
path = Path('/my/directory/filename.txt')
path.parent.mkdir(parents=True, exist_ok=True) 
# path.parent ~ os.path.dirname(path)

This recursively creates the directory and does not raise an exception if the directory already exists.

(just as os.makedirs got an exist_ok flag starting from python 3.2 e.g os.makedirs(path, exist_ok=True))

@Acumenus 2013-01-16 17:31:19

Python 3.5+:

import pathlib
pathlib.Path('/my/directory').mkdir(parents=True, exist_ok=True) 

pathlib.Path.mkdir as used above recursively creates the directory and does not raise an exception if the directory already exists. If you don't need or want the parents to be created, skip the parents argument.

Python 3.2+:

Using pathlib:

If you can, install the current pathlib backport named pathlib2. Do not install the older unmaintained backport named pathlib. Next, refer to the Python 3.5+ section above and use it the same.

If using Python 3.4, even though it comes with pathlib, it is missing the useful exist_ok option. The backport is intended to offer a newer and superior implementation of mkdir which includes this missing option.

Using os:

import os
os.makedirs(path, exist_ok=True)

os.makedirs as used above recursively creates the directory and does not raise an exception if the directory already exists. It has the optional exist_ok argument only if using Python 3.2+, with a default value of False. This argument does not exist in Python 2.x up to 2.7. As such, there is no need for manual exception handling as with Python 2.7.

Python 2.7+:

Using pathlib:

If you can, install the current pathlib backport named pathlib2. Do not install the older unmaintained backport named pathlib. Next, refer to the Python 3.5+ section above and use it the same.

Using os:

import os
try: 
    os.makedirs(path)
except OSError:
    if not os.path.isdir(path):
        raise

While a naive solution may first use os.path.isdir followed by os.makedirs, the solution above reverses the order of the two operations. In doing so, it prevents a common race condition having to do with a duplicated attempt at creating the directory, and also disambiguates files from directories.

Note that capturing the exception and using errno is of limited usefulness because OSError: [Errno 17] File exists, i.e. errno.EEXIST, is raised for both files and directories. It is more reliable simply to check if the directory exists.

Alternative:

mkpath creates the nested directory, and does nothing if the directory already exists. This works in both Python 2 and 3.

import distutils.dir_util
distutils.dir_util.mkpath(path)

Per Bug 10948, a severe limitation of this alternative is that it works only once per python process for a given path. In other words, if you use it to create a directory, then delete the directory from inside or outside Python, then use mkpath again to recreate the same directory, mkpath will simply silently use its invalid cached info of having previously created the directory, and will not actually make the directory again. In contrast, os.makedirs doesn't rely on any such cache. This limitation may be okay for some applications.


With regard to the directory's mode, please refer to the documentation if you care about it.

@Charles L. 2013-04-26 05:52:17

This answer covers pretty much every special case as far as I can tell. I plan on wrapping this in a "if not os.path.isdir()" though since I expect the directory to exist almost every time and I can avoid the exception that way.

@jpmc26 2014-04-29 22:39:07

@CharlesL. An exception is probably cheaper than the disk IO of the check, if your reason is performance.

@kwarunek 2014-09-19 10:31:21

@jpmc26 but makedirs does additional stat, umask, lstat when only checking to throw OSError.

@penguin359 2015-12-30 21:54:49

@CharesL Anytime there's a directory separator, os.makedirs() starts off with a stat() so a pre-check with os.path.isdir() is just extra effort.

@sleepycal 2016-01-08 15:20:49

This is the wrong answer, as it introduces a potential FS race cond. See answer from Aaron Hall.

@farmir 2016-04-27 07:20:54

as @sleepycal has said, this suffers from a similar race condition as the accepted answer. If between raising the error and checking os.path.isdir someone else deletes the folder, you will raise the wrong, outdated, and confusing error that folder exists.

@nealmcb 2017-10-22 16:21:27

The concern about race conditions noted by @sleepycal is addressed in the comments on Heikki Toivonen's answer for many (most) use cases, including mine. If you're worried that someone else is going to remove the directory, you're still going to need to check that before using it later on. Creating multiple places where you check for what you'll need later just adds complications.

@smci 2019-01-08 09:08:00

On Windows, what about permissions errors when parent directory is owned by Administrator, or else a special directory e.g. C:\Program Files or ...(x86) or Documents and Settings, and hence creating directory fails? Does this code propagate the exact permissions error?

@Acumenus 2019-01-08 17:13:50

@smci It doesn't seem so, but at least on Linux the available permissions can then be checked. I don't know about Windows.

@fitzl 2019-01-09 06:03:48

Note that NotADirectoryError: [Errno 20] Not a directory: ... is raised by pathlib.Path('/my/directory').mkdir(parents=True, exist_ok=True) in cases where a file exists matching the proposed name of a directory in the path, e.g., if my is a file.

@Steffi Keran Rani J 2018-04-28 16:00:40

Call the function create_dir() at the entry point of your program/project.

import os

def create_dir(directory):
    if not os.path.exists(directory):
        print('Creating Directory '+directory)
        os.makedirs(directory)

create_dir('Project directory')

@Michael Strobel 2017-02-09 03:56:05

I use os.path.exists(), here is a Python 3 script that can be used to check if a directory exists, create one if it does not exist, and delete it if it does exist (if desired).

It prompts users for input of the directory and can be easily modified.

@alissonmuller 2016-02-08 04:08:06

I saw Heikki Toivonen and A-B-B's answers and thought of this variation.

import os
import errno

def make_sure_path_exists(path):
    try:
        os.makedirs(path)
    except OSError as exception:
        if exception.errno != errno.EEXIST or not os.path.isdir(path):
            raise

@kavadias 2014-07-14 15:29:07

The relevant Python documentation suggests the use of the EAFP coding style (Easier to Ask for Forgiveness than Permission). This means that the code

try:
    os.makedirs(path)
except OSError as exception:
    if exception.errno != errno.EEXIST:
        raise
    else:
        print "\nBE CAREFUL! Directory %s already exists." % path

is better than the alternative

if not os.path.exists(path):
    os.makedirs(path)
else:
    print "\nBE CAREFUL! Directory %s already exists." % path

The documentation suggests this exactly because of the race condition discussed in this question. In addition, as others mention here, there is a performance advantage in querying once instead of twice the OS. Finally, the argument placed forward, potentially, in favour of the second code in some cases --when the developer knows the environment the application is running-- can only be advocated in the special case that the program has set up a private environment for itself (and other instances of the same program).

Even in that case, this is a bad practice and can lead to long useless debugging. For example, the fact we set the permissions for a directory should not leave us with the impression permissions are set appropriately for our purposes. A parent directory could be mounted with other permissions. In general, a program should always work correctly and the programmer should not expect one specific environment.

@Victoria Stuart 2017-12-16 04:16:07

I found this Q/A and I was initially puzzled by some of the failures and errors I was getting. I am working in Python 3 (v.3.5 in an Anaconda virtual environment on an Arch Linux x86_64 system).

Consider this directory structure:

└── output/         ## dir
   ├── corpus       ## file
   ├── corpus2/     ## dir
   └── subdir/      ## dir

Here are my experiments/notes, which clarifies things:

# ----------------------------------------------------------------------------
# [1] https://stackoverflow.com/questions/273192/how-can-i-create-a-directory-if-it-does-not-exist

import pathlib

""" Notes:
        1.  Include a trailing slash at the end of the directory path
            ("Method 1," below).
        2.  If a subdirectory in your intended path matches an existing file
            with same name, you will get the following error:
            "NotADirectoryError: [Errno 20] Not a directory:" ...
"""
# Uncomment and try each of these "out_dir" paths, singly:

# ----------------------------------------------------------------------------
# METHOD 1:
# Re-running does not overwrite existing directories and files; no errors.

# out_dir = 'output/corpus3'                ## no error but no dir created (missing tailing /)
# out_dir = 'output/corpus3/'               ## works
# out_dir = 'output/corpus3/doc1'           ## no error but no dir created (missing tailing /)
# out_dir = 'output/corpus3/doc1/'          ## works
# out_dir = 'output/corpus3/doc1/doc.txt'   ## no error but no file created (os.makedirs creates dir, not files!  ;-)
# out_dir = 'output/corpus2/tfidf/'         ## fails with "Errno 20" (existing file named "corpus2")
# out_dir = 'output/corpus3/tfidf/'         ## works
# out_dir = 'output/corpus3/a/b/c/d/'       ## works

# [2] https://docs.python.org/3/library/os.html#os.makedirs

# Uncomment these to run "Method 1":

#directory = os.path.dirname(out_dir)
#os.makedirs(directory, mode=0o777, exist_ok=True)

# ----------------------------------------------------------------------------
# METHOD 2:
# Re-running does not overwrite existing directories and files; no errors.

# out_dir = 'output/corpus3'                ## works
# out_dir = 'output/corpus3/'               ## works
# out_dir = 'output/corpus3/doc1'           ## works
# out_dir = 'output/corpus3/doc1/'          ## works
# out_dir = 'output/corpus3/doc1/doc.txt'   ## no error but creates a .../doc.txt./ dir
# out_dir = 'output/corpus2/tfidf/'         ## fails with "Errno 20" (existing file named "corpus2")
# out_dir = 'output/corpus3/tfidf/'         ## works
# out_dir = 'output/corpus3/a/b/c/d/'       ## works

# Uncomment these to run "Method 2":

#import os, errno
#try:
#       os.makedirs(out_dir)
#except OSError as e:
#       if e.errno != errno.EEXIST:
#               raise
# ----------------------------------------------------------------------------

Conclusion: in my opinion, "Method 2" is more robust.

[1] How can I create a directory if it does not exist?

[2] https://docs.python.org/3/library/os.html#os.makedirs

@iPhynx 2016-06-08 12:52:46

You can use os.listdir for this:

import os
if 'dirName' in os.listdir('parentFolderPath')
    print('Directory Exists')

@Georgy 2019-05-16 12:28:35

This doesn't answer the question

@Aaron Hall 2015-01-22 23:45:59

Check if a directory exists and create it if necessary?

The direct answer to this is, assuming a simple situation where you don't expect other users or processes to be messing with your directory:

if not os.path.exists(d):
    os.makedirs(d)

or if making the directory is subject to race conditions (i.e. if after checking the path exists, something else may have already made it) do this:

import errno
try:
    os.makedirs(d)
except OSError as exception:
    if exception.errno != errno.EEXIST:
        raise

But perhaps an even better approach is to sidestep the resource contention issue, by using temporary directories via tempfile:

import tempfile

d = tempfile.mkdtemp()

Here's the essentials from the online doc:

mkdtemp(suffix='', prefix='tmp', dir=None)
    User-callable function to create and return a unique temporary
    directory.  The return value is the pathname of the directory.

    The directory is readable, writable, and searchable only by the
    creating user.

    Caller is responsible for deleting the directory when done with it.

New in Python 3.5: pathlib.Path with exist_ok

There's a new Path object (as of 3.4) with lots of methods one would want to use with paths - one of which is mkdir.

(For context, I'm tracking my weekly rep with a script. Here's the relevant parts of code from the script that allow me to avoid hitting Stack Overflow more than once a day for the same data.)

First the relevant imports:

from pathlib import Path
import tempfile

We don't have to deal with os.path.join now - just join path parts with a /:

directory = Path(tempfile.gettempdir()) / 'sodata'

Then I idempotently ensure the directory exists - the exist_ok argument shows up in Python 3.5:

directory.mkdir(exist_ok=True)

Here's the relevant part of the documentation:

If exist_ok is true, FileExistsError exceptions will be ignored (same behavior as the POSIX mkdir -p command), but only if the last path component is not an existing non-directory file.

Here's a little more of the script - in my case, I'm not subject to a race condition, I only have one process that expects the directory (or contained files) to be there, and I don't have anything trying to remove the directory.

todays_file = directory / str(datetime.datetime.utcnow().date())
if todays_file.exists():
    logger.info("todays_file exists: " + str(todays_file))
    df = pd.read_json(str(todays_file))

Path objects have to be coerced to str before other APIs that expect str paths can use them.

Perhaps Pandas should be updated to accept instances of the abstract base class, os.PathLike.

@Evil Exists 2017-07-05 23:15:27

import os
if os.path.isfile(filename):
    print "file exists"
else:
    "Your code here"

Where your code here is use the (touch) command

This will check if the file is there if it is not then it will create it.

@Heikki Toivonen 2011-02-17 17:17:25

Using try except and the right error code from errno module gets rid of the race condition and is cross-platform:

import os
import errno

def make_sure_path_exists(path):
    try:
        os.makedirs(path)
    except OSError as exception:
        if exception.errno != errno.EEXIST:
            raise

In other words, we try to create the directories, but if they already exist we ignore the error. On the other hand, any other error gets reported. For example, if you create dir 'a' beforehand and remove all permissions from it, you will get an OSError raised with errno.EACCES (Permission denied, error 13).

@isaaclw 2012-04-13 19:46:56

What's the common reason against this? ie: why do people prefer the if statement to the try/except? Is it that try/except is more dangerous if you miss the error?

@Heikki Toivonen 2012-05-07 18:23:20

The accepted answer is actually dangerous because it has a race-condition. It is simpler, though, so if you are unaware of the race-condition, or think it won't apply to you, that would be your obvious first pick.

@Acumenus 2013-01-16 17:13:21

Raising the exception only when exception.errno != errno.EEXIST will unintentionally ignore the case when path exists but is a non-directory object such as a file. The exception should ideally be raised if the path is a non-directory object.

@Navin 2013-02-09 15:36:16

Note that the above code is equivalent to os.makedirs(path,exist_ok=True)

@Acumenus 2013-02-14 17:46:44

@Navin The exist_ok parameter was introduced in Python 3.2. It is not present in Python 2.x. I will incorporate it into my answer.

@Josh Segall 2013-04-08 03:37:19

@Navin, A-B-B: it looks like recent 2.x builds ignores the EEXIST error, essentially the same as the code above. I checked in 2.7.4.

@Josh Segall 2013-04-08 04:22:08

Nevermind, that's only for intermediate paths. The leaf directory will still raise an exception in 2.7

@jpmc26 2014-04-29 22:41:14

@HeikkiToivonen Technically speaking, if another program is modifying the directories and files at the same time your program is, your entire program is one giant race condition. What's to stop another program from just deleting this directory after the code creates it and before you actually put files in it?

@Acumenus 2015-02-24 16:44:28

@jpmc26, technically speaking, one can use one or more of: permissions, ownership, advisory lock, mandatory lock.

@jpmc26 2015-02-24 18:45:19

@A-B-B Ownership would only work if you don't need different instances of an application reading the same files, in which case you can avoid the race condition without it by just being conscious of what you're accessing. Same for permissions. Is there an atomic way of creating the directory and obtaining a lock? If not, then that won't help with the specific case presented here. If so, this solution doesn't incorporate it, which means it's just as "dangerous" as the other.

@A. Binzxxxxxx 2015-07-13 15:52:45

consider the usage of isdir instead of exists - depends on usecase

@gerrit 2016-05-25 14:07:22

In Python 3.3 and above, you can use FileExistsError.

@Clément 2017-05-31 02:41:30

Downvoted. The source code of mkdirs actually warns against checking for EEXIST: github.com/python/cpython/blob/…

@rtkaleta 2017-07-04 13:41:50

@Clément You're looking at Python 3.x source code, the answer given above applies to Python 2.7: github.com/python/cpython/blob/2.7/Lib/os.py#L153

@Clément 2017-07-04 14:59:49

@rtkaleta: That doesn't matter. The issue might not have been fixed in 2.7, but it exists nonetheless.

@Davis Herring 2017-10-02 23:58:43

@jpmc26: It's not that there are no race conditions; it's that your program doesn't contain two different places where you might detect what (due to races) is really the same condition.

@jpmc26 2017-10-03 00:27:41

@DavisHerring What you say isn't exactly true. An existence check occurs any time you access the directory or its contents. So if you create the directory and then open a file in it, you're still vulnerable to the same race condition. The answer is good; it's just still not safe if another program is modifying stuff, which is what Heikki (not the answer) asserted.

@Davis Herring 2017-10-03 00:47:20

@jpmc26: The races are materially worse the other way: with if not path.exists(...) a concurrent mkdir can give you an exception (despite the check). More broadly put, the check doesn't protect you from having to deal with any race condition, so it's strictly inferior to include it.

@jpmc26 2017-10-03 00:59:06

@DavisHerring This answer doesn't prevent you from having to deal with any race conditions, either. If your program is competing with another program for control of a directory, you're doing it wrong.

@Davis Herring 2017-10-03 02:03:28

@jpmc26: It prevents the mkdir race (in that you have two clear cases and obvious code paths to deal with them). Contention doesn't have to be a "battle": it's easily useful to have several programs create different parts of a directory.

@euccas 2017-01-03 22:33:47

In Python3, os.makedirs supports setting exist_ok. The default setting is False, which means an OSError will be raised if the target directory already exists. By setting exist_ok to True, OSError (directory exists) will be ignored and the directory will not be created.

os.makedirs(path,exist_ok=True)

In Python2, os.makedirs doesn't support setting exist_ok. You can use the approach in heikki-toivonen's answer:

import os
import errno

def make_sure_path_exists(path):
    try:
        os.makedirs(path)
    except OSError as exception:
        if exception.errno != errno.EEXIST:
            raise

@Ralph Schwerdt 2016-12-03 16:00:13

If you consider the following:

os.path.isdir('/tmp/dirname')

means a directory (path) exists AND is a directory. So for me this way does what I need. So I can make sure it is folder (not a file) and exists.

@Georgy 2019-05-16 12:32:31

How does this answer a question of creating a directory?

@Dennis Golomazov 2016-09-13 21:44:56

You can use mkpath

# Create a directory and any missing ancestor directories. 
# If the directory already exists, do nothing.

from distutils.dir_util import mkpath
mkpath("test")    

Note that it will create the ancestor directories as well.

It works for Python 2 and 3.

@Pod 2016-09-27 08:26:39

distutils.dir_util is not a part of the distutil public API and has problems in multi threaded environments: bugs.python.org/issue10948

@Acumenus 2016-10-21 04:38:43

Yes. As noted in the first message of the bug, the problem with distutils.dir_util.mkpath is that if you create a directory, then delete it from inside or outside Python, then use mkpath again, mkpath will simply use its invalid cached info of having previously created the directory, and will not actually make the directory again. In contrast, os.makedirs doesn't rely on any such cache.

@Aaron Hall 2015-01-22 23:49:18

Insights on the specifics of this situation

You give a particular file at a certain path and you pull the directory from the file path. Then after making sure you have the directory, you attempt to open a file for reading. To comment on this code:

filename = "/my/directory/filename.txt"
dir = os.path.dirname(filename)

We want to avoid overwriting the builtin function, dir. Also, filepath or perhaps fullfilepath is probably a better semantic name than filename so this would be better written:

import os
filepath = '/my/directory/filename.txt'
directory = os.path.dirname(filepath)

Your end goal is to open this file, you initially state, for writing, but you're essentially approaching this goal (based on your code) like this, which opens the file for reading:

if not os.path.exists(directory):
    os.makedirs(directory)
f = file(filename)

Assuming opening for reading

Why would you make a directory for a file that you expect to be there and be able to read?

Just attempt to open the file.

with open(filepath) as my_file:
    do_stuff(my_file)

If the directory or file isn't there, you'll get an IOError with an associated error number: errno.ENOENT will point to the correct error number regardless of your platform. You can catch it if you want, for example:

import errno
try:
    with open(filepath) as my_file:
        do_stuff(my_file)
except IOError as error:
    if error.errno == errno.ENOENT:
        print 'ignoring error because directory or file is not there'
    else:
        raise

Assuming we're opening for writing

This is probably what you're wanting.

In this case, we probably aren't facing any race conditions. So just do as you were, but note that for writing, you need to open with the w mode (or a to append). It's also a Python best practice to use the context manager for opening files.

import os
if not os.path.exists(directory):
    os.makedirs(directory)
with open(filepath, 'w') as my_file:
    do_stuff(my_file)

However, say we have several Python processes that attempt to put all their data into the same directory. Then we may have contention over creation of the directory. In that case it's best to wrap the makedirs call in a try-except block.

import os
import errno
if not os.path.exists(directory):
    try:
        os.makedirs(directory)
    except OSError as error:
        if error.errno != errno.EEXIST:
            raise
with open(filepath, 'w') as my_file:
    do_stuff(my_file)

@gone 2008-11-07 19:00:01

Try the os.path.exists function

if not os.path.exists(dir):
    os.mkdir(dir)

@Blair Conrad 2008-11-07 19:01:39

I was going to comment on the question, but do we mean os.mkdir? My python (2.5.2) has no os.path.mkdir....

@Serge S. 2012-05-21 15:14:12

There is no os.path.mkdir() method. os.path module implements some useful functions on pathnames.

@Chris Johnson 2015-10-14 21:30:09

This is terrible advice. See stackoverflow.com/a/5032238/763269.

@Antti Haapala 2015-03-11 20:50:01

In Python 3.4 you can also use the brand new pathlib module:

from pathlib import Path
path = Path("/my/directory/filename.txt")
try:
    if not path.parent.exists():
        path.parent.mkdir(parents=True)
except OSError:
    # handle error; you can also catch specific errors like
    # FileExistsError and so on.

@Acumenus 2017-12-06 17:58:13

@JanuszSkonieczny pypi.python.org/pypi/pathlib2 is the newer backport. The older one is unmaintained.

@Janusz Skonieczny 2017-12-11 15:43:16

As it it stated in the first line of the readme ;P. But the old backport is still valid for the answer here. And there is not naming headache. No need to explain why and when to use pathlib and where pathlib2 for the new users, and I think pros here will figure out the deprecation ;)

@crimsonstone 2009-01-14 17:57:32

I would personally recommend that you use os.path.isdir() to test instead of os.path.exists().

>>> os.path.exists('/tmp/dirname')
True
>>> os.path.exists('/tmp/dirname/filename.etc')
True
>>> os.path.isdir('/tmp/dirname/filename.etc')
False
>>> os.path.isdir('/tmp/fakedirname')
False

If you have:

>>> dir = raw_input(":: ")

And a foolish user input:

:: /tmp/dirname/filename.etc

... You're going to end up with a directory named filename.etc when you pass that argument to os.makedirs() if you test with os.path.exists().

@MrWonderful 2014-02-18 20:07:53

If you use 'isdir' only, won't you still have a problem when you attempt to create the directory and a file with the same name already exists?

@Damian Yerrick 2015-07-25 15:35:12

@MrWonderful The resulting exception when creating a directory over an existing file would correctly reflect the problem back to the caller.

@Chris Johnson 2015-10-14 21:32:33

This is bad advice. See stackoverflow.com/a/5032238/763269.

@Ali Afshar 2008-11-07 21:23:03

I have put the following down. It's not totally foolproof though.

import os

dirname = 'create/me'

try:
    os.makedirs(dirname)
except OSError:
    if os.path.exists(dirname):
        # We are nearly safe
        pass
    else:
        # There was an error on creation, so make sure we know about it
        raise

Now as I say, this is not really foolproof, because we have the possiblity of failing to create the directory, and another process creating it during that period.

@Ali Afshar 2008-11-07 21:33:43

@Chris Johnson 2018-06-25 21:23:02

Two problems: (1) you need to check the sub-error condition of OSError before deciding to check os.path.exists - see stackoverflow.com/a/5032238/763269, and (2) success on os.path.exists does not mean the directory exists, just that the path exists - could be a file, or a symlink, or other file system object.

Related Questions

Sponsored Content

44 Answered Questions

[SOLVED] How do I find all files containing specific text on Linux?

41 Answered Questions

[SOLVED] How to merge two dictionaries in a single expression?

12 Answered Questions

[SOLVED] How can I make a time delay in Python?

62 Answered Questions

[SOLVED] Get the source directory of a Bash script from within the script itself

  • 2008-09-12 20:39:56
  • Jiaaro
  • 1532112 View
  • 4568 Score
  • 62 Answer
  • Tags:   bash directory

30 Answered Questions

[SOLVED] How can I add an empty directory to a Git repository?

  • 2008-09-22 16:41:03
  • Laurie Young
  • 856766 View
  • 3955 Score
  • 30 Answer
  • Tags:   git directory git-add

21 Answered Questions

[SOLVED] How do I list all files of a directory?

  • 2010-07-08 19:31:22
  • duhhunjonn
  • 3542941 View
  • 3474 Score
  • 21 Answer
  • Tags:   python directory

11 Answered Questions

[SOLVED] How to find if directory exists in Python

  • 2012-01-19 21:03:20
  • David542
  • 999440 View
  • 1005 Score
  • 11 Answer
  • Tags:   python directory

28 Answered Questions

[SOLVED] How do I check if a list is empty?

  • 2008-09-10 06:20:11
  • Ray Vega
  • 2415644 View
  • 3235 Score
  • 28 Answer
  • Tags:   python list

37 Answered Questions

[SOLVED] How do I check whether a file exists without exceptions?

13 Answered Questions

[SOLVED] Find current directory and file's directory

  • 2011-02-28 01:51:21
  • John Howard
  • 2590162 View
  • 1921 Score
  • 13 Answer
  • Tags:   python directory

Sponsored Content