By Nelson


2009-08-23 21:29:29 8 Comments

What's the proper way to declare custom exception classes in modern Python? My primary goal is to follow whatever standard other exception classes have, so that (for instance) any extra string I include in the exception is printed out by whatever tool caught the exception.

By "modern Python" I mean something that will run in Python 2.5 but be 'correct' for the Python 2.6 and Python 3.* way of doing things. And by "custom" I mean an Exception object that can include extra data about the cause of the error: a string, maybe also some other arbitrary object relevant to the exception.

I was tripped up by the following deprecation warning in Python 2.6.2:

>>> class MyError(Exception):
...     def __init__(self, message):
...         self.message = message
... 
>>> MyError("foo")
_sandbox.py:3: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6

It seems crazy that BaseException has a special meaning for attributes named message. I gather from PEP-352 that attribute did have a special meaning in 2.5 they're trying to deprecate away, so I guess that name (and that one alone) is now forbidden? Ugh.

I'm also fuzzily aware that Exception has some magic parameter args, but I've never known how to use it. Nor am I sure it's the right way to do things going forward; a lot of the discussion I found online suggested they were trying to do away with args in Python 3.

Update: two answers have suggested overriding __init__, and __str__/__unicode__/__repr__. That seems like a lot of typing, is it necessary?

11 comments

@Eugene Yarmash 2020-02-29 13:02:52

To define your own exceptions correctly, there are a few best practices that you should follow:

  • Define a base class inheriting from Exception. This will allow to easily catch any exceptions related to the project:

    class MyProjectError(Exception):
        """A base class for MyProject exceptions."""
    

    Organizing the exception classes in a separate module (e.g. exceptions.py) is generally a good idea.

  • To create a specific exception, subclass the base exception class.

  • To add support for extra argument(s) to a custom exception, define a custom __init__() method with a variable number of arguments. Call the base class's __init__(), passing any positional arguments to it (remember that BaseException/Exception expect any number of positional arguments):

    class CustomError(MyProjectError):
        def __init__(self, *args, **kwargs):
            super().__init__(*args)
            self.foo = kwargs.get('foo')
    

    To raise such exception with an extra argument you can use:

     raise CustomError('Something bad happened', foo='foo')
    

This design adheres to the Liskov substitution principle, since you can replace an instance of a base exception class with an instance of a derived exception class. Also, it allows you to create an instance of a derived class with the same parameters as the parent.

@Luke Savefrogs 2020-07-14 05:23:39

Really like this design... I feel like it's a lot cleaner than the ones in the other answers.

@loutre 2020-07-17 22:19:30

LSP adhesion should be mandatory, that's why I prefer this answer to the others.

@Macintosh Fan 2020-07-02 20:37:02

A really simple approach:

class CustomError(Exception):
    pass

raise CustomError("Hmm, seems like this was custom coded...")

Or, have the error raise without printing __main__ (may look cleaner and neater):

class CustomError(Exception):
    __module__ = Exception.__module__

raise CustomError("Improved CustomError!")

@Yaroslav Nikitenko 2019-06-10 20:40:04

See a very good article "The definitive guide to Python exceptions". The basic principles are:

  • Always inherit from (at least) Exception.
  • Always call BaseException.__init__ with only one argument.
  • When building a library, define a base class inheriting from Exception.
  • Provide details about the error.
  • Inherit from builtin exceptions types when it makes sense.

There is also information on organizing (in modules) and wrapping exceptions, I recommend to read the guide.

@logicOnAbstractions 2019-10-05 18:09:39

This is a good example of why on SO I usually check the most upvoted answer, but the most recent ones as well. Usefull addition, thanks.

@Eugene Yarmash 2020-02-29 13:12:35

Always call BaseException.__init__ with only one argument. Seems like unneeded constraint, since it actually accepts any number of arguments.

@Yaroslav Nikitenko 2020-02-29 13:56:46

@EugeneYarmash I agree, now I don't understand that. I don't use it anyway. Maybe I should reread the article and expand my answer.

@Yaroslav Nikitenko 2020-03-03 16:51:37

@EugeneYarmash I read the article again. It is stated that in case of several arguments the C implementation calls "return PyObject_Str(self->args);" It means that one string should work better than several ones. Did you check that?

@fameman 2018-11-25 17:14:25

As of Python 3.8 (2018, https://docs.python.org/dev/whatsnew/3.8.html), the recommended method is still:

class CustomExceptionName(Exception):
    """Exception raised when very uncommon things happen"""
    pass

Please don't forget to document, why a custom exception is neccessary!

If you need to, this is the way to go for exceptions with more data:

class CustomExceptionName(Exception):
    """Still an exception raised when uncommon things happen"""
    def __init__(self, message, payload=None):
        self.message = message
        self.payload = payload # you could add more args
    def __str__(self):
        return str(self.message) # __str__() obviously expects a string to be returned, so make sure not to send any other data types

and fetch them like:

try:
    raise CustomExceptionName("Very bad mistake.", "Forgot upgrading from Python 1")
except CustomExceptionName as error:
    print(str(error)) # Very bad mistake
    print("Detail: {}".format(error.payload)) # Detail: Forgot upgrading from Python 1

payload=None is important to make it pickle-able. Before dumping it, you have to call error.__reduce__(). Loading will work as expected.

You maybe should investigate in finding a solution using pythons return statement if you need much data to be transferred to some outer structure. This seems to be clearer/more pythonic to me. Advanced exceptions are heavily used in Java, which can sometimes be annoying, when using a framework and having to catch all possible errors.

@kevlarr 2019-03-26 16:46:13

At the very least, the current docs indicate this is the way to do it (at least without the __str__) rather than other answers that use super().__init__(...).. Just a shame that overrides for __str__ and __repr__ are probably necessary just for better "default" serializing.

@Roel Schroeven 2019-08-06 15:21:54

Honest question: Why is it important for exceptions to be pickle-able? What are the use cases for dumping and loading exceptions?

@logicOnAbstractions 2019-10-06 02:58:48

@RoelSchroeven: I had to parallelize code once. Ran fine single process, but aspects of some of its classes were not serializable (lambda function being passed as objects). Took me some time figuring it out & fixing it. Meaning someone later may end up needing your code to be serialize, be unable to do it, and have to dig up why... My issue wasn't unpickeable errors, but I can see it causing similar problems.

@omkaartg 2018-07-22 05:27:45

Try this Example

class InvalidInputError(Exception):
    def __init__(self, msg):
        self.msg = msg
    def __str__(self):
        return repr(self.msg)

inp = int(input("Enter a number between 1 to 10:"))
try:
    if type(inp) != int or inp not in list(range(1,11)):
        raise InvalidInputError
except InvalidInputError:
    print("Invalid input entered")

@frnknstn 2012-04-22 18:18:17

With modern Python Exceptions, you don't need to abuse .message, or override .__str__() or .__repr__() or any of it. If all you want is an informative message when your exception is raised, do this:

class MyException(Exception):
    pass

raise MyException("My hovercraft is full of eels")

That will give a traceback ending with MyException: My hovercraft is full of eels.

If you want more flexibility from the exception, you could pass a dictionary as the argument:

raise MyException({"message":"My hovercraft is full of animals", "animal":"eels"})

However, to get at those details in an except block is a bit more complicated. The details are stored in the args attribute, which is a list. You would need to do something like this:

try:
    raise MyException({"message":"My hovercraft is full of animals", "animal":"eels"})
except MyException as e:
    details = e.args[0]
    print(details["animal"])

It is still possible to pass in multiple items to the exception and access them via tuple indexes, but this is highly discouraged (and was even intended for deprecation a while back). If you do need more than a single piece of information and the above method is not sufficient for you, then you should subclass Exception as described in the tutorial.

class MyError(Exception):
    def __init__(self, message, animal):
        self.message = message
        self.animal = animal
    def __str__(self):
        return self.message

@mtraceur 2018-04-20 22:36:04

"but this will be deprecated in the future" - is this still intended for deprecation? Python 3.7 still seems to happily accept Exception(foo, bar, qux).

@frnknstn 2018-05-02 08:20:56

It haven't seen any recent work to depricate it since the last attempt failed due to the pain of transitioning, but that usage is still discouraged. I will update my answer to reflect that.

@neves 2018-05-08 17:48:42

@frnknstn, why it is discouraged? Looks like a nice idiom for me.

@frnknstn 2018-05-10 08:09:53

@neves for a start, using tuples to store exception information has no benefit over using a dictionary to do the same. If you are interested in the reasoning behind the exception changes, take a look at PEP352

@liberforce 2019-04-17 16:25:09

The relevant section of PEP352 is "Retracted Ideas".

@gahooa 2009-08-23 21:55:23

Maybe I missed the question, but why not:

class MyException(Exception):
    pass

Edit: to override something (or pass extra args), do this:

class ValidationError(Exception):
    def __init__(self, message, errors):

        # Call the base class constructor with the parameters it needs
        super(ValidationError, self).__init__(message)

        # Now for your custom code...
        self.errors = errors

That way you could pass dict of error messages to the second param, and get to it later with e.errors


Python 3 Update: In Python 3+, you can use this slightly more compact use of super():

class ValidationError(Exception):
    def __init__(self, message, errors):

        # Call the base class constructor with the parameters it needs
        super().__init__(message)

        # Now for your custom code...
        self.errors = errors

@jiakai 2017-08-01 02:54:22

However an exception defined like this would not be pickable; see the discussion here stackoverflow.com/questions/16244923/…

@Robino 2017-09-15 13:39:34

@jiakai means "picklable". :-)

@ddleon 2020-02-27 15:13:39

Following the documentation of python for user-defined exceptions, the names that are mentioned in the __init__ function are incorrect. Instead of (self,message,error) it is (self,expression,message). The attribute expression is the input expression in which the error occurred and the message is an explanation of the error.

@asthasr 2020-04-22 21:15:11

That is a misunderstanding, @ddleon. The example in the docs that you are referring to is for a particular use case. There is no significance to the name of the subclass's constructor arguments (nor their number).

@M. Utku ALTINKAYA 2009-08-23 21:46:54

You should override __repr__ or __unicode__ methods instead of using message, the args you provide when you construct the exception will be in the args attribute of the exception object.

@Aaron Hall 2014-11-14 21:09:22

"Proper way to declare custom exceptions in modern Python?"

This is fine, unless your exception is really a type of a more specific exception:

class MyException(Exception):
    pass

Or better (maybe perfect), instead of pass give a docstring:

class MyException(Exception):
    """Raise for my specific kind of exception"""

Subclassing Exception Subclasses

From the docs

Exception

All built-in, non-system-exiting exceptions are derived from this class. All user-defined exceptions should also be derived from this class.

That means that if your exception is a type of a more specific exception, subclass that exception instead of the generic Exception (and the result will be that you still derive from Exception as the docs recommend). Also, you can at least provide a docstring (and not be forced to use the pass keyword):

class MyAppValueError(ValueError):
    '''Raise when my specific value is wrong'''

Set attributes you create yourself with a custom __init__. Avoid passing a dict as a positional argument, future users of your code will thank you. If you use the deprecated message attribute, assigning it yourself will avoid a DeprecationWarning:

class MyAppValueError(ValueError):
    '''Raise when a specific subset of values in context of app is wrong'''
    def __init__(self, message, foo, *args):
        self.message = message # without this you may get DeprecationWarning
        # Special attribute you desire with your Error, 
        # perhaps the value that caused the error?:
        self.foo = foo         
        # allow users initialize misc. arguments as any other builtin Error
        super(MyAppValueError, self).__init__(message, foo, *args) 

There's really no need to write your own __str__ or __repr__. The builtin ones are very nice, and your cooperative inheritance ensures that you use it.

Critique of the top answer

Maybe I missed the question, but why not:

class MyException(Exception):
    pass

Again, the problem with the above is that in order to catch it, you'll either have to name it specifically (importing it if created elsewhere) or catch Exception, (but you're probably not prepared to handle all types of Exceptions, and you should only catch exceptions you are prepared to handle). Similar criticism to the below, but additionally that's not the way to initialize via super, and you'll get a DeprecationWarning if you access the message attribute:

Edit: to override something (or pass extra args), do this:

class ValidationError(Exception):
    def __init__(self, message, errors):

        # Call the base class constructor with the parameters it needs
        super(ValidationError, self).__init__(message)

        # Now for your custom code...
        self.errors = errors

That way you could pass dict of error messages to the second param, and get to it later with e.errors

It also requires exactly two arguments to be passed in (aside from the self.) No more, no less. That's an interesting constraint that future users may not appreciate.

To be direct - it violates Liskov substitutability.

I'll demonstrate both errors:

>>> ValidationError('foo', 'bar', 'baz').message

Traceback (most recent call last):
  File "<pyshell#10>", line 1, in <module>
    ValidationError('foo', 'bar', 'baz').message
TypeError: __init__() takes exactly 3 arguments (4 given)

>>> ValidationError('foo', 'bar').message
__main__:1: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6
'foo'

Compared to:

>>> MyAppValueError('foo', 'FOO', 'bar').message
'foo'

@Kos 2018-01-03 18:21:50

Hello from 2018! BaseException.message is gone in Python 3, so the critique only holds for old versions, right?

@Aaron Hall 2018-01-03 19:02:34

@Kos The critique about Liskov Substitutability is still valid. The semantics of the first argument as a "message" are also arguably questionable, but I don't think I'll argue the point. I'll give this more of a look when I have more free time.

@Jacquot 2018-03-18 23:39:33

FWIW, for Python 3 (at least for 3.6+), one would redefine the __str__ method of MyAppValueError instead of relying on the message attribute

@cowbert 2018-07-14 03:18:28

why avoid passing a dict as a positional argument? It retains all of the original semantics, including (__repr__/__str__) and the user can just parse the dict via .args[0] per frnknstn's answer? (You are noting this in the docstring aren't you?)

@ostergaard 2018-09-16 13:52:05

@AaronHall Could you expand on the benefit of sub-classing ValueError rather than Exception? You state that this is what is meant by the docs but a direct reading doesn't support that interpretation and in the Python Tutorial under User-defined Exceptions it clearly makes it the users choice: "Exceptions should typically be derived from the Exception class, either directly or indirectly." Hence keen to understand if your view is justifiable, please.

@Aaron Hall 2018-09-16 21:17:20

@ostergaard Can't answer in full right now, but in short, the user gets the additional option of catching ValueError. This makes sense if it's in the category of Value Errors. If it's not in the category of Value Errors, I'd argue against it on the semantics. There's room for some nuance and reasoning on the part of the programmer, but I much prefer specificity when applicable. I'll update my answer to better tackle the subject some time soon.

@ostergaard 2018-09-28 13:12:38

@AaronHall thanks, that does make sense and I agree. Though I believe it's a stretch to ascribe that meaning to the docs.

@Eugene Yarmash 2020-02-28 15:34:59

I don't see much sense in following the Liskov substitution principle with custom exceptions. You raise a specific exception to indicate a specific condition. Why would you ever need to substitute an instance of a base exception class with an instance of a derived exception class?

@Aaron Hall 2020-02-28 15:51:40

@EugeneYarmash Why? If you're raising a general exception, perhaps you later want to raise a subclass and not break users still catching the more general exception, then you want substitutability that doesn't break the user.

@Eugene Yarmash 2020-02-28 18:02:33

@AaronHall I've never needed this in practice. Anyway, I guess your example violates LSP as well: if your code uses raise ValueError(msg), you can't replace this with raise ValidationError(msg) as the latter requires one more parameter.

@Aaron Hall 2020-02-28 18:13:19

@EugeneYarmash Now that's an interesting point that I have long considered! Is a subclass Liskov Substitutable (LS) if it disallows the same invocation of the constructor in the child as in the parent? Maybe it strictly isn't LS. But in most cases I envisage, it shouldn't break anyone. However, I suppose it could, if you're catching and instantiating the type(error) instead of the class by name. Can you find me a source to cite on that? (Any peer-reviewed paper that discusses it is preferable to not.)

@Anakhand 2020-06-29 14:39:14

Related to what @Eugene said, does the Liskov substitution principle apply to constructors? (See also: this, this.) Specifically, in the case of exceptions, it could very well be that I decide to replace a generic expression with a more specific one, but in that case, I'll also make sure to provide the necessary arguments—otherwise, it's a half-baked job.

@mykhal 2013-08-07 16:23:56

see how exceptions work by default if one vs more attributes are used (tracebacks omitted):

>>> raise Exception('bad thing happened')
Exception: bad thing happened

>>> raise Exception('bad thing happened', 'code is broken')
Exception: ('bad thing happened', 'code is broken')

so you might want to have a sort of "exception template", working as an exception itself, in a compatible way:

>>> nastyerr = NastyError('bad thing happened')
>>> raise nastyerr
NastyError: bad thing happened

>>> raise nastyerr()
NastyError: bad thing happened

>>> raise nastyerr('code is broken')
NastyError: ('bad thing happened', 'code is broken')

this can be done easily with this subclass

class ExceptionTemplate(Exception):
    def __call__(self, *args):
        return self.__class__(*(self.args + args))
# ...
class NastyError(ExceptionTemplate): pass

and if you don't like that default tuple-like representation, just add __str__ method to the ExceptionTemplate class, like:

    # ...
    def __str__(self):
        return ': '.join(self.args)

and you'll have

>>> raise nastyerr('code is broken')
NastyError: bad thing happened: code is broken

@Lennart Regebro 2009-08-23 21:58:47

No, "message" is not forbidden. It's just deprecated. You application will work fine with using message. But you may want to get rid of the deprecation error, of course.

When you create custom Exception classes for your application, many of them do not subclass just from Exception, but from others, like ValueError or similar. Then you have to adapt to their usage of variables.

And if you have many exceptions in your application it's usually a good idea to have a common custom base class for all of them, so that users of your modules can do

try:
    ...
except NelsonsExceptions:
    ...

And in that case you can do the __init__ and __str__ needed there, so you don't have to repeat it for every exception. But simply calling the message variable something else than message does the trick.

In any case, you only need the __init__ or __str__ if you do something different from what Exception itself does. And because if the deprecation, you then need both, or you get an error. That's not a whole lot of extra code you need per class. ;)

@Yaroslav Nikitenko 2019-06-10 20:05:14

It's interesting that Django exceptions don't inherit from a common base. docs.djangoproject.com/en/2.2/_modules/django/core/exception‌​s Do you have a good example when catching all exceptions from a specific application is needed? (maybe it is useful only for some specific types of applications).

@Yaroslav Nikitenko 2019-06-10 20:44:29

I found a good article on this topic, julien.danjou.info/python-exceptions-guide . I think that Exceptions should be subclassed primarily domain-based, not application-based. When your app is about HTTP protocol, you derive from HTTPError. When part of your app is TCP, you derive that part's exceptions from TCPError. But if your app spans a lot of domains (file, permissions, etc), the reason to have a MyBaseException diminishes. Or is it to protect from 'layer violation'?

Related Questions

Sponsored Content

23 Answered Questions

[SOLVED] Is there a way to create multiline comments in Python?

11 Answered Questions

[SOLVED] python exception message capturing

8 Answered Questions

[SOLVED] Manually raising (throwing) an exception in Python

  • 2010-01-12 21:07:40
  • TIMEX
  • 1842119 View
  • 2355 Score
  • 8 Answer
  • Tags:   python exception

23 Answered Questions

[SOLVED] Is there a way to run Python on Android?

13 Answered Questions

[SOLVED] How do you test that a Python function throws an exception?

26 Answered Questions

[SOLVED] Pythonic way to create a long multi-line string

13 Answered Questions

[SOLVED] What's the canonical way to check for type in Python?

  • 2008-09-30 11:00:10
  • Herge
  • 916962 View
  • 1335 Score
  • 13 Answer
  • Tags:   python types

8 Answered Questions

[SOLVED] How to print an exception in Python?

14 Answered Questions

[SOLVED] Proper way to use **kwargs in Python

20 Answered Questions

[SOLVED] jQuery Ajax error handling, show custom exception messages

Sponsored Content