By Dick Lucas


2012-11-30 04:12:53 8 Comments

filter, map, and reduce work perfectly in Python 2. Here is an example:

>>> def f(x):
        return x % 2 != 0 and x % 3 != 0
>>> filter(f, range(2, 25))
[5, 7, 11, 13, 17, 19, 23]

>>> def cube(x):
        return x*x*x
>>> map(cube, range(1, 11))
[1, 8, 27, 64, 125, 216, 343, 512, 729, 1000]

>>> def add(x,y):
        return x+y
>>> reduce(add, range(1, 11))
55

But in Python 3, I receive the following outputs:

>>> filter(f, range(2, 25))
<filter object at 0x0000000002C14908>

>>> map(cube, range(1, 11))
<map object at 0x0000000002C82B70>

>>> reduce(add, range(1, 11))
Traceback (most recent call last):
  File "<pyshell#8>", line 1, in <module>
    reduce(add, range(1, 11))
NameError: name 'reduce' is not defined

I would appreciate if someone could explain to me why this is.

Screenshot of code for further clarity:

IDLE sessions of Python 2 and 3 side-by-side

5 comments

@Yogendra Singh 2018-12-18 07:35:26

Here are the examples of Filter, map and reduce functions.

numbers = [10,11,12,22,34,43,54,34,67,87,88,98,99,87,44,66]

//Filter

oddNumbers = list(filter(lambda x: x%2 != 0, numbers))

print(oddNumbers)

//Map

multiplyOf2 = list(map(lambda x: x*2, numbers))

print(multiplyOf2)

//Reduce

The reduce function, since it is not commonly used, was removed from the built-in functions in Python 3. It is still available in the functools module, so you can do:

from functools import reduce

sumOfNumbers = reduce(lambda x,y: x+y, numbers)

print(sumOfNumbers)

@Bikash Singh 2018-06-25 02:20:24

Since the reduce method has been removed from the built in function from Python3, don't forget to import the functools in your code. Please look at the code snippet below.

import functools
my_list = [10,15,20,25,35]
sum_numbers = functools.reduce(lambda x ,y : x+y , my_list)
print(sum_numbers)

@Dimitris Fasarakis Hilliard 2016-08-31 04:00:39

As an addendum to the other answers, this sounds like a fine use-case for a context manager that will re-map the names of these functions to ones which return a list and introduce reduce in the global namespace.

A quick implementation might look like this:

from contextlib import contextmanager    

@contextmanager
def noiters(*funcs):
    if not funcs: 
        funcs = [map, filter, zip] # etc
    from functools import reduce
    globals()[reduce.__name__] = reduce
    for func in funcs:
        globals()[func.__name__] = lambda *ar, func = func, **kwar: list(func(*ar, **kwar))
    try:
        yield
    finally:
        del globals()[reduce.__name__]
        for func in funcs: globals()[func.__name__] = func

With a usage that looks like this:

with noiters(map):
    from operator import add
    print(reduce(add, range(1, 20)))
    print(map(int, ['1', '2']))

Which prints:

190
[1, 2]

Just my 2 cents :-)

@javadba 2018-10-05 04:22:25

python as a language is a mess - but it has v good to excellent libraries: numpy, pandas, statsmodels and friends.. I had been buliding convenience libraries like you show here to reduce the pain of the native language - but have lost the energy and try not to stray far from a data.frame / datatable, or xarray. But kudos for trying..

@Joshua D. Boyd 2012-11-30 04:17:44

The functionality of map and filter was intentionally changed to return iterators, and reduce was removed from being a built-in and placed in functools.reduce.

So, for filter and map, you can wrap them with list() to see the results like you did before.

>>> def f(x): return x % 2 != 0 and x % 3 != 0
...
>>> list(filter(f, range(2, 25)))
[5, 7, 11, 13, 17, 19, 23]
>>> def cube(x): return x*x*x
...
>>> list(map(cube, range(1, 11)))
[1, 8, 27, 64, 125, 216, 343, 512, 729, 1000]
>>> import functools
>>> def add(x,y): return x+y
...
>>> functools.reduce(add, range(1, 11))
55
>>>

The recommendation now is that you replace your usage of map and filter with generators expressions or list comprehensions. Example:

>>> def f(x): return x % 2 != 0 and x % 3 != 0
...
>>> [i for i in range(2, 25) if f(i)]
[5, 7, 11, 13, 17, 19, 23]
>>> def cube(x): return x*x*x
...
>>> [cube(i) for i in range(1, 11)]
[1, 8, 27, 64, 125, 216, 343, 512, 729, 1000]
>>>

They say that for loops are 99 percent of the time easier to read than reduce, but I'd just stick with functools.reduce.

Edit: The 99 percent figure is pulled directly from the What’s New In Python 3.0 page authored by Guido van Rossum.

@Xiao 2014-07-22 03:25:54

You do not need to create extra functions in list comprehensions. Just use [i*i*i for i in range(1,11)]

@Joshua D. Boyd 2014-09-02 18:52:51

You are absolutely correct. I kept the function in the list comprehension examples to keep it looking similar to the filter/map examples.

@Breezer 2016-01-17 15:13:27

i**3 is also equivalent of i*i*i

@syntonym 2016-03-18 11:06:09

@Breezer actually i**3 will call i.__pow__(3) and i*i*i i.__mul__(i).__mul__(i) (or something like that). With ints it doesn't matter but with numpy numbers/custom classes it might even produce different results.

@John Strood 2018-08-08 07:14:04

@JoshuaD.Boyd But which is faster? A map or generator?

@javadba 2018-10-05 04:17:51

I have noticed that whenever we hear that "Guido made decision X" that pain is a likely outcome. This is a great example: list(list(list(.. ))) to do what was already verbose in python.

@nhahtdh 2012-11-30 04:22:03

You can read about the changes in What's New In Python 3.0. You should read it thoroughly when you move from 2.x to 3.x since a lot has been changed.

The whole answer here are quotes from the documentation.

Views And Iterators Instead Of Lists

Some well-known APIs no longer return lists:

  • [...]
  • map() and filter() return iterators. If you really need a list, a quick fix is e.g. list(map(...)), but a better fix is often to use a list comprehension (especially when the original code uses lambda), or rewriting the code so it doesn’t need a list at all. Particularly tricky is map() invoked for the side effects of the function; the correct transformation is to use a regular for loop (since creating a list would just be wasteful).
  • [...]

Builtins

  • [...]
  • Removed reduce(). Use functools.reduce() if you really need it; however, 99 percent of the time an explicit for loop is more readable.
  • [...]

@javadba 2018-02-27 05:29:01

Adding list(map(...) everywhere .. how in the world is that helping readability.. python can't seem to handle progressive / streaming application of functional combinators. Other languages I can chain a dozen operations against a collection in a row and it's readable. Here? what do you want - a dozen way nested in ??

@MatrixManAtYrService 2018-06-27 15:08:29

If you're working in an imperative context, then a for-loop is probably the more readable option. But there are good reasons to prefer a functional context--and breaking from that to go back to procedural can be pretty darn ugly.

@Imperishable Night 2018-10-05 02:19:00

@javadba Are you sure in a "streaming application" you need to add the list call at all? I thought the meaning of "streaming" is "no list is created at all; process each element of the input fully before moving on to the next".

@Imperishable Night 2018-10-05 02:22:38

@MatrixManAtYrService If you are sure the python 2 behavior is what you need, you can always just redefine map.

@Minato 2018-11-08 12:44:43

I still cant grasp how a readability argument leads to such a change. If it was for performance reasons I might understand...

@Kaos 2019-10-25 16:37:38

readability is one thing, but it makes (list, dict, ...)comprehensions unusable. Can't put an ordinary for loop in the middle of a list comprehension, now can I.. ugh.

@dmonopoly 2019-11-13 15:52:03

A "quick fix" (read: hack) is to use list(map...) but notice the "better fix" is to use a list comprehension instead - like [Foo(x) for x in mylist]. This doesn't lead to adding list() everywhere and longer term may be better. (@javadba FYI)

@javadba 2019-11-13 20:08:55

@dmonopoly I specifically mentioned applying a dozen transformations and that I do not want to have a 12 way nested list comprehension - which is the only option in standard python afaik

Related Questions

Sponsored Content

39 Answered Questions

[SOLVED] How do I check whether a file exists without exceptions?

36 Answered Questions

[SOLVED] How to get the current time in Python

  • 2009-01-06 04:54:23
  • user46646
  • 3142922 View
  • 2701 Score
  • 36 Answer
  • Tags:   python datetime time

29 Answered Questions

[SOLVED] Finding the index of an item given a list containing it in Python

  • 2008-10-07 01:39:38
  • Eugene M
  • 3608715 View
  • 2962 Score
  • 29 Answer
  • Tags:   python list indexing

18 Answered Questions

[SOLVED] What are metaclasses in Python?

42 Answered Questions

[SOLVED] How do I merge two dictionaries in a single expression?

23 Answered Questions

[SOLVED] Does Python have a ternary conditional operator?

25 Answered Questions

[SOLVED] How do I concatenate two lists in Python?

63 Answered Questions

[SOLVED] Calling an external command from Python

10 Answered Questions

[SOLVED] Does Python have a string 'contains' substring method?

Sponsored Content