[SOLVED] Why can I use a list index as an indexing variable in a for loop?

I have the following code:

``````a = [0,1,2,3]

for a[-1] in a:
print(a[-1])
``````

The output is:

``````0
1
2
2
``````

I'm confused about why a list index can be used as an indexing variable in a for loop. @gerrit 2019-04-12 11:12:14

The answer by TrebledJ explains the technical reason of why this is possible.

Why would you want to do this though?

Suppose I have an algorithm that operates on an array:

``````x = np.arange(5)
``````

And I want to test the result of the algorithm using different values of the first index. I can simply skip the first value, reconstructing an array every time:

``````for i in range(5):
print(np.r_[i, x[1:]].sum())
``````

This will create a new array on every iteration, which is not ideal, in particular if the array is large. To reuse the same memory on every iteration, I can rewrite it as:

``````for i in range(5):
x = i
print(x.sum())
``````

Which is probably clearer than the first version too.

But that is exactly identical to the more compact way to write this:

``````for x in range(5):
print(x.sum())
``````

all of the above will result in:

``````10
11
12
13
14
``````

Now this is a trivial "algorithm", but there will be more complicated purposes where one might want to test changing a single (or multiple, but that complicating things due to assignment unpacking) value in an array to a number of values, preferably without copying the entire array. In this case, you may want to use an indexed value in a loop, but be prepared to confuse anyone maintaining your code (including yourself). For this reason, the second version explicitly assigning `x = i` is probably preferable, but if you prefer the more compact `for x in range(5)` style, this should be a valid use case. @TrebledJ 2019-04-12 04:05:44

List indexes such as `a[-1]` in the expression `for a[-1] in a` are valid as specified by the `for_stmt` (and specifically the `target_list`) grammar token, where `slicing` is a valid target for assignment.

"Huh? Assignment? What has that got to do with my output?"

Indeed, it has everything to do with the output and result. Let's dive into the documentation for a `for-in` loop:

``````for_stmt ::=  "for" target_list "in" expression_list ":" suite
``````

The expression list is evaluated once; it should yield an iterable object. An iterator is created for the result of the `expression_list`. The suite is then executed once for each item provided by the iterator, in the order returned by the iterator. Each item in turn is assigned to the target list using the standard rules for assignments (see Assignment statements), and then the suite is executed.

N.B. the suite refers to the statement(s) under the for-block, `print(a[-1])` in our particular case.

Let's have a little fun and extend the print statement:

``````a = [0, 1, 2, 3]
for a[-1] in a:
print(a, a[-1])
``````

This gives the following output:

``````[0, 1, 2, 0] 0    # a[-1] assigned 0
[0, 1, 2, 1] 1    # a[-1] assigned 1
[0, 1, 2, 2] 2    # a[-1] assigned 2
[0, 1, 2, 2] 2    # a[-1] assigned 2 (itself)
``````

Here, `a[-1]` changes on each iteration and we see this change propagated to `a`. Again, this is possible due to `slicing` being a valid target.

A good argument made by Ev. Kounis regards the first sentence of the quoted doc above: "The expression list is evaluated once". Does this not imply that the expression list is static and immutable, constant at `[0, 1, 2, 3]`? Shouldn't `a[-1]` thus be assigned `3` at the final iteration?

No, [the expression list is] evaluated once to create an iterable object. But that iterable object still iterates over the original data, not a copy of it.

The following code demonstrates how an iterable `it` lazily yields elements of a list `x`.

``````x = [1, 2, 3, 4]
it = iter(x)
print(next(it))    # 1
print(next(it))    # 2
print(next(it))    # 3
x[-1] = 0
print(next(it))    # 0
``````

(code inspired by Kounis')

If evaluation was eager, we could expect `x[-1] = 0` to have zero effect on `it` and expect `4` to be printed. This is clearly not the case and goes to show that by the same principle, our `for`-loop lazily yields numbers from `a` following assignments to `a[-1]` on each iteration. @Christian Dean 2019-04-12 04:21:02

Essentially, the important thing to note here is that Python considers `a[-1]` to be a valid form of the left-hand side of an assignment statement (e.g. `a[-1] = 1` is valid grammar). Thus `a[-1]` is a valid "variable" name, because as the documentation stated, it evaluates the binding variable(s) in a for loop declaration as it would the left-hand side of an assignment. This answer is missing an important point, namely how the grammar defines `target_list`. For reasons that are utterly obscure to me, the grammar explicitly (rather than “accidentally”) allows the target list to contain slicing expressions (the syntax for the `for` loop simply recycles normal assignment targets, which seems odd). @gerrit 2019-04-12 10:58:46

@KonradRudolph To me it seems rather "accidentally" or automatically than explicitly: `target list` is the LHS of any assignment (isn't it?), so even if it is silly in the context of a for loop, anything that can be a LHS is allowed here: `for (x, *y) in [range(5), range(6)]`, `for () in ()`, etc. Suppose that I have `x = np.zeros(L)`, then I can do `for x[:] in [f(a), f(b)]` and ensure (1) that I reuse the memory allocated by `x`, and (2) that I confuse the hell out of anybody reading my code including myself. @Ev. Kounis 2019-04-12 11:04:12

doesn't the fact that `a[-1]` on the last iteration is `2` contradict the documentation in that "the expression list is evaluated once"? @gerrit Fair enough. What I mean was: why does the rule for `for_stmt` refer to `target_list` at all? Wouldn’t it simply make more sense to refer put `identifier ("," identifier)* [","]` here instead? @Ev.Kounis No, it’s evaluated once to create an iterable object. But that iterable object still iterates over the original data, not a copy of it. @Ev. Kounis 2019-04-12 11:23:18 @jeremycg 2019-04-12 20:21:24

It works in list comprehensions, which is (to me) unexpected given the scoping of list comprehension loop variables: `[a[-1] for a[-1] in a]` @recnac 2019-04-12 04:03:52

It is an interesting question, and you can understand it by that:

``````for v in a:
a[-1] = v
print(a[-1])

print(a)
``````

actually `a` becomes: `[0, 1, 2, 2]` after loop

Output:

``````0
1
2
2
[0, 1, 2, 2]
``````

I hope that helps you, and comment if you have further questions. : ) @Nathan 2019-04-12 04:14:04

(This is more of a long comment than an answer - there are a couple of good ones already, especially @TrebledJ's. But I had to think of it explicitly in terms of overwriting variables that already have values before it clicked for me.)

``````x = 0
l = [1, 2, 3]
for x in l:
print(x)
``````

you wouldn't be surprised that `x` is overridden each time through the loop. Even though `x` existed before, its value isn't used (i.e. `for 0 in l:`, which would throw an error). Rather, we assign the values from `l` to `x`.

When we do

``````a = [0, 1, 2, 3]

for a[-1] in a:
print(a[-1])
``````

even though `a[-1]` already exists and has a value, we don't put that value in but rather assign to `a[-1]` each time through the loop. @AmirAShabani 2019-04-12 04:17:41

Somehow I thought the variable used in `for loop` is immutable. +1 for pointing it out. @Tom Karzes 2019-04-12 03:51:09

`a[-1]` refers to the last element of `a`, in this case `a`. The `for` loop is a bit unusual in that it is using this element as the loop variable. It's not evaluating that element upon loop entry, but rather it is assigning to it on each iteration through the loop.

So first `a[-1]` gets set to 0, then 1, then 2. Finally, on the last iteration, the `for` loop retrieves `a` which at that point is `2`, so the list ends up as `[0, 1, 2, 2]`.

A more typical `for` loop uses a simple local variable name as the loop variable, e.g. `for x ...`. In that case, `x` is set to the next value upon each iteration. This case is no different, except that `a[-1]` is set to the next value upon each iteration. You don't see this very often, but it's consistent. @AmirAShabani 2019-04-12 03:53:10

How does it actually `gets set to 0, then 1, then 2 and finally 2`? That's the confusing part! @gameon67 2019-04-12 03:53:43 @Nathan 2019-04-12 03:56:41

I expected it would be like saying `for 3 in a: print(a[-1])` (because `a[-1]` was `3` at the start of the loop) and give an error, but clearly that's not the case. You're obviously correct about what's happening, but I'm surprised this evaluates this way. @Tom Karzes 2019-04-12 04:05:30

I've expanded the answer to explain more precisely how this works. Hopefully people will understand it this time. @Christian Dean 2019-04-12 04:29:27

"The for loop is a bit unusual in that it is using this element as the loop variable." - This may be where the confusion is coming from. Python isn't using the element `a[-1]` evaluates to, rather, it's using the index `a[-1]` itself. If you rephrase that statement, your answer becomes much more clear. @Tom Karzes 2019-04-12 04:48:16

@ChristianDean Ok, I added a clarifying sentence after that one. Hopefully it's clearer now. @Christian Dean 2019-04-12 04:49:40

Yeah, that's better! @TomKarzes @blhsing 2019-04-12 04:08:52

The left expression of a `for` loop statement gets assigned with each item in the iterable on the right in each iteration, so

``````for n in a:
print(n)
``````

is just a fancy way of doing:

``````for i in range(len(a)):
n = a[i]
print(n)
``````

Likewise,

``````for a[-1] in a:
print(a[-1])
``````

is just a fancy way of doing:

``````for i in range(len(a)):
a[-1] = a[i]
print(a[-1])
``````

where in each iteration, the last item of `a` gets assigned with the next item in `a`, so when the iteration finally comes to the last item, its value got last assigned with the second-last item, `2`.

[SOLVED] How do I list all files of a directory?

• 2010-07-08 19:31:22
• duhhunjonn
• 3310652 View
• 3475 Score
• Tags:   python directory

[SOLVED] Accessing the index in 'for' loops?

• 2009-02-06 22:47:54
• Joan Venge
• 1798986 View
• 3119 Score
• Tags:   python loops list

[SOLVED] Finding the index of an item given a list containing it in Python

• 2008-10-07 01:39:38
• Eugene M
• 3291383 View
• 2711 Score
• Tags:   python list indexing