By mb567


2018-06-13 14:25:30 8 Comments

I have three lists X, Y, Z as follows:

X: [1, 1, 2, 3, 4, 5, 5, 5]
Y: [3, 3, 2, 6, 7, 1, 1, 2]
Z: [0, 0, 1, 1, 2, 3, 3, 4]

I am trying to remove both duplicated set of values at the same index of the lists get a reduced list as follows, all three list will always have the same length initially and at the end as well:

X: [2, 3, 4, 5]
Y: [2, 6, 7, 2]
Z: [1, 1, 2, 4]

I tried using the zip(X, Y, Z) function but I can't index it and the dict.fromkeys only removes one of the duplicates and leaves the other in the new list. I want to be able to remove both.

Any help is appreciated!

4 comments

@Laurent H. 2018-06-13 15:20:48

Here is my solution without any import, but still short and easily readable:

X = [1, 1, 2, 3, 4, 5, 5, 5]
Y = [3, 3, 2, 6, 7, 1, 1, 2]
Z = [0, 0, 1, 1, 2, 3, 3, 4]

zipped = list(zip(X, Y, Z))
X, Y, Z = zip(*[i for i in zipped if zipped.count(i) == 1])
X, Y, Z = list(X), list(Y), list(Z)    

print(X, Y, Z, sep='\n')
# [2, 3, 4, 5]
# [2, 6, 7, 2]
# [1, 1, 2, 4]

@jpp 2018-06-13 14:31:02

Using collections.Counter and zip, you can count unique triplets.

Then remove duplicates via a generator comprehension.

from collections import Counter

X = [1, 1, 2, 3, 4, 5, 5, 5]
Y = [3, 3, 2, 6, 7, 1, 1, 2]
Z = [0, 0, 1, 1, 2, 3, 3, 4]

c = Counter(zip(X, Y, Z))

X, Y, Z = zip(*(k for k, v in c.items() if v == 1))

print(X, Y, Z, sep='\n')

(2, 3, 4, 5)
(2, 6, 7, 2)
(1, 1, 2, 4)

Note if ordering is a requirement and you are not using Python 3.6+, you can create an "OrderedCounter" instead by subclassing collections.OrderedDict.

@jpp 2018-06-13 14:32:40

Downvoter care to comment?

@Jared Goguen 2018-06-13 14:39:54

correct me if I'm wrong, but this does not guarantee that the order of the triplets will be preserved - albeit, the triplets themselves will be. I'm not the down-voter, just an observation

@jpp 2018-06-13 14:41:43

@JaredGoguen, Good point: in Python 3.6+ it would, not in <3.6. You could construct an OrderedCounter (subclassing collections.OrderedDict), but not sure if order is a requirement and OP's Python version.

@Jared Goguen 2018-06-13 14:42:59

that is a side effect of the CPython implementation and is not, in general, guaranteed in 3.6. It will however be guaranteed in 3.7

@Jared Goguen 2018-06-13 14:44:07

of course, we don't even know if preserving order is required by the OP

@shanmuga 2018-06-13 14:38:15

Not the best possible approach

>>> from collections import Counter
>>> zipped_items = list(zip(x,y,z))
>>> counts = Counter(zipped_items)
>>> filtered_items = [item for item in zipped_items if counts[item] == 1]
>>> x1, y1, z1 = [ list(map(lambda x: x[i], filtered_items))
... for i in range(3)]

@koPytok 2018-06-13 14:31:56

It's convenient to use pandas library for the task. Just create dataframe using the lists and apply df.drop_duplicates with keep=False (means remove all duplicated rows):

import pandas as pd

dct = {
"X": [1, 1, 2, 3, 4, 5, 5, 5],
"Y": [3, 3, 2, 6, 7, 1, 1, 2],
"Z": [0, 0, 1, 1, 2, 3, 3, 4],
}
d = pd.DataFrame(dct)
d.drop_duplicates(keep=False)

Related Questions

Sponsored Content

35 Answered Questions

[SOLVED] Making a flat list out of list of lists in Python

52 Answered Questions

[SOLVED] Calling an external command in Python

16 Answered Questions

[SOLVED] What are metaclasses in Python?

25 Answered Questions

[SOLVED] Finding the index of an item given a list containing it in Python

  • 2008-10-07 01:39:38
  • Eugene M
  • 2685825 View
  • 2316 Score
  • 25 Answer
  • Tags:   python list

28 Answered Questions

[SOLVED] How to concatenate two lists in Python?

  • 2009-11-12 07:04:09
  • y2k
  • 1459272 View
  • 1696 Score
  • 28 Answer
  • Tags:   python list

27 Answered Questions

[SOLVED] How do I check if a list is empty?

  • 2008-09-10 06:20:11
  • Ray Vega
  • 1811427 View
  • 2809 Score
  • 27 Answer
  • Tags:   python list is-empty

23 Answered Questions

23 Answered Questions

[SOLVED] Difference between append vs. extend list methods in Python

6 Answered Questions

[SOLVED] How to get the number of elements in a list in Python?

  • 2009-11-11 00:30:54
  • y2k
  • 2572518 View
  • 1582 Score
  • 6 Answer
  • Tags:   python list

7 Answered Questions

Sponsored Content