By Mottie


2009-12-25 04:28:23 8 Comments

I have an array of numbers that I need to make sure are unique. I found the code snippet below on the internet and it works great until the array has a zero in it. I found this other script here on Stack Overflow that looks almost exactly like it, but it doesn't fail.

So for the sake of helping me learn, can someone help me determine where the prototype script is going wrong?

Array.prototype.getUnique = function() {
 var o = {}, a = [], i, e;
 for (i = 0; e = this[i]; i++) {o[e] = 1};
 for (e in o) {a.push (e)};
 return a;
}

More answers from duplicate question:

Similar question:

30 comments

@Ravi Sharma 2020-01-27 20:24:41

The simplest method to find unique element using filter method:

var A1 = [2,2,4,5,5,6,8,8,9];

var uniqueA1 = A1.filter(function(element){
    return A1.indexOf(element) == A1.lastIndexOf(element);
});

console.log(uniqueA1);

Here the logic is that the first and the last occurrence of an element are the same then this element occurs only once, so the array elements which pass the callback condition are included in the new array and displayed.

Output: 4,6,9

@Gherciu Gheorghe 2020-01-24 09:51:06

The easiest way is to transform values into strings to filter also nested objects values.

function uniq(arg = []) {
  var stringifyedArg = arg.map(value => JSON.stringify(value))
  if (typeof value === 'object')
     return stringifyedArg.indexOf(JSON.stringify(value)) === index
  return self.indexOf(value) === index
}

console.log(uniq([21, 'twenty one', 21])) // [21, 'twenty one']
console.log(uniq([{ a: 21 }, { a: 'twenty one' }, { a: 21 }])) // [{a: 21}, {a: 'twenty one'}]

@Vahid Akhtar 2020-01-01 10:49:39

Multiple way to remove duplicate elements from an Array using

  • set
  • filter
  • forEach
  • lodash(third party library)
  • for loop

const names = ['XYZ', 'ABC', '123', 'ABC', 'ACE', 'ABC', '123'];

// set
let unique = [...new Set(names)];
console.log(unique);

// filter
let x = (names) => names.filter((val,i) => names.indexOf(val) === i)
console.log(x(names));


// forEach
function removeDuplicates(names) {
  let unique = {};
  names.forEach(function(i) {
    if(!unique[i]) {
      unique[i] = true;
    }
  });
  return Object.keys(unique);
}

console.log(removeDuplicates(names));

Using lodash

npm i lodash

code

import _ from 'lodash';

let uniqueVal = _.uniq (names);
console.dir(uniqueVal);

Using simple for loop

const names = ['XYZ', 'ABC', '123', 'ABC', 'ACE', 'ABC', '123'];
let unique=[];
names.sort();
let len = names.length;
for(let i=0;i<len;i++){
    if(names[i]!==names[i+1])
    unique.push(names[i])
}
console.log(unique)

@Erçin Dedeoğlu 2019-12-23 11:01:42

Code:

function RemoveDuplicates(array) {
    return array.filter(function (value, index, self) {
        return self.indexOf(value) === index;
    });
}

Usages:

var arr = ["a","a","b","c","d", "d"];

console.log(RemoveDuplicates(arr));

Result:

0: "a"
1: "b"
2: "c"
3: "d"

@Dev Yego 2019-11-21 18:59:45

Found this sweet snippet from a post by Changhui Xu for those looking to get unique objects. I haven't measured its performance against the other alternatives though.

const array = [{
    name: 'Joe',
    age: 17
  },
  {
    name: 'Bob',
    age: 17
  },
  {
    name: 'Tom',
    age: 25
  },
  {
    name: 'John',
    age: 22
  },
  {
    name: 'Jane',
    age: 20
  },
];

const distinctAges = [...new Set(array.map(a => a.age))];

console.log(distinctAges)

@DutchPrime 2019-12-17 19:19:19

How can i do this while keeping the name and age index inside the array?

@Dev Yego 2019-12-19 07:13:02

@DutchPrime, that's the caveat. The resulting array is smaller than the original array so indexes would clearly be off. I feel this can be a great question in itself, how about throwing it to the community as I also do more research on it.

@Damian Kacprzak 2019-12-13 13:26:47

A simple solution if you want to check unique values ​​in an indefinite number of parameters using es6.

function uniteUnique(arr, ...rest) {
      const newArr = arr.concat(rest).flat()
      return [...new Set(newArr)]
   }

console.log(uniteUnique([1, 3, 2], [5, 2, 1, 4], [2, 1]))
console.log(uniteUnique([1, 2, 3], [5, 2, 1]))

I hope it will be useful to someone.

@WesleyAC 2019-11-15 17:45:26

You don't need .indexOf() at all; you can do this O(n):

function SelectDistinct(array) {
    var seenIt = {};

    return array.filter(function (val) {
        if (seenIt[val]) 
            return false;
        return seenIt[val] = true;
    });
}

var hasDuplicates = [1,2,3,4,5,5,6,7,7];
console.log(SelectDistinct(hasDuplicates)) //[1,2,3,4,5,6,7]

If you don't want to use .filter():

function SelectDistinct(array) {
    var seenIt = {};
    var distinct = [];

    for (var i in array) {
        if (!seenIt[array[i]]) {
            seenIt[array[i]] = true;
            distinct.push(array[i]);
        }
    }

    return distinct;
}

@Ashikur Rahman 2019-11-17 06:23:20

Using filter and splice

myArray.filter((el,i)=>![...myArray].splice(0,i).includes(el))

Using Set with prototype from

Array.from(new Set(myArray))

Using Set with spread operator

[...new Set(myArray)]

@Naveed Ali 2019-11-13 17:01:09

   var elems = ['f', 'a','b','f', 'c','d','e','f','c', 'n', 'n'];

    elems.sort();

    elems.forEach(function (value, index, arr){

        let first_index = arr.indexOf(value);
        let last_index = arr.lastIndexOf(value);

         if(first_index === last_index){

         console.log('unique items in array ' + value);

         }else{

         console.log('Duplicate item in array ' + value);             

         }

    });

@Salem Megiddo 2019-10-31 03:24:02

Reduce it!!!

This alternative instead of deduplicating explicitly it will take the array and reduce it so that each value of the array can be iterated and destructured in an accumulative behavior, ignoring the already included values by exploiting the persistence of the array because of the recursiveness.

['a', 1, 'a', 2, '1'].reduce((accumulator, currentValue) => accumulator.includes(currentValue) ? accumulator : [...accumulator, currentValue], [])

Test Example:

var array = ['a', 1, 'a', 2, '1'];
const reducer = (accumulator, currentValue) => accumulator.includes(currentValue) ? accumulator : [...accumulator, currentValue];

console.log(
  array.reduce(reducer, [])
);

Conclusion

By far more elegant and useful when boring for-each approach wants to be avoided (not that it is not useful).

No need for external libraries like Underscore.js, JQuery or Lo-Dash, nor the trouble to create any built-in function to achieve the desired deduplicated effect.

Oh, and HEY!, it can be done as a one-liner!!!


This answered was possible thanks to ES5 (ECMAScript 2015) include() and reduce().

@Shmulik 2019-10-30 08:10:16

pure js in one single line:

const uniqueArray = myArray.filter((elem, pos) => myArray.indexOf(elem) == pos); 

@AMTerp 2020-01-04 04:25:34

This is O(n^2) performance, no? Other answers provide O(n), some also in one line and pure JS.

@Krishnadas PC 2019-05-17 08:18:58

Now using sets you can remove duplicates and convert them back to the array.

var names = ["Mike","Matt","Nancy", "Matt","Adam","Jenny","Nancy","Carl"];

console.log([...new Set(names)])

Another solution is to use sort & filter

var names = ["Mike","Matt","Nancy", "Matt","Adam","Jenny","Nancy","Carl"];
var namesSorted = names.sort();
const result = namesSorted.filter((e, i) => namesSorted[i] != namesSorted[i+1]);
console.log(result);

@noor 2019-10-18 19:28:39

If you want to only get the unique elements and remove the elements which repeats even once, you can do this:

let array = [2, 3, 4, 1, 2, 8, 1, 1, 2, 9, 3, 5, 3, 4, 8, 4];
function removeDuplicates(inputArray) {
  let output = [];
  for (value of array) {
    countObject[value] = (countObject[value] || 0) + 1;
  }
  for (key in countObject) {
    if (countObject[key] === 1) {
      output.push(key);
    }
  }
  return output;
}

console.log(removeDuplicates(array));

@Nikki Luzader 2019-10-09 04:45:40

You can try this:

function removeDuplicates(arr){
  var temp = arr.sort();
  for(i = 0; i < temp.length; i++){
    if(temp[i] == temp[i + 1]){
      temp.splice(i,1);
      i--;
    }
  }
  return temp;
}

@Dave 2019-09-06 15:23:55

This has been answered a lot, but it didn't address my particular need.

Many answers are like this:

a.filter((item, pos, self) => self.indexOf(item) === pos);

But this doesn't work for arrays of complex objects.

Say we have an array like this:

const a = [
 { age: 4, name: 'fluffy' },
 { age: 5, name: 'spot' },
 { age: 2, name: 'fluffy' },
 { age: 3, name: 'toby' },
];

If we want the objects with unique names, we should use array.prototype.findIndex instead of array.prototype.indexOf:

a.filter((item, pos, self) => self.findIndex(v => v.name === item.name) === pos);

@Jason 2013-07-18 01:52:17

It appears we have lost Rafael's answer, which stood as the accepted answer for a few years. This was (at least in 2017) the best-performing solution if you don't have a mixed-type array:

Array.prototype.getUnique = function(){
    var u = {}, a = [];
    for (var i = 0, l = this.length; i < l; ++i) {
        if (u.hasOwnProperty(this[i])) {
            continue;
        }
        a.push(this[i]);
        u[this[i]] = 1;
    }
return a;
}

If you do have a mixed-type array, you can serialize the hash key:

Array.prototype.getUnique = function() {
    var hash = {}, result = [], key; 
    for ( var i = 0, l = this.length; i < l; ++i ) {
        key = JSON.stringify(this[i]);
        if ( !hash.hasOwnProperty(key) ) {
            hash[key] = true;
            result.push(this[i]);
        }
    }
    return result;
}

@Suman Patra 2019-07-19 12:18:21

[...new Set(duplicates)]

This is the simplest one and referenced from MDN Web Docs.

const numbers = [2,3,4,4,2,3,3,4,4,5,5,6,6,7,5,32,3,4,5]
console.log([...new Set(numbers)]) // [2, 3, 4, 5, 6, 7, 32]

@id.ot 2019-07-19 18:59:39

While this code may solve the question, including an explanation of how and why this solves the problem would really help to improve the quality of your post, and probably result in more up-votes. Remember that you are answering the question for readers in the future, not just the person asking now. Please edit your answer to add explanation, and give an indication of what limitations and assumptions apply.

@bob 2019-07-15 13:04:05

Deduplication usually requires an equality operator for the given type. However, using an eq function stops us from utilizing a Set to determine duplicates in an efficient manner, because Set falls back to ===. As you know for sure, === doesn't work for reference types. So we're kind if stuck, right?

The way out is simply using a transformer function that allows us to transform a (reference) type into something we can actually lookup using a Set. We could use a hash function, for instance, or JSON.stringify the data structure, if it doesn't contain any functions.

Often we only need to access a property, which we can then compare instead of the Object's reference.

Here are two combinators that meet these requirements:

const dedupeOn = k => xs => {
  const s = new Set();

  return xs.filter(o =>
    s.has(o[k])
      ? null
      : (s.add(o[k]), o[k]));
};

const dedupeBy = f => xs => {
  const s = new Set();

  return xs.filter(x => {
    const r = f(x);
    
    return s.has(r)
      ? null
      : (s.add(r), x);
  });
};

const xs = [{foo: "a"}, {foo: "b"}, {foo: "A"}, {foo: "b"}, {foo: "c"}];

console.log(
  dedupeOn("foo") (xs)); // [{foo: "a"}, {foo: "b"}, {foo: "A"}, {foo: "c"}]

console.log(
  dedupeBy(o => o.foo.toLowerCase()) (xs)); // [{foo: "a"}, {foo: "b"}, {foo: "c"}]

With these combinators we're extremely flexible in handling all kinds of deduplication issues. It's not the fastes approach, but the most expressive and most generic one.

@Nizmox 2019-07-03 06:38:41

A lot of people have already mentioned using...

[...new Set(arr)];

And this is a great solution, but my preference is a solution that works with .filter. In my opinion filter is a more natural way to get unique values. You're effectively removing duplicates, and removing elements from an array is exactly what filter is meant for. It also lets you chain off of .map, .reduce and other .filter calls. I devised this solution...

const unique = () => {
  let cache;  
  return (elem, index, array) => {
    if (!cache) cache = new Set(array);
    return cache.delete(elem);
  };
};

myArray.filter(unique());

The caveat is that you need a closure, but I think this is a worthy tradeoff. In terms of performance, it is more performant than the other solutions I have seen posted that use .filter, but worse performing than [...new Set(arr)].

See also my github package youneek

@Kamil Kiełczewski 2018-11-28 19:45:33

Magic

a.filter(e=>!(t[e]=e in t)) 

O(n) performance; we assume your array is in a and t={}. Explanation here (+Jeppe impr.)

let t={}, unique= a=> a.filter(e=>!(t[e]=e in t));

// "stand-alone" version working with global t:
// a1.filter((t={},e=>!(t[e]=e in t)));

// Test data
let a1 = [5,6,0,4,9,2,3,5,0,3,4,1,5,4,9];
let a2 = [[2, 17], [2, 17], [2, 17], [1, 12], [5, 9], [1, 12], [6, 2], [1, 12]];
let a3 = ['Mike', 'Adam','Matt', 'Nancy', 'Adam', 'Jenny', 'Nancy', 'Carl'];

// Results
console.log(JSON.stringify( unique(a1) ))
console.log(JSON.stringify( unique(a2) ))
console.log(JSON.stringify( unique(a3) ))

@Ondřej Želazko 2019-01-08 14:16:10

this look so super cool, that without a solid explanation i fell you're gonna mine bitcoins when i run this

@Ondřej Želazko 2019-01-09 09:49:28

what i meant is that you should expand your answer with some explanation and commented deconstruction of it. don't expect people will find useful answers like this. (though it really looks cool a probably works)

@Jeppe 2019-01-13 20:21:15

Not magic, but is much like the "Set"-answers, using O(1) key-lookups in the dictionary. Do you need to increment the counters though? How about "e=>!(t[e]=e in t)". Nice answer though.

@Kamil Kiełczewski 2019-01-14 03:32:29

@Jeppe when I run your improvement then I experience aha effect (before I don't know that I can use in operator outside the other construction than for loop :P) - Thank you - I appreciate it and will give +2 to your other good answers.

@philipp 2019-03-01 12:31:15

doesn't that create a global variable t which keeps alive after the filtering…??

@ford04 2019-03-10 22:29:50

This really is a tricky solution. Unfortunately, you have to declare t outside the expression unless you want to define a global variable and/or rely on your module bundler to hide the global. It looks fancy, but do yourself (and your dev team) a favor and just write a two liner then: let t = {}; a.filter(e=>!(t[e]=e in t))

@Shridhar Sagari 2019-05-10 10:08:40

I have a simple example where we can remove objects from array having repeated id in objects,

  let data = new Array({id: 1},{id: 2},{id: 3},{id: 1},{id: 3});
  let unique = [];
  let tempArr = [];
  console.log('before', data);
  data.forEach((value, index) => {
    if (unique.indexOf(value.id) === -1) {
      unique.push(value.id);
    } else {
      tempArr.push(index);    
    }
  });
  tempArr.reverse();
  tempArr.forEach(ele => {
    data.splice(ele, 1);
  });
  console.log(data);

@Riajul Islam 2019-05-03 10:49:55

I think this this is most easiest way to get unique item from array.

var arr = [1,2,4,1,4];
arr = Array.from(new Set(arr))
console.log(arr)

@Firas Abd Alrahman 2019-04-08 21:29:49

This solution should be very fast, and will work in many cases.

  1. Convert the indexed array items to object keys
  2. Use Object.keys function

    var indexArray = ["hi","welcome","welcome",1,-9];
    var keyArray = {};
    indexArray.forEach(function(item){ keyArray[item]=null; });
    var uniqueArray = Object.keys(keyArray);
    

@Ali 2017-09-03 09:18:54

Making an array of unique arrays, using field[2] as an Id:

const arr = [
  ['497', 'Q0', 'WTX091-B06-138', '0', '1.000000000', 'GROUP001'],
  ['497', 'Q0', 'WTX091-B09-92', '1', '0.866899288', 'GROUP001'],
  ['497', 'Q0', 'WTX091-B09-92', '2', '0.846036819', 'GROUP001'],
  ['497', 'Q0', 'WTX091-B09-57', '3', '0.835025326', 'GROUP001'],
  ['497', 'Q0', 'WTX091-B43-79', '4', '0.765068215', 'GROUP001'],
  ['497', 'Q0', 'WTX091-B43-56', '5', '0.764211464', 'GROUP001'],
  ['497', 'Q0', 'WTX091-B44-448', '6', '0.761701704', 'GROUP001'],
  ['497', 'Q0', 'WTX091-B44-12', '7', '0.761701704', 'GROUP001'],
  ['497', 'Q0', 'WTX091-B49-128', '8', '0.747434800', 'GROUP001'],
  ['497', 'Q0', 'WTX091-B18-17', '9', '0.746724770', 'GROUP001'],
  ['497', 'Q0', 'WTX091-B19-374', '10', '0.733379549', 'GROUP001'],
  ['497', 'Q0', 'WTX091-B19-344', '11', '0.731421782', 'GROUP001'],
  ['497', 'Q0', 'WTX091-B09-92', '12', '0.726450470', 'GROUP001'],
  ['497', 'Q0', 'WTX091-B19-174', '13', '0.712757036', 'GROUP001']
];


arr.filter((val1, idx1, arr) => !!~val1.indexOf(val1[2]) &&
  !(arr.filter((val2, idx2) => !!~val2.indexOf(val1[2]) &&
    idx2 < idx1).length));

console.log(arr);

@chinmayan 2018-04-09 03:49:15

We can do this using ES6 sets:

var duplicatedArray = [1, 2, 3, 4, 5, 1, 1, 1, 2, 3, 4];
var uniqueArray = Array.from(new Set(duplicatedArray));

console.log(uniqueArray);

//The output will be

uniqueArray = [1,2,3,4,5];

@tjacks3 2018-08-23 14:45:53

I have a solution that uses es6 reduce and find array helper methods to remove duplicates.

let numbers = [2, 2, 3, 3, 5, 6, 6];

const removeDups = array => {
  return array.reduce((acc, inc) => {
    if (!acc.find(i => i === inc)) {
      acc.push(inc);
    }
    return acc;
  }, []);
}

console.log(removeDups(numbers)); /// [2,3,5,6]

@shunryu111 2018-10-11 13:30:32

I had a slightly different problem where I needed to remove objects with duplicate id properties from an array. this worked.

let objArr = [{
  id: '123'
}, {
  id: '123'
}, {
  id: '456'
}];

objArr = objArr.reduce((acc, cur) => [
  ...acc.filter((obj) => obj.id !== cur.id), cur
], []);

console.log(objArr);

@Junaid Khan 2018-12-12 12:06:21

var numbers = [1, 1, 2, 3, 4, 4];

function unique(dupArray) {
  return dupArray.reduce(function(previous, num) {

    if (previous.find(function(item) {
        return item == num;
      })) {
      return previous;
    } else {
      previous.push(num);
      return previous;
    }
  }, [])
}

var check = unique(numbers);
console.log(check);

@Vahid Akhtar 2019-02-09 15:17:54

Es6 based solution...

var arr = [2, 3, 4, 2, 3, 4, 2];
const result = [...new Set(arr)];
console.log(result);

@Nidhal Ben Tahar 2019-02-18 14:20:03

Sometimes I need to get unique occurrences from an array of objects. Lodash seems like a nice helper but I don't think filtering an array justifies adding a dependency to a project.

Let's assume the comparison of two objects poses on comparing a property, an id for example.

const a = [{id: 3}, {id: 4}, {id: 3}, {id: 5}, {id: 5}, {id: 5}];

Since we all love one line snippets, here is how it can be done:

a.reduce((acc, curr) => acc.find(e => e.id === curr.id) ? acc : [...acc, curr], [])

Related Questions

Sponsored Content

94 Answered Questions

[SOLVED] How do I remove a particular element from an array in JavaScript?

  • 2011-04-23 22:17:18
  • Walker
  • 6266968 View
  • 7826 Score
  • 94 Answer
  • Tags:   javascript arrays

39 Answered Questions

[SOLVED] For-each over an array in JavaScript?

50 Answered Questions

45 Answered Questions

[SOLVED] Sort array of objects by string property value

42 Answered Questions

[SOLVED] How do I remove a property from a JavaScript object?

43 Answered Questions

[SOLVED] Loop through an array in JavaScript

23 Answered Questions

[SOLVED] Get the current URL with JavaScript?

  • 2009-06-23 19:26:45
  • dougoftheabaci
  • 2748146 View
  • 2921 Score
  • 23 Answer
  • Tags:   javascript url

17 Answered Questions

[SOLVED] How to insert an item into an array at a specific index (JavaScript)?

38 Answered Questions

[SOLVED] How do you get a timestamp in JavaScript?

Sponsored Content