30 comments

@khizer 2020-04-19 20:24:42

Using Array.prototype.includes() method to remove duplicates:

(function() {
    const array = [1, 1, 2, 2, 3, 5, 5, 2];
    let uniqueValues = [];
    array.map(num => {
        if (Number.isInteger(num) && !uniqueValues.includes(num)) {
            uniqueValues.push(num)
        }
    });
    console.log(uniqueValues)
}());

@TLindig 2013-01-21 12:46:24

With JavaScript 1.6 / ECMAScript 5 you can use the native filter method of an Array in the following way to get an array with unique values:

function onlyUnique(value, index, self) {
  return self.indexOf(value) === index;
}

// usage example:
var a = ['a', 1, 'a', 2, '1'];
var unique = a.filter(onlyUnique);

console.log(unique); // ['a', 1, 2, '1']

The native method filter will loop through the array and leave only those entries that pass the given callback function onlyUnique.

onlyUnique checks, if the given value is the first occurring. If not, it must be a duplicate and will not be copied.

This solution works without any extra library like jQuery or prototype.js.

It works for arrays with mixed value types too.

For old Browsers (<ie9), that do not support the native methods filter and indexOf you can find work arounds in the MDN documentation for filter and indexOf.

If you want to keep the last occurrence of a value, simple replace indexOf by lastIndexOf.

With ES6 it could be shorten to this:

// usage example:
var myArray = ['a', 1, 'a', 2, '1'];
var unique = myArray.filter((v, i, a) => a.indexOf(v) === i);

console.log(unique); // unique is ['a', 1, 2, '1']

Thanks to Camilo Martin for hint in comment.

ES6 has a native object Set to store unique values. To get an array with unique values you could do now this:

var myArray = ['a', 1, 'a', 2, '1'];

let unique = [...new Set(myArray)];

console.log(unique); // unique is ['a', 1, 2, '1']

The constructor of Set takes an iterable object, like Array, and the spread operator ... transform the set back into an Array. Thanks to Lukas Liese for hint in comment.

@Mottie 2013-01-21 14:02:57

+1 Thanks for sharing! The only issue I see is that IE versions < 9 don't have an array indexOf function (obviously), which is why the other answers are using loops.

@TLindig 2013-02-13 09:42:37

@Motti Exactly, because <=ie8 do not have JavaScript 1.6 support. If you follow the link filter you will find at bottom a table with Browser compatibility.

@Jack Franzen 2013-11-23 10:11:12

This solution will run much slower, unfortunately. You're looping twice, once with filter and once with index of

@TLindig 2013-11-23 17:40:45

@JackFranzen Slower than what? The solution from Rafael? Rafaels solution do not work for mixed type arrays. For my example ['a', 1, 'a', 2, '1'] you would get ['a', 1, 2]. But this is not what I expected. BTW, much slower is very relative.

@Jack Franzen 2013-11-26 17:47:27

I see. You need to hash out the number one to be ###1 or something unique so that the hash key doesnt mess up. Afterwards, you need to convert back. It's unbelievable that'd you'd have a data set where it'd be impossible to make one of these keys, that's just disorganization

@L S 2016-02-17 14:41:09

While I'm aware the asker specified a solution for a list of numbers, I want to point out that this may not work with other data types. I.e. strings of differing case or objects.

@Camilo Martin 2016-07-24 08:43:19

In modern JS: .filter((v,i,a)=>a.indexOf(v)==i) (fat arrow notation).

@TLindig 2016-09-19 09:49:18

@Nico: show me one that does it better! you will not found one. In a unsorted array each entry must be compared with all other entries. So every solution will get exponentially slower.

@Nico 2016-09-24 11:25:07

@TLindig: try to get O(n) solution here @ codility.com/programmers/task/odd_occurrences_in_array

@Dan Green-Leipciger 2016-11-10 21:33:39

Probably shouldn't put personally identifiable info on Stack Overflow

@Lukas Liesis 2016-11-19 15:07:06

let unique_values = [...new Set(random_array)]; developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/…

@Dan Nissenbaum 2017-01-22 21:01:53

How about sorting the array first, then finding unique values? Seems this should be O(n log n) - from stackoverflow.com/a/8306539/368896

@Dan Nissenbaum 2017-01-22 21:08:11

For a much more detailed answer, including many possibilities - such as sorting first, and dealing with varying data types - see stackoverflow.com/a/9229821/368896

@Jakub Synowiec 2017-02-26 22:36:38

While this might be the shortest solution using a Set as a cache (and checking for values with Set.has() method) in a loop is ~2 times faster than spreading a Set. This might be important for very large arrays.

@BradGreens 2017-09-17 21:35:29

let unique = [...new Set(myArray)] threw a type error for me in TypeScript - anyone have an example for that?

@monzonj 2017-10-26 06:57:49

@BradGreens. Because TypeScript is strong-typed, you cannot do this kind of hacks. Try Array.from(new Set(myArray));

@Manuel 2017-10-31 10:40:11

How would Set be used with an array of dictionaries where a specific key should be unique?

@Arthur Tacca 2017-12-09 18:52:47

@Nico This answer is O(n²), which is polynomial time, not exponential time. Exponential growth is e.g. O(2^n).

@Nico 2017-12-12 10:04:43

I stand corrected my dudes it's polynomial - not exponential - and there is a big difference.

@jakubiszon 2018-02-15 23:44:15

Similar function finding duplicated values is also useful sometimes function onlyDuplicates(value, index, self) { return self.indexOf(value) !== index; } note - note how each duplicated value is returned as many times as it is duplicated

@Cristi Mihai 2018-08-21 13:24:16

Interestingly enough, the arrow function method is the fastest (but varies on your dataset): jsben.ch/remd9

@Mahesh 2018-10-05 04:39:31

Just an heads up [...new Set(myArray)] this fails on IE 11.

@Adam Katz 2019-09-11 21:31:16

Jason's answer solves the mixed-type array issue with Rafael's solution. That answer now has both versions to compare.

@Salem Megiddo 2019-10-31 03:32:30

You could also use the reduce alternative

@WesleyAC 2019-11-15 18:03:08

Use .filter() but instead of using .indexOf(), which is a linear search through the same array, just track which elements you've "seen" in a key/value hash {} and filter accordingly. My answer is on page three if you want to see.

@Pavel Nazarov 2020-03-04 10:03:46

Why third parameter "self" is needed when original array reference exists in this case? I am agree when filter is "on the fly" array constructing, for instance, after map or string.split

@Carl 2020-03-27 00:40:00

This answer should be removed. It is adding O(n^2) complexity to God knows how many copy pasta codebases out there.

@Geetanshu Gulati 2020-04-29 10:55:08

You can use Set operator to get unique values from an array

const uniqueArray = [...new Set([1, 1, 1])];

console.log(uniqueArray) // [1]

@noor 2019-10-18 19:28:39

If you want to only get the unique elements and remove the elements which repeats even once, you can do this:

let array = [2, 3, 4, 1, 2, 8, 1, 1, 2, 9, 3, 5, 3, 4, 8, 4];

function removeDuplicates(inputArray) {
  let output = [];
  let countObject = {};

  for (value of array) {
    countObject[value] = (countObject[value] || 0) + 1;
  }

  for (key in countObject) {
    if (countObject[key] === 1) {
      output.push(key);
    }
  }

  return output;
}

console.log(removeDuplicates(array));

@Ravi Sharma 2020-01-27 20:24:41

The simplest method to find unique element using filter method:

var A1 = [2, 2, 4, 5, 5, 6, 8, 8, 9];

var uniqueA1 = A1.filter(function(element) {
  return A1.indexOf(element) == A1.lastIndexOf(element);
});

console.log(uniqueA1); // Output: [4,6,9]

Here the logic is that the first and the last occurrence of an element are the same then this element occurs only once, so the array elements which pass the callback condition are included in the new array and displayed.

@gildniy 2020-09-10 00:52:50

This is one of the approaches using reduce map

const arr = [{id: '1'},{id: '4'},{id: '2'},{id: '1'},{id: '3'},{id: '1'},{id: '1'},{id: '5'}]

let uniqueArr = arr.reduce((arr, item) => {
  const uniq = arr.filter(i => i.id !== item.id)
    return [...uniq, item]
  }, [])

@Roman 2020-07-01 04:54:45

The task is to get a unique array from an array consisted of arbitrary types (primitive and non primitive).

The approach based on using new Map(...) is not new. Here it is leveraged by JSON.stringify(...), JSON.parse(...) and [].map method. The advantages are universality (applicability for an array of any types), short ES6 notation and probably performance for this case:

const dedupExample = [
    { a: 1 },
    { a: 1 },
    [ 1, 2 ],
    [ 1, 2 ],
    1,
    1,
    '1',
    '1'
]

const getUniqArr = arr => {
    const arrStr = arr.map(item => JSON.stringify(item))
    return [...new Set(arrStr)]
        .map(item => JSON.parse(item))
}

console.info(getUniqArr(dedupExample))
   /* [ {a: 1}, [1, 2], 1, '1' ] */

@Didier68 2020-07-02 08:55:31

in my solution, I sort data before filtering :

const uniqSortedArray = dataArray.sort().filter((v, idx, t) => idx==0 || v != t[idx-1]); 

@debugmode 2020-06-29 07:56:45

How about using set?

   let productPrice = [230,560,125,230,678,45,230,125,127];

   let tempData = new Set(productPrice);
   let uniqeProductPrice = [...tempData];

   uniqeProductPrice.forEach((item)=>{
      console.log(item)
    });

@Endless 2020-08-07 01:06:01

how about using just simple Set instead of casting it into an array? new Set(arr).forEach

@Gherciu Gheorghe 2020-01-24 09:51:06

The easiest way is to transform values into strings to filter also nested objects values.

const uniq = (arg = []) => {
  const stringifyedArg = arg.map(value => JSON.stringify(value))
  return arg.filter((value, index, self) => {
    if (typeof value === 'object')
      return stringifyedArg.indexOf(JSON.stringify(value)) === index
    return self.indexOf(value) === index
  })
}

    console.log(uniq([21, 'twenty one', 21])) // [21, 'twenty one']
    console.log(uniq([{ a: 21 }, { a: 'twenty one' }, { a: 21 }])) // [{a: 21}, {a: 'twenty one'}]

@Max Makhrov 2017-03-27 12:24:52

I split all answers to 4 possible solutions:

  1. Use object { } to prevent duplicates
  2. Use helper array [ ]
  3. Use filter + indexOf
  4. Bonus! ES6 Sets method.

Here's sample codes found in answers:

Use object { } to prevent duplicates

function uniqueArray1( ar ) {
  var j = {};

  ar.forEach( function(v) {
    j[v+ '::' + typeof v] = v;
  });

  return Object.keys(j).map(function(v){
    return j[v];
  });
} 

Use helper array [ ]

function uniqueArray2(arr) {
    var a = [];
    for (var i=0, l=arr.length; i<l; i++)
        if (a.indexOf(arr[i]) === -1 && arr[i] !== '')
            a.push(arr[i]);
    return a;
}

Use filter + indexOf

function uniqueArray3(a) {
  function onlyUnique(value, index, self) { 
      return self.indexOf(value) === index;
  }

  // usage
  var unique = a.filter( onlyUnique ); // returns ['a', 1, 2, '1']

  return unique;
}

Use ES6 [...new Set(a)]

function uniqueArray4(a) {
  return [...new Set(a)];
}

And I wondered which one is faster. I've made sample Google Sheet to test functions. Note: ECMA 6 is not avaliable in Google Sheets, so I can't test it.

Here's the result of tests: enter image description here

I expected to see that code using object { } will win because it uses hash. So I'm glad that tests showed the best results for this algorithm in Chrome and IE. Thanks to @rab for the code.

Update 2020

Google Script enabled ES6 Engine. Now I tested the last code with Sets and it appeared faster than the object method.

@liberborn 2017-09-14 13:18:31

The option "filter + indexOf" is extremely slow on arrays over 100.000 items. I had to use "object map" approach however it breaks original sorting.

@fletchsod 2019-03-20 15:32:58

Is higher number slower, not faster?

@Max Makhrov 2019-03-20 15:38:25

@ fletchsod , numbers are the time in ms to run the code.

@Jay Dadhania 2020-04-11 12:35:45

The numbers are wong! Google apps script runs on their server, not on the client's browser - and therefore the performance on Chrome/IE (or virtually any browser) will be same!

@Jay Dadhania 2020-04-11 12:41:06

While it is possible to run client-side JS from Google Script (either via google.script.host or google.script.run - not sure which), the answer clearly states that "ECMA 6 is not avaliable in Google Sheets" - therefore we can safely assume that the poster used server-side Google Script for this - and thus the answer shows the performance of the Google Server, not of the browser!!

@Alex Craft 2020-04-27 20:02:09

This should have better performance than the variant with list.indexOf

function uniq(list) { return [...new Set(list)] }

@Yair Nevet 2020-04-28 03:53:57

what about: const uniqueItems = [...new Set(list)]

@Alex Craft 2020-04-28 21:02:28

@YairNevet do you know if it's stable - i.e. does not reorder original list?

@Saurabh Goel 2020-04-23 16:58:30

array.forEach for traversing each element through the array and the if condition checking if the new array contains the element already. Hope this helps!!

    var arr = [1, 2, 2, 3, 4, 5, 6, 7, 7, 7, 7, 8, 9, 4, 3, 3, 5];
    let arr1 = [];
    arr.forEach(elem => {
        if (!arr1.includes(elem)) {
            arr1.push(elem);
        }
    });
    console.log(arr1);

@Brian 2020-04-23 20:40:49

Hello! While this code may solve the question, including an explanation of how and why this solves the problem would really help to improve the quality of your post, and probably result in more up-votes. Remember that you are answering the question for readers in the future, not just the person asking now. Please edit your answer to add explanations and give an indication of what limitations and assumptions apply.

@shubhs 2020-03-14 19:12:21

Use this without using any function and time complexity of O(n)

function uni(arr){
   var i = 0,uk = 0;
    for (j = 1; j < arr.length; j++){
        if(arr[i] != arr[j]){
            uk++;
            i = j;
        }

    }
    return uk + 1;
}

@emiel187 2020-05-08 07:21:53

This returns a number, but not an array with unique values.

@Shreyansh Sharma 2020-02-29 15:30:58

For an object-based array with some unique id's, I have a simple solution through which you can sort in linear complexity

function getUniqueArr(arr){
    const mapObj = {};
    arr.forEach(a => { 
       mapObj[a.id] = a
    })
    return Object.values(mapObj);
}

@Sukanta Bala 2020-02-21 07:35:38

The simplest answer is :

const array = [1, 1, 2, 2, 3, 5, 5, 2];
const uniqueArray = [...new Set(array)];
console.log(uniqueArray); // [1, 2, 3, 5]

@Thanwa Ch. 2020-04-02 16:55:52

Simple but it doesn't work with an array of objects.

@Vahid Akhtar 2020-01-01 10:49:39

Multiple way to remove duplicate elements from an Array using

  • set
  • filter
  • forEach
  • lodash(third party library)
  • for loop

const names = ['XYZ', 'ABC', '123', 'ABC', 'ACE', 'ABC', '123'];

// set
let unique = [...new Set(names)];
console.log(unique);

// filter
let x = (names) => names.filter((val,i) => names.indexOf(val) === i)
console.log(x(names));


// forEach
function removeDuplicates(names) {
  let unique = {};
  names.forEach(function(i) {
    if(!unique[i]) {
      unique[i] = true;
    }
  });
  return Object.keys(unique);
}

console.log(removeDuplicates(names));

Using lodash

npm i lodash

code

import _ from 'lodash';

let uniqueVal = _.uniq (names);
console.dir(uniqueVal);

Using simple for loop

const names = ['XYZ', 'ABC', '123', 'ABC', 'ACE', 'ABC', '123'];
let unique=[];
names.sort();
let len = names.length;
for(let i=0;i<len;i++){
    if(names[i]!==names[i+1])
    unique.push(names[i])
}
console.log(unique)

@Erçin Dedeoğlu 2019-12-23 11:01:42

Code:

function RemoveDuplicates(array) {
    return array.filter(function (value, index, self) {
        return self.indexOf(value) === index;
    });
}

Usages:

var arr = ["a","a","b","c","d", "d"];

console.log(RemoveDuplicates(arr));

Result:

0: "a"
1: "b"
2: "c"
3: "d"

@Emile Bergeron 2020-06-10 04:29:22

This is the same as the most upvoted answer and it adds nothing new or relevant.

@Dev Yego 2019-11-21 18:59:45

Found this sweet snippet from a post by Changhui Xu for those looking to get unique objects. I haven't measured its performance against the other alternatives though.

const array = [{
    name: 'Joe',
    age: 17
  },
  {
    name: 'Bob',
    age: 17
  },
  {
    name: 'Tom',
    age: 25
  },
  {
    name: 'John',
    age: 22
  },
  {
    name: 'Jane',
    age: 20
  },
];

const distinctAges = [...new Set(array.map(a => a.age))];

console.log(distinctAges)

@DutchPrime 2019-12-17 19:19:19

How can i do this while keeping the name and age index inside the array?

@Dev Yego 2019-12-19 07:13:02

@DutchPrime, that's the caveat. The resulting array is smaller than the original array so indexes would clearly be off. I feel this can be a great question in itself, how about throwing it to the community as I also do more research on it.

@Damian Kacprzak 2019-12-13 13:26:47

A simple solution if you want to check unique values ​​in an indefinite number of parameters using es6.

function uniteUnique(arr, ...rest) {
      const newArr = arr.concat(rest).flat()
      return [...new Set(newArr)]
   }

console.log(uniteUnique([1, 3, 2], [5, 2, 1, 4], [2, 1]))
console.log(uniteUnique([1, 2, 3], [5, 2, 1]))

I hope it will be useful to someone.

@WesleyAC 2019-11-15 17:45:26

You don't need .indexOf() at all; you can do this O(n):

function SelectDistinct(array) {
    var seenIt = {};

    return array.filter(function (val) {
        if (seenIt[val]) 
            return false;
        return seenIt[val] = true;
    });
}

var hasDuplicates = [1,2,3,4,5,5,6,7,7];
console.log(SelectDistinct(hasDuplicates)) //[1,2,3,4,5,6,7]

If you don't want to use .filter():

function SelectDistinct(array) {
    var seenIt = {};
    var distinct = [];

    for (var i in array) {
        if (!seenIt[array[i]]) {
            seenIt[array[i]] = true;
            distinct.push(array[i]);
        }
    }

    return distinct;
}

@Ashikur Rahman 2019-11-17 06:23:20

Using filter and splice

myArray.filter((el,i)=>![...myArray].splice(0,i).includes(el))

Using Set with prototype from

Array.from(new Set(myArray))

Using Set with spread operator

[...new Set(myArray)]

@Naveed Ali 2019-11-13 17:01:09

   var elems = ['f', 'a','b','f', 'c','d','e','f','c', 'n', 'n'];

    elems.sort();

    elems.forEach(function (value, index, arr){

        let first_index = arr.indexOf(value);
        let last_index = arr.lastIndexOf(value);

         if(first_index === last_index){

         console.log('unique items in array ' + value);

         }else{

         console.log('Duplicate item in array ' + value);             

         }

    });

@Salem Megiddo 2019-10-31 03:24:02

Reduce it!!!

This alternative instead of deduplicating explicitly it will take the array and reduce it so that each value of the array can be iterated and destructured in an accumulative behavior, ignoring the already included values by exploiting the persistence of the array because of the recursiveness.

['a', 1, 'a', 2, '1'].reduce((accumulator, currentValue) => accumulator.includes(currentValue) ? accumulator : [...accumulator, currentValue], [])

Test Example:

var array = ['a', 1, 'a', 2, '1'];
const reducer = (accumulator, currentValue) => accumulator.includes(currentValue) ? accumulator : [...accumulator, currentValue];

console.log(
  array.reduce(reducer, [])
);

Conclusion

By far more elegant and useful when boring for-each approach wants to be avoided (not that it is not useful).

No need for external libraries like Underscore.js, JQuery or Lo-Dash, nor the trouble to create any built-in function to achieve the desired deduplicated effect.

Oh, and HEY!, it can be done as a one-liner!!!


This answered was possible thanks to ES5 (ECMAScript 2015) include() and reduce().

@Shmulik 2019-10-30 08:10:16

pure js in one single line:

const uniqueArray = myArray.filter((elem, pos) => myArray.indexOf(elem) == pos); 

@AMTerp 2020-01-04 04:25:34

This is O(n^2) performance, no? Other answers provide O(n), some also in one line and pure JS.

@Krishnadas PC 2019-05-17 08:18:58

Now using sets you can remove duplicates and convert them back to the array.

var names = ["Mike","Matt","Nancy", "Matt","Adam","Jenny","Nancy","Carl"];

console.log([...new Set(names)])

Another solution is to use sort & filter

var names = ["Mike","Matt","Nancy", "Matt","Adam","Jenny","Nancy","Carl"];
var namesSorted = names.sort();
const result = namesSorted.filter((e, i) => namesSorted[i] != namesSorted[i+1]);
console.log(result);

@Nikki Luzader 2019-10-09 04:45:40

You can try this:

function removeDuplicates(arr){
  var temp = arr.sort();
  for(i = 0; i < temp.length; i++){
    if(temp[i] == temp[i + 1]){
      temp.splice(i,1);
      i--;
    }
  }
  return temp;
}

@Dave 2019-09-06 15:23:55

This has been answered a lot, but it didn't address my particular need.

Many answers are like this:

a.filter((item, pos, self) => self.indexOf(item) === pos);

But this doesn't work for arrays of complex objects.

Say we have an array like this:

const a = [
 { age: 4, name: 'fluffy' },
 { age: 5, name: 'spot' },
 { age: 2, name: 'fluffy' },
 { age: 3, name: 'toby' },
];

If we want the objects with unique names, we should use array.prototype.findIndex instead of array.prototype.indexOf:

a.filter((item, pos, self) => self.findIndex(v => v.name === item.name) === pos);

@Thanwa Ch. 2020-04-02 17:33:27

Great solution, beware that a new array will return from a function. (it doesn't modify itself)

@Jason 2013-07-18 01:52:17

It appears we have lost Rafael's answer, which stood as the accepted answer for a few years. This was (at least in 2017) the best-performing solution if you don't have a mixed-type array:

Array.prototype.getUnique = function(){
    var u = {}, a = [];
    for (var i = 0, l = this.length; i < l; ++i) {
        if (u.hasOwnProperty(this[i])) {
            continue;
        }
        a.push(this[i]);
        u[this[i]] = 1;
    }
return a;
}

If you do have a mixed-type array, you can serialize the hash key:

Array.prototype.getUnique = function() {
    var hash = {}, result = [], key; 
    for ( var i = 0, l = this.length; i < l; ++i ) {
        key = JSON.stringify(this[i]);
        if ( !hash.hasOwnProperty(key) ) {
            hash[key] = true;
            result.push(this[i]);
        }
    }
    return result;
}

@ifelse.codes 2019-07-19 12:18:21

[...new Set(duplicates)]

This is the simplest one and referenced from MDN Web Docs.

const numbers = [2,3,4,4,2,3,3,4,4,5,5,6,6,7,5,32,3,4,5]
console.log([...new Set(numbers)]) // [2, 3, 4, 5, 6, 7, 32]

@id.ot 2019-07-19 18:59:39

While this code may solve the question, including an explanation of how and why this solves the problem would really help to improve the quality of your post, and probably result in more up-votes. Remember that you are answering the question for readers in the future, not just the person asking now. Please edit your answer to add explanation, and give an indication of what limitations and assumptions apply.

Related Questions

Sponsored Content

Sponsored Content