Getting rid of duplicate values in a list/array is quite a common software development task. I remember in the past when I first faced this problem it was not very straight forward to do it in vanilla JS. You would think that such a common operation would be handled with out of the box method or command but somehow it was not included in the older Ecma Script specifications.

The old ES5 method

So to achieve desired results your best bet was using combined array.filter and array.sort or similar approach. One that would need to iterate over the whole list of values and get rid of duplicates one entry after another. Something along these lines of code:

function uniq(array) {
    return array.sort().filter(function(item, pos, arr) {
        return !pos || item != arr[pos - 1];
    })
}

So what used to happen here was we were sorting the array first and then compare each item with its predecessor to see if they differ and remove item if they do not. Obviously this would work with primitive values only as array.sort does not put objects in order as they appear all the same to this method. Handling object values would require even more code so lot of people turned to methods provided by 3rd party libs like lodash or jQuery which offered much easier ways to do the list clean up.

Modern ES6/7 array de-duplication

Luckily for us front end developers ES6 update brought us many new functionalities and data types. We can now de-duplicate arrays (or any other lists) with this neat one liner:

uniqueOnly = [...new Set(array)];

We use the Set type which by definition is a list of unique items so if we pass an array into its constructor it does old the hard work of making sure all the values that end up in the created set are unique. It does it equally well for both primitives and more advanced data types like objects. Then we use another new functionality – the spread operator () to assign this newly created set back to an array. This makes de-duplicating arrays in modern js much easier and fun.

Leave a comment

Your email address will not be published. Required fields are marked *