How to remove duplicates from a slice in Go

In this article, we'll explore how to remove duplicate elements from a slice in Go. A slice is a dynamic, flexible view into the elements of an array, and addressing duplicates can be a common task. We will start with basic implementations and then move to more advanced and efficient techniques.

Basic Method: Using a Map

One of the straightforward ways to remove duplicates is using a map to keep track of the elements you've already seen.

package main

import (
    "fmt"
)

func removeDuplicates(slice []int) []int {
    dedupMap := make(map[int]bool)
    result := []int{}
    for _, value := range slice {
        if !dedupMap[value] {
            dedupMap[value] = true
            result = append(result, value)
        }
    }
    return result
}

func main() {
    sampleSlice := []int{1, 2, 2, 3, 4, 4, 5}
    fmt.Println("Original Slice:", sampleSlice)
    fmt.Println("Slice without duplicates:", removeDuplicates(sampleSlice))
}

The above code snippet defines a function that iterates through the slice, using a map to store each unique element, ensuring duplicates are filtered out in the process.

Intermediate Method: Removing Duplicates with Custom Function

With deeper understanding, you can build reusable functions or methods encapsulating duplicate removal.

package main

import (
    "fmt"
)

func removeDuplicatesString(slice []string) []string {
    dedupMap := make(map[string]bool)
    result := []string{}
    for _, value := range slice {
        if !dedupMap[value] {
            dedupMap[value] = true
            result = append(result, value)
        }
    }
    return result
}

func main() {
    sampleSlice := []string{"apple", "banana", "apple", "grape", "banana"}
    fmt.Println("Original Slice:", sampleSlice)
    fmt.Println("Slice without duplicates:", removeDuplicatesString(sampleSlice))
}

This function operates similarly to the integer example but handles a slice of strings, promoting code reuse across various data types.

Advanced Method: Using Set Data Structure

While Go doesn’t have a built-in set type, we can use a map to mimic set behavior effectively. Here we handle a more complex data manipulation by applying the functionality to structures or atypical scenarios.

package main

import (
    "fmt"
)

type person struct {
    name string
    age  int
}

func removeDuplicatePersons(slice []person) []person {
    dedupMap := make(map[person]bool)
    result := []person{}
    for _, value := range slice {
        if !dedupMap[value] {
            dedupMap[value] = true
            result = append(result, value)
        }
    }
    return result
}

func main() {
    sampleSlice := []person{
        {name: "Alice", age: 30},
        {name: "Bob", age: 25},
        {name: "Alice", age: 30},
        {name: "Tom", age: 40},
    }
    fmt.Println("Original Slice:", sampleSlice)
    fmt.Println("Slice without duplicates:", removeDuplicatePersons(sampleSlice))
}

In the advanced example above, an identical approach is taken to handle deduplication of slices composed of a struct, demonstrating the flexibility of map-based duplicate removal even for composite data types.

By understanding and implementing these techniques, you can efficiently handle duplicate elements in slices across various use cases in Go.