Parsing Logs and CSVs Using Strings in Go

Parsing logs and CSV files efficiently is a common requirement in software development. In this article, we will explore how to perform these tasks using the Go programming language, focusing on the usage of strings.

Basic Concepts

To start with, let's explore the basic Go functionalities for handling strings, which will be foundational in parsing files.

package main

import (
    "fmt"
    "strings"
)

func main() {
    input := "error: something went wrong"
    // Basic string splitting
    parts := strings.Split(input, ": ")
    fmt.Println(parts[0]) // Output: "error"
    fmt.Println(parts[1]) // Output: "something went wrong"
}

Intermediate: Parsing CSVs

Go's standard library provides encoding/csv package to handle CSV files, which can be combined with string operations for additional processing.

package main

import (
    "encoding/csv"
    "fmt"
    "os"
    "strings"
)

func main() {
    file, err := os.Open("data.csv")
    if err != nil {
        panic(err)
    }
    defer file.Close()

    reader := csv.NewReader(file)
    records, err := reader.ReadAll()
    if err != nil {
        panic(err)
    }

    for _, record := range records {
        fmt.Println(strings.Join(record, ", "))
    }
}

This snippet opens a CSV file, reads all the data into memory, and then uses strings.Join to print each line in a modified format.

Advanced: Parsing Log Files

Log files can sometimes be more complex than CSV files because they may have various formats and delimiters. Let's explore a common form of log parsing where each line is processed individually.

package main

import (
    "bufio"
    "fmt"
    "os"
    "strings"
)

func main() {
    file, err := os.Open("logs.txt")
    if err != nil {
        panic(err)
    }
    defer file.Close()

    scanner := bufio.NewScanner(file)
    for scanner.Scan() {
        processLogLine(scanner.Text())
    }

    if err := scanner.Err(); err != nil {
        fmt.Fprintln(os.Stderr, "reading standard input:", err)
    }
}

func processLogLine(line string) {
    // Split the log by a delimiter
    parts := strings.Split(line, " ")
    for _, part := range parts {
        fmt.Println(part)
    }
}

In the example above, each log line is read separately and processed using strings.Split function based on spaces or any other delimiter.

String handling in Go is powerful and, when combined with the encoding/csv and bufio packages, offers efficient means to parse and handle text data like logs and CSVs.