How to programmatically upload files to AWS S3 using Go

Uploading files to AWS S3 (Simple Storage Service) programmatically can be a necessary operation in building cloud-native applications. Go, known for its efficiency and concurrency support, can be a great choice for performing such file uploads.

In this article, you'll learn how to upload files to S3 using Go. We'll break this process into steps and provide code examples you can use immediately.

Prerequisites

• AWS SDK for Go.
• An AWS S3 bucket.
• Access credentials (AWS Access Key ID and Secret Access Key).
• Basic understanding of Go programming.

Step 1: Setting Up Your Go Environment

First, ensure you have Go installed on your machine. If not, download and install it from the official Go downloads page. You'll also need to set up a project directory for your Go application:

$ mkdir my-s3-uploader
$ cd my-s3-uploader
$ go mod init my-s3-uploader

Step 2: Installing AWS SDK for Go

Use the following command to install the AWS SDK for Go:

$ go get github.com/aws/aws-sdk-go-v2@latest
$ go get github.com/aws/aws-sdk-go-v2/config@latest
$ go get github.com/aws/aws-sdk-go-v2/service/s3@latest

Step 3: Writing the Code

Let's write a Go program to upload a file to S3. In your project directory, create a file named main.go and paste the following code:

package main

import (
    "context"
    "fmt"
    "log"
    "os"

    "github.com/aws/aws-sdk-go-v2/aws"
    "github.com/aws/aws-sdk-go-v2/config"
    "github.com/aws/aws-sdk-go-v2/service/s3"
)

func main() {
    file, err := os.Open("file_to_upload.txt")
    if err != nil {
        log.Fatalf("Failed to open file: %v", err)
    }
    defer file.Close()

    cfg, err := config.LoadDefaultConfig(context.TODO(), config.WithRegion("us-west-2"))
    if err != nil {
        log.Fatalf("Unable to load SDK config, %v", err)
    }

    svc := s3.NewFromConfig(cfg)

    input := &s3.PutObjectInput{
        Bucket: aws.String("your-bucket-name"),
        Key:    aws.String("file_to_upload.txt"),
        Body:   file,
    }

    _, err = svc.PutObject(context.TODO(), input)
    if err != nil {
        log.Fatalf("Unable to upload file: %v", err)
    }

    fmt.Println("Successfully uploaded file to S3")
}

Step 4: Running the Application

Before running your application, make sure you have set your AWS credentials in your environment. You can set them using environment variables:

$ export AWS_ACCESS_KEY_ID="your-access-key-id"
$ export AWS_SECRET_ACCESS_KEY="your-secret-access-key"

Finally, run your application:

$ go run main.go

If everything is set up correctly, your file should be uploaded to the specified S3 bucket.

Conclusion

We've covered the process of uploading a file to AWS S3 using Go. By using the AWS SDK for Go, developers can leverage Go's strengths while handling file uploads to the cloud. Remember to handle potential errors and manage AWS credentials securely in a production setting.