I was trying to upload an image to aws (Amazon Web Service) for the first time and couldn’t find a comprehensive guide, I had to search for every bit of the process. Here are my findings I hope it helps you 😇.

The code example is not the ideal way to structure your application, config variables should be gotten from environment variable. The goal is to upload any file to an S3 bucket, not to structure a web application for production 🙂.

First of all you need an aws account. You can create on here. You get a free trial which is sufficient for you to learn how to upload images over there.

You might want to know why you would want to upload images on aws. Most likely you want to upload to a storage server so that your storage is managed independent of your application. Multiple services or application can access your stored files independent of each other. There are other alternatives like google cloud storage and digital ocean as well, but we are going to be focusing on aws here. If you need to know how to upload to those other platforms you could drop a comment and i’ll give it a shot. You could google more on the “why”, let’s continue with the tutorial.

After creating the account, go to your console and select the s3 service in the storage category. On the s3 console, create a new bucket with a proper name that is url friendly, because it will form part of the url to your files. Complete the form and create your new bucket.

Now that we have a bucket to store our files, let’s write some code.

Here is our server, with one handler that captures the file uploaded to the route /.

package main
import (
   "fmt"
   "log"
)
func handler(w http.ResponseWriter, r *http.Request) {
  maxSize := int64(1024000) // allow only 1MB of file size
  err := r.ParseMultipartForm(maxSize)
  if err != nil {
    log.Println(err)
    fmt.Fprintf(w, "Image too large. Max Size: %v", maxSize)
    return
  }
  file, fileHeader, err := r.FormFile("profile_picture")
  if err != nil {
    log.Println(err)
    fmt.Fprintf(w, "Could not get uploaded file")
    return
  }
  defer file.Close()
  // create an AWS session which can be
  // reused if we're uploading many files
  s, err := session.NewSession(&aws.Config{
  Region: aws.String("us-east-2"),
  Credentials: credentials.NewStaticCredentials(
    "secret-id", // id
    "secret-key",   // secret
    ""),  // token can be left blank for now
  })
  if err != nil {
    fmt.Fprintf(w, "Could not upload file")
  }
  fileName, err := UploadFileToS3(s, file, fileHeader)
  if err != nil {
    fmt.Fprintf(w, "Could not upload file")
  }
  fmt.Fprintf(w, "Image uploaded successfully: %v", fileName)
}
func main() {
  http.HandleFunc("/", handler)
  log.Println("Upload server started")
  log.Fatal(http.ListenAndServe(":8080", nil))
}

Before we continue you need to install the aws go sdk package from github. It can be achieved by running this on you terminal:

$ go get github.com/aws/aws-sdk-go/...

This will install the go sdk on your system, ready to be used in the application.

Now we have the go sdk we need to use it for uploading the files we captured from request form. We are going to create one extra function that will receive an aws session, a multipart.File, and a multipart.FileHeader objects. This function will upload the file to aws and return the file name and an error if there was any. We will also update our handler to call the function and return the file url to the user.

Here is what the code now looks like:

package main

import (
    "net/http"
    "log"
    "fmt"
    "mime/multipart"
    "bytes"
    "path/filepath"

    "github.com/globalsign/mgo/bson"
    "github.com/aws/aws-sdk-go/aws"
    "github.com/aws/aws-sdk-go/aws/credentials"
    "github.com/aws/aws-sdk-go/aws/session"
    "github.com/aws/aws-sdk-go/service/s3"
)

// UploadFileToS3 saves a file to aws bucket and returns the url to // the file and an error if there's any
func UploadFileToS3(s *session.Session, file multipart.File, fileHeader *multipart.FileHeader) (string, error) {
  // get the file size and read
  // the file content into a buffer
  size := fileHeader.Size
  buffer := make([]byte, size)
  file.Read(buffer)

  // create a unique file name for the file
  tempFileName := "pictures/" + bson.NewObjectId().Hex() + filepath.Ext(fileHeader.Filename)

  // config settings: this is where you choose the bucket,
  // filename, content-type and storage class of the file
  // you're uploading
  _, err := s3.New(s).PutObject(&s3.PutObjectInput{
     Bucket:               aws.String("test-bucket"),
     Key:                  aws.String(tempFileName),
     ACL:                  aws.String("public-read"),// could be private if you want it to be access by only authorized users
     Body:                 bytes.NewReader(buffer),
     ContentLength:        aws.Int64(int64(size)),
     ContentType:        aws.String(http.DetectContentType(buffer)),
     ContentDisposition:   aws.String("attachment"),
     ServerSideEncryption: aws.String("AES256"),
     StorageClass:         aws.String("INTELLIGENT_TIERING"),
  })
  if err != nil {
     return "", err
  }
  return tempFileName, err
}

func handler(w http.ResponseWriter, r *http.Request) {
    maxSize := int64(1024000) // allow only 1MB of file size
    err := r.ParseMultipartForm(maxSize)
    if err != nil {
       log.Println(err)
       fmt.Fprintf(w, "Image too large. Max Size: %v", maxSize)
       return
    }

    file, fileHeader, err := r.FormFile("profile_picture")
    if err != nil {
        log.Println(err)
        fmt.Fprintf(w, "Could not get uploaded file")
        return
    }
    defer file.Close()

    // create an AWS session which can be
    // reused if we're uploading many files
    s, err := session.NewSession(&aws.Config{
        Region: aws.String("us-east-2"),
        Credentials: credentials.NewStaticCredentials(
            "secret-id", // id
            "secret-key",   // secret
            ""),  // token can be left blank for now
    })
    if err != nil {
        fmt.Fprintf(w, "Could not upload file")
    }
    fileName, err := UploadFileToS3(s, file, fileHeader)
    if err != nil {
        fmt.Fprintf(w, "Could not upload file")
    }
    fmt.Fprintf(w, "Image uploaded successfully: %v", fileName)
}

func main() {
    http.HandleFunc("/", handler)
    log.Println("Upload server started")
    log.Fatal(http.ListenAndServe(":8080", nil))
}

As you can see from the complete code, the UploadFileToS3() receives the file size and creates a new buffer with the file size and reads the content of the file to the buffer. Next a random file name is generated to be stored on the aws bucket (You don’t want file names clashing…), and the extension of the file is appended to the end of the filename.

Next, we create a new S3 bucket instance from the session passed to the function and pass some information to the PutObject function that takes in a PutObjectInput type. The PutObjectInput type sets up the information needed to add the file to the bucket. These information includes the name of the bucket, which is the name of the bucket we created earlier in the aws S3 console, the key, which holds the name of the file, ACL is the type of permission given to the file (it can be public-read or private or any other permissions listed here), then the body which is the actual content of the file in the buffer and then the size of the file. Make sure to pass the actual size of the file.

The upload function receives a session, let’s talk about it.

We are creating a session which can be reused every time we want to upload a file, hence we are passing it to the upload function. Creating the session requires some credentials like the bucket region, the secret id, and the secret key. These can be obtained from the aws IAM console. If you don’t know how to generate it read this.

These credentials are passed to the session.NewSession() as an aws.Config values. this session can now be passed to the upload function.

Here is a link to the complete code on github.

This article was originally posted on medium. You can check it out here