Basic AWS S3 operations using C#

Created by
Saturday, June 4, 2016

What is AWS S3?  AWS S3 is Amazon's cloud storage solution. Noted as “Amazon Simple Storage Service is storage for the internet”. AWS provide a very simple file explorer style interface which you can access directly in any modern browser.

 

Could it be a storage solution for my software applications? Most certainly! S3 makes it so simple to put any file type to and from different buckets. Bucket (bucket: A bucket is a logical unit of storage in object storage service, Simple Storage Solution S3. Buckets are used to store objects, which consist of data and metadata that describes the data.) Once included in your project you can read and write practically anything to and from your S3 buckets.

Data: My files themselves.

MetaData: Key info related to my file.

Sending files to and from S3 couldn't be any simpler using their SDK. A few of which are listed below. For the purposes of this article we are going to write an object (blob) to an s3 bucket using C# and the .NET SDK provided by Amazon. It’s good to note that AWS S3 provide many other SDK’s for different environments as listed below.

Android Browser iOS
JAVA .NET Node.js
PHP Python Ruby
Go C++  

Let's get Started: Setting up

In this feature we will be using a simple console application to write a file from our desktop to an Amazon bucket. Open visual studio create a new C# Console Application. Next we are going to install the Amazon SDK for .NET. Right click on the solution and select the nuget package manager. Alternatively you could open the package manager console and key in the following ‘Install-Package AWSSDK’.

Place a temporary .txt file on your desktop with some sample text inside.

The fun stuff: Coding

We begin with the configuration needed to grant access to our S3 solutions. It may be a good idea for you to store these in another class or configuration file and interface it  so that you can inject it into any area of your application you desire. To do this we need to create an AmazonS3Client object [_s3Client] and assign it our personal key, secret and region endpoint.

using Amazon;
using Amazon.Runtime;

namespace LMS.AWS.Services
{
    public sealed class AWSConfig
    {
        public static BasicAWSCredentials GetCredentials()
        {
            var key = "YOURACCESSKEYHERE";
            var secret = "youRs3CretKeyheR3_youRs3CretKeyheR3";

            return new BasicAWSCredentials(key, secret);
        }

        public RegionEndpoint GetRegion()
        {
            // Your amazon region
            var region = RegionEndpoint.USWest2;
            return region;
        }
    }
}

Now lets create an operations class with a method we will use for writing objects to the bucket. It's probably best practise to keep all your operations in the one class to avoid repeating yourself. In the future we will use this class for getting objects, listing them etc. I comment the most important components of this class through the code.

 

 

using System;
using System.Collections.Generic;
using System.IO;
using Amazon;
using Amazon.S3;
using Amazon.S3.IO;
using Amazon.S3.Model;
using LMS.Common.Classes;

namespace LMS.AWS.Services
{
    public class AWSS3Operations : LoggingBase, IAWSS3Operations
    {
        //private readonly AmazonS3Client _s3Client = new AmazonS3Client(AWSConfig.GetCredentials(), RegionEndpoint.USWest2);

        /// <summary>
        ///     Write an obejct to the given AWS Bucket
        /// </summary>
        /// <param name="bucketName"></param>
        /// <param name="keyName"></param>
        /// <param name="contentBody"></param>
        /// <param name="contentType"></param>
        public void WriteAnObject(AmazonS3Client _s3Client, string bucketName, string keyName, string contentBody, string contentType)
        {
            try
            {
                // Put request including metadata along with the content
                var putRequest = new PutObjectRequest
                {
                    BucketName = bucketName,
                    Key = keyName,
                    ContentBody = contentBody,
                    ContentType = contentType
                };
                
                // Include Metadata, Date object added
                //putRequest.Metadata.Add("DateAdded", DateTime.Now.ToString());

                var response = _s3Client.PutObject(putRequest);
            }
            catch (AmazonS3Exception amazonS3Exception)
            {
                // Catch AWS exception.
                if (amazonS3Exception.ErrorCode != null && (amazonS3Exception.ErrorCode.Equals("InvalidAccessKeyId") ||
                                                            amazonS3Exception.ErrorCode.Equals("InvalidSecurity")))
                {
                    LoggerService.Error(string.Format("AWS access error. AWS ErrorCode: {0} AWS ErrorType: {1}",
                        amazonS3Exception.ErrorCode,
                        amazonS3Exception.ErrorType));
                }
                else
                {
                    LoggerService.Error(
                        string.Format("AWS ListObjects unknown fatal error. AWS ErrorCode: {0} AWS ErrorType: {1}",
                            amazonS3Exception.ErrorCode,
                            amazonS3Exception.ErrorType));
                }
            }
        }

        /// <summary>
        ///     List the objects in a bucket
        /// </summary>
        /// <param name="bucket"></param>
        /// <param name="prefix"></param>
        public List<string> ListObjects(AmazonS3Client _s3Client, string bucket, string prefix)
        {
            var items = new List<string>();

            try
            {
                var request = new ListObjectsRequest
                {
                    BucketName = bucket,
                    Prefix = prefix,
                    MaxKeys = 2
                };

                do
                {
                    var response = _s3Client.ListObjects(request);

                    foreach (var entry in response.S3Objects)
                    {
                        items.Add(entry.Key);
                        // Write to a view or return a list
                    }
                }
                while (request != null);
            }
            catch (AmazonS3Exception amazonS3Exception)
            {
                // Catch AWS exception.
                if (amazonS3Exception.ErrorCode != null && (amazonS3Exception.ErrorCode.Equals("InvalidAccessKeyId") ||
                                                            amazonS3Exception.ErrorCode.Equals("InvalidSecurity")))
                {
                    LoggerService.Error(string.Format("AWS access error. AWS ErrorCode: {0} AWS ErrorType: {1}",
                        amazonS3Exception.ErrorCode,
                        amazonS3Exception.ErrorType));
                }
                else
                {
                    LoggerService.Error(
                        string.Format("AWS ListObjects fatal error. aws errorcode: {0} aws errortype: {1}",
                            amazonS3Exception.ErrorCode,
                            amazonS3Exception.ErrorType));
                }
            }

            return items;
        }

        /// <summary>
        ///     AWS Get an object
        /// </summary>
        /// <param name="bucketName"></param>
        /// <param name="keyName">Friendly Name</param>
        public void GetObject(AmazonS3Client _s3Client, string bucketName, string keyName)
        {
            try
            {
                using (_s3Client)
                {
                    GetObjectRequest request = new GetObjectRequest
                    {
                        BucketName = bucketName,
                        Key = keyName
                    };

                    using (GetObjectResponse response = _s3Client.GetObject(request))
                    {
                        string dest = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Desktop), keyName);
                        if (!File.Exists(dest))
                        {
                            response.WriteResponseStreamToFile(dest);
                        }
                    }
                }
            }
            catch (AmazonS3Exception amazonS3Exception)
            {
                // Catch AWS exception.
                if (amazonS3Exception.ErrorCode != null && (amazonS3Exception.ErrorCode.Equals("InvalidAccessKeyId") ||
                                                            amazonS3Exception.ErrorCode.Equals("InvalidSecurity")))
                {
                    LoggerService.Error(string.Format("AWS access error. AWS ErrorCode: {0} AWS ErrorType: {1}",
                        amazonS3Exception.ErrorCode,
                        amazonS3Exception.ErrorType));
                }
                else
                {
                    LoggerService.Error(
                        string.Format("AWS GetObject attempt fatal error. aws errorcode: {0} aws errortype: {1}",
                            amazonS3Exception.ErrorCode,
                            amazonS3Exception.ErrorType));
                }
            }
        }

        /// <summary>
        ///     AWS Read an object
        /// </summary>
        /// <param name="bucketName"></param>
        /// <param name="keyName"></param>
        /// <returns></returns>
        public string ReadDataObject(AmazonS3Client _s3Client, string bucketName, string keyName)
        {
            var responseBody = "";

            using (_s3Client)
            {
                var request = new GetObjectRequest
                {
                    BucketName = bucketName,
                    Key = keyName
                };

                using (var response = _s3Client.GetObject(request))
                using (var responseStream = response.ResponseStream)
                using (var reader = new StreamReader(responseStream))
                {
                    var title = response.Metadata["x-amz-meta-title"];

                    responseBody = reader.ReadToEnd();
                }
            }

            return responseBody;
        }


        /// <summary>
        ///     Finds an objects existance.
        /// </summary>
        /// <param name="bucketName"></param>
        /// <param name="keyName"></param>
        public bool FindObject(AmazonS3Client _s3Client, string bucketName, string keyName)
        {
            var response = false;

            try
            {
                var s3FileInfo = new S3FileInfo(_s3Client, bucketName, keyName);
                if (s3FileInfo.Exists)
                {
                    response = true;
                }
            }
            catch (AmazonS3Exception amazonS3Exception)
            {
                // Catch AWS exception.
                if (amazonS3Exception.ErrorCode != null && (amazonS3Exception.ErrorCode.Equals("InvalidAccessKeyId") ||
                                                            amazonS3Exception.ErrorCode.Equals("InvalidSecurity")))
                {
                    LoggerService.Error(string.Format("AWS access error. AWS ErrorCode: {0} AWS ErrorType: {1}",
                        amazonS3Exception.ErrorCode,
                        amazonS3Exception.ErrorType));
                }
                else
                {
                    LoggerService.Error(
                        string.Format("AWS ListObjects unknown fatal error. AWS ErrorCode: {0} AWS ErrorType: {1}",
                            amazonS3Exception.ErrorCode,
                            amazonS3Exception.ErrorType));
                }
            }

            return response;
        }
    }
}

You will notice that i pass through an AmazonS3Exception whenever an exception is caught. This is up to you personally but i do find it useful to log detailed exceptions that provide S3 detailed errors rather than standard exception types.

Simple as that! You can apply the fundamentals of these methods to create more complex operations depending on your requirements.

 

References:

http://docs.aws.amazon.com/AmazonS3/latest/dev/Welcome.html

http://searchaws.techtarget.com/definition/AWS-bucket

https://aws.amazon.com/tools/

https://www.nuget.org/packages/AWSSDK/

http://docs.aws.amazon.com/AmazonS3/latest/dev/Welcome.html

Tags:
aws
s3
c#
.net
7
Rating
1 2 3 4 5

Save

Saved

Report

Reported
Reasons
Cancel
0 comments