CRUD Bucket S3 AWS

CRUD model for object handling implemented with Systems Manager Parameter Store, S3 Bucket, Api-Gateway, Serverless-Framework, Lambda, NodeJs, aws-sdk-v3, among others others. AWS services are deployed locally. The project code and documentation (except technical docs) have been developed in English.

Table of Contents 📜

View

Section 1) Description, configuration and technologies

1.0) Description 🔝

View

1.0.0) General Description

This app is divided into several functionalities/components. The first component or connection layer (/bucket) is the interaction with aws-sdk and the bucket. It is modularized in such a way that we have .js files for creating s3 clients, bucket reading, bucket writing, etc. Then for the application layer (/helpers) we have header validations, request body, date formats, authentication, etc. Next, the controller/view layer (/controllers) is defined by the CRUD operations possible in each of the defined lambdas.

1.0.1) Architecture and Operation Description

The image of the aws architecture used describes the general flow of the app. Any request to the bucket starts from a client (Postman, server, etc.).
Step 0 : This request is received by the api-gateway and will only be validated if the correct x-api-key is found within the headers of said request.
Steps 1A, 1B, etc. : All these steps correspond to an endpoint with its specific resource. For example, for uploadObject (1A) it is http://localhost:4000/dev/upload-object .... check these endpoints in endpoints section. Each lambda performs x-api-key and Bearer token verification, among others.
Steps 2 : The lambdas perform the corresponding ssm validations with the System Manager Parameter Store.. they validate tokens, values for the s3 bucket, etc.
Steps 3 : The lambdas perform the necessary requests and operations against the s3 bucket (reading, updating, deleting and inserting objects).
Clarifications : This operation is emulated within the same network and in a local environment with the corresponding serverless plugins.

1.1) Project Execution 🔝

View

Once a work environment has been created through an ide, we clone the project

git clone https://github.com/andresWeitzel/CRUD_Bucket_S3_AWS

We position ourselves on the project

cd 'projectName'

We install the LTS version of Nodejs(v18)
We install the Serverless Framework globally if we haven't already. I recommend version three since it's free and doesn't ask for credentials. The latest version (four) can be used without any problems, although it requires a paid version.

npm install -g serverless@3

We check the installed version of Serverless

sls -v

We install all the necessary packages

npm i

The ssm variables used in the project are kept to simplify the configuration process of the project. It is recommended to add the corresponding file (serverless_ssm.yml) to the .gitignore.
The following script configured in the package.json of the project is in charge of
Starting serverless-offline (serverless-offline)

"scripts": {
"serverless-offline": "sls offline start",
"start": "npm run serverless-offline"
},

We run the app from the terminal.

npm start

If we get a message indicating that port 9324 is already in use, we can terminate all dependent processes.

npx kill-port 9324

We launch the app from terminal.

npm start

1.2) Setting up the project from scratch 🔝

View

We create a work environment through an IDE, after creating a folder we position ourselves on it

cd 'projectName'

We install the LTS version of Nodejs(v18)
We install the Serverless Framework globally if we haven't already. I recommend version three since it's free and doesn't ask for credentials. The latest version (four) can be used without any problems, although it requires a paid version.

npm install -g serverless@3

Check the installed Serverless version

sls -v

Initialize a serverless template

serverless create --template aws-nodejs

Initialize an npm project

npm init -y

Install local S3

npm install serverless-s3-local --save-dev

Install the s3 Client

npm install @aws-sdk/client-s3

Install serverless offline

npm i serverless-offline --save-dev

Install serverless ssm

npm i serverless-offline-ssm --save-dev

The ssm variables used in the project are kept to simplify the configuration process of the project. It is recommended to add the corresponding file (serverless_ssm.yml) to the .gitignore.
The following script configured in the package.json of the project is in charge of
Starting serverless-offline (serverless-offline)

"scripts": {
"serverless-offline": "sls offline start",
"start": "npm run serverless-offline"
},

We run the app from the terminal.

npm start

If we get a message indicating that port 9324 is already in use, we can terminate all dependent processes.

npx kill-port 9324

We launch the app from terminal.

npm start

1.3) Technologies 🔝

View

Technology	Version	Purpose
SDK	4.3.2	Automatic Module Injection for Lambdas
Serverless Framework Core v3	3.23.0	AWS Core Services
Systems Manager Parameter Store (SSM)	3.0	Management of Environment Variables
Amazon Api Gateway	2.0	API Manager, Authentication, Control and Processing
Amazon S3	3.0	Object Container
NodeJS	14.18.1	JS Library
VSC	1.72.2	IDE
Postman	10.11	Http Client
CMD	10	Command Prompt
Git	2.29.1	Version Control

Plugins

Plugin	Description
Serverless Plugin	Libraries for Modular Definition
serverless-offline	This serverless plugin emulates AWS λ and API Gateway on-premises
serverless-offline-ssm	Finds environment variables that match SSM parameters at build time and replaces them from a file
serverless-s3-local	Serverless plugin to run S3 clones locally

Extensions

Extension
Prettier - Code formatter
YAML - Autoformatter .yml (alt+shift+f)

Section 2) Endpoints and Examples.

2.0) Endpoints and Resources 🔝

View

2.1.0) Variables in Postman

Variable	Initial Value	Current Value
base_url	http://localhost:4000	http://localhost:4000
x-api-key	f98d8cd98h73s204e3456998ecl9427j	f98d8cd98h73s204e3456998ecl9427j
bearer_token	Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c	Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c

2.1.1) Upload an object to the s3 bucket

Request | Code snippet

curl --location 'http://localhost:4000/dev/upload-object' \
--header 'x-api-key: f98d8cd98h73s204e3456998ecl9427j' \
--header 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c' \
--header 'Content-Type: application/json' \
--data '{
    "type":"image",
    "format":"png",
    "description":"5000 × 3061 png",
    "url":"https://www.bing.com/images/search?view=detailV2&ccid=Tf4BFI68&id=D66EF5BFB7DA0A645A70240C32CB8664E8F8BF09&thid=OIP.Tf4BFI6846neirVSebC0vAHaEi&mediaurl=https%3a%2f%2flogos-download.com%2fwp-content%2fuploads%2f2016%2f09%2fNode_logo_NodeJS.png&cdnurl=https%3a%2f%2fth.bing.com%2fth%2fid%2fR.4dfe01148ebce3a9de8ab55279b0b4bc%3frik%3dCb%252f46GSGyzIMJA%26pid%3dImgRaw%26r%3d0&exph=3061&expw=5000&q=jpg+nodejs&simid=608055434302923247&FORM=IRPRST&ck=2FF3D39CAEF945F20B996CF6042F88A6&selectedIndex=1&ajaxhist=0&ajaxserp=0"
}'

Response

{
    "message": {
        "type": "image",
        "format": "png",
        "description": "5000 × 3061 png",
        "url": "https://www.bing.com/images/search?view=detailV2&ccid=Tf4BFI68&id=D66EF5BFB7DA0A645A70240C32CB8664E8F8BF09&thid=OIP.Tf4BFI6846neirVSebC0vAHaEi&mediaurl=https%3a%2f%2flogos-download.com%2fwp-content%2fuploads%2f2016%2f09%2fNode_logo_NodeJS.png&cdnurl=https%3a%2f%2fth.bing.com%2fth%2fid%2fR.4dfe01148ebce3a9de8ab55279b0b4bc%3frik%3dCb%252f46GSGyzIMJA%26pid%3dImgRaw%26r%3d0&exph=3061&expw=5000&q=jpg+nodejs&simid=608055434302923247&FORM=IRPRST&ck=2FF3D39CAEF945F20B996CF6042F88A6&selectedIndex=1&ajaxhist=0&ajaxserp=0",
        "uuid": 104851112
    }
}

2.1.2) Get an object from the bucket based on its uuid

Request | Code snippet

curl --location 'http://localhost:4000/dev/get-object/103053674' \
--header 'x-api-key: f98d8cd98h73s204e3456998ecl9427j' \
--header 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c' \
--header 'Content-Type: application/json' \
--data ''

Response

{
    "message": {
        "type": "image",
        "format": "jpg",
        "description": "1000 × 1261 png",
        "url": "https://www.bing.com/images/search?view=detailV2&ccid=Tf4BFI68&id=D66EF5BFB7DA0A645A70240C32CB8664E8F8BF09&thid=OIP.Tf4BFI6846neirVSebC0vAHaEi&mediaurl=https%3a%2f%2flogos-download.com%2fwp-content%2fuploads%2f2016%2f09%2fNode_logo_NodeJS.png&cdnurl=https%3a%2f%2fth.bing.com%2fth%2fid%2fR.4dfe01148ebce3a9de8ab55279b0b4bc%3frik%3dCb%252f46GSGyzIMJA%26pid%3dImgRaw%26r%3d0&exph=3061&expw=5000&q=jpg+nodejs&simid=608055434302923247&FORM=IRPRST&ck=2FF3D39CAEF945F20B996CF6042F88A6&selectedIndex=1&ajaxhist=0&ajaxserp=0",
        "uuid": 103053674
    }
}

2.1.3) Update an object in the s3 bucket

Request | Code snippet

curl --location --request PUT 'http://localhost:4000/dev/edit-object/104851112' \
--header 'x-api-key: f98d8cd98h73s204e3456998ecl9427j' \
--header 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c' \
--header 'Content-Type: application/json' \
--data '{
    "type":"image",
    "format":"jpg",
    "description":"1200 × 1201 png",
    "url":"https://www.bing.com/images/search?view=detailV2&ccid=Tf4BFI68&id=D66EF5BFB7DA0A645A70240C32CB8664E8F8BF09&thid=OIP.Tf4BFI6846neirVSebC0vAHaEi&mediaurl=https%3a%2f%2flogos-download.com%2fwp-content%2fuploads%2f2016%2f09%2fNode_logo_NodeJS.png&cdnurl=https%3a%2f%2fth.bing.com%2fth%2fid%2fR.4dfe01148ebce3a9de8ab55279b0b4bc%3frik%3dCb%252f46GSGyzIMJA%26pid%3dImgRaw%26r%3d0&exph=3061&expw=5000&q=jpg+nodejs&simid=608055434302923247&FORM=IRPRST&ck=2FF3D39CAEF945F20B996CF6042F88A6&selectedIndex=1&ajaxhist=0&ajaxserp=0"
}'

Response

{
    "message": {
        "type": "image",
        "format": "jpg",
        "description": "1200 × 1201 png",
        "url": "https://www.bing.com/images/search?view=detailV2&ccid=Tf4BFI68&id=D66EF5BFB7DA0A645A70240C32CB8664E8F8BF09&thid=OIP.Tf4BFI6846neirVSebC0vAHaEi&mediaurl=https%3a%2f%2flogos-download.com%2fwp-content%2fuploads%2f2016%2f09%2fNode_logo_NodeJS.png&cdnurl=https%3a%2f%2fth.bing.com%2fth%2fid%2fR.4dfe01148ebce3a9de8ab55279b0b4bc%3frik%3dCb%252f46GSGyzIMJA%26pid%3dImgRaw%26r%3d0&exph=3061&expw=5000&q=jpg+nodejs&simid=608055434302923247&FORM=IRPRST&ck=2FF3D39CAEF945F20B996CF6042F88A6&selectedIndex=1&ajaxhist=0&ajaxserp=0",
        "uuid": 104851112
    }
}

2.1.4) Delete an object from the bucket

Request | Code snippet

curl --location --request DELETE 'http://localhost:4000/dev/delete-object/104851112' \
--header 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c' \
--header 'x-api-key: f98d8cd98h73s204e3456998ecl9427j' \
--header 'Content-Type: application/json'

Response

{
    "message": "Removed object with uuid 104851112 successfully."
}

Section 3) Functionality Testing and References

3.0) Functionality Testing 🔝

View

Name		Name	Last commit message	Last commit date
Latest commit History 88 Commits
.github/workflows		.github/workflows
doc/assets		doc/assets
src		src
translation		translation
.gitattributes		.gitattributes
.gitignore		.gitignore
.jshintrc		.jshintrc
.prettierrc		.prettierrc
LICENSE		LICENSE
README.md		README.md
package-lock.json		package-lock.json
package.json		package.json
serverless.yml		serverless.yml
serverless_ssm.yml		serverless_ssm.yml

License

andresWeitzel/CRUD_Amazon_S3_AWS

Folders and files

Latest commit

History

Repository files navigation

CRUD Bucket S3 AWS

Table of Contents 📜

Section 1) Description, Configuration, and Technologies

Section 2) Endpoints and Examples

Section 3) Functionality Testing and References

Section 1) Description, configuration and technologies

1.0) Description 🔝

1.0.0) General Description

1.0.1) Architecture and Operation Description

1.1) Project Execution 🔝

1.2) Setting up the project from scratch 🔝

1.3) Technologies 🔝

Plugins

Extensions

Section 2) Endpoints and Examples.

2.0) Endpoints and Resources 🔝

2.1.0) Variables in Postman

2.1.1) Upload an object to the s3 bucket

Request | Code snippet

Response

2.1.2) Get an object from the bucket based on its uuid

Request | Code snippet

Response

2.1.3) Update an object in the s3 bucket

Request | Code snippet

Response

2.1.4) Delete an object from the bucket

Request | Code snippet

Response

Section 3) Functionality Testing and References

3.0) Functionality Testing 🔝

Watch video

3.1) References 🔝

Bucket Configuration

Tools

AWS-SDK

API Gateway

Libraries

About

Topics

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases 6

Packages 0

Uh oh!

Contributors 2

Uh oh!

Languages

Packages