Notes from my AWS training

A couple of weeks ago, on December 8-10, I took a three day class for AWS training. Really intensive, but definitely worth it. The week after, I got the AWS Developer Associate certificate 🎉

I took some notes during training that I don’t want to lose, so I decided to write them on a post, for future review and in hopes that they can help someone else. Please beware these notes are pretty scarce and might not make much sense without the presentation slides and videos from the training.

Day One

AWS offers solutions for IaaS, PaaS and SaaS. Some examples are:

IaaS (EC2, ECS)
PaaS (DynamoDB, Lambda)
SaaS (AWS Marketplace apps)

On cloud hosting:

VMs need a guest OS to run (slow boot times, and resource overhead)
Containers need orchestration (non-trivial)
Server-less -> 🚀

AWS launched in 2006 with only 3 services: EC2, S3, and SQS

All services provide an API to perform any tasks.

AWS CLI and AWS management console both use the REST API
Everything you can do in the AWS console can be done via API
API requests must be signed (Signature Version 4)

Observability

Monitoring, logging, tracing
CloudWatch -> logs, events
CloudTrail -> auditing tool, not real-time
X-Ray -> service maps, identify performance bottlenecks

IAM Policies

Only three services support inline policies: S3, SQS, OpsWorks
Use managed policies when possible
With IAM, everything is deny by default
1. Explicit deny
2. Explicit allow

IAM Policy Simulator -> Troubleshooting of permission issues

IAM Role === Identity (cannot have two identities at once)

Trusted Advisor (support)

Save money
Security tips
Identify security risks

Identities:

User
Role

GetSessionToken API call for MFA (multi-factor authentication)

STS AssumeRole operation -> no MFA

S3

11 nines -> durability
4 nines -> availability
S3 is an object store (not files)
Bucket names must be globally unique, always lowercase
- 1024 characters max
- Must be DNS compatible

S3 versioning:

Must delete all versions one by one
Deleting a file inserts a “delete mark”

S3 select:

individual document basis
query data inside documents (eg. JSON documents)

Athena -> cross-document queries with actual SQL statements

S3 pre-signed urls

avoid leechers
temporary urls that expire after a threshold

Day Two

Neptune -> Graph database

Aurora -> Serverless relational database (MySQL or Postgres)

DynamoDB

Partitions up to 10GB
Primary Key -> Partition Key + Sort Key (optional)
400Kb per item max, including attribute names
Throughput is divided among partitions
5 max Local Secondary Indexes (LSI) before table creation
20 max Global Secondary Indexes (GSI) -> new PK/SK, can be added at any time
Sparse index: when GSI PK does not exist on an item in the table

DynamoDB Streams

DynamoDB Streams -> S3 (JSON)
DynamoDB Streams -> SNS
DynamoDB Streams -> Lambda

Global tables -> multi-master arrangement

DynamoDB Transactions

10 items max
4MB max

Batch -> 16MB

DynamoDB Accelerator (DAX)

write-through cache (latency)
write around for heavy write applications

AWS Secrets Manager

key/value store
rotate secrets

AWS Lambda

Max duration: 15 min
Lambda Service (managed)
- poll for events
- pull invokation
Reserved concurrency
- limit number of concurrent invocations
- max concurrent invocations
Provisioned concurrency
- warm lambda instances
- avoid cold starts
- runs continuously
Canary deploys with aliases and versions
- 1% - 99%, 10% - 90%, etc
Lambda destination for async invocations: success / failure
Initial burst: 500-3000 cus
- +500/minute afterwards
- 1000 cus max/region by default

SALAD

Remember LAMP (Linux, Apache, MySQL, and PHP)?

SALAD is the new stack:

S3
API Gateway
Lambda
?
DynamoDB

API Gateway (REST)

Sell API keys on AWS marketplace
10k requests/sec
Throttle + daily quotes

Cloud Formation -> Infrastructure as code

SQS

Idempotence -> write consumers for no issues if duplicate messages are received
14 days max time for messages to stay in queue
SQS -> 1:1 only one consumer can get a message
Dead Letter Queue (DQL) for failed messages
SQS FIFO -> MessageDeduplicationId + SendMsg
When using Lambda consumers, polling is handled by Lambda Service

SNS

1:N multiple subscribers per topic
Retry policy
SNS filters -> avoid invocations
Order of messages is not guaranteed
FIFO supported on SNS since Dec 2020 re:Invent
No persistence
256Kb max per message

Use SNS to fanout:

                                      ┌───────────┐       ┌───────────┐
                                      │           │       │           │
                                 ┌───▶│    SQS    │──────▶│Consumer A │
┌───────────┐   ┌───────────┐    │    │           │       │           │
│           │   │           │    │    └───────────┘       └───────────┘
│ Producer  │──▶│    SNS    │────┤                                     
│           │   │           │    │    ┌───────────┐       ┌───────────┐
└───────────┘   └───────────┘    │    │           │       │           │
                                 └───▶│    SQS    │──────▶│Consumer B │
                                      │           │       │           │
                                      └───────────┘       └───────────┘

Amazon MQ

Runs Apache MQ
Managed service
MQTT for IoT
- AWS IoT Broker message -> MQTT

Step Functions

Allow for manual approval steps
State saved for one year max

ElastiCache

Memcached: multi-threaded, single availability zone
Redis: multi-az

Containers

Docker: only one process per container
Container = runtime engine + dependencies + code + configuration
ECR
- Public images since Dec 2020
- Integrated with Docker CLI
- ECR ~= Docker Hub