libaws

libaws

aws should be easy

github.com/nathants/libaws

why

aws is amazing, but it's hard to see the forest for the trees.

aws should:

it should be easy for a lambda to react to:

it should be easy to create:

how

declare and deploy groups of related aws infrastructure as infrastructure sets:

what

a simpler way to declare aws infrastructure that is easy to use and extend.

there are two ways to use it:

the primary entrypoints are:

infra-ensure is a positive assertion. it asserts that some named infrastructure exists, and is configured correctly, creating or updating it if needed.

many other entrypoints exist, and can be explored by type. they fall into two categories:

aws sdk, pulumi, terraform, cloudformation, and serverless

compared to the full aws api, systems declared as infrastructure sets:

if you want to use the full aws api, there are many great tools:

readme index

install

cli

go install github.com/nathants/libaws@latest

export PATH=$PATH:$(go env GOPATH)/bin

go api

go get github.com/nathants/libaws@latest

tldr

define an infrastructure set

>> cd examples/simple/go/s3 && tree
.
├── infra.yaml
└── main.go
name: test-infraset-${uid}

s3:
  test-bucket-${uid}:
    attr:
      - acl=private

lambda:
  test-lambda-${uid}:
    entrypoint: main.go
    attr:
      - concurrency=0
      - memory=128
      - timeout=60
    policy:
      - AWSLambdaBasicExecutionRole
    trigger:
      - type: s3
        attr:
          - test-bucket-${uid}
package main

import (
	"context"
	"fmt"
	"github.com/aws/aws-lambda-go/events"
	"github.com/aws/aws-lambda-go/lambda"
)

func handleRequest(_ context.Context, e events.S3Event) (events.APIGatewayProxyResponse, error) {
	for _, record := range e.Records {
		fmt.Println(record.S3.Object.Key)
	}
	return events.APIGatewayProxyResponse{StatusCode: 200}, nil
}

func main() {
	lambda.Start(handleRequest)
}

ensure the infrastructure set

view the infrastructure set

depth based colors by yaml

trigger the infrastructure set

quickly update lambda code

delete the infrastructure set

usage

explore the cli

>> libaws -h | grep ensure | head

codecommit-ensure             - ensure a codecommit repository
dynamodb-ensure               - ensure a dynamodb table
ec2-ensure-keypair            - ensure a keypair
ec2-ensure-sg                 - ensure a sg
ecr-ensure                    - ensure ecr image
iam-ensure-ec2-spot-roles     - ensure iam ec2 spot roles that are needed to use ec2 spot
iam-ensure-instance-profile   - ensure an iam instance-profile
iam-ensure-role               - ensure an iam role
iam-ensure-user-api           - ensure an iam user with api key
iam-ensure-user-login         - ensure an iam user with login

explore a cli entrypoint

>> libaws s3-ensure -h

ensure a s3 bucket

example:
 - libaws s3-ensure test-bucket acl=public versioning=true

optional attrs:
 - acl=VALUE        (values = public | private, default = private)
 - versioning=VALUE (values = true | false,     default = false)
 - metrics=VALUE    (values = true | false,     default = false)
 - cors=VALUE       (values = true | false,     default = false)
 - ttldays=VALUE    (values = 0 | n,            default = 0)

setting 'cors=true' uses '*' for allowed origins. to specify one or more explicit origins, do this instead:
 - corsorigin=http://localhost:8080
 - corsorigin=https://example.com

Usage: s3-ensure [--preview] NAME [ATTR [ATTR ...]]

Positional arguments:
  NAME
  ATTR

Options:
  --preview, -p
  --help, -h             display this help and exit

explore the go api

package main

import (
	"github.com/nathants/libaws/lib"
)

func main() {
    lib. (TAB =>)
      |--------------------------------------------------------------------------------|
      |f AcmClient func() *acm.ACM (Function)                                          |
      |f AcmClientExplicit func(accessKeyID string, accessKeySecret string, region stri|
      |f AcmListCertificates func(ctx context.Context) ([]*acm.CertificateSummary, erro|
      |f Api func(ctx context.Context, name string) (*apigatewayv2.Api, error) (Functio|
      |f ApiClient func() *apigatewayv2.ApiGatewayV2 (Function)                        |
      |f ApiClientExplicit func(accessKeyID string, accessKeySecret string, region stri|
      |f ApiList func(ctx context.Context) ([]*apigatewayv2.Api, error) (Function)     |
      |f ApiListDomains func(ctx context.Context) ([]*apigatewayv2.DomainName, error) (|
      |f ApiUrl func(ctx context.Context, name string) (string, error) (Function)      |
      |f ApiUrlDomain func(ctx context.Context, name string) (string, error) (Function)|
      |...                                                                             |
      |--------------------------------------------------------------------------------|
}

explore simple examples

explore complex examples

explore external examples

infrastructure set

an infrastructure set is defined by yaml or go struct and contains:

typical usage

design

tradeoffs

infra.yaml

use an infra.yaml file to declare an infrastructure set. the schema is as follows:

name: VALUE
lambda:
  VALUE:
    entrypoint: VALUE
    policy:     [VALUE ...]
    allow:      [VALUE ...]
    attr:       [VALUE ...]
    require:    [VALUE ...]
    env:        [VALUE ...]
    include:    [VALUE ...]
    trigger:
      - type: VALUE
        attr: [VALUE ...]
s3:
  VALUE:
    attr: [VALUE ...]
dynamodb:
  VALUE:
    key:  [VALUE ...]
    attr: [VALUE ...]
sqs:
  VALUE:
    attr: [VALUE ...]
vpc:
  VALUE:
    security-group:
      VALUE:
        rule: [VALUE ...]
keypair:
  VALUE:
    pubkey-content: VALUE
instance-profile:
  VALUE:
    allow: [VALUE ...]
    policy: [VALUE ...]

environment variable substitution

anywhere in infra.yaml you can substitute environment variables from the caller's environment:

the following variables are defined during deployment, and are useful in allow declarations:

name

defines the name of the infrastructure set.

s3

defines a s3 bucket:

dynamodb

defines a dynamodb table:

sqs

defines a sqs queue:

keypair

defines an ec2 keypair.

vpc

defines a default-like vpc with an internet gateway and public access.

security group

defines a security group on a vpc

instance profile

defines an ec2 instance profile.

lambda

defines a lambda.

entrypoint

defines the code of the lambda. it is one of:

attr

defines lambda attributes. the following can be defined:

policy

defines policies on the lambda's iam role.

allow

defines allows on the lambda's iam role.

env

defines environment variables on the lambda:

include

defines extra content to include in the lambda zip:

require

defines dependencies to install with pip in the virtualenv zip.

trigger

defines triggers for the lambda:

trigger types

ses

defines an ses email receiving trigger.

api

defines an apigateway v2 http api:

websocket

defines an apigateway v2 websocket api:

s3

defines an s3 trigger:

dynamodb

defines a dynamodb trigger:

sqs

defines a sqs trigger:

schedule

defines a schedule trigger:

ecr

defines an ecr trigger:

bash completion

source completions.d/libaws.sh

extending

drop down to the aws go sdk and implement what you need.

extend an existing mutative operation or add a new one.

you will find examples in cmd/ and lib/ that can provide a good place to start.

you can reuse many existing operations like:

alternatively, lift and shift to other infrastructure automation tooling. ls and describe operations will give you all the information you need.

testing

run all integration tests aws with tox:

export LIBAWS_TEST_ACCOUNT=$ACCOUNT_NUM

tox

run one integration test aws with tox:

export LIBAWS_TEST_ACCOUNT=$ACCOUNT_NUM

tox -- bash -c 'cd examples/simple/python/api/ && python test.py'