Email Setup

From "help me set up email" to an app that sends mail through Amazon SES. Start step 1 as soon as the user asks. This is AWS whatever the app is deployed on; another mail provider only when the user names one, and then it is the four SMTP_ lines in production.env (see elements man email).

What to say: only the quoted lines on this page, word for word. Nothing else reaches the user: no record names, keys, file names, config, tool names, reasoning, or what comes next.

How to run: each block is one command and stands alone. It reads what it needs from .elements/id, the env file and AWS, so nothing carries over in shell variables, and every aws call names --profile elements. No scripts around them, no set -e, no exit.

1. Sign In

Terminal
aws sts get-caller-identity --profile elements --query Account --output text

An account number means signed in. Anything else: do step 1 of AWS (opens in a new tab), then come back here.

2. One Question

Ask with a question tool when there is one (options "Just testing" and "Real users"). Otherwise send it as its own message:

Is this email for testing, or for real users?

Testing sends from the user's own address to the user's own address, and to nobody else. Real users needs a domain the user owns.

3. Testing

What email address should the test mail go to?

Then, with that address:

Terminal
aws sesv2 create-email-identity --profile elements --email-identity <address> --query VerifiedForSendingStatus --output text

Amazon sent you an email. Click the link in it, then come back.

Terminal
aws sesv2 get-email-identity --profile elements --email-identity <address> --query VerifiedForSendingStatus --output text

True means verified; False means the link is not clicked yet, so say the line above again. The address is also the from in step 5. Go to step 5.

4. Real Users

A domain in deploy.domain that is not under elements.online is the domain. Otherwise:

Which domain should your email come from? It needs to be one you own.

A user without one gets step 3 instead, with this first:

Without your own domain, mail lands in spam or is refused. Let's set up testing for now.

With the domain, this asks Amazon for its signing records and prints where the domain's DNS is hosted:

Terminal
D=<domain>
aws sesv2 create-email-identity --profile elements --email-identity $D >/dev/null 2>&1
for t in $(aws sesv2 get-email-identity --profile elements --email-identity $D --query 'DkimAttributes.Tokens' --output text); do echo "CNAME $t._domainkey.$D $t.dkim.amazonses.com"; done
[ -z "$(dig +short TXT _dmarc.$D)" ] && echo "TXT _dmarc.$D \"v=DMARC1; p=none;\""
Z=$(dig +noall +answer +authority SOA $D | awk '$4=="SOA" {print $1; exit}'); echo "zone ${Z%.} $(dig +short NS $Z | head -1)"

The last line names the DNS zone and its name server, which says which of these runs. A record that already exists is left as it is.

awsdns (Route 53):

Terminal
D=<domain>; ZN=<zone>
Z=$(aws route53 list-hosted-zones-by-name --profile elements --dns-name $ZN --query "HostedZones[?Name=='$ZN.'].Id" --output text)
for t in $(aws sesv2 get-email-identity --profile elements --email-identity $D --query 'DkimAttributes.Tokens' --output text); do
  aws route53 change-resource-record-sets --profile elements --hosted-zone-id $Z --change-batch "{\"Changes\":[{\"Action\":\"UPSERT\",\"ResourceRecordSet\":{\"Name\":\"$t._domainkey.$D\",\"Type\":\"CNAME\",\"TTL\":1800,\"ResourceRecords\":[{\"Value\":\"$t.dkim.amazonses.com\"}]}}]}" >/dev/null
done
[ -z "$(dig +short TXT _dmarc.$D)" ] && aws route53 change-resource-record-sets --profile elements --hosted-zone-id $Z --change-batch "{\"Changes\":[{\"Action\":\"CREATE\",\"ResourceRecordSet\":{\"Name\":\"_dmarc.$D\",\"Type\":\"TXT\",\"TTL\":1800,\"ResourceRecords\":[{\"Value\":\"\\\"v=DMARC1; p=none;\\\"\"}]}}]}" >/dev/null
echo done

digitalocean.com, with doctl signed in (see DigitalOcean (opens in a new tab)):

Terminal
D=<domain>; ZN=<zone>; n() { x=$1.$D.; echo ${x%.$ZN.}; }
for t in $(aws sesv2 get-email-identity --profile elements --email-identity $D --query 'DkimAttributes.Tokens' --output text); do
  doctl compute domain records create $ZN --context elements --record-type CNAME --record-name $(n $t._domainkey) --record-data $t.dkim.amazonses.com. --record-ttl 1800 >/dev/null
done
[ -z "$(dig +short TXT _dmarc.$D)" ] && doctl compute domain records create $ZN --context elements --record-type TXT --record-name $(n _dmarc) --record-data 'v=DMARC1; p=none;' --record-ttl 1800 >/dev/null
echo done

Anything else: the user adds them. Fill in the records the first block printed, as a table of type, name and value:

Add these records where you manage <domain>'s DNS, then come back.

Then ask Amazon to let the account send to anyone. A new account can only send to addresses it has verified until Amazon reviews it:

Terminal
D=<domain>
aws sesv2 put-account-details --profile elements --production-access-enabled --mail-type TRANSACTIONAL --website-url https://$D --contact-language EN \
  --use-case-description "Transactional mail only (sign-up confirmation, password reset, account notifications) to people who sign up at https://$D. No marketing, no purchased lists. Bounces and complaints go to the account suppression list."

Wait for the records. Each run waits at most four minutes; run it again until it prints SUCCESS:

Terminal
D=<domain>
for i in 1 2 3 4 5 6 7 8; do S=$(aws sesv2 get-email-identity --profile elements --email-identity $D --query 'DkimAttributes.Status' --output text); [ "$S" = SUCCESS ] && break; sleep 30; done
echo $S

The from in step 5 is no-reply@<domain>.

5. Connect the App

Set email.from in config.jsoc to the address. Then this makes a sending key that can do nothing but send mail and writes it to production.env:

Terminal
APP=$(cat .elements/id); R=$(aws configure get region --profile elements); F=config/env/production.env
aws iam create-user --profile elements --user-name elements-$APP-email --tags Key=elements:app,Value=$APP >/dev/null 2>&1
aws iam put-user-policy --profile elements --user-name elements-$APP-email --policy-name send --policy-document '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":"ses:SendRawEmail","Resource":"*"}]}'
read -r KEY SECRET <<< "$(aws iam create-access-key --profile elements --user-name elements-$APP-email --query 'AccessKey.[AccessKeyId,SecretAccessKey]' --output text)"
k=$(printf %s 11111111 | openssl dgst -sha256 -mac HMAC -macopt "key:AWS4$SECRET" | awk '{print $NF}')
for m in $R ses aws4_request SendRawEmail; do k=$(printf %s $m | openssl dgst -sha256 -mac HMAC -macopt hexkey:$k | awk '{print $NF}'); done
touch $F; grep -v -e '^EMAIL_LIVE=' -e '^SMTP_' $F > $F.tmp; mv $F.tmp $F
printf 'EMAIL_LIVE=true\nSMTP_HOST=email-smtp.%s.amazonaws.com\nSMTP_PORT=587\nSMTP_USER=%s\nSMTP_PASSWORD=%s\n' $R $KEY "$({ printf '\004'; printf %s $k | xxd -r -p; } | base64)" >> $F
echo ok

The password is derived from the key, the way Amazon's mail servers expect. It lives only in production.env. Development keeps logging mail to the server log instead of sending it.

6. Look at Them

With templates in app/emails, each gets a preview route, the way elements man email shows, with sample values. Look at each at desktop and phone width and fix what breaks the rules in elements man style/email. Then open each one in the user's browser, one at a time:

Here's your <name> email. Anything you'd like changed?

Change what the user asks for and reload. With no templates, skip this step.

7. Send One

Testing only, to the user's address. A new key takes a minute or two to work; on 535 wait 30 seconds and run it again.

Terminal
A=<address>; F=config/env/production.env; v() { grep "^$1=" $F | cut -d= -f2-; }
printf 'From: %s\r\nTo: %s\r\nSubject: Email works\r\n\r\nYour app can send email.\r\n' $A $A | curl -sS --url smtp://$(v SMTP_HOST):587 --ssl-reqd --user "$(v SMTP_USER):$(v SMTP_PASSWORD)" --mail-from $A --mail-rcpt $A -T - && echo sent

8. Deploy

With machines in config.jsoc:

Terminal
elements deploy -json

No errors, testing:

Email is set up. Check your inbox for a test message. Your app sends to you only, until you set up email for real users.

No errors, real users:

Email is set up, from no-reply@<domain>. Amazon reviews new senders, usually within a day, and emails you when it's done. Until then, mail only reaches addresses you've verified.

With no machines yet, the same line without deploying; the next deploy carries it.

Teardown

Confirm with the user first.

Terminal
APP=$(cat .elements/id); U=elements-$APP-email
for k in $(aws iam list-access-keys --profile elements --user-name $U --query 'AccessKeyMetadata[].AccessKeyId' --output text); do aws iam delete-access-key --profile elements --user-name $U --access-key-id $k; done
aws iam delete-user-policy --profile elements --user-name $U --policy-name send
aws iam delete-user --profile elements --user-name $U
F=config/env/production.env; grep -v -e '^EMAIL_LIVE=' -e '^SMTP_' $F > $F.tmp; mv $F.tmp $F

Then aws sesv2 delete-email-identity --profile elements --email-identity <address or domain>, and the DNS records by hand.