adamdabbracci

[adamdabbracci]

Does DNS resolution not work in AWS App Runner?

If you don’t know what AWS App Runner, it’s a really easy way to deploy Docker apps without all the messy nonsense of an ECS cluster. You point it to an ECR image, tell it the port, and you’re off! It’ll even deploy new versions of your image when they’re pushed... for $1/month/app(?).

Anyway. I have a UI built with Next.JS (sidenote: I clearly don’t understand SPA development these days, because Next needs a Node server??) To avoid CORS issues calling the API directly, I opted to use Next’s redirect functionality as a sort of reverse proxy. It worked in a local instance pointing to the deployed API.

When I deployed it to App Runner, it looked like the API calls were all being resolved to localhost, instead of the fully-qualified URL I had hard-coded. This, of course, resulted in a connection error, because nothing else was running on that image. Even if I changed it to something like Google, it would resolve to localhost.

I gave up trying to resolve this and am instead getting into the messy nonsense of an ECS deployment...

Update 12/27: More proof

I setup Auth0 for my app, and it worked great locally. When I deployed it, I got the following when the Auth0 client attempted to pull down the JWKS list:

requests.exceptions.ConnectionError: HTTPSConnectionPool(host='dev-p1j473ay.us.auth0.com', port=443): Max retries exceeded with url: /.well-known/jwks.json (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fcfbf9b7eb0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')) 12-27-2021 03:35:30 PM         raise ConnectionError(e, request=request)

There don’t appear to be any VPC settings on App Runner, so I’m not sure where to go from here.

[adamdabbracci]

How do I anonymously pull a public ECR image?

I must be missing something.

I’m trying to pull a Docker image from a public ECR repo in AWS and run it on a Lambda. They make it sound like this should be as easy as using the Docker Hub:

For image pulls, Amazon ECR Public accepts both anonymous pulls and pulls using an authentication token.

I published my image as a public ECR image, which gives you a very strange image URL format that includes the “default” registry alias:

public.ecr.aws/<ALIAS>/<IMAGE_URL>:latest

However, when I try to do the usual docker pull <IMAGE_URL>, I get this error:

Error response from daemon: pull access denied for <IMAGE_URL> repository does not exist or may require 'docker login': denied: Your authorization token has expired. Reauthenticate and try again.

I also tried with the ECR endpoints from this other page (AWS is so good at spreading one topic’s worth of docs across their site) and got:

Error response from daemon: error parsing HTTP 404 response body: invalid character '<' looking for beginning of value: "<UnknownOperationException/>\n"

It appears you actually need a temporary AWS authentication token. That’s a problem if you’re trying to make a programmatic call or or offer the image to someone who (gasp!) doesn’t use AWS.

Like I said, I must be missing something.

[adamdabbracci]

Pass AWS credentials to Docker when running locally

Sometimes, you just need to run your Docker images locally, using your real AWS credentials, and don’t want to mess around with IAM roles. Here’s a simple little script that will pass them for you:

AWS_ACCESS_KEY_ID=$(aws --profile default configure get aws_access_key_id)

AWS_SECRET_ACCESS_KEY=$(aws --profile default configure get aws_secret_access_key)

docker build . -t my-app-image

docker run -it --rm \  

-e AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID \  

-e AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY \   

my-app-image

[adamdabbracci]

Use Sharp With AWS Lambda

Dealing with images is hard. If you’re using Node as your backend language, Sharp will make it easy.

However, if you’re doing this on Lambda, you’ll need to compile the Sharp binary for the Lambda runtime before deploying it. It’s a pain in the ass. This script will compile the right binary, deploy to a “dev” environment in your Lambda application, then re-install the local version so you can keep editing:

rm -rf node_modules/sharp && SHARP_IGNORE_GLOBAL_LIBVIPS=1 npm install --arch=x64 --platform=linux sharp && sls deploy --stage dev && rm -rf node_modules/sharp && npm install

I add it to my package.json file as a script so I can run this command:

npm run deploy-dev

[adamdabbracci]

The Best Database Diagramming Tool

If you need a quick database diagram made without all of the nonsense that comes with most tools, check out quickdatabasediagrams.com. They have a free tier that lets you do pretty much everything - WITHOUT SIGNING UP. It’s super useful, and their support team is amazing.

[adamdabbracci]

RDS Proxy with Lambda

Amazon recently announced that the RDS Proxy service was in General Availability and therefore can be used for Prod. I am essite. I’m going to document my experience as I go through the process.

As usual, Amazon’s documentation is... lacking. It provides paragraph on paragraph of words, but at the end you’re still wondering how to actually use the Proxy in any meaningful way. There was only one dedicated page that I could find:

https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/rds-proxy.html

However, some others have created good tutorials and explanations of how and why to use it:

https://www.percona.com/community-blog/2020/01/07/a-first-look-at-amazon-rds-proxy/

https://idk.dev/improving-application-availability-with-amazon-rds-proxy/

Once you have the Proxy setup, you can simply connect to it without needing any special changes. Just point your ORM at the proxy URL instead of the Reader URL and you’re off to the races.

Function Speeds

The speed was... impressive to say the least. A cold start of a very simple Node function took around 300ms. Subsequent calls took less than 10ms. This is much better than the 5 second cold starts (and slow calls) previously seen.

Caveats

You can’t change the DB instance size once the proxy is created, meaning you can’t scale your database tier up. It’s not a big deal as long as you plan for the right amount of traffic.

The proxy can’t be public, which means you need special logic when running your serverless applications offline. I detailed my solution below.

There is still a cold start if the RDS Proxy has shut down. You can specify the idle time when configuring the proxy. Keep in mind that you pay for the time it’s active, so this could become a cost that doesn’t scale well with your infrastructure.

Working Locally or with Serverless Offline

When using the excellent Serverless Offline plugin (or just executing locally), you need to perform special logic to determine whether your Lambda function is running in AWS or locally. Thankfully, the Serverless framework provides a variable for that:

https://www.serverless.com/plugins/serverless-offline#the-processenvis_offline-variable

So you can do some magic like:

host: (process.env.IS_OFFLINE === "false") ? `YOUR_READER_URL` : process.env.DB_CREDS_HOST,

Weird/Infuriating Issue With VPC Setup

I tried using the same VPC as an existing database so I didn’t have to setup a bunch of new stuff for this test. For some reason, when trying to save the RDS proxy, I’m getting this error:

Minimum 2 AZs are needed to be covered by subnets but found only 1 AZs, requested subnet 'subnet-0b504b629635cdee0' is in an unsupported availability zone 'us-east-1e

I confirmed that the subnet in question was in us-east-1e. You can’t change the AZ for a subnet once it’s created, so I made a new one, in the same VPC, with the same Route Table, using private IPs... but they refused to show up in the list when creating a new Proxy. So I deleted the entire database and started a new one and allowed it to create it’s own VPC settings.

[adamdabbracci]

AWS Amplify + React Native PubSub

12:06:I’ve been learning React Native by building a Todo app. The backend is maintained by AWS Amplify. Today I’m addng PubSub and calling it programmatically from the API on behalf of the user. The tutorial appears to include a manual CLI step so it’s going to be a learning experience. The below is a real time log of my efforts.

Note: Amplify Auth is best served plain. Aka use Javascript, not Typescript if you want to use the out of the box Auth HOC. It’s written in JS and will fail in a Typescript enabled project.

11:40: Looking at the tutorial here to understand whats happening at a high level: https://docs.amplify.aws/lib/pubsub/getting-started/q/platform/js#aws-iot

11:48: Setting up IoT Core per the instructions: https://console.aws.amazon.com/iot/home?region=us-east-1

Pricing is based on how long a device is connected, which is great:

Connectivity pricing: $0.08 (per million minutes of connection)

For example, in the US East (N. Virginia) region you pay $0.042 per device per year (1 connection * $0.08/1,000,000 minutes of connection * 525,600 minutes/year) for 24/7 connectivity. In order to maintain connectivity, devices may send keep-alive (“Ping”) messages at frequencies ranging from 20 minutes to every 30s, and you do not incur any additional cost for these messages. See additional connectivity pricing details >>

11:55: I think you can use the endpoint that’s configured here:

https://console.aws.amazon.com/iot/home?region=us-east-1#/settings

12:06: Setup an “allow all�� IoT Policy since I’m just testing this for now.

12:13: Added the policy to my user following these instructions:

Allowing your Amazon Cognito Authenticated Role to access IoT Services

For your Cognito Authenticated Role to be able to interact with AWS IoT it may be necessary to update its permissions, if you haven’t done this before. One way of doing this is to log to your AWS Console, select CloudFormation from the available services. Locate the parent stack of your solution: it is usually named <SERVICE-NAME>-<CREATION_TIMESTAMP>.   Select the Resources tab and tap on AuthRole Physical ID.   The IAM console will be opened in a new tab. Once there, tap on the button Attach Policies, then search AWSIoTDataAccess and AWSIoTConfigAccess, select them and tap on Attach policy. 

I got caught up in something else and never finished this. Sorry :(

[adamdabbracci]

403 on AWS S3 Presigned URL POST/PUT

Things to check:

- CORS settings on the bucket are allowing your domain and methods

- The bucket name being used to generate the presigned URL matches the actual bucket name

- The account being used to generate the presigned URL has access to the S3 bucket. If you’re running the project locally, make sure you don’t have multiple named profiles and using the wrong one

- The role being used to sign it must have access for S3. For example, make sure the policy attached to it has READ/WRITE perms

- There has to be a AssumeRole Trust relation between the Role and S3. For example:

{  "Version": "2012-10-17",  "Statement": [    {      "Effect": "Allow",      "Principal": {        "Service": "ecs-tasks.amazonaws.com"      },      "Action": "sts:AssumeRole"    },    {      "Effect": "Allow",      "Principal": {        "Service": "s3.amazonaws.com"      },      "Action": "sts:AssumeRole"    }  ] }

[adamdabbracci]

Node Server Crashing At Launch

Problem 1 = Too many processes

One of my side projects suddenly started crashing whenever I would launch the Node server. No error, no message, just stop cold. At first, it was only one of two projects that had a bunch of shared dependencies. So naturally I assumed it was something specific to that API. I spent hours trying to figure it out - the other service continued to work every time I rebooted it. 

Turns out the problem wasn’t the code at all. For some reason, Node doesn’t terminate correctly 100% of the time, leaving ghost instances running in the background. Eventually, they start to compete for resources and your new servers can’t get any threads to run!

Solution = turn it on and off again by rebooting your computer..

Problem 2: Circular references

Check your references - it’s really helpful to use the delta tool in VS code and see where references were added. In my case, I had a class referencing itself...

[adamdabbracci]

Get A Free Shopify Store

Want to get started with Shopify (you know, the leading e-commerce platform)? Well now’s your lucky chance. I’m building free Shopify stores, from scratch, for anyone who visits my site: http://www.adamdabbracci.com/#/shopify. When I say “free” I mean “free as in $0″. I don’t retain ownership of the site and I don’t charge you any sort of fees. 100% free. Check it out!

[adamdabbracci]

Javascript Snippets: Date Countdown with FlipClock.js and Countdown.js

Countdowns are great to get your customers pumped up for an upcoming sale or event. I needed a clock for a Black Friday sale, but most of the clock libraries I found were either ugly or difficult to use. I stumbled across FlipClock.js, which blew the rest of the libraries out of the water. It had a gorgeous interface (that can be customized!), a simple API, and a drop-in JS file that just... worked. It offers the ability to countdown, but requires the parameter in seconds. This made it difficult to have a countdown to a date in the future, so I had to find a way to calculate the “seconds” from the time someone loads a page to the desired date. Enter Countdown.js, which calculates the difference between any two dates. Combining these two libraries is super simple!

THE LIBRARIES

http://flipclockjs.com/

http://countdownjs.org/

THE HTML

<div class="your-clock"></div>

THE CSS

You will need to download the CSS from the FlipClock site.

THE JAVASCRIPT

Once you include the Javascript files from both libraries, the actual script is really easy to implement. We first calculate the timespan between when the page is loaded and the date we want to count do. We also need to tell countdown.js to return the value in seconds and pass that value into the FlipClick initialization.

You will need to change the second Date() object to the date you want to count to. The format is (Year, Month, Day), but you can add time and timezones if you want.

var timespan = countdown(new Date(), new Date(2015, 10, 27), countdown.SECONDS);      console.log(timespan);

     var clock = $('.your-clock').FlipClock(timespan.seconds, {          countdown:true,          clockFace: 'DailyCounter'      });

And that’s it!

[adamdabbracci]

Shopify Tricks: Customizeable Message Of The Day

A "Message of the Day” header can be a powerful tool for driving conversions or providing customer-centric information above the fold.  In this post we’ll look at how to make a customizeable message header.

MVMT uses their message to convey important information - free shipping.

The Code

The HTML is pretty straightforward here. Open your template file (probably called theme.liquid  and add the following code:

   {%if settings.header_motd_enable %}    <section class="motd">      <h3>{{settings.header_motd}}</h3>    </section>    {% endif %}

The CSS can be added anywhere - I usually add mine to the bottom of one of the theme’s main CSS Liquid file:

/*message of the day*/ .motd {  background-image: url({{'header_motd_bg.jpg'}});  text-align:center;  height: {{settings.header_motd_height}}px;

} .motd h3 {  color: {{settings.header_motd_color}};  vertical-align: middle;  padding-top: {{settings.header_motd_paddingtop}}px;  font-weight:bold; }

Finally, we have to add the settings in order to make this easily customizeable by the end user, without any code. Open your settings_schema.json file and add the following snippet in between major nodes:

{   "name":"Message of the Day",   "settings": [     {       "type":"checkbox",       "id":"header_motd_enable",       "label":"Enable message of the day"     },     {       "type":"text",       "id":"header_motd",       "label":"Message of the day"     },     {       "type":"color",       "id":"header_motd_color",       "label":"Color"     },     {       "type":"number",       "id":"header_motd_height",       "label":"Height"     },     {       "type":"number",       "id":"header_motd_paddingtop",       "label":"Top padding"     },     {       "type": "image",       "id": "header_motd_bg.jpg",       "label": "MOTD Background"     }    ] },

Hit save on all of those files and you should see this in your settings:

Enter those values and your message will show up:

[adamdabbracci]

Shopify Tricks: Minimum Order

There are no built-in options to limit the minimum order amount on your store, so I found a neat workaround that should do the trick. The following code will only display the "Checkout" button on the cart if the order amount is over $15 (but you can change it to any number you would like).

{% if cart.total_price >= 1500 %}

<button type="submit" name="checkout" class="btn"> <span class="icon icon-cart"></span> Checkout </button>

{% else %}

<font color="red">There is a minimum order of $15.</font> {% endif %}

If you want a value other than $15, feel free to change 1500 to whatever value you would like. Make sure to change the display text $15 to match, otherwise you'll confuse your customers!

[adamdabbracci]

Shopify Tricks: Featured Items On Cart

I've had a few clients ask if I could put a list of "featured items" on their shopping cart to entice customers to buy more items. There are a few apps that recommend related items, but for those who want a more simple (and free) approach, this is for you.

The following instructions will let you put a text-based list of items in any collection on your shopping cart page (or anywhere else!).

1) Go to https://[YOUR_SHOP_NAME].myshopify.com/admin/themes

2) Click "Edit HTML/CSS" on your current theme.

3) Select "cart.liquid" (or any page you want to add this to) from under the Templates dropdown

4) Add the following snippet of code, replacing YOUR_COLLECTION_NAME with the name of your featured items collection:

<h4>Recommended Products:</h4> {% assign collectionName = 'YOUR_COLLECTION_NAME' %} {% tablerow product in collections[collectionName].products %} <a href="{{product.url}}">{{product.title}}</a><br> {% endtablerow %}

The result looks like this:

If your theme has the standard Shopify checkout box, you can align the list with your "Checkout" button like this:

using the following snippet:

<div class="cart-row"> <div class="grid">

{% if settings.cart_notes_enable %} <div class="grid-item large--one-half"> <label for="cartSpecialInstructions">Special instructions for seller</label> <textarea name="note" class="input-full" id="cartSpecialInstructions">{{ cart.note }}</textarea> </div>

{% else %} <div class="grid-item large--one-half"> <h4>Recommended Products:</h4> {% assign collectionName = 'YOUR_COLLECTION_NAME' %} {% tablerow product in collections[collectionName].products %} <a href="{{product.url}}">{{product.title}}</a><br> {% endtablerow %} </div> {%endif%} <div class="grid-item text-right large--one-half">

<div class="cart-subtotal"> Subtotal <span class="h1 cart-subtotal--price"> {% include 'price' with cart.total_price %} </span> </div>

<input type="submit" name="update" class="btn-secondary update-cart" value="Update cart">

<button type="submit" name="checkout" class="btn"> <span class="icon icon-cart"></span> Checkout </button>

{% if additional_checkout_buttons %} <p>{{ content_for_additional_checkout_buttons }}</p> {% endif %}

</div>

</div> </div>

If you want Shopify consulting, please feel free to email me.