Page MenuHomePhabricator

[terraform] run migration on aws deploy script only when necessary
AbandonedPublic

Authored by will on Aug 18 2024, 10:29 PM.
Tags
None
Referenced Files
Unknown Object (File)
Wed, Sep 25, 4:12 PM
Unknown Object (File)
Wed, Sep 25, 4:12 PM
Unknown Object (File)
Wed, Sep 25, 4:11 PM
Unknown Object (File)
Sun, Sep 22, 7:53 AM
Unknown Object (File)
Sep 12 2024, 4:44 PM
Unknown Object (File)
Sep 9 2024, 8:00 AM
Unknown Object (File)
Sep 5 2024, 11:14 PM
Unknown Object (File)
Aug 30 2024, 5:50 PM
Subscribers

Details

Summary

This includes changes to the script that would allow it to only run the migration
when there is an upgrade to the database version. If the old and intended db versions are the same, a terraform apply occurs.

It fails when the db version is intended to be downgraded. When db version is not specified as a label,
the script will ask the user if they would like the force an upgrade as the intended db version is unknown.

Depends on D13103

Test Plan

Ran this starting with a keyserver running db version 66 on my own custom image. Tested by upgrading to another custom image
running on db version 68. This change was noted by the script and it prompted the user if they would like to go forward with the migration.

Trying to go backwards was aborted. And trying the commapp/keyserver:1.0.105 resulted in the prompt asking if the user would like
to force an image upgrade which results in the complete migration process.

Diff Detail

Repository
rCOMM Comm
Branch
script_migration
Lint
No Lint Coverage
Unit
No Test Coverage

Event Timeline

services/terraform/self-host/aws-deploy.sh
121–125

This check is here as the user could be enabling mariadb access through terraform.tfvars.json. Running the authorize command for a duplicate ingress rule granting access would result in a harmless error when trying to authorize access. However, we don't want to disable this access afterwards even if temporary before the terraform apply is run. Therefore, mariadb_authorized is checked later to determine if access needs to be revoked only if access was granted temporarily by the script.

will requested review of this revision.Aug 18 2024, 10:45 PM
services/terraform/self-host/aws-deploy.sh
43

Will remove this extra line in next rebase

ashoat requested changes to this revision.Aug 20 2024, 12:06 PM

Love how thoughtfully you considered the steps here

Mostly minor comments, but requested changes because of the number of them, and because I'd like to understand the part of the logic that's handled "inside" Terraform a bit better

services/terraform/self-host/aws-deploy.sh
149

I'm guessing =~ is a RegExp match and ! at the front negates it... but what does -? mean here?

Seems like copy-pasted code from somewhere that was supposed to handle negative numbers, but in our case version numbers are positive only

167

Capitalize Docker

177

Why is there a space after index?

179

Same question about -?

188–189

Weird indentation here again

195–217

Indentation issues here

220

Nit – I find it helpful to use some kind of "more important" style for these header comments. One example in our codebase is the keyserver's Dockerfile (link), but we don't really have a standard / consistent style for it

276

If I recall correctly, some of the migration logic was handled "inside" Terraform. I vaguely recall that the terraform apply would internally make sure to bring up the secondary services only after the primary completed the migration.

Two things here:

  1. It would be helpful to the reader of this script to have some explanation of that, and a pointer to where the code for it is
  2. How does the "internal" logic inside Terraform differentiate between the terraform apply here, versus the one on line 197? In the former case, we want to wait on the primary before bringing the secondary online; in contrast, in the latter case we want to immediately update all tasks
This revision now requires changes to proceed.Aug 20 2024, 12:06 PM
services/terraform/self-host/aws-deploy.sh
276

Regarding the "internal" logic, also curious how we make sure that traffic to the primary isn't enabled until we can confirm it's up and we're ready to start the secondary

Abandoning this revision. With our new approach of using JS based logic to block secondary nodes during migrations, this logic is no longer necessary in the deploy script.

https://linear.app/comm/issue/ENG-9084/consider-replacing-aws-deploysh-with-js-logic-in-keyserver

services/terraform/self-host/aws-deploy.sh
149

Yeah. -? can be removed.

Removing in next rebase