Page MenuHomePhabricator
Differential D5847

[Nix] Provide default for squadcal_url.json
ClosedPublic
Actions

Authored by • jon on Dec 8 2022, 11:43 AM.
Tags
None
Referenced Files
Unknown Object (File)
Wed, Mar 12, 5:26 PM
Unknown Object (File)
Fri, Mar 7, 2:27 PM
Unknown Object (File)
Feb 26 2025, 2:27 PM
Unknown Object (File)
Feb 26 2025, 2:27 PM
Unknown Object (File)
Feb 26 2025, 2:27 PM
Unknown Object (File)
Feb 26 2025, 2:27 PM
Unknown Object (File)
Feb 26 2025, 2:26 PM
Unknown Object (File)
Feb 26 2025, 2:20 PM
View All Files
Subscribers
ashoat
tomek

Details

Reviewers
atul
varun
ashoat
Commits
rCOMM147886f8a769: [Nix] Provide default for squadcal_url.json
Summary

Add squalcal_url.json default with other facts.

squalcal_url.json is being used a fallback router for "/" in
this regard.

May be redundant with D5846, but aligns with intent.

https://linear.app/comm/issue/ENG-2260

Depends on D5846

Test Plan
rm keyserver/facts/*.json

nix develop

# Assert files were written, and content matches scripts/templates
ls keyserver/facts
cat keyserver/facts/*.json

Diff Detail

Repository
rCOMM Comm
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

• jon created this revision.Dec 8 2022, 11:43 AM
Herald added subscribers: tomek, ashoat. · View Herald TranscriptDec 8 2022, 11:43 AM
Harbormaster completed remote builds in B14071: Diff 19260.Dec 8 2022, 11:55 AM
• jon requested review of this revision.Dec 8 2022, 11:55 AM
ashoat added a comment.Dec 9 2022, 6:51 AM

Not redundant – the migration won't run when initializing a new environment

ashoat requested changes to this revision.Dec 13 2022, 9:56 AM
ashoat added inline comments.
scripts/templates/squadcal_url.json
6 ↗(On Diff #19260)

This needs proxy: "apache"

This revision now requires changes to proceed.Dec 13 2022, 9:56 AM
• jon removed a parent revision: D5846: [Keyserver] Migrate to non-apache config, update native.Dec 15 2022, 9:14 AM
• jon updated this revision to Diff 19400.Dec 15 2022, 9:17 AM
• jon added a parent revision: D5846: [Keyserver] Migrate to non-apache config, update native.

Address feedback

• jon marked an inline comment as done.Dec 15 2022, 9:17 AM
ashoat accepted this revision.Dec 15 2022, 9:27 AM

I would really love for the Test Plan to actually test the Apache configuration to make sure it works. Is this handled somewhere else, or are we just hoping it works, and hoping that your reviewers will catch any errors?

This revision is now accepted and ready to land.Dec 15 2022, 9:27 AM
Harbormaster completed remote builds in B14182: Diff 19400.Dec 15 2022, 9:29 AM
• jon retitled this revision from [Nix] Provide default for squalcal_url.json to [Nix] Provide default for squadcal_url.json.Dec 15 2022, 9:40 AM
• jon updated this revision to Diff 19793.Dec 20 2022, 10:35 AM

Update template with settings from D5846

• jon removed a parent revision: D5846: [Keyserver] Migrate to non-apache config, update native.Dec 20 2022, 10:35 AM

I would really love for the Test Plan to actually test the Apache configuration to make sure it works. Is this handled somewhere else, or are we just hoping it works, and hoping that your reviewers will catch any errors?

Tested this with the migration workflow in D5846. Just couldn't do my testing until I was able to re-create the apache workflow reliably.

The native iOS application seems to dynamically pick the changes from D5846, so this shouldn't be too much of an issue after D5846 lands.

Harbormaster completed remote builds in B14536: Diff 19793.Dec 20 2022, 10:45 AM
Closed by commit rCOMM147886f8a769: [Nix] Provide default for squadcal_url.json (authored by • jon). · Explain WhyDec 20 2022, 10:54 AM
This revision was automatically updated to reflect the committed changes.
• jon added a commit: rCOMM147886f8a769: [Nix] Provide default for squadcal_url.json.

Revision Contents
Changeset List

PathSize
scripts/
5 lines
templates/
7 lines

Diff 19798

View Options

scripts/create_url_facts.sh

FROM node:16.13-bullseye FROM node:16.13-bullseye
#------------------------------------------------------------------------------- #-------------------------------------------------------------------------------
# STEP 0: INSTALL PREREQS # STEP 0: INSTALL PREREQS
# Install prereqs first so we don't have to reinstall them if anything changes # Install prereqs first so we don't have to reinstall them if anything changes
#------------------------------------------------------------------------------- #-------------------------------------------------------------------------------
# We use rsync in the prod-build yarn script # We add Debian's unstable repo since it's the only way to get mysqldump
RUN echo "deb http://deb.debian.org/debian unstable main non-free contrib" \
>> /etc/apt/sources.list
# We need rsync in the prod-build yarn script
# We need mysql-client so we can use mysqldump for backups
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \ RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \
rsync \ rsync \
mysql-client \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
#------------------------------------------------------------------------------- #-------------------------------------------------------------------------------
# STEP 1: DEVOLVE PRIVILEGES # STEP 1: DEVOLVE PRIVILEGES
# Create another user to run the rest of the commands # Create another user to run the rest of the commands
#------------------------------------------------------------------------------- #-------------------------------------------------------------------------------
RUN useradd -m comm RUN useradd -m comm
USER comm USER comm
WORKDIR /home/comm/app WORKDIR /home/comm/app
#------------------------------------------------------------------------------- #-------------------------------------------------------------------------------
# STEP 2: INSTALL NVM # STEP 2: SET UP MYSQL BACKUPS
# Prepare the system to properly handle mysqldump backups
#-------------------------------------------------------------------------------
# Prepare the directory that will hold the backups
RUN mkdir /home/comm/mysqldump
# We install mysql-client 8.0 above but use it with MySQL 5.7. Unfortunately,
# we haven't been able to figure out a way to install mysql-client 5.7 on
# Debian bullseye. Instead, we configure mysqldump 8.0 to work with MySQL 5.7
RUN echo "[mysqldump]\ncolumn-statistics=0" > /home/comm/.my.cnf
#-------------------------------------------------------------------------------
# STEP 3: INSTALL NVM
# We use nvm to make sure we're running the right Node version # We use nvm to make sure we're running the right Node version
#------------------------------------------------------------------------------- #-------------------------------------------------------------------------------
ENV NVM_DIR /home/comm/.nvm ENV NVM_DIR /home/comm/.nvm
RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh \ RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh \
| bash | bash
#------------------------------------------------------------------------------- #-------------------------------------------------------------------------------
# STEP 3: YARN CLEANINSTALL # STEP 4: YARN CLEANINSTALL
# We run yarn cleaninstall before copying most of the files in for build caching # We run yarn cleaninstall before copying most of the files in for build caching
#------------------------------------------------------------------------------- #-------------------------------------------------------------------------------
# Copy in package.json and yarn.lock files # Copy in package.json and yarn.lock files
COPY --chown=comm package.json yarn.lock . COPY --chown=comm package.json yarn.lock .
COPY --chown=comm keyserver/package.json keyserver/.flowconfig keyserver/ COPY --chown=comm keyserver/package.json keyserver/.flowconfig keyserver/
COPY --chown=comm lib/package.json lib/.flowconfig lib/ COPY --chown=comm lib/package.json lib/.flowconfig lib/
COPY --chown=comm web/package.json web/.flowconfig web/ COPY --chown=comm web/package.json web/.flowconfig web/
COPY --chown=comm native/package.json native/.flowconfig native/ COPY --chown=comm native/package.json native/.flowconfig native/
COPY --chown=comm landing/package.json landing/.flowconfig landing/ COPY --chown=comm landing/package.json landing/.flowconfig landing/
# Copy in files needed for patch-package and pod-patch # Copy in files needed for patch-package and pod-patch
COPY --chown=comm patches patches/ COPY --chown=comm patches patches/
COPY --chown=comm native/ios/pod-patch native/ios/pod-patch/ COPY --chown=comm native/ios/pod-patch native/ios/pod-patch/
COPY --chown=comm native/ios/Podfile native/ios/ COPY --chown=comm native/ios/Podfile native/ios/
# Actually run yarn # Actually run yarn
RUN yarn cleaninstall RUN yarn cleaninstall
#------------------------------------------------------------------------------- #-------------------------------------------------------------------------------
# STEP 4: WEBPACK BUILD # STEP 5: WEBPACK BUILD
# We do this first so Docker doesn't rebuild when only keyserver files change # We do this first so Docker doesn't rebuild when only keyserver files change
#------------------------------------------------------------------------------- #-------------------------------------------------------------------------------
COPY --chown=comm lib lib/ COPY --chown=comm lib lib/
COPY --chown=comm landing landing/ COPY --chown=comm landing landing/
RUN yarn workspace landing prod RUN yarn workspace landing prod
COPY --chown=comm web web/ COPY --chown=comm web web/
RUN yarn workspace web prod RUN yarn workspace web prod
#------------------------------------------------------------------------------- #-------------------------------------------------------------------------------
# STEP 5: COPY IN SOURCE FILES # STEP 6: COPY IN SOURCE FILES
# We run this later so the above layers are cached if only source files change # We run this later so the above layers are cached if only source files change
#------------------------------------------------------------------------------- #-------------------------------------------------------------------------------
COPY --chown=comm . . COPY --chown=comm . .
#------------------------------------------------------------------------------- #-------------------------------------------------------------------------------
# STEP 6: RUN BUILD SCRIPTS # STEP 7: RUN BUILD SCRIPTS
# We need to populate keyserver/dist, among other things # We need to populate keyserver/dist, among other things
#------------------------------------------------------------------------------- #-------------------------------------------------------------------------------
# Babel transpilation of keyserver src # Babel transpilation of keyserver src
RUN yarn workspace keyserver prod-build RUN yarn workspace keyserver prod-build
#------------------------------------------------------------------------------- #-------------------------------------------------------------------------------
# STEP 7: RUN THE SERVER # STEP 8: RUN THE SERVER
# Actually run the Node.js keyserver using nvm # Actually run the Node.js keyserver using nvm
#------------------------------------------------------------------------------- #-------------------------------------------------------------------------------
EXPOSE 3000 EXPOSE 3000
WORKDIR /home/comm/app/keyserver WORKDIR /home/comm/app/keyserver
CMD bash/run-prod.sh CMD bash/run-prod.sh
View Options

scripts/templates/squadcal_url.json

FROM node:16.13-bullseye FROM node:16.13-bullseye
#------------------------------------------------------------------------------- #-------------------------------------------------------------------------------
# STEP 0: INSTALL PREREQS # STEP 0: INSTALL PREREQS
# Install prereqs first so we don't have to reinstall them if anything changes # Install prereqs first so we don't have to reinstall them if anything changes
#------------------------------------------------------------------------------- #-------------------------------------------------------------------------------
# We use rsync in the prod-build yarn script # We add Debian's unstable repo since it's the only way to get mysqldump
RUN echo "deb http://deb.debian.org/debian unstable main non-free contrib" \
>> /etc/apt/sources.list
# We need rsync in the prod-build yarn script
# We need mysql-client so we can use mysqldump for backups
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \ RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \
rsync \ rsync \
mysql-client \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
#------------------------------------------------------------------------------- #-------------------------------------------------------------------------------
# STEP 1: DEVOLVE PRIVILEGES # STEP 1: DEVOLVE PRIVILEGES
# Create another user to run the rest of the commands # Create another user to run the rest of the commands
#------------------------------------------------------------------------------- #-------------------------------------------------------------------------------
RUN useradd -m comm RUN useradd -m comm
USER comm USER comm
WORKDIR /home/comm/app WORKDIR /home/comm/app
#------------------------------------------------------------------------------- #-------------------------------------------------------------------------------
# STEP 2: INSTALL NVM # STEP 2: SET UP MYSQL BACKUPS
# Prepare the system to properly handle mysqldump backups
#-------------------------------------------------------------------------------
# Prepare the directory that will hold the backups
RUN mkdir /home/comm/mysqldump
# We install mysql-client 8.0 above but use it with MySQL 5.7. Unfortunately,
# we haven't been able to figure out a way to install mysql-client 5.7 on
# Debian bullseye. Instead, we configure mysqldump 8.0 to work with MySQL 5.7
RUN echo "[mysqldump]\ncolumn-statistics=0" > /home/comm/.my.cnf
#-------------------------------------------------------------------------------
# STEP 3: INSTALL NVM
# We use nvm to make sure we're running the right Node version # We use nvm to make sure we're running the right Node version
#------------------------------------------------------------------------------- #-------------------------------------------------------------------------------
ENV NVM_DIR /home/comm/.nvm ENV NVM_DIR /home/comm/.nvm
RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh \ RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh \
| bash | bash
#------------------------------------------------------------------------------- #-------------------------------------------------------------------------------
# STEP 3: YARN CLEANINSTALL # STEP 4: YARN CLEANINSTALL
# We run yarn cleaninstall before copying most of the files in for build caching # We run yarn cleaninstall before copying most of the files in for build caching
#------------------------------------------------------------------------------- #-------------------------------------------------------------------------------
# Copy in package.json and yarn.lock files # Copy in package.json and yarn.lock files
COPY --chown=comm package.json yarn.lock . COPY --chown=comm package.json yarn.lock .
COPY --chown=comm keyserver/package.json keyserver/.flowconfig keyserver/ COPY --chown=comm keyserver/package.json keyserver/.flowconfig keyserver/
COPY --chown=comm lib/package.json lib/.flowconfig lib/ COPY --chown=comm lib/package.json lib/.flowconfig lib/
COPY --chown=comm web/package.json web/.flowconfig web/ COPY --chown=comm web/package.json web/.flowconfig web/
COPY --chown=comm native/package.json native/.flowconfig native/ COPY --chown=comm native/package.json native/.flowconfig native/
COPY --chown=comm landing/package.json landing/.flowconfig landing/ COPY --chown=comm landing/package.json landing/.flowconfig landing/
# Copy in files needed for patch-package and pod-patch # Copy in files needed for patch-package and pod-patch
COPY --chown=comm patches patches/ COPY --chown=comm patches patches/
COPY --chown=comm native/ios/pod-patch native/ios/pod-patch/ COPY --chown=comm native/ios/pod-patch native/ios/pod-patch/
COPY --chown=comm native/ios/Podfile native/ios/ COPY --chown=comm native/ios/Podfile native/ios/
# Actually run yarn # Actually run yarn
RUN yarn cleaninstall RUN yarn cleaninstall
#------------------------------------------------------------------------------- #-------------------------------------------------------------------------------
# STEP 4: WEBPACK BUILD # STEP 5: WEBPACK BUILD
# We do this first so Docker doesn't rebuild when only keyserver files change # We do this first so Docker doesn't rebuild when only keyserver files change
#------------------------------------------------------------------------------- #-------------------------------------------------------------------------------
COPY --chown=comm lib lib/ COPY --chown=comm lib lib/
COPY --chown=comm landing landing/ COPY --chown=comm landing landing/
RUN yarn workspace landing prod RUN yarn workspace landing prod
COPY --chown=comm web web/ COPY --chown=comm web web/
RUN yarn workspace web prod RUN yarn workspace web prod
#------------------------------------------------------------------------------- #-------------------------------------------------------------------------------
# STEP 5: COPY IN SOURCE FILES # STEP 6: COPY IN SOURCE FILES
# We run this later so the above layers are cached if only source files change # We run this later so the above layers are cached if only source files change
#------------------------------------------------------------------------------- #-------------------------------------------------------------------------------
COPY --chown=comm . . COPY --chown=comm . .
#------------------------------------------------------------------------------- #-------------------------------------------------------------------------------
# STEP 6: RUN BUILD SCRIPTS # STEP 7: RUN BUILD SCRIPTS
# We need to populate keyserver/dist, among other things # We need to populate keyserver/dist, among other things
#------------------------------------------------------------------------------- #-------------------------------------------------------------------------------
# Babel transpilation of keyserver src # Babel transpilation of keyserver src
RUN yarn workspace keyserver prod-build RUN yarn workspace keyserver prod-build
#------------------------------------------------------------------------------- #-------------------------------------------------------------------------------
# STEP 7: RUN THE SERVER # STEP 8: RUN THE SERVER
# Actually run the Node.js keyserver using nvm # Actually run the Node.js keyserver using nvm
#------------------------------------------------------------------------------- #-------------------------------------------------------------------------------
EXPOSE 3000 EXPOSE 3000
WORKDIR /home/comm/app/keyserver WORKDIR /home/comm/app/keyserver
CMD bash/run-prod.sh CMD bash/run-prod.sh