[CI] Add `identity_build` workflow to Buildkite/GH Actions
ClosedPublic
Actions

Authored by atul on Jun 29 2022, 12:01 PM.

Details

Reviewers

• jacek
tomek
• abosh
varun
• karol

Commits

rCOMM53b4f3fab8d8: [CI] Add `identity_build` workflow to Buildkite/GH Actions

Summary

This was previously handled as part of the "Services CI" workflow that will be deprecated later in this stack. This diff allows us to increase concurrency as we move over to the autoscaling EC2 instances.

Differences on the Buildkite side:

We use the actual shell command instead of yarn blah because the autoscaling instances are intended to be as minimal as possible and don't include node, yarn, etc.

We use the --no-cache flag to force a new complete build on each run. The previous "Services CI" workflow depended heavily on caches, because running 3-4 complete builds would take a long time. Now, we can separate these out and take advantage of the concurrency provided by the autoscaling instances.

Depends on D4390

Test Plan

Buildkite: Copy/pasted from what I have in the Buildkite GUI that has been "active" for the last 2-3 days without issue.

GitHub: Will see what happens when it lands, but fairly confident it should be good to go.

Diff Detail

Repository

rCOMM Comm

Lint

Lint Not Applicable

Unit

Tests Not Applicable

Event Timeline

atul created this revision.Jun 29 2022, 12:01 PM

Herald added subscribers: • adrian, ashoat. · View Herald TranscriptJun 29 2022, 12:01 PM

atul edited the summary of this revision. (Show Details)Jun 29 2022, 12:12 PM

atul edited the summary of this revision. (Show Details)

Harbormaster completed remote builds in B10110: Diff 13975.Jun 29 2022, 12:13 PM

atul requested review of this revision.Jun 29 2022, 12:13 PM

atul added a child revision: D4405: [CI] Add `tunnelbroker_build` workflow to Buildkite/GH Actions.Jun 29 2022, 1:07 PM

atul added a reviewer: varun.Jun 29 2022, 6:25 PM

• abosh mentioned this in D4390: [CI] Add `blob_build` workflow to Buildkite/GH Actions.Jun 30 2022, 7:39 AM

varun accepted this revision.Jun 30 2022, 9:55 AM

varun added inline comments.

.github/workflows/identity_build.yaml
19 ↗	(On Diff #13975)	nit: maybe something less verbose here like `build service`

This revision is now accepted and ready to land.Jun 30 2022, 9:55 AM

atul added 1 blocking reviewer(s): tomek.Jun 30 2022, 12:31 PM

This revision now requires review to proceed.Jun 30 2022, 12:31 PM

atul added inline comments.Jun 30 2022, 2:04 PM

.github/workflows/identity_build.yaml
19 ↗	(On Diff #13975)	I tried to keep this consistent with the "names" of steps we use in the other GitHub Actions workflows where we just match the name of the command. It's also convenient in the GH actions workflow run page to see exactly what command was run for each step without having to "expand" it.

We use the --no-cache flag to force a new complete build on each run. The previous "Services CI" workflow depended heavily on caches, because running 3-4 complete builds would take a long time. Now, we can separate these out and take advantage of the concurrency provided by the autoscaling instances.

It isn't too intuitive that avoiding cache would improve performance. So basically, using caches stopped us from using autoscaling, and now the builds will take more time that can be shared between more instances, so overall the performance will be better, right? Do we know how the resources usage will grow after this change?

• karol accepted this revision.Jul 1 2022, 7:17 AM

This revision is now accepted and ready to land.Jul 1 2022, 7:17 AM

rebase after cherrypicking and before landing

It isn't too intuitive that avoiding cache would improve performance. So basically, using caches stopped us from using autoscaling, and now the builds will take more time that can be shared between more instances, so overall the performance will be better, right? Do we know how the resources usage will grow after this change?

Ah so two separate things. One is that we want to avoid using caches because there was some strange behavior observed during the Keyserver docker work with caches. Avoiding caches will strictly hurt performance, but we still deemed it the correct thing to do.

However, if we were to avoid the caches and do 4 full builds in a single workflow, it might take 20 minutes. By breaking them into 4 separate workflows that take 5 minutes each for example, we can get complete builds while still ensuring that the wait time is acceptable thanks to autoscaling.

This revision was landed with ongoing or failed builds.Jul 1 2022, 12:49 PM

Closed by commit rCOMM53b4f3fab8d8: [CI] Add `identity_build` workflow to Buildkite/GH Actions (authored by atul). · Explain Why

This revision was automatically updated to reflect the committed changes.

atul added a commit: rCOMM53b4f3fab8d8: [CI] Add `identity_build` workflow to Buildkite/GH Actions.

Harbormaster completed remote builds in B10208: Diff 14087.Jul 1 2022, 12:54 PM