[services] Add docker-compose overrides for tests
ClosedPublic
Actions

Authored by bartek on Oct 17 2023, 5:57 AM.

Details

Reviewers

varun
michal
kamil
• jon

Commits

rCOMMf8477adb619e: [services] Add docker-compose overrides for tests

Summary

Adds a new docker-compose config that works together with the existing one, and overrides some properties.
Using both configs together gives a test configuration (docker compose -f docker-compose.yml -f docker-compose.tests.yml [COMMAND]).

Most of the config is overriding the environment variables to correctly point to other service containers. The test-commons.env provides environment shared by all containers, that is normally proviced by Nix when outside docker.
Also, a few other properties needed to be overridden:

Reset custom platform option - tests will compile using the machine native architecture
Build Identity service with pre-generated OPAQUE keypair - D9508
Run Blob service in "auto-deletion" mode - D9509
Terraform inside commtest container needs to have custom localstack address
Added dedicated volume for build artifacts to speed up builds when running tests multiple tiems in local environment

Depends on D9511

Test Plan

Added dedicated command to run all tests in local docker:

yarn run-commtest-in-docker

Its equivalent is going to be used on Buildkite

Diff Detail

Repository

rCOMM Comm

Lint

No Lint Coverage

Unit

No Test Coverage

Event Timeline

bartek created this revision.Oct 17 2023, 5:57 AM

bartek held this revision as a draft.

Herald added a reviewer: • jon. · View Herald TranscriptOct 17 2023, 5:57 AM

Herald added subscribers: will, tomek, ashoat. · View Herald Transcript

bartek published this revision for review.Oct 17 2023, 6:08 AM

bartek added inline comments.

services/docker-compose.tests.yml
59 ↗	(On Diff #32078)	example of single service log level override
services/test-commons.env
6 ↗	(On Diff #32078)	For tests, all services can be run in debug mode

Harbormaster completed remote builds in B23287: Diff 32078.Oct 17 2023, 6:14 AM

bartek added a child revision: D9520: [CI] Add Buildkite steps for Commtest.Oct 18 2023, 3:16 AM

michal accepted this revision.Oct 18 2023, 3:37 AM

This revision is now accepted and ready to land.Oct 18 2023, 3:37 AM

Rebase on latest changes. Besides that:

Fixed an issue that tunnelbroker container failed to start because of long init of rabbitmq:
- Added healthcheck to rabbitmq
- Added restart: always to tunnelbroker
Fine grained RUST_LOG to avoid spam
Reworked platform settings for containers:
- Made linux/amd64 default platform
- It can be overriden by PLATFORM env variable. It is set by the yarn script
- The previous !reset solution didn't work on CI because of outdated docker-compose executable
  - Lack of these settings made local dev failing on M1 mac due to arch mismatch