Page MenuHomePhabricator

[docker] Switch base image from `alpine:3.14` to `ubuntu:20.04`
ClosedPublic

Authored by atul on Feb 15 2022, 3:29 PM.
Tags
None
Referenced Files
F3752373: D3212.id9703.diff
Fri, Jan 10, 12:31 AM
F3752333: D3212.id9703.diff
Fri, Jan 10, 12:26 AM
F3743994: D3212.id9704.diff
Thu, Jan 9, 2:54 PM
F3743977: D3212.id9704.diff
Thu, Jan 9, 2:50 PM
Unknown Object (File)
Wed, Jan 8, 5:17 AM
Unknown Object (File)
Wed, Jan 8, 5:17 AM
Unknown Object (File)
Wed, Jan 8, 5:17 AM
Unknown Object (File)
Wed, Jan 8, 5:17 AM

Details

Summary

This is Ubuntu "minimal" or whatever according to: https://ubuntu.com/blog/minimal-ubuntu-released

Test Plan

Putting this up to see what happens with the CI

Diff Detail

Repository
rCOMM Comm
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

Harbormaster returned this revision to the author for changes because remote builds failed.Feb 15 2022, 3:47 PM
Harbormaster failed remote builds in B6855: Diff 9701!

update docker-base for backup, blob, and tunnelbroker

Harbormaster returned this revision to the author for changes because remote builds failed.Feb 15 2022, 4:19 PM
Harbormaster failed remote builds in B6856: Diff 9702!

fix libcurl4-openssl-dev dep

Harbormaster returned this revision to the author for changes because remote builds failed.Feb 15 2022, 4:37 PM
Harbormaster failed remote builds in B6857: Diff 9703!

fix ubuntu version and add protoc via apt-get

Harbormaster returned this revision to the author for changes because remote builds failed.Feb 15 2022, 5:37 PM
Harbormaster failed remote builds in B6858: Diff 9704!
atul requested review of this revision.Feb 15 2022, 7:19 PM

What are we removing from project.pbxproj? Also you were able run yarn run-tunnelbroker-service yarn build-tunnelbroker-base yarn build-base-image successfully?

This revision is now accepted and ready to land.Feb 15 2022, 7:25 PM
This revision now requires review to proceed.Feb 15 2022, 7:25 PM

What are we removing from project.pbxproj?

Ah sorry, I actually expected the CI to fail and thought this diff would remain in a draft state so I didn't clean it up. But, I'll rebase once https://phabricator.ashoat.com/D3197 is landed and there should no longer be any change to project.pbxproj. I'll remove it from the diff.

Also you were able run yarn run-tunnelbroker-service yarn build-tunnelbroker-base yarn build-base-image successfully?

Haven't tried all of those yet. I just ran yarn build-all in the services directory. I'm kind of working under the assumption that if everything builds everything should continue to behave as before. Maybe that's not the case?

This revision is now accepted and ready to land.Feb 15 2022, 7:34 PM
This revision now requires review to proceed.Feb 15 2022, 7:34 PM
atul planned changes to this revision.Feb 15 2022, 7:38 PM

Now that everything is building, going to take a look at a few things before re-requesting review

  • Removing unrelated changes to /project.pbxproj
  • Seeing why we need to apt-get protobuf-compiler and it's not picking up protoc from path like it did on Alpine?
  • See if we can avoid sudo

retrigger CI, will plan changes

atul planned changes to this revision.Feb 15 2022, 8:38 PM

Screen Shot 2022-02-15 at 9.14.39 PM.png (586×1 px, 167 KB)

Nice IntelliSense/Go-To-Def/etc work as they did on debian:slim

cut sudo, will plan changes

revert pbxproj changes, ready for review

services/base-image/Dockerfile
10 ↗(On Diff #9711)

Without this we're prompted to select a timezone, but we aren't able to interact with the prompt and select one:

Setting up tzdata (2021e-0ubuntu0.20.04) ...
debconf: unable to initialize frontend: Dialog
debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.)
debconf: falling back to frontend: Readline
Configuring tzdata
------------------
 
Please select the geographic area in which you live. Subsequent configuration
questions will narrow this down by presenting a list of cities, representing
the time zones in which they are located.
 
  1. Africa      4. Australia  7. Atlantic  10. Pacific  13. Etc
  2. America     5. Arctic     8. Europe    11. SystemV
  3. Antarctica  6. Asia       9. Indian    12. US
Geographic area: # Received cancellation signal, interrupting
10 ↗(On Diff #9711)

-y flag from man apt-get:

-y, --yes, --assume-yes
      Automatic yes to prompts; assume "yes" as answer to all prompts and run non-interactively. If an
      undesirable situation, such as changing a held package, trying to install an unauthenticated package or
      removing an essential package occurs then apt-get will abort. Configuration Item: APT::Get::Assume-Yes.

Did you perform any measurements when using ubuntu vs alpine? IIRC alpine was much faster when it came to installing packages for example.
I understand you want to change the distro, I'm just not sure whether ubuntu is the right choice.

services/base-image/Dockerfile
10 ↗(On Diff #9711)

I know this is needed but I'd split this into separate lines

This revision now requires changes to proceed.Feb 16 2022, 2:10 AM
atul requested review of this revision.EditedFeb 16 2022, 9:19 AM

Did you perform any measurements when using ubuntu vs alpine?

Running yarn build-all from scratch in the services directory took about the same amount of time on my machine and the CI machine with Alpine and Ubuntu. Maybe the difference is more pronounced on x86 Macs?

IIRC alpine was much faster when it came to installing packages for example.

I wasn't able to reproduce this issue. My understanding was that the packages were only going to get installed once and cached afterward?

I understand you want to change the distro, I'm just not sure whether ubuntu is the right choice.

I think we agreed on Ubuntu during the Monday meeting. Is there an alternative distro you think we should go with? The primary reasons were:

  • Issues with Alpine's use of musl which has partial ABI compatibility with the more widely used glibc. Specifically, this caused issues with the VSCode C++ language server which has experimental (read: broken) support for musl. This is the big one since it blocks a more sane dev environment
  • Ubuntu is more widely used/team members have more familiarity with it/there's more documentation (official + stackoverflow type stuff) available online
  • Perception that there's a stronger security team behind Ubuntu (I think @jimpo mentioned this and probably has more context)
services/base-image/Dockerfile
10 ↗(On Diff #9711)

From what I read, it's best practice to set that environment variable for the single command: https://serverfault.com/a/797318

services/base-image/Dockerfile
10 ↗(On Diff #9711)

Thanks for linking to that, I'll update this diff with changes

Maybe the difference is more pronounced on x86 Macs?

That could be the case.

Anyway, we can defer rebuilding images and pushing them to the docker hub to someone who doesn't see a difference.

Basically, my first choice was ubuntu but I switched to alpine because it was recommended for docker images and significantly faster.

This revision is now accepted and ready to land.Feb 21 2022, 11:28 PM
services/base-image/Dockerfile
10 ↗(On Diff #9711)

Looks like you can keep it in one RUN command but split it into separate lines like @karol-bisztyga suggested by appending each line with \

rebase before making changes

@atul This should run rm -rf /var/lib/apt/lists/* after the apt-get install. See https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#apt-get. "In addition, when you clean up the apt cache by removing /var/lib/apt/lists it reduces the image size, since the apt cache is not stored in a layer."

@jimpo ah sorry I missed that last paragraph, I’ll put up a diff addressing that and link to it from here