Page MenuHomePhabricator

[DRAFT] [services] Create a logs directory in the base image
AbandonedPublic

Authored by karol on Jul 21 2022, 3:47 AM.
Tags
None
Referenced Files
F3301080: D4596.diff
Sun, Nov 17, 11:47 PM
Unknown Object (File)
Fri, Nov 8, 8:28 PM
Unknown Object (File)
Fri, Nov 8, 4:41 PM
Unknown Object (File)
Fri, Nov 8, 8:13 AM
Unknown Object (File)
Tue, Nov 5, 2:05 AM
Unknown Object (File)
Oct 16 2024, 9:21 AM
Unknown Object (File)
Sep 26 2024, 6:14 PM
Unknown Object (File)
Sep 26 2024, 6:13 PM

Details

Summary

Depends on D4595

As discussed in the Comm channel, we'd like to perform logging in services and save the logs to files so that when the service crashes, we're able to track down what might've happened more easily.

In order to do this, I think it would be good to create a separate space (directory) in the docker containers' file systems.

I was thinking of just a directory /logs. I think this should be in the base image because it's going to be used in the same way across all the services that are built on top of the base image. If we decide this is a good idea, I'm going to rebuild the base image and push it to the hub, then land this.

Test Plan

None for now, once the idea is accepted, only then will I push this change to the hub.

Diff Detail

Repository
rCOMM Comm
Lint
No Lint Coverage
Unit
No Test Coverage

Event Timeline

Harbormaster returned this revision to the author for changes because remote builds failed.Jul 21 2022, 3:54 AM
Harbormaster failed remote builds in B10717: Diff 14756!
karol edited the test plan for this revision. (Show Details)
karol added reviewers: tomek, ashoat, max.

Publishing the draft. CI failed for the services but that's expected. When I push the new version of the base image to the hub, it will pass.

max requested changes to this revision.Jul 21 2022, 7:16 AM

Maybe I've missed something, but the good practice is to forward stdout and stderr to the log files instead of explicitly using only files. It's not flexible, we can't use an aggregation, and it's not effective in development mode. Regarding Docker context in this diff, as I know we are switching to nix and should consider this when making such changes.
It's a good idea to make those discussions in Linear so everyone on the team can read the context about this. Personally, I think we should use a flag or parameter in the configuration file to use the file logging like:

log-files-dir=/logs

And if this parameter is defined we are logging in to the files instead of stdout and stderr through glog.

This revision now requires changes to proceed.Jul 21 2022, 7:16 AM
ashoat requested changes to this revision.Jul 22 2022, 8:52 AM

Agree with @max on most points. Some thoughts:

  1. I think we should be using stdout / stderr for logs because it integrates with other things well. For instance, to monitor keyserver logs I use docker compose logs --follow --timestamps to see the logs from stdout / stderr
  2. I wouldn't worry about Nix for production deployments quite yet
  3. Agree Linear is a good place for discussion, probably a bit better than chat since it's easier to search and reference later
  4. If we do decide to have a directory like this, I agree it would be good to make it runtime configurable as @max suggested
  5. Last piece, I think generally /var/log is the place to put logs on a UNIX-y system

Ok, I think this diff can be abandoned