Page MenuHomePhabricator

[Nix] Add comm binary cache to flake
ClosedPublic

Authored by jon on Aug 11 2022, 2:58 PM.
Tags
None
Referenced Files
Unknown Object (File)
Sun, Nov 3, 7:45 PM
Unknown Object (File)
Sun, Nov 3, 7:45 PM
Unknown Object (File)
Sun, Nov 3, 7:44 PM
Unknown Object (File)
Sun, Nov 3, 7:26 PM
Unknown Object (File)
Fri, Oct 18, 8:41 AM
Unknown Object (File)
Thu, Oct 17, 11:24 AM
Unknown Object (File)
Thu, Oct 17, 7:19 AM
Unknown Object (File)
Fri, Oct 11, 6:08 AM

Details

Summary

Add comm cachix binary cache to the flake

This allows for commands such as nix develop to check the binary cache
in addition to nixpkgs' cache.nixos.org.

I've already uploaded the dev shell for x86_64-linux, x86_64-darwin (intel), and
aarch64-darwin (m1).

Cachix has a limited free tier for public caches, since we don't have anything
sensitive, I just created a comm cache.

If nix is not able to find a package in either cache, it will just fall back
to building from source (which is the current workflow).

Test Plan

Follow instructions added in D4817, to allow your user to modify substitutor behavior

Then, from a new machine (or run nix-collect-garbage with all gcroots removed):

nix develop
# assert that nodejs-16.3.0 came from cachix

Diff Detail

Repository
rCOMM Comm
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

added @ashoat as this brings adds an optional dependency on the cachix service

ashoat requested changes to this revision.Aug 11 2022, 10:01 PM

Looks like pricing is based on GiB of storage. How many GiB of storage will this change require for our Cachix cluster? How much do you think it will cost?

Please also see question here

This revision now requires changes to proceed.Aug 11 2022, 10:01 PM

Looks like pricing is based on GiB of storage. How many GiB of storage will this change require for our Cachix cluster? How much do you think it will cost?

I pushed all of our comm-specific builds to the cache, and its 52MB across the three platforms. I don't see usage going above more than ~2GiB unless we start pinning a bunch of crap.

Users have a free 5 GB limit for open source projects.

Since I've been upstreaming most of the bigger work, I don't think we'll ever hit this limit for the dev shell. But if we make use of the service and feel like it provides value, then we should probably start a subscription just to ensure it's longevity

This revision is now accepted and ready to land.Aug 12 2022, 10:09 AM

By the way, next time, please hit re-request review so this shows up on my queue. You've gotten some version of this feedback at least 5 times now... it's your responsibility as the diff author to manage your reviewers' queue.

If it needs to be reviewed, it should not be red. If it should not be reviewed yet, it should be red.

You've gotten some version of this feedback at least 5 times now... it's your responsibility as the diff author to manage your reviewers' queue.

Yea, but I have github behavior ingrained in me after a few thousand hours of usage. There there's a notification on any activity on a PR.

I'm trying....

There's a notification on every diff as well. We're not talking about managing notifs, we're talking about managing your reviewers' queues.

We're not talking about managing notifs, we're talking about managing your reviewers' queues.

For github, the notification feed https://github.com/notifications is the queue.

Nice! Will be really nice to not have to wait forever for node in particular to build on new machines (eg clean CI environment).

I've already uploaded the dev shell for x86_64-linux, x86_64-darwin (intel), and aarch64-darwin (m1).

Is this a manual process? Are there some credentials that team members need to have access to in order to make changes/upload things?

Does this cache need to be maintained manually? If we make a change to our nix environment will the cache be updated automatically?

Assuming you have a project with default.nix you can build it and push:
$ nix-build | cachix push mycache
It’s recommended to set up Continuous Integration to push for every branch of every project.

(https://docs.cachix.org/getting-started#pushing-binaries-with-cachix)

Guessing we'll need to add a GitHub Actions workflow to do this for us on push to master?

Is this a manual process?

I did it manual, but it's relatively easy to automate https://docs.cachix.org/pushing

Are there some credentials that team members need to have access to in order to make changes/upload things?

Yea, I can create some, there's just some api keys which only need to be set once. And the workflow looks like cachix auth <token>; cachix push comm <store path>

Does this cache need to be maintained manually? If we make a change to our nix environment will the cache be updated automatically?

This should only be needed if we change the flake.nix, the nix/dev-shell.nix, or nix/overlay.nix.

I did it manually the first time to just "prime it". Likely we would just have some CI job which unilaterally pushes up to cachix. The client is "smart" enough to only send things which are not present in the cache.

Guessing we'll need to add a GitHub Actions workflow to do this for us on push to master?

That would be the most sustainable moving forward.

Likely the job would just be:

nix develop --profile dev-profile # create a nix profile which contains the dev shell
cachix push comm dev-profile # hand the dev shell over to cachix, it will be able to determine what needs to be uploaded if not present already
In D4816#139499, @jon wrote:

We're not talking about managing notifs, we're talking about managing your reviewers' queues.

For github, the notification feed https://github.com/notifications is the queue.

I agree that Phabricator's notification handling is subpar. If you're @'ed in a diff or something happens in a diff you're subscribed to, you basically rely on emails and the Recent Activity tab for any sort of updates. Additionally, yes, GitHub's notifications queue is great. But @ashoat's queue is huge and it is harder to sift through emails for diffs that don't show up on his Phabricator queue, so I also understand what he's saying.

I think for the future, just remember any time you're submitting a comment, there should probably be an action attached to it (like something red, blue, or green). Red for Changes Planned or requesting revision, blue for updating or re-requesting review, and green for accepting a revision (I may have gotten these colors wrong, but the different colors help me remember updates in a revision). And if you're commenting on your OWN diff, you should almost always have an action attached to it so it will show up in reviewers' queues.

So basically any time you comment anything, try and see if you can attach an action to it. This mental check helped me in the past to remember to re-request review, etc.

Credentials should be backed up in 1Password

For github, the notification feed https://github.com/notifications is the queue.

@jon, your comment here is concerning... if you are confused about the difference between an notification "queue" and the Phabricator "queue", please sync with @abosh to learn more. Based on your past history of using Phabricator I would hope you understand more than you are letting on (and if so, your comment was completely unnecessary).

Based on your past history of using Phabricator I would hope you understand more than you are letting on (and if so, your comment was completely unnecessary).

My current understanding:

  • Activity feed: all activty
  • Email notifications: all activity on items you're subscribed to, an author of, or subscriber to
  • Phabricator queue: facilitated by explicit revision actions, or some state-machine logic on CI status. Still not super clear on best practices here. Mostly seem to get it right when just updating diffs and then submitting responses at the same time. Seems like "just commenting, and submitting responses without submitting revision status" is causing the most issues here.

I'm just going to watch some videos and read some tutorials on best practices.

This revision was automatically updated to reflect the committed changes.