Page MenuHomePhabricator

[Nix] Auto-generate url json configuration files
ClosedPublic

Authored by jon on Sep 30 2022, 2:21 PM.
Tags
None
Referenced Files
F3401242: D5274.id17550.diff
Mon, Dec 2, 11:06 AM
Unknown Object (File)
Sun, Dec 1, 7:43 PM
Unknown Object (File)
Sun, Dec 1, 7:42 PM
Unknown Object (File)
Sun, Dec 1, 7:42 PM
Unknown Object (File)
Sun, Dec 1, 7:42 PM
Unknown Object (File)
Sun, Dec 1, 7:42 PM
Unknown Object (File)
Tue, Nov 26, 4:48 PM
Unknown Object (File)
Tue, Nov 26, 3:27 PM
Subscribers

Details

Reviewers
ashoat
atul
abosh
varun
Group Reviewers
Restricted Owners Package(Owns No Changed Paths)
Commits
rCOMM618c72de9519: [Nix] Auto-generate url json configuration files
Summary

Instead of users having to write this manually, just create it.

https://linear.app/comm/issue/ENG-1936

Test Plan
# move old directory
test -d keyserver/facts && mv keyserver/facts{,.bak}

nix develop

# assert files were written
cat keyserver/facts/{commapp,landing}_url.json

# move old directory back
test -d keyserver/facts.bak && rm -r keyserver/facts && mv keyserver/facts{.bak,}

Diff Detail

Repository
rCOMM Comm
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

Owners added a reviewer: Restricted Owners Package.Sep 30 2022, 2:21 PM
ashoat requested changes to this revision.Sep 30 2022, 6:01 PM

Very incomplete test plan. You need to test the app thoroughly. Think about how you can test it more and add a whole bunch of things to the test plan

scripts/create_url_facts.sh
11 ↗(On Diff #17256)

Use [[ and ]] instead of [ and ] has been discussed to death... please use [[, unless you have a good reason (which you should call out... you're going to get this feedback every time)

scripts/templates/commapp_url.json
4 ↗(On Diff #17256)

I don't think this is right. Can you update your test plan to include registering / logging in / playing around with / refreshing the app? (Also the landing page)

6 ↗(On Diff #17256)

You're missing a proxy config. I have no idea how you got this to work. Worries me that your test plan was incomplete

This revision now requires changes to proceed.Sep 30 2022, 6:01 PM

I don't think this is right. Can you update your test plan to include registering / logging in / playing around with / refreshing the app? (Also the landing page)

I would probably need to pair with someone to see that all of these workflows are satisfied. I'm being pulled in a lot of different directions, and don't really have context on "what you should be able to do". The current dev docs lack any type of user workflows.

scripts/templates/commapp_url.json
6 ↗(On Diff #17256)

I don't understand. If you're referring to apache proxying, that should only be needed if we need to forward traffic from port 80. If the port is specified, then proxying shouldn't be needed.

Unless I'm missing something

scripts/create_url_facts.sh
1 ↗(On Diff #17256)

Remove the space after shebang please

scripts/templates/commapp_url.json
6 ↗(On Diff #17256)

Yeah, I'm referring to Apache proxying. See here

In D5274#156440, @jon wrote:

The current dev docs lack any type of user workflows.

See the Development section

Fix commapp_url.json configuration, and rebase on master

@atul and I paired, and from a cleaned repository (e.g. git clean -xfd && rm -rf ~/.local/share/MariaDB), we were able to post a message after creating a user.

ashoat requested changes to this revision.Oct 13 2022, 1:20 PM
ashoat added inline comments.
scripts/create_url_facts.sh
11 ↗(On Diff #17256)

Feedback has not been addressed...

This revision now requires changes to proceed.Oct 13 2022, 1:20 PM

Please update your test plan to reflect what you tested with @atul. Please also add testing landing to your test plan, as it appears that it wasn't tested

scripts/templates/landing_url.json
5 ↗(On Diff #17550)

Why is there no "proxy": "none" here?

jon marked 4 inline comments as done.

Use test instead of [

jon marked 3 inline comments as done.

Apply whitespace feedback

scripts/create_url_facts.sh
11 ↗(On Diff #17256)

just used test to avoid [ vs [[ issue.

scripts/templates/commapp_url.json
4 ↗(On Diff #17256)

I was able login, create a thread, navigate to the calendar, refresh the page fine.

I can update the test plan with the end-to-end workflow

scripts/templates/landing_url.json
5 ↗(On Diff #17550)

if the basePath is the same as the baseRoutePath, then you don't need the proxy pass for different routes.

The configuration from the docs aren't needed if these routes are the same

still need to update test plan

just used test to avoid [ vs [[ issue.

Is test basically just a third variety here? Can we update our code to only use one variety of shell-scripting test? The goal of insisting on [[ instead of [ is consistency across the codebase... seems like we're getting even further away from that by introducing a third category of condition to sneakily avoid the feedback about [[ vs. [. Let me know if I'm misinterpreting here!

ashoat requested changes to this revision.Oct 17 2022, 8:31 AM
ashoat added inline comments.
scripts/templates/landing_url.json
5 ↗(On Diff #17550)

if the basePath is the same as the baseRoutePath, then you don't need the proxy pass for different routes.

I don't follow what you're talking about here. It seems to reflect a misunderstanding about what proxy does.

I think what happened is that you found proxy was not necessary during testing, and then you went ahead and made some assumptions about what it does instead of diving deeper to understand what's going on.

I usually don't have time to do a deep-dive in the code to clarify misconceptions, but in this case I wrote the code and I can take a second to help you out. In the future, it's my expectation that you'll be able to do this sort of "deep-dive" on your own.

Let's start by answering the question, "What does "proxy": "none": do?" You seem to have the perception that it decides whether to set up a "proxy pass", but that's not quite accurate.

A good way to understand what something does it git grep. Here's what I see when I run git grep in keyserver:

ashoat@ashoatmbp2021 [~/src/comm/keyserver]$ git grep proxy
src/session/cookies.js:  const { proxy } = getAppURLFactsFromRequestURL(req.originalUrl);
src/session/cookies.js:  if (proxy === 'none') {
src/session/cookies.js:  } else if (proxy === 'apache') {
src/utils/security-utils.js:  const { https, proxy } = getAppURLFactsFromRequestURL(req.originalUrl);
src/utils/security-utils.js:    (proxy === 'none' && req.protocol !== 'https') ||
src/utils/security-utils.js:    (proxy === 'apache' && req.get('X-Forwarded-SSL') !== 'on')
src/utils/urls.js:  +proxy?: 'apache' | 'none', // defaults to apache
src/utils/urls.js:    const { proxy } = urlFacts;
src/utils/urls.js:      proxy: validProxies.has(proxy) ? proxy : 'apache',

Let's look three all three of these files:

  1. In the first file, getRequestIPAddress seems to use the proxy config to decide where to get the requester's IP address from. If proxy is set to apache, then we look for the X-Forwarded-For header. Otherwise, we just use the default req.socket.remoteAddress from Node.
  2. In the second file, assertSecureRequest seems to use the proxy config to figure out whether the requester is using HTTPS. If proxy is set to apache, then we look for the X-Forwarded-SSL header. Otherwise, we just the default req.protocol === 'https' from Node.
  3. The third file doesn't actually use proxy... it's just plumbing code for the config. So we can conclude it's irrelevant.

Now that we've looked at the code, we can start to understand more about proxy and why it matters. Now we have two questions to answer:

  1. What does proxy do for landing? Why doesn't it seem to have any impact during testing?
  2. What should landing_url.json contain?

For the first question, we can see clearly in the second file that https: false means that code doesn't even look at proxy. As for the first file, the getRequestIPAddress function is called from three places: fetchViewerForSocket, fetchViewerForJSONRequest, and fetchViewerForHomeRequest. None of these three callsites are accessed by the landing code in keyserver, so that explains the behavior you saw.

Next, let's ask "what should landing_url.json contain?" We can answer this by considering what proxy is meant to do.

  1. First, we can observe that the default is Apache. The third file in our git grep above confirms that. We can ask ourselves "are we using an Apache proxy here?" If the answer is no, we probably should avoid this default.
  2. Next, we can ask ourselves "is it possible that this config will be used by landing code in the future?" Here again I think the answer is "definitely yes"... if at any point the landing page needs to know the requester's IP address, it will need proxy to be correctly configured.

With this we can confidently conclude that you need to make a change here.

scripts/templates/landing_url.json
5 ↗(On Diff #17550)

This would have been a lot easier if the non-apache workflow was already documented. I don't have the context as to how apache + keyserver + commapp + landing should interact. Part of the reason why I really wanted to pair with someone was to have this context while iterating on the "desired solution". We just found a working solution which fit the original intent of "generate something which can be used for development".

Having a lot of the tickets be, "you need to become intimately familiar with every use case for every workflow" is just setting me up for failure and despair.

jon marked an inline comment as done.

Apply feedback

ashoat requested changes to this revision.Oct 24 2022, 7:58 AM

Close!

scripts/templates/landing_url.json
6 ↗(On Diff #17847)

This isn't valid

This revision now requires changes to proceed.Oct 24 2022, 7:58 AM
jon added inline comments.
scripts/templates/landing_url.json
6 ↗(On Diff #17847)

Yea, sorry was still waking up

Use proper value for proxy

Awesome, let's get this landed!

atul accepted this revision as: Restricted Owners Package.Oct 25 2022, 6:16 AM
This revision is now accepted and ready to land.Oct 25 2022, 6:16 AM

retested this on top of the node changes, seems to work for web app and landing page