Page MenuHomePhabricator

[keyserver] Update CORS
ClosedPublic

Authored by michal on Oct 6 2023, 7:02 AM.
Tags
None
Referenced Files
F3339235: D9396.diff
Thu, Nov 21, 7:15 PM
F3332644: D9396.id32146.diff
Thu, Nov 21, 12:41 AM
Unknown Object (File)
Sun, Nov 17, 3:39 PM
Unknown Object (File)
Fri, Nov 15, 7:37 PM
Unknown Object (File)
Sun, Nov 10, 3:17 PM
Unknown Object (File)
Fri, Nov 8, 11:52 AM
Unknown Object (File)
Fri, Nov 8, 11:30 AM
Unknown Object (File)
Fri, Nov 8, 12:28 AM
Subscribers

Details

Summary

ENG-4768
We want the web app to call squadcal instead of web.comm.app which means we need to setup new cors headers so the browser doesn't reject the request. We can use the express middleware from cors library to automatically handle adding cors headers to requests and handling preflight requests (where the browser first sends and OPTION request).

Test Plan

To test this:

  1. First we need to start hosting the web app on a different origin, so the browser will require a CORS header for the requests. We can do it like this:
    • Edit /etc/hosts file and add 127.0.0.1 test.localhost line. Thanks to this we can navigate to test.localhost:3000/comm/ and still get redirected to 127.0.0.1 (note: for some reason this continued to work for me even without this line on some browsers)
    • Modify domains in webapp_cors.json and commapp_url.json to http://test.localhost:3000
  2. Then we want to make the the webapp connect to the keyserver endpoints (squadCalRouter). Modify the default dev urlPrefix in web/redux/default-state.js to http://localhost:3000 (remove the /comm)
    • We also need to change the proxy in squadcal_url.json to "none" because otherwise keyserver tries to determine the request api like it was redirected from apache and fails. As I don't have a proxy setup I need to change this setting.
  3. Load the web app
    • Without CORS middleware the initial-redux-state request fails with Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource
    • With CORS middleware (changes in this diff) the request goes through

I have also confirmed that after removing the upload-endpoint-specific CORS setup, uploads still load on the web app (both media uploaded before the CORS changes and after the changes).
Also tested if both nix develop and keyserver migration created the new file.

Adding @ashoat as reviewer because of new dependency.

Diff Detail

Repository
rCOMM Comm
Lint
No Lint Coverage
Unit
No Test Coverage

Event Timeline

Harbormaster returned this revision to the author for changes because remote builds failed.Oct 6 2023, 7:17 AM
Harbormaster failed remote builds in B23054: Diff 31743!

Move the import inside of the flow-types/npm/cors_v2.x.x.js to declare module block to fix the flow error.

michal published this revision for review.Oct 11 2023, 9:13 AM
This revision is now accepted and ready to land.Oct 16 2023, 3:03 AM
This revision now requires review to proceed.Oct 16 2023, 3:04 AM

Resigning since it looks like others have/will take a look, feel free to re-add if there's something specific I can be helpful with

ashoat requested changes to this revision.Oct 17 2023, 5:05 AM
ashoat added inline comments.
keyserver/src/keyserver.js
68 ↗(On Diff #31929)

I don't think using commAppURLFacts.baseDomain is a good long-term approach here, since we'll want people to be able to set up a keyserver without specifying commAppURLFacts. I know you had some plans to figure config out as a follow-up... can you link the task you're tracking that in?

139 ↗(On Diff #31929)

It appears that you're setting CORS for both commAppRouter and squadCalRouter, but I'm not sure why you need to.

I think you should only set CORS for one if you don't need it for both. I suspect you just need to set CORS for the future keyserver endpoints (squadCalRouter) to allow for calls from commAppURLFacts.baseDomain, but I may be wrong.

Let me know if I'm missing something!

This revision now requires changes to proceed.Oct 17 2023, 5:05 AM
keyserver/src/uploads/uploads.js
175–180 ↗(On Diff #31929)

Can you explain why these were removed?

keyserver/src/keyserver.js
68 ↗(On Diff #31929)

That's true, I haven't considered that. I think we could just default to web.comm.app if commAppURLFacts doesn't exist?

  • for dev environments it will still be overridden to localhost
  • for staging setup we can still override it for the staging web hosting
  • for your production keyserver it will be a no-op override and for other keyservers it will default to production web.comm.app
139 ↗(On Diff #31929)

Yeah, that's done in D9452 where I split keyserver and "web hosting" endpoints.

keyserver/src/uploads/uploads.js
175–180 ↗(On Diff #31929)

They are handled by the top-level config. The development url facts already contains the setup for localhost:3000 cors

Default cors to https://web.comm.app

ashoat requested changes to this revision.Oct 18 2023, 7:06 AM
ashoat added inline comments.
keyserver/src/keyserver.js
68 ↗(On Diff #31929)

commAppURLFacts is used for configuring the web app hosting. There should be no expectation that the keyserver and the web app are hosted in the same place. I don't think it makes sense to use commAppURLFacts.baseDomain here. Can you identify a solution that doesn't use commAppURLFacts.baseDomain, and either implement it, or create a follow-up task / follow-up diff for it?

139 ↗(On Diff #31929)

It's confusing to introduce issues in one diff and resolve it in a later diff. I'm not even on the review for D9452. There should be no expectation that a reviewer needs to read a later diff in order to understand an earlier diff, ESPECIALLY when they're not even on the review for the later diff.

Please address this issue here.

keyserver/src/uploads/uploads.js
175–180 ↗(On Diff #31929)

I need a deeper explanation. If localhost:3000 was just connecting to localhost:3000, why did we even have this in the first place?

High-level feedback – if somebody asks a question in code review, it's probably because something is non-obvious. Responding with a couple sentences like this is frankly not helpful, and will lead to your work being delayed further.

This revision now requires changes to proceed.Oct 18 2023, 7:06 AM

Perhaps the URL that the keyserver expects for the web app should be something configurable via getCommConfig? We could preload / cache it similar to how we do with the URL facts today.

Use getCommConfig for the cors domain. Added the migrations/nix setup for it because it's required for the media download (as we are removing the already existing upload-specific cors handling). Amend the test plan for generation and use of config file on dev environment.

Regarding setting up the cors on commAppRouter - after testing it actually should to be there so that this diff can be tested. Currently the dev urlPrefix for the web app is hardcoded to localhost:300/comm (which I'm changing in D9451) and the web app uses the commAppRouter endpoints (also changed in D9451). This means that if I try to test from the subdomain.localhost:3000 the cors issues still fail. This shouldn't be a problem on normal dev setup with web app hosted on localhost (without subdomains), but in my opinion we should keep the diff as it currently is.

keyserver/src/keyserver.js
56–57 ↗(On Diff #32265)

I don't think we need to do any additional caching/preloading but I might be missing something.

keyserver/src/uploads/uploads.js
175–180 ↗(On Diff #31929)

This code has been introduced in D7249 and there is more context there. In summary: uploads in development mode use the ip address instead of localhost (http://xx.xx.xx.xx:3000/comm/upload/123/123abc). From what I understand this is because of native media download issues.

The web app is hosted on localhost and using fetch() with the media url containing the ip address fails with a cors issue.

Now that we handle cors globally we can remove this upload-specific code.

ashoat requested changes to this revision.Oct 23 2023, 5:55 AM

Regarding setting up the cors on commAppRouter - after testing it actually should to be there so that this diff can be tested. Currently the dev urlPrefix for the web app is hardcoded to localhost:300/comm (which I'm changing in D9451) and the web app uses the commAppRouter endpoints (also changed in D9451). This means that if I try to test from the subdomain.localhost:3000 the cors issues still fail. This shouldn't be a problem on normal dev setup with web app hosted on localhost (without subdomains), but in my opinion we should keep the diff as it currently is.

I don't follow your explanation here. Please read my feedback below, where I spent 30min going into the code and trying to explain to you what your response should have been to my other concerns.

Please try to understand the feedback, and then try explaining this again in a way that doesn't force me to do any further research.

Please also take some time to explain why subdomain.localhost:3000 is a relevant concern. I don't understand why you're bringing up this weird access pattern.

keyserver/src/database/migration-config.js
584 ↗(On Diff #32265)

Can you add a comment explaining why you're checking this?

Alternately, instead of a comment it might be more readable to define a function.

Something like:

function isDockerEnvironment(): boolean {
  return !!process.env.COMM_DATABASE_HOST;
}
keyserver/src/uploads/uploads.js
175–180 ↗(On Diff #31929)

In summary: uploads in development mode use the ip address instead of localhost (http://xx.xx.xx.xx:3000/comm/upload/123/123abc). From what I understand this is because of native media download issues.

This explanation was still insufficient for me, even after asking for a deeper explanation. As a result, I had to do further research, which is why this diff's review was further delayed.

I'd like to repeat what I said in my last comment:

High-level feedback – if somebody asks a question in code review, it's probably because something is non-obvious. Responding with a couple sentences like this is frankly not helpful, and will lead to your work being delayed further.

@michal, please understand the feedback here. It is the responsibility of the diff author to make the review as easy as possible. Your diff reviewers are generally more senior people, in this case the CEO of the company. I do not have as much time as you, and forcing me to do independent research to review your diff is a recipe for (1) frustrating me, (2) making your diff review take longer.

In the future, please try *much* harder to find deeper explanations. Your current efforts are insufficient.

With the high-level feedback out of the way, let's get into specifics:

From what I understand this is because of native media download issues.

"Native media download issues" is extremely vague. This is not a helpful answer because it doesn't really explain anything, and just forces the reader to ask more questions.

To answer my own question, I had to take a look at getUploadURL: link. We can see that there is a special condition that checks ip.v4.sync() there, which is why the IP address is used instead of localhost. Looking at git blame, we can see that was introduced in D6587, which provides an excellent explanation. We need to use the IP address in order to support devs testing the mobile app from their physical device.

This is the explanation you should have provided.

Now that we handle cors globally we can remove this upload-specific code.

Once again there is simply not enough detail in this statement. You need to write and explain more.

  1. What testing did you do to make sure this change is safe? Why isn't it listed in the Test Plan?
  2. How does "handling CORS globally" address the original issue?

Here is the explanation you should have given instead of this single sentence:

This CORS rule is only necessary for the dev environment on web. On native CORS rules are ignored, and in production getUploadURL will just use the URL from the URL facts (the keyserver URL), which in the past was the same URL that the web app is being served on, and in the future will be handled by the CORS rules introduced in this diff.

Now that we've narrowed down the purpose of this CORS rule, let's see if we still need it for the dev environment on web.

In the dev environment we always access the web app via http://localhost:3000. This is why these CORS rule were explicitly written to reference that address. However, the CORS rules we're introducing should actually protect this case. That's because the upload URL is a keyserver URL, and the web app must already be set up to be able to access keyserver URLs via CORS rules on the keyserver endpoints.

As such, we can conclude that the new CORS rules obviate the need for these old rules. Additionally, I've tested and confirmed that media uploads load fine in the web app under this new setup.

This revision now requires changes to proceed.Oct 23 2023, 5:55 AM

Removed CORS middleware from commAppRouter and amended the test plan so it doesn't require it. Amended the test plan to test the removal of existing CORS settings for upload endpoints. Defined a function for checking docker environment for clarity.

michal edited the test plan for this revision. (Show Details)
ashoat added inline comments.
keyserver/flow-typed/npm/cors_v2.x.x.js
6–21 ↗(On Diff #32462)

Can we indent consistently? I know this was pulled from flow-typed (before some slight modifications) but it would be good to match at least within the file. Two space indents preferred

keyserver/src/keyserver.js
72 ↗(On Diff #32462)

Can we call this keyserverCorsOptions for clarity?

This revision is now accepted and ready to land.Oct 27 2023, 7:59 AM

Indentation and naming fixes

(indentation fixes, I forgot to commit them)