Paths

Table of Contentst

[services] Tunnelbroker - Get AWS region from aws/config only.
ClosedPublic
Actions

Authored by • max on Feb 7 2022, 4:49 AM.

Details

Reviewers

• karol
tomek
varun
• jim
ashoat

Commits

rCOMMb0729e01eb14: [services] Tunnelbroker - Get AWS region from aws/config only.

Summary

Switch to use the AWS region from ~/.aws/config file instead of looking it at the ~/.aws/credentials first.
The main reason for that, AWS CLI sets the region inside the ~/.aws/config file and we need both config and credentials to follow the AWS CLI approach.

Linear task: ENG-559

Test Plan

Tunnelbroker successfuly connect to AWS using ~/.aws/credentials for credentials and ~/.aws/config for the region:

cat ~/.aws/config            
[default]
region = us-east-2

cat ~/.aws/credentials
[default]
aws_access_key_id = .....
aws_secret_access_key = .....

Diff Detail

Repository

rCOMM Comm

Branch

D3119

Lint

No Lint Coverage

Unit

No Test Coverage

Event Timeline

• max created this revision.Feb 7 2022, 4:49 AM

Herald added subscribers: • karol, atul, • adrian and 2 others. · View Herald TranscriptFeb 7 2022, 4:49 AM

• max edited the summary of this revision. (Show Details)Feb 7 2022, 4:51 AM

• max edited the test plan for this revision. (Show Details)

• max added reviewers: • karol, tomek.

• max retitled this revision from [services] Tunnelbroker - Get aws region from aws/config only. to [services] Tunnelbroker - Get AWS region from aws/config only..

Harbormaster completed remote builds in B6583: Diff 9331.Feb 7 2022, 4:54 AM

• max requested review of this revision.Feb 7 2022, 4:54 AM

How about that thing I raised in the issue?

I think we could use ~/.aws/config as long as we manage to automate it on the cloud during deployment. I don't think we should explicitly mount volumes to the docker containers.

services/docker-compose.yml
24 ↗	(On Diff #9331)	Do we still need this?

This revision now requires changes to proceed.Feb 7 2022, 7:25 AM

ashoat added reviewers: varun, • jim.Feb 7 2022, 7:11 PM

• max planned changes to this revision.Feb 8 2022, 10:56 AM

I don't love the idea of having a third config file (.aws/credentials & tunnelbroker.ini already). Why is .aws/credentials not good enough? What does the CLI have to do with the application config?

In D3119#83034, @jimpo wrote:

I don't love the idea of having a third config file (.aws/credentials & tunnelbroker.ini already). Why is .aws/credentials not good enough? What does the CLI have to do with the application config?

The problem is that AWS cli operates both .aws/credentials and .aws/config. We have credentials in .aws/credentials and region in .aws/config. And we need both credentials and region provided.
If we use .aws/credentials only and the user will use AWS CLI it will create .aws/config for the region anyway.

• max marked an inline comment as done.Feb 8 2022, 7:25 PM

• max added inline comments.

services/docker-compose.yml
24 ↗	(On Diff #9331)	Do we still need this? Does AWS c++ SDK work without mounting the credentials file inside the container? How it will get the credentials? Does it related to EC2 only?

• max requested review of this revision.Feb 8 2022, 7:25 PM

• max marked an inline comment as done.

• karol added inline comments.Feb 9 2022, 4:07 AM

services/docker-compose.yml
24 ↗	(On Diff #9331)	Yes, sorry, we need this locally.

• karol accepted this revision.Feb 9 2022, 4:08 AM

This revision is now accepted and ready to land.Feb 9 2022, 4:08 AM

Herald added a reviewer: ashoat. · View Herald TranscriptFeb 9 2022, 4:08 AM

Herald added a subscriber: • benschac. · View Herald Transcript

This revision now requires review to proceed.Feb 9 2022, 4:08 AM

• jim accepted this revision.Feb 9 2022, 8:23 AM

• jim added inline comments.Feb 9 2022, 2:26 PM

services/tunnelbroker/docker-server/contents/server/src/Tools/AwsTools.cpp
16 ↗	(On Diff #9331)	What if we keep this code and also mount in the .aws/config. Then in prod it can get the region using the role credentials without needing to set it in .aws/config and it will use the one in .aws/config if set by the CLI?

@geekbrother, what do you think of @jimpo's suggestion above? If it makes sense, you could make a separate task for it. Also curious for @karol-bisztyga's take

This revision is now accepted and ready to land.Feb 9 2022, 10:20 PM

In D3119#83783, @ashoat wrote:

@geekbrother, what do you think of @jimpo's suggestion above? If it makes sense, you could make a separate task for it. Also curious for @karol-bisztyga's take

It's makes sense, but in this way we need to skip the AWS CLI approach and parse ~/.aws/crdentials file by ourselves.

I've created a task for discussion about that and assigned @jimpo and @karol-bisztyga as well for the discussion there.

services/tunnelbroker/docker-server/contents/server/src/Tools/AwsTools.cpp
16 ↗	(On Diff #9331)	What if we keep this code and also mount in the .aws/config. Then in prod it can get the region using the role credentials without needing to set it in .aws/config and it will use the one in .aws/config if set by the CLI? I am not sure it is worth it, because seems that method doesn't fulfill `region` properly.