[services] Make AWS SDK versions consistent
ClosedPublic
Actions

Authored by bartek on Apr 13 2023, 12:43 AM.

Details

Reviewers

varun
• jon
tomek

Commits

rCOMM1207793603ad: [services] Make AWS SDK versions consistent

Summary

This unblocks ENG-3102.

Bumped blob, backup and feature-flag services AWS SDK versions to match these from identity service.
There is a newer version of AWS SDKs (0.55.0 for aws-{config,types} / 0.25.1 for aws-sdk-*) but they contain too many renames and breaking changes. Such bump should be done in a separate diff if requested.

Two significant breaking changes here, related to introduction of "Endpoint 2.0":

aws_config Endpoint is now a simple endpoint_url() that takes a string.
S3 now defaults to virtual bucket addressing, which is not supported by localstack.

Change details can be found in AWS changelog (although they're grouped by date, not version number): https://github.com/awslabs/aws-sdk-rust/blob/main/CHANGELOG.md#january-13th-2023

See the first two bullet points of January 13th 2023

Test Plan

CI
Ran cargo build for each service
Ran integration tests for blob and backup services

Diff Detail

Repository

rCOMM Comm

Lint

Lint Not Applicable

Unit

Tests Not Applicable

Event Timeline

bartek created this revision.Apr 13 2023, 12:43 AM

bartek held this revision as a draft.

Herald added subscribers: atul, ashoat. · View Herald TranscriptApr 13 2023, 12:43 AM

bartek published this revision for review.Apr 13 2023, 12:52 AM

Harbormaster completed remote builds in B18299: Diff 25077.Apr 13 2023, 1:01 AM

tomek accepted this revision.Apr 13 2023, 4:03 AM

This revision is now accepted and ready to land.Apr 13 2023, 4:03 AM

S3 now defaults to virtual bucket addressing, which is not supported by localstack

Is this concerning to us? Will this prevent us from testing eg. blob service in our local environment?

In D7409#220765, @ashoat wrote:

S3 now defaults to virtual bucket addressing, which is not supported by localstack

Is this concerning to us? Will this prevent us from testing eg. blob service in our local environment?

Nope, it's just the change of API defaults - I adjusted the config accordingly. You can read more here: https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html
Basically, Localstack understands "Path-style requests" as these new virtual addresses are probably difficult to achieve on localhost (perhaps would need some changes to /etc/hosts).
As long as S3-SDK supports both ways, we're safe (should be long, I read somewhere that they delayed the deprecation of path-style addresses)

Closed by commit rCOMM1207793603ad: [services] Make AWS SDK versions consistent (authored by bartek). · Explain WhyApr 13 2023, 6:36 AM

This revision was automatically updated to reflect the committed changes.

bartek added a commit: rCOMM1207793603ad: [services] Make AWS SDK versions consistent.

There is a newer version of AWS SDKs (0.55.0 for aws-{config,types} / 0.25.1 for aws-sdk-*) but they contain too many renames and breaking changes.

Huh, that's kind of surprising. My perception was that AWS was pretty hardcore about avoiding breaking changes if at all possible barring any sort of serious security issue.