Page MenuHomePhabricator

[tunnelbroker] Hyper service for WebSocket connections
ClosedPublic

Authored by bartek on Sep 12 2023, 1:53 PM.
Tags
None
Referenced Files
F3373748: D9158.id31116.diff
Tue, Nov 26, 11:38 AM
F3373461: D9158.diff
Tue, Nov 26, 9:49 AM
Unknown Object (File)
Fri, Nov 22, 9:46 PM
Unknown Object (File)
Wed, Nov 20, 5:10 PM
Unknown Object (File)
Wed, Nov 20, 5:10 PM
Unknown Object (File)
Wed, Nov 20, 5:10 PM
Unknown Object (File)
Wed, Nov 20, 5:08 PM
Unknown Object (File)
Tue, Oct 29, 3:57 AM
Subscribers

Details

Summary

Implementation basing on tokio-tungstenite doesn't work well with upgrading HTTP connection to WebSocket. This diff implements hyper service for WebSocket connections, using hyper-tungstenite library.
By doing that, we also get health check for free. In the future, we can support more "normal" HTTP requests on the same port as websocket.

A great series of articles describing Tower / Hyper services under the hood: https://www.fpcomplete.com/blog/axum-hyper-tonic-tower-part2/

The hyper-tungstenite library does more or less stuff that is described in this article: https://levelup.gitconnected.com/handling-websocket-and-http-on-the-same-port-with-rust-f65b770722c9

Depends on D9156.

Test Plan

Tunnelbroker builds. Actual test plan is in the next diff.

Diff Detail

Repository
rCOMM Comm
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

bartek held this revision as a draft.
bartek published this revision for review.Sep 12 2023, 2:12 PM
bartek added inline comments.
services/tunnelbroker/src/websockets/mod.rs
60 ↗(On Diff #30991)

This is done in the next diff D9160

Awesome that we get the health check for free! I assume we'll need to implement something on the client for that to work, yes? The Tunnelbroker client in keyserver currently uses the ws library for Node.js

Awesome that we get the health check for free! I assume we'll need to implement something on the client for that to work, yes? The Tunnelbroker client in keyserver currently uses the ws library for Node.js

We tested that with @kamil and fortunately no client changes are needed. The client library is already handling upgrade requests properly.

I'm asking specifically about the health check. It's my understanding that the WebSocket protocol does not have any built-in health checks. Given we're using the Node.js ws library directly on the keyserver side, it seems like we'll either need to switch to using a compatible (Rust-based?) client, OR we'll need to do some additional work on the keyserver to implement this health check. Is that accurate?

This health check is only needed by AWS to check if the instance endpoint is healthy. It's more related to ENG-4470.

It's my understanding that the WebSocket protocol does not have any built-in health checks.

That's right. The check endpoint is pure HTTP, it has nothing in common with WebSocket, except port number. If you really want to check health on client side, you can simply fetch('http://tunnelbroker.host:websocket_port/health').


Or, if you mean a kind of ping-pong heartbeat, this is unrelated to this particular health check, I see there's a task for it (ENG-4313) - that can be implemented e.g. as a separate tunnelbroker message type and sent repeatedly after the connection (websocket session) is already established.

This revision is now accepted and ready to land.Sep 13 2023, 6:10 AM