Page MenuHomePhabricator

Implement native iOS code to fetch data from blob service and keyserver code to upload notification payload if it exceeds APNs limits
ClosedPublic

Authored by marcin on Jul 29 2023, 4:01 AM.
Tags
None
Referenced Files
Unknown Object (File)
Tue, Nov 12, 12:15 PM
Unknown Object (File)
Sun, Nov 10, 10:54 PM
Unknown Object (File)
Sun, Nov 10, 8:49 PM
Unknown Object (File)
Sun, Nov 10, 7:30 PM
Unknown Object (File)
Sun, Nov 10, 6:08 PM
Unknown Object (File)
Sun, Nov 10, 6:02 PM
Unknown Object (File)
Sun, Nov 10, 4:11 PM
Unknown Object (File)
Sun, Nov 10, 11:41 AM
Subscribers

Details

Summary

This differential implements native iOS code in Objective-C that can fetch data (any data - not
necessarily notif payload) from blob service. Additionally it implements first usage of this code in
NotificationService where large notification payload is downloaded from blob service, AES decrypted and
persisted in flat file and redux (if app is in the background). Finally the keyserver code that upload large notification payload to the blob service is implemented.

Test Plan
  1. Build iOS app.
  2. Background or kill the app.
  3. Send large notifications to the app. I used 16000 character notifications for testing.
  4. Ensure notification is displayed shortly after it was sent.
  5. Kill your local keyserver
  6. Open the app. Ensure that (entire!!!) message content is available in the thread.

Diff Detail

Repository
rCOMM Comm
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

Remove unecessary line left by oversight

native/ios/NotificationService/NSEBlobServiceClient.mm
5 ↗(On Diff #29228)

This time limit matches the one we introduced and justified for Android. Android has 20 seconds to process notif, iOS has 30. Theoretically we could increase it to 20 but maybe it is better to keep behaviour similar between platforms?

13 ↗(On Diff #29228)

Why to make this class singleton?
This class is mostly responsible for executing operations on NSURLSession instance. According to Apple Docs, NSURLSession instances should be shared: https://developer.apple.com/documentation/foundation/url_loading_system/fetching_website_data_into_memory?language=objc.

Additionally I have done some experimentation and established that NSE uses different threads to process subsequent notifications but it uses the same process for a prolonged period of time. Once I had sent first notification to iOS device I was seeing the same PID being logged to the console for each notifications for > 15 minutes. Therefore I think it is better to create this object once end just keep it then to create it for every invocation.

15 ↗(On Diff #29228)

This way of implementing singleton is inspired by our CommSecureStoreIOSWrapper.

24 ↗(On Diff #29228)

Why to pass mainQueue here? According to apple docs: https://developer.apple.com/documentation/foundation/nsurlsession/1411597-sessionwithconfiguration, passing nothing makes NSURLSession instance create its own queue and thread to process request completion handlers. We don't want to create new threads when we process notif so I pass mainQueue here since it is always associated with the main thread which is always present so we don't create new threads when processing notif.

32 ↗(On Diff #29228)

Why synchronous? Unfortunately I don't have any Apple docs to support it - just my own experimentation. Basically I tried to make this call async in two ways:

  1. Process notification payload and call notification completionHandler in data task completionHandler. I observed that even though we download large notification and call notification completionHandler on a different thread NSE still waits before it starts to process new notification. I sent two notifs: one large and one small. Small notification was displayed afer large notification processing completed even though it took place on separate thread.
  2. Download and process notification payload on a separate thread, but call notification completionHandler immediately on the same thread NSE received notification. I observed that after calling completionHandler NSE dropped the download task and large notification payload was not persisted.

So my conclusion is that NSE doesn't process new notification until we call completionHandler of previous notification no matter the thread it takes place. Therefore there is no good way to make this call async in NSE.

native/ios/NotificationService/NotificationService.mm
91 ↗(On Diff #29228)

I made some of NSE methods static since I want to call them in a block and avoid this issue: https://bytes.vokal.io/objc-block-capture-weakself/. Those methods were stateless anyway .

native/ios/NotificationService/NSEBlobServiceClient.mm
5 ↗(On Diff #29228)

I'd leave 15 - Shouldn't notif processing be as quick as possible?

13 ↗(On Diff #29228)

Yeah, singleton seems to be good option here

native/ios/NotificationService/NotificationService.mm
13 ↗(On Diff #29228)

Technically I'd prefer this constant to be defined in the AES Crypto module, however I'm not sure if ObjC <-> Swift won't complicate it too much - might be not worth it

280 ↗(On Diff #29228)
  1. Do we have to create this object with each call? Can it be static or singleton? I had the same concern for Android: https://phab.comm.dev/D8566?id=28842#inline-54921
  2. Can we give this variable a meaningful name?
  1. Remove AES constant definition and use new AESCryptoModuleObjCCompat methods
  2. Make AESCryptoModuleObjCCompat object static and initialized only once.
tomek added 1 blocking reviewer(s): bartek.

Seems reasonable, but I don't know Objective-C that well

native/ios/NotificationService/NSEBlobServiceClient.mm
20 ↗(On Diff #29438)

Do we have a task for it? We probably can mention it in the code comment.

native/ios/NotificationService/NotificationService.mm
59–61 ↗(On Diff #29438)

How do we get the encryptionKey?

bartek added inline comments.
native/ios/NotificationService/NotificationService.mm
59–61 ↗(On Diff #29438)

We do it in D8529 by calling generateKey() when uploading the notif to blob service

This revision is now accepted and ready to land.Aug 3 2023, 12:52 AM
native/ios/NotificationService/NSEBlobServiceClient.mm
20 ↗(On Diff #29438)

This TODO section is about authentication with COMM_ACCESS_TOKEN. I was advices not to land this feature until COMM_ACCESS_TOKEN staff is resolved and utilised in this stack. So I don't think we need a task for it since authentication is a part of this task.

native/ios/NotificationService/NotificationService.mm
59–61 ↗(On Diff #29438)

The encryptionKey is sent by the keyserver in olm-encrypted notification. It is delivered as base 64 string an converted to bytes there since AESCryptoModule requires bytes.

Share blob client code with the main app.

native/ios/Comm/CommIOSNotifications/CommIOSNotificationsBlobClient.mm
87 ↗(On Diff #30792)

This signature got me confused.
General idea of Obj-C/Swift naming is that it can be readable in human language, e.g a few lines below there is

binaryAuthObject = [NSJSONSerialization dataWithJSONObject:jsonAuthObject
                                                       options:0
                                                         error:&jsonError];

// From NSJSONSerialization get data, with given JSON object, options and error
marcin retitled this revision from Implement native iOS code to fetch data from blob service to Implement native iOS code to fetch data from blob service and keyserver code to upload notification payload if it exceeds APNs limits.
marcin edited the summary of this revision. (Show Details)
marcin edited the test plan for this revision. (Show Details)
keyserver/src/push/send.js
1029–1030 ↗(On Diff #38139)

Why is that the case?

native/ios/Comm/CommIOSServices/CommIOSBlobClient.mm
5–10 ↗(On Diff #38139)

Would that mean that it is impossible to use the local blob service?

12 ↗(On Diff #38139)

What unit do we use here?

Fix two errors:

  1. Call signal semaphore after receiving response with error code.
  2. Check against entire range of successful response codes - not just 200

Simplify blob client API and make it match the Android one.

Some nits below, but I didn't review closely.

This code deals with a lot of parallelism stuff, so we should be very careful in reviewing it. Probably same goes for the comparable Android code.

keyserver/src/push/send.js
1032 ↗(On Diff #38193)

Capitalize Notifs

native/ios/NotificationService/NotificationService.mm
889–896 ↗(On Diff #38193)

Seems like very narrow columns here. Can we stick with 80 chars?

Also typo in instancess

marcin added inline comments.
keyserver/src/push/send.js
1029–1030 ↗(On Diff #38139)

The context is here: https://linear.app/comm/issue/ENG-3797/download-notification-payload-from-blob-service-if-its-size-exceeded#comment-4d35b32c. My understanding is that it was a business decision that was made when MacOS notifications were first introduced. The reason here was probably the fact that initially MacOS notifications were handled entirely inside a main desktop process where there is no access to redux so messageInfos were simply unnecessary. Technically it would be possible to transfer messageInfos from notification to renderer proces using ipc but it wasn't implemented. Making messageInfos useful in MacOS notifications is an option that we can consider but it is out of the scope of this differential. cc @michal

native/ios/Comm/CommIOSServices/CommIOSBlobClient.mm
5–10 ↗(On Diff #38139)

Using local blob service will be possible after replacing one of the blobServiceAddress above. Developer willing to use local blob service has to hardcode its address anyway in lib/facts/blob-service.js.

12 ↗(On Diff #38139)

Its seconds. This constant is used a couple of lines below where setting request timeout which require timeout value in seconds: https://developer.apple.com/documentation/foundation/nsurlsessionconfiguration/1408259-timeoutintervalforrequest

atul added inline comments.
native/ios/Comm/CommIOSServices/CommIOSBlobClient.mm
20–110 ↗(On Diff #38193)

Completely unsolicited thought, but does this code need to be in Objective C? At a high level this seems like a LOT of potentially error-prone code just to make an HTTP request?

I'm not suggesting we do it, but code to make an HTTP request in Swift would be a lot simpler thanks to eg async/await

f1e2ee.png (816×1 px, 314 KB)

tomek added inline comments.
keyserver/src/push/send.js
1093–1095 ↗(On Diff #38193)

It's probably safer to spread first

native/ios/Comm/CommIOSServices/CommIOSBlobClient.mm
75 ↗(On Diff #38193)

Is it a good practice to check it like this?

95 ↗(On Diff #38193)

Is it possible for some exceptions to be thrown before this call? Can we execute it in some kind of finally block?

This revision is now accepted and ready to land.Mar 22 2024, 10:09 AM
native/ios/Comm/CommIOSServices/CommIOSBlobClient.mm
20–110 ↗(On Diff #38193)

I will create a follow up task to rewrite it in Swift. However I don't think it is actionable right now. The purpose of this project was to refresh most of the existing code. Rewriting in swift would require to add some headers with bindings do additional testing which could delay this project further.

75 ↗(On Diff #38193)

I was unable to find any instance method of NSHTTPURLResponse to check if the request was successful or not. It is pretty low level API so I think we have to check maunally.

native/ios/Comm/CommIOSServices/CommIOSBlobClient.mm
20–110 ↗(On Diff #38193)

Eh not sure we need a task, it was more just commentary rather than something to prioritize

Address Tomek suggestions

Remove unnecessary dispatch_semaphore_signal since it is in the finally statement.