Page MenuHomePhabricator

[native] Implement thumbhash generation on Android
ClosedPublic

Authored by bartek on May 10 2023, 6:08 AM.
Tags
None
Referenced Files
Unknown Object (File)
Tue, Apr 16, 11:26 AM
Unknown Object (File)
Tue, Apr 16, 7:54 AM
Unknown Object (File)
Mon, Apr 15, 6:56 PM
Unknown Object (File)
Mon, Apr 15, 3:54 PM
Unknown Object (File)
Mon, Apr 15, 3:42 PM
Unknown Object (File)
Sun, Apr 14, 8:25 AM
Unknown Object (File)
Sun, Apr 14, 7:56 AM
Unknown Object (File)
Sat, Apr 13, 3:54 PM
Subscribers

Details

Summary

Implemented the generateThumbHash() function on iOS.
Used code from the thumbhash repository: https://github.com/evanw/thumbhash

Didn't convert the original library's Java code to Kotlin because it's unnecessary - interop between these languages works flawlessly. Conversion could potentially introduce bugs.

As gathering bitmap pixels from URI is a bit tricky, I put some in-code comments

Depends on D7779

Test Plan

One easy way is to hook this to expo-image-picker result (e.g. avatar picker) and provide picked uri to the function. It should usually return a 43 char long string (sometimes longer if there is alpha channel).

Diff Detail

Repository
rCOMM Comm
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

bartek held this revision as a draft.
bartek published this revision for review.May 10 2023, 6:29 AM
bartek added inline comments.
native/expo-modules/thumbhash/android/src/main/java/app/comm/android/thumbhash/ThumbHash.java
5 ↗(On Diff #26341)

This file is copy-pasted from the original repo (only removed unused code) - you can skip it

native/expo-modules/thumbhash/android/src/main/java/app/comm/android/thumbhash/ThumbHash.java
5 ↗(On Diff #26341)

The code is licensed under MIT, which requires attribution and including the license. That means we should avoid copy-paste. Can we find a way to include this as a dependency? One approach would be to add it to package.json:

"thumbhash": "git+https://git@github.com/evanw/thumbhash.git",

And then to symlink from node_modules/thumbhash into this folder.

Alternately, we could do the whole third_party folder thing and include the LICENSE in there.

native/expo-modules/thumbhash/android/src/main/java/app/comm/android/thumbhash/ThumbHash.java
5 ↗(On Diff #26341)

Expo-image has the same copy-paste with single-line mention of the original author: https://github.com/expo/expo/blob/d4b007465df538de2ae5abb36dc768caf9865e97/packages/expo-image/ios/Thumbhash.swift#L4

If we really want to add the licensing stuff, the git+ dependency would be very cumbersome to link it to the native code. The only reliable and not overcomplicated way, on which we won't spend too much time, would be to create the third_party folder directly in the expo-modules/thumbhash/{ios,android}/ directories.

native/expo-modules/thumbhash/android/src/main/java/app/comm/android/thumbhash/ThumbHash.java
5 ↗(On Diff #26341)

If we really want to add the licensing stuff, the git+ dependency would be very cumbersome to link it to the native code.

Not sure why? Should be possible with symlinks to just reference the files you want in node_modules

The only reliable and not overcomplicated way, on which we won't spend too much time, would be to create the third_party folder directly in the expo-modules/thumbhash/{ios,android}/ directories.

That seems fine

Move ThumbHash code to third party dir and add LICENSE

native/expo-modules/thumbhash/android/src/main/java/app/comm/android/thumbhash/ThumbHash.java
5 ↗(On Diff #26341)

If we really want to add the licensing stuff, the git+ dependency would be very cumbersome to link it to the native code.

Not sure why? Should be possible with symlinks to just reference the files you want in node_modules

  • We're in monorepo so we cannot be sure that hoisting won't break symlinks (even with future dependency changes). We could use something that expo-autolinking does for resolving correct paths but this requires some groovy + ruby scripting for gradle/cocoapods which adds complexity
  • The upstream repo can change (unless we fork it - that might be a good idea)
  • Files in the repo are package-private (no public modifier) so I'd need to patch-package them (or use a fork)
native/expo-modules/thumbhash/android/src/main/java/app/comm/android/thumbhash/ThumbHash.java
5 ↗(On Diff #26341)

The current solution seems fine, but just following up here:

The upstream repo can change (unless we fork it - that might be a good idea)

We generally set a specific commit hash / tag / branch when we do this, eg. take a look at how we depend on olm in `native:

"olm": "git+https://github.com/CommE2E/olm.git#v0.0.9",

Files in the repo are package-private (no public modifier) so I'd need to patch-package them (or use a fork)

Not familiar with this or why it matters... if you're checking out a Git repo just to access some file in it, I don't think package.json needs to be considered. For olm in native, I believe we include a Git repo that doesn't even have a package.json

native/expo-modules/thumbhash/android/src/main/java/app/comm/android/thumbhash/ThumbHash.java
5 ↗(On Diff #26341)

Not familiar with this or why it matters... if you're checking out a Git repo just to access some file in it, I don't think package.json needs to be considered. For olm in native, I believe we include a Git repo that doesn't even have a package.json

I mean native Swift/Java files.

Fortunately on web, we can simply add the thumbhash npm dependency which has all stuff that we need

atul added a reviewer: ashoat.

Defer to @ashoat (setting as blocking) on license/attribution/dependency management.

Didn't convert the original library's Java code to Kotlin because it's unnecessary - interop between these languages works flawlessly.

Can confirm that Java <> Kotlin interop is super solid.

native/expo-modules/thumbhash/android/src/main/java/app/comm/android/thumbhash/ThumbhashModule.kt
46–47 ↗(On Diff #26434)

Should we define 100 as some sort of constant outside of generateThumbHash(...) at "the top" of the file?

62–65 ↗(On Diff #26434)

Should we define 100 as some sort of constant outside of generateThumbHash(...)

85–86 ↗(On Diff #26434)

Shouldn't we be using bitmap.width and bitmap.height instead of this.width and this.height?

I'm guessing that bitmaps will have the same dimensions regardless of configuration, so it probably doesn't really matter. But it feels like we should use the dimensions from the object we're calling getPixels(...) on?

native/expo-modules/thumbhash/android/src/main/java/thirdparty/ThumbHash.java
148 ↗(On Diff #26434)

Figure we should add newline at end of file to make sure it's a "valid" file

Can you clarify why you named the folder third_party in D7779 but thirdparty here?

\We need a consistent way to identify these folders so we find all of the LICENSEs we are using for attribution reasons. Can we stick with a consistent pattern of always naming the folder third_party?

This revision is now accepted and ready to land.May 15 2023, 12:24 PM

Can you clarify why you named the folder third_party in D7779 but thirdparty here?

Because it's Java package name convention. Folder name should correspond to package name which should not contain underscores

native/expo-modules/thumbhash/android/src/main/java/app/comm/android/thumbhash/ThumbhashModule.kt
85–86 ↗(On Diff #26434)

It doesn't really matter, but yeah, your way seems more correct

Address feedback: newline, size as constant, use correct dimensions