Page MenuHomePhabricator

[services] Export cpp tools as CMake project
ClosedPublic

Authored by jon on Jun 19 2022, 11:24 AM.
Tags
None
Referenced Files
Unknown Object (File)
Fri, Nov 1, 9:01 PM
Unknown Object (File)
Sun, Oct 27, 6:55 AM
Unknown Object (File)
Sun, Oct 27, 6:55 AM
Unknown Object (File)
Sun, Oct 27, 6:55 AM
Unknown Object (File)
Sun, Oct 27, 6:55 AM
Unknown Object (File)
Sun, Oct 27, 6:55 AM
Unknown Object (File)
Sun, Oct 27, 6:55 AM
Unknown Object (File)
Sun, Oct 27, 6:55 AM

Details

Summary

Export a cmake project

Depends on D3867

Test Plan

Checkout https://phab.comm.dev/D4452 (which should have all of the file paths normalized), then run:

nix develop && cd native/cpp/CommonCpp/Tools && mkdir build && cd build && cmake .. && make

Diff Detail

Repository
rCOMM Comm
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Harbormaster returned this revision to the author for changes because remote builds failed.Jun 19 2022, 11:29 AM
Harbormaster failed remote builds in B9806: Diff 13570!
jon retitled this revision from Export cpp tools as CMake project to [services] Export cpp tools as CMake project.
Harbormaster returned this revision to the author for changes because remote builds failed.Jun 19 2022, 1:50 PM
Harbormaster failed remote builds in B9811: Diff 13575!

btw iOS build had the following error:

error: Build input file cannot be found: '/opt/homebrew/var/buildkite-agent/builds/comms-MacBook-Air-local-1/comm/ios-build/native/cpp/CommonCpp/grpc/_generated/tunnelbroker.grpc.pb.cc' (in target 'NotificationService' from project 'Comm')

but xcodebuild just hung instead of exiting.

Harbormaster returned this revision to the author for changes because remote builds failed.Jun 22 2022, 8:26 PM
Harbormaster failed remote builds in B9907: Diff 13699!

Update diff with changes to protobuf CMake Project

Harbormaster returned this revision to the author for changes because remote builds failed.Jul 3 2022, 9:12 AM
Harbormaster failed remote builds in B10220: Diff 14107!
jon edited the test plan for this revision. (Show Details)

The CI is fixed in a later build. It was omitted to keep this diff minimal.

native/cpp/CommonCpp/Tools/CMakeLists.txt
8–17 ↗(On Diff #14107)

Parentheses issues again...

Resigning since I'm usually not a good person to set as a first-pass reviewer. Exceptions here

native/cpp/CommonCpp/Tools/CMakeLists.txt
8–17 ↗(On Diff #14107)

maybe we can just add a linter? https://github.com/cmake-lint/cmake-lint

native/cpp/CommonCpp/Tools/CMakeLists.txt
8–17 ↗(On Diff #14107)

That would be amazing!! @jonringer-comm what do you think of adding that?

@yayabosh has been working on adding ShellCheck to our CI across Buildkite / GitHub Actions / lint-staged, so he can help with any pointers. (As can @atul)

@jonringer-comm – if you're down, can you create a Linear task to track?

atul requested changes to this revision.EditedJul 10 2022, 9:09 AM

Apart from the parenthesis issue (which should be addressed before landing), it looks like Android and iOS aren't building?

Ideally every diff should pass CI, but there have been exceptions in the past when making broad changes (eg React Native upgrade). Are we expecting this diff to pass CI, or is this one of those situations where the stack will be landed "atomically" on master to avoid breaking things?


edit: JK, answered in the next diff (D4296):

Please review, CI gates are fixed in later diffs

This revision now requires changes to proceed.Jul 10 2022, 9:09 AM
native/cpp/CommonCpp/Tools/CMakeLists.txt
1 ↗(On Diff #14107)

Don't know what the best/common practices are (or if it matters) but the CMake docs seem to consistently use PascalCase project names (eg CommTools)

(https://cmake.org/cmake/help/latest/guide/tutorial/Adding%20a%20Library.html)

2 ↗(On Diff #14107)

Is there a reason we're setting the minimum version of CMake to 3.4? I see that we're using 3.4 elsewhere in the codebase, but it seems pretty old?

The highest minimum version that we have in the repo is 3.16 (services/[blob/tunnelbroker]/CMakeLists.txt), should we push up all the cmake_minimum_required in the repo to match that (where possible, I think Android CMake version is dependent on NDK)?

8–13 ↗(On Diff #14107)

Would it be possible to do this on a directory basis instead of a file-by-file basis?

I don't know what the best practices are for this sort of thing, but it doesn't seem ideal that a developer introducing a new file would A. need to pinpoint this part of the build B. make a change by hand?

28 ↗(On Diff #14107)

We can probably remove this comment?

If you think it's a good idea to leave this comment in, we should probably clean it up and/or provide some more detail.

43 ↗(On Diff #14107)

Not sure if we need this comment, but if we decide to leave it in we should probably cut the redundancy b/w "For development purposes" and "if doing development"

49 ↗(On Diff #14107)

Not sure if we need this comment

jon added inline comments.
native/cpp/CommonCpp/Tools/CMakeLists.txt
1 ↗(On Diff #14107)

CMake is case insensitive for most "cmake" synatx. However, naming conventions are up to the user/team to decide. For example, gRPC, Folly, Olm, double-conversion, AWSSDK, glog, protobuf, fbjni are all other dependencies we consume.

I will say that PascalCase seems to be the most common, but it really just seems like "what ever the first person typed in is what got established, and we no longer want to deviate from the convention".

2 ↗(On Diff #14107)

If we want to use the project files with android, we are limited to cmake<=3.10. AFAICT, there's not a significant difference (that I've encountered yet) between 3.4 and 3.12.

8–13 ↗(On Diff #14107)

We can do FILE(GLOB ./*.h), to grab everything, but there is a gotcha, as the file list is generated on the first time you invoke cmake. So if you were to add a file, it's not obvious that you need to clear the cache, then run cmake again. Where as just enumerating the items is just being a bit more explicit.

I prefer enumerating the items if it's less than ~5 or so files. But I can change

8–17 ↗(On Diff #14107)

I'm fine with adding a linter.

Parentheses issues again...

Really thought I knocked them all out, must have missed an arc diff

@jonringer-comm – if you're down, can you create a Linear task to track?

https://linear.app/comm/issue/ENG-1373/add-cmake-linter-ci-check

28 ↗(On Diff #14107)

I've just been commenting when things are not "super obvious", maybe just a knee jerk reaction with my nix work.

43 ↗(On Diff #14107)

I'm not really sure how I feel about this comment + block of code. Although it seems like it would be super useful, so that you can just re-use existing projects within a code base, I can't find any documentation on doing so....

49 ↗(On Diff #14107)

This one is fair enough. I think I thought it was necessary as export(<target>) and install(EXPORT <target>) seem really semantically similar.

jon marked 7 inline comments as done.

Fix parenthesis

native/cpp/CommonCpp/Tools/CMakeLists.txt
1 ↗(On Diff #14107)

Ah gotcha, thanks for explaining and listing those examples.

"what ever the first person typed in is what got established, and we no longer want to deviate from the convention"

Makes sense, so we'll just make sure to stick with this naming convention going forward.

2 ↗(On Diff #14107)

If we want to use the project files with android, we are limited to cmake<=3.10

Ah gotcha, IIRC one of the recent versions of react-native increases that limit to a later version of Cmake (something >= 3.11).

That should greatly simplify the gRPC installation step in our Android build: https://grpc.io/blog/cmake-improvements/

gRPC installation is currently one of the bottlenecks slowing down our Android build time, so we'd also benefit from faster CI runs and whatnot.

native/cpp/CommonCpp/Tools/CMakeLists.txt
8–17 ↗(On Diff #14107)

Really thought I knocked them all out, must have missed an arc diff

Are you using a JetBrains IDE by chance? Looks like CLion formats CMake files like this:

10bbe9.png (132×484 px, 19 KB)

It got me because I have "Format on Save" enabled.

native/cpp/CommonCpp/Tools/CMakeLists.txt
1 ↗(On Diff #14381)

Could we try something like

project(comm-tools
        LANGUAGES CXX)

Don't think it makes much difference, but allegedly a best practice:

The project() command does much more than just populate a few variables. One of its important responsibilities is to check the compilers for each enabled language and ensure they are able to compile and link successfully. Problems with the compiler and linker setup are then caught very early.

native/cpp/CommonCpp/Tools/CMakeLists.txt
25 ↗(On Diff #14381)

Should we prefix with this PUBLIC since any target dependent on comm-tools would need Folly for EG folly::Optional<std::string> get(const std::string key) const (exposed by CommSecureStore)

Public dependencies specify that not only does library A use library B internally, it also uses B in its interface. This means that A cannot be used without B, so anything that uses A will also have a direct dependency on B. An example of this would be a function defined in library A which has at least one parameter of a type defined and implemented in library B, so code cannot call the function from A without providing a parameter whose type comes from B.


Sorry for keeping these trickling in. Trying to learn CMake as I review these so making a note of things as I run into them.

(This isn't on @jonringer-comm's queue right now – might be worth hitting Request Changes so he sees it)

jon added inline comments.
native/cpp/CommonCpp/Tools/CMakeLists.txt
1 ↗(On Diff #14381)

Yea, it probably would be good to do this. I'll try to go through the diffs again :(

8–17 ↗(On Diff #14107)

I'm using vim, because... I'm a terrible human being. :)

atul requested changes to this revision.Jul 14 2022, 9:30 AM

I think it's close

native/cpp/CommonCpp/Tools/CMakeLists.txt
28 ↗(On Diff #14381)

If we're going to include this comment, we should probably capitalize the first word to keep things consistent.

43 ↗(On Diff #14381)

Can we cut the redundancy here?

52 ↗(On Diff #14381)

Should this be comm-tools?

This revision now requires changes to proceed.Jul 14 2022, 9:30 AM
jon marked 2 inline comments as done.

Don't attempt to normalize headers

atul requested changes to this revision.EditedAug 11 2022, 9:13 AM

Nice, will accept right after the CMake Lint issue is addressed (comment on line 46 is too long, should be easy to cut 2 characters)

atuls-MacBook-Pro:build atul$ make
[ 50%] Building CXX object CMakeFiles/comm-tools.dir/WorkerThread.cpp.o
[100%] Linking CXX static library libcomm-tools.a
[100%] Built target comm-tools
This revision now requires changes to proceed.Aug 11 2022, 9:13 AM
jon marked 6 inline comments as done.

Address comment

native/cpp/CommonCpp/Tools/CMakeLists.txt
25 ↗(On Diff #14381)

I believe PUBLIC is the default unless specified otherwise.

28 ↗(On Diff #14381)

I think

43 ↗(On Diff #14381)

I don't think so, it's a bit different on each line. There's a difference between things being the same, and things being similar. In this case I think it's just similar.

52 ↗(On Diff #14381)

${PROJECT_NAME}-targets.cmake is a CMake convention with install(EXPORT I can't change it. I think I can just leave this line off altogether because it's the default.

native/cpp/CommonCpp/Tools/CMakeLists.txt
43 ↗(On Diff #14381)

I mean the redundancy between "for development purposes" and "if doing development"

It could be like

"For development purposes, can point CMake to this directory"

Looks good, followed Test Plan correctly!

This revision is now accepted and ready to land.Aug 11 2022, 12:00 PM
jon added inline comments.
native/cpp/CommonCpp/Tools/CMakeLists.txt
43 ↗(On Diff #14381)

I just removed the comment altogether. It's canonical cmake.

But like most things cmake, hard to know why any particular line is there :/