Page MenuHomePhabricator

[Services] Update blob to reference new directory
ClosedPublic

Authored by jon on Aug 31 2022, 11:11 AM.
Tags
None
Referenced Files
F3536318: D4998.id16510.diff
Wed, Dec 25, 6:19 PM
F3531735: D4998.diff
Wed, Dec 25, 7:57 AM
Unknown Object (File)
Sat, Dec 21, 11:55 AM
Unknown Object (File)
Sat, Dec 14, 11:48 PM
Unknown Object (File)
Wed, Dec 4, 7:12 PM
Unknown Object (File)
Nov 21 2024, 5:19 PM
Unknown Object (File)
Nov 14 2024, 6:50 PM
Unknown Object (File)
Nov 14 2024, 6:46 PM
Subscribers

Details

Summary

Update blob references to protobuf directory

https://linear.app/comm/issue/ENG-1231

Depends on D4996

Test Plan
nix develop
cd services/blob
rm -rf build && mkdir build && cd build && cmake .. && make -j

Diff Detail

Repository
rCOMM Comm
Branch
jonringer/relocate-protos (branched from master)
Lint
No Lint Coverage
Unit
No Test Coverage

Event Timeline

This revision is now accepted and ready to land.Aug 31 2022, 5:01 PM
karol added a reviewer: tomek.

what of the dockerfile?

This revision now requires changes to proceed.Sep 1 2022, 1:46 AM

what of the dockerfile?

Will be in a later diff

Then we should either have a linear task for it or a diff already up.

This revision now requires changes to proceed.Sep 2 2022, 12:07 AM

Rebase with corrosion fixes

karol requested changes to this revision.EditedSep 5 2022, 5:41 AM

Could you please address my comment? Thanks!

This revision now requires changes to proceed.Sep 5 2022, 5:41 AM

Could you please address my comment? Thanks!

https://phab.comm.dev/D5044 Is in the stack, which handles docker

what of the dockerfile?

This diff doesn't have any effect on the docker workflow.

Getting the following:

atuls-MacBook-Pro:blob atul$ rm -rf build && mkdir build && cd build && cmake .. && make -j
-- The C compiler identification is Clang 11.1.0
-- The CXX compiler identification is Clang 11.1.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /nix/store/84bdn74mjlh7cx3yypi71zapb75lfbmx-clang-wrapper-11.1.0/bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /nix/store/84bdn74mjlh7cx3yypi71zapb75lfbmx-clang-wrapper-11.1.0/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Warning (dev) at /nix/store/cg3f9hs6zdag48yi8zdpg3kwq9ffcpbj-cmake-3.23.2/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:438 (message):
  The package name passed to `find_package_handle_standard_args` (Protobuf)
  does not match the name of the calling package (protobuf).  This can lead
  to problems in calling code that expects `find_package` result variables
  (e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
  /nix/store/cg3f9hs6zdag48yi8zdpg3kwq9ffcpbj-cmake-3.23.2/share/cmake-3.23/Modules/Findprotobuf.cmake:650 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
  CMakeLists.txt:19 (find_package)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Found Protobuf: /nix/store/r5nv2l4hiraggzawpjhxhbqgnc93p0a9-protobuf-3.15.8/lib/libprotobuf.dylib (found version "3.15.8") 
-- Found ZLIB: /nix/store/lkgrb3lfingdcxr9fzc847c5wihk1w73-zlib-1.2.12/lib/libz.dylib (found version "1.2.12") 
-- Found OpenSSL: /nix/store/1yhkn4rmhxpxdpvf10m10x5z60sxwhkr-openssl-1.1.1q/lib/libcrypto.dylib (found version "1.1.1q")  
-- Could NOT find c-ares (missing: c-ares_DIR)
-- Found c-ares: /nix/store/0i2sxmly7haf1fqgbkjx4104rh0s0ps9-c-ares-1.18.1/include (found version "1.18.1") 
-- Found Threads: TRUE  
-- Found RE2 via CMake.
-- Found Boost: /nix/store/7rrgbk62knkpf4jngbrmzpjc002j2q65-boost-1.79.0-dev/lib/cmake/Boost-1.79.0/BoostConfig.cmake (found suitable version "1.79.0", minimum required is "1.51.0") found components: context filesystem program_options regex system thread 
-- Found folly: /nix/store/27vgfzvc113qcc165fviqbf5crfvj8rz-folly-2022.07.04.00
-- Found AWS SDK for C++, Version: 1.9.238, Install Root:, Platform Prefix:, Platform Dependent Libraries: pthread;curl
-- Components specified for AWSSDK: s3;core;dynamodb, application will be depending on libs: aws-cpp-sdk-s3;aws-cpp-sdk-dynamodb;aws-cpp-sdk-core
-- Try finding aws-cpp-sdk-core
-- Found aws-cpp-sdk-core
-- Try finding aws-cpp-sdk-dynamodb
-- Found aws-cpp-sdk-dynamodb
-- Try finding aws-cpp-sdk-s3
-- Found aws-cpp-sdk-s3
-- Could NOT find Boost: missing: REQUIRE (found /nix/store/7rrgbk62knkpf4jngbrmzpjc002j2q65-boost-1.79.0-dev/lib/cmake/Boost-1.79.0/BoostConfig.cmake (found suitable version "1.79.0", minimum required is "1.40"))
CMake Warning (dev) at /nix/store/cg3f9hs6zdag48yi8zdpg3kwq9ffcpbj-cmake-3.23.2/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:438 (message):
  The package name passed to `find_package_handle_standard_args` (Protobuf)
  does not match the name of the calling package (protobuf).  This can lead
  to problems in calling code that expects `find_package` result variables
  (e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
  /nix/store/cg3f9hs6zdag48yi8zdpg3kwq9ffcpbj-cmake-3.23.2/share/cmake-3.23/Modules/Findprotobuf.cmake:650 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
  /Users/atul/comm/shared/protos/CMakeLists.txt:5 (find_package)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Found Boost: /nix/store/7rrgbk62knkpf4jngbrmzpjc002j2q65-boost-1.79.0-dev/lib/cmake/Boost-1.79.0/BoostConfig.cmake (found suitable version "1.79.0", minimum required is "1.40") found components: program_options 
CMake Warning (dev) at /nix/store/cg3f9hs6zdag48yi8zdpg3kwq9ffcpbj-cmake-3.23.2/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:438 (message):
  The package name passed to `find_package_handle_standard_args` (Protobuf)
  does not match the name of the calling package (protobuf).  This can lead
  to problems in calling code that expects `find_package` result variables
  (e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
  /nix/store/cg3f9hs6zdag48yi8zdpg3kwq9ffcpbj-cmake-3.23.2/share/cmake-3.23/Modules/Findprotobuf.cmake:650 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
  /Users/atul/comm/services/lib/src/CMakeLists.txt:26 (find_package)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Configuring done
-- Generating done
-- Build files have been written to: /Users/atul/comm/services/blob/build
[  5%] Generate protobuf files for blob
[ 16%] Building CXX object protos/CMakeFiles/comm-blob-grpc.dir/blob.pb.cc.o
[ 16%] Building CXX object protos/CMakeFiles/comm-blob-grpc.dir/blob.grpc.pb.cc.o
In file included from /Users/atul/comm/services/blob/build/protos/blob.grpc.pb.cc:5:
/Users/atul/comm/services/blob/build/protos/blob.pb.h:17:2: error: This file was generated by an older version of protoc which is
In file included from /Users/atul/comm/services/blob/build/protos/blob.pb.cc:4:
#error This file was generated by an older version of protoc which is/Users/atul/comm/services/blob/build/protos/blob.pb.h
:17 ^:
2: error: /Users/atul/comm/services/blob/build/protos/blob.pb.h:18This file was generated by an older version of protoc which is:2
: error: incompatible with your Protocol Buffer headers. Please
#error This file was generated by an older version of protoc which is
#error incompatible with your Protocol Buffer headers. Please ^

 ^
/Users/atul/comm/services/blob/build/protos/blob.pb.h:/Users/atul/comm/services/blob/build/protos/blob.pb.h18::192::2 : error: error: incompatible with your Protocol Buffer headers. Pleaseregenerate this file with a newer version of protoc.

#error incompatible with your Protocol Buffer headers. Please#error regenerate this file with a newer version of protoc.

 ^ ^

/Users/atul/comm/services/blob/build/protos/blob.pb.h:19:2: error: regenerate this file with a newer version of protoc.
#error regenerate this file with a newer version of protoc.
 ^
/Users/atul/comm/services/blob/build/protos/blob.pb.cc:853:11: error: no member named 'Swap' in 'google::protobuf::internal::ArenaStringPtr'
  holder_.Swap(&other->holder_, &::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), GetArena());
  ~~~~~~~ ^
/Users/atul/comm/services/blob/build/protos/blob.pb.cc:1048:14: error: no member named 'Swap' in 'google::protobuf::internal::ArenaStringPtr'
  datachunk_.Swap(&other->datachunk_, &::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), GetArena());
  ~~~~~~~~~~ ^
/Users/atul/comm/services/blob/build/protos/blob.pb.cc:1248:11: error: no member named 'Swap' in 'google::protobuf::internal::ArenaStringPtr'
  holder_.Swap(&other->holder_, &::PROTOBUF_NAMESPACE_ID::internal::GetEmptyStringAlreadyInited(), GetArena());
  ~~~~~~~ ^
6 errors generated.
make[2]: *** [protos/CMakeFiles/comm-blob-grpc.dir/blob.pb.cc.o] Error 1
make[2]: *** Waiting for unfinished jobs....
3 errors generated.
make[2]: *** [protos/CMakeFiles/comm-blob-grpc.dir/blob.grpc.pb.cc.o] Error 1
make[1]: *** [protos/CMakeFiles/comm-blob-grpc.dir/all] Error 2
make: *** [all] Error 2
atul requested changes to this revision.Sep 7 2022, 8:31 AM
This revision now requires changes to proceed.Sep 7 2022, 8:31 AM

In file included from /Users/atul/comm/services/blob/build/protos/blob.pb.cc:4:
#error This file was generated by an older version of protoc which is/Users/atul/comm/services/blob/build/protos/blob.pb.h

This looks related to the version of protoc on your PATH, this should be a non-issue after https://phab.comm.dev/D5077

Hm, doesn't

-- Found Protobuf: /nix/store/r5nv2l4hiraggzawpjhxhbqgnc93p0a9-protobuf-3.15.8/lib/libprotobuf.dylib (found version "3.15.8")

indicate that it's pulling from nix store?

The non-nix version I have in my path is also 3.15.8:

39e27d.png (168×958 px, 93 KB)

indicate that it's pulling from nix store?

It indicates that find_library() found ProtobufConfig.cmake in the nix store. But somehow something from the environment is bleeding into the build.

Either way, please retry the build, I rebased on top of D5077

Rebased locally and still ran into the same issue.

Spun up a Debian and macOS VM to try things from a "clean slate." I can confirm that things build correctly on Debian: https://gist.githubusercontent.com/atulsmadhugiri/863e5b3421849f83b56d35ffa25fe4c5/raw/5222191d00b4a4eecf30074bdbfd1575e70d6df3/debian_%2520D4998

Still waiting for the macOS VM to build node 16.13 from scratch, will accept/request changes on this diff depending on how that goes.

Ah, looks like something is weird with my local dev environment. I'll spend some time trying to fix things up.

I was, however, able to get things to build properly on a macOS VM:

Screen Shot 2022-09-07 at 7.54.46 PM.png (1×2 px, 1 MB)

Apologies for requesting changes when it was an issue on my end.. will review the rest of your diffs in that VM for now.

This revision is now accepted and ready to land.Sep 8 2022, 4:31 AM