diff --git a/docs/nix_dev_env.md b/docs/nix_dev_env.md --- a/docs/nix_dev_env.md +++ b/docs/nix_dev_env.md @@ -37,9 +37,50 @@ ./scripts/install_nix.sh ``` +## Workflow prerequisites + +On macOS, [installing Xcode](./nix_mobile_setup.md#xcode) is a prerequisite for all workflows. + +- [Web prerequisites](./nix_web_setup.md#nix-web-requisities) + - [React Dev Tools Chrome extension](./nix_web_setup.md#react-dev-tools-chrome-extension) + - [Redux Dev Tools Chrome extension](./nix_web_setup.md#redux-dev-tools-chrome-extension) +- [Mobile prerequisites](./nix_mobile_setup.md#nix-mobile-prerequisites) + - [Xcode](./nix_mobile_setup.md#xcode) + - [Xcode settings](./nix_mobile_setup.md#xcode-settings) + - [Android Studio](./nix_mobile_setup.md#android-studio) + - [Android SDK](./nix_mobile_setup.md#android-sdk) + - [Flipper](./nix_mobile_setup.md#flipper) + - [Android Emulator](./nix_mobile_setup.md#android-emulator) + # Development environment -Run `nix develop` to create a dev environment. +Run `nix develop` to create a dev environment. Nix will handle installation of all remaining dependencies not mentioned in "Workflow prerequisites". + +## Development workflows + +- [Web workflows](./nix_web_workflows.md#development) + - [Flow typechecker](./nix_web_workflows.md#flow-typechecker) + - [Running keysever](./nix_web_workflows.md#running-keyserver) + - [Running web app](./nix_web_workflows.md#running-web-app) + - [Running landing page](./nix_web_workflows.md#running-landing-page) + - [Debugging](./nix_web_workflows.md#debugging) + - [React Developer Tools](./nix_web_workflows.md#react-developer-tools) + - [Redux Developer Tools](./nix_web_workflows.md#redux-developer-tools) + - [Debugging JavaScript](./nix_web_workflows.md#debugging-javascript) +- [Mobile workflows](./nix_mobile_workflows.md#mobile-workflows) + - [Running mobile app on iOS Simulator](./nix_mobile_workflows.md#running-mobile-app-on-ios-simulator) + - [Running mobile app on Android Emulator](./nix_mobile_workflows.md#running-mobile-app-on-android-emulator) + - [Running mobile app on physical iOS devices](./nix_mobile_workflows.md#running-mobile-app-on-physical-ios-devices) +- [Shared workflows](./nix_shared_workflows.md#shared-workflows) + - [Codegen](./nix_shared_workflows.md#codegen) + - [Codegen for JSI](./nix_shared_workflows.md#codegen-for-jsi) + - [Codegen for gRPC](./nix_shared_workflows.md#codegen-for-grpc) + - [Working with Phabricator](./nix_shared_workflows.md#working-with-phabricator) + - [Creating a new diff](./nix_shared_workflows.md#creating-a-new-diff) + - [Updating a diff](./nix_shared_workflows.md#updating-a-diff) + - [Working with a stack](./nix_shared_workflows.md#working-with-a-stack) + - [Committing a diff](./nix_shared_workflows.md#committing-a-diff) + - [Final notes](./nix_shared_workflows.md#final-notes) ## How Nix introduces dependencies to a development environment diff --git a/docs/nix_mobile_setup.md b/docs/nix_mobile_setup.md new file mode 100644 --- /dev/null +++ b/docs/nix_mobile_setup.md @@ -0,0 +1,142 @@ +# Nix mobile prerequisites + +## Xcode + +Go to the [Mac App Store](https://apps.apple.com/us/app/xcode/id497799835) to install Xcode, or if you already have it, to update it to the latest version. + +Once Xcode is installed, open it up. If you are prompted, follow the instructions to install any “Additional Required Components”. + +Finally, you need to make sure that the Xcode “Command Line Tools” are installed. You can do this by running: + +``` +xcode-select --install +``` + +### Xcode settings + +First, in Xcode, open the Comm workspace `native/ios/Comm.xcworkspace`. Make sure that you’re signed into Xcode with an Apple Developer account (either the Comm developer team’s or your own). You can see any accounts currently associated with Xcode by navigating to Xcode → Preferences → Accounts. + +Next, you’ll want to ensure that the Comm project is configured with a valid Team, Bundle Identifier, and Provisioning Profile. To access these settings, navigate to View → Navigators → Project, and select the “Comm” project in the left sidebar. Then, select “Comm” from the “TARGETS” list, and navigate to the “Signing & Capabilities” tab. You should verify the following settings: + +#### Using a Comm development team Apple Developer account + +- Team + - Comm Technologies, Inc. +- Bundle Identifier + - app.comm +- Provisioning Profile + - Make sure that the Provisioning Profile exists + +#### Using a Personal Apple Developer account + +- Team + - Set this to a valid “Team”, which can just be your personal Apple Developer account. “Comm Technologies, Inc.” may be chosen by default, but it’s not valid if you’re using a personal Apple Developer account +- Bundle Identifier + - Pick a unique [Bundle Identifier](https://developer.apple.com/documentation/xcode/preparing-your-app-for-distribution) +- Provisioning Profile + - Make sure that the Provisioning Profile exists + +## Homebrew + +Install [Homebrew](https://brew.sh/), a package manager for macOS. + +# Nix Android prerequisites + +## Android Studio + +Start by downloading and installing [Android Studio](https://developer.android.com/studio/index.html) for your platform. When prompted to choose an installation type, select “Custom”. + +You’ll be prompted to select a JDK installation. If your SDKMAN!-sourced JDK doesn’t appear in the dropdown, you can find the absolute path to your installed JDK with the following command: + +``` +sdk home java 11.0.13-zulu +``` + +Make sure you check the boxes for the following: + +#### Intel x86-64: + +- `Android SDK` +- `Android SDK Platform` +- `Performance (Intel ® HAXM)` +- `Android Virtual Device` + +#### Apple silicon: + +- `Android SDK` +- `Android SDK Platform` +- `Android Virtual Device` + +### Android SDK + +Android Studio installs the latest Android SDK by default, but since React Native uses the Android 11 SDK specifically, we’ll need to install it using Android Studio’s SDK Manager. You can access the SDK Manager from the “Welcome to Android Studio” screen that pops up when you first open the application, from More Actions → SDK Manager. If you already have a project open, you can access it from Tools → SDK Manager. + +Once you have the SDK Manager open, select the “SDK Platforms” tab, and then check the box for “Show Package Details”. Now expand the “Android 11 (R)” section, and make sure the following subsections are checked: + +- `Android SDK Platform 30` + +#### Intel x86-64: + +- `Intel x86 Atom_64 System Image` or `Google APIs Intel x86 Atom System Image` + +#### Apple silicon: + +- `Google Play ARM 64 v8a System Image` + +Next, select the “SDK Tools” tab, and check the box for “Show Package Details”. Refer to `native/android/build.gradle` for specific tool versions and install the following: + +- Android SDK Build-Tools +- NDK +- CMake version 3.18.1 + +To finish the SDK Manager step, click “Apply” to download and install everything you’ve selected. + +## Flipper + +Flipper is a debugging tool for mobile applications from Facebook. We use it for JavaScript debugging using the Hermes runtime, and also use several plugins such as the React Dev Tools. You can download the latest version of Flipper for MacOS [here](https://www.facebook.com/fbflipper/public/mac). + +### Flipper settings + +After opening Flipper, click the gear icon in the bottom left and navigate to “Settings”. + +Let’s set the Android SDK path. To find the path, open Android Studio and navigate to Preferences → Appearance & Behavior → System Settings → Android SDK. The explicit path of your Android SDK is defined in “Android SDK Location”. Use this path in Flipper Settings as the “Android SDK Location”. + +Also, enable the option for “React Native keyboard shortcuts” below. + +### Flipper plugins + +Flipper has a plugin system that allows teams to integrate additional debugging tools into Flipper. We currently only use one plugin, which is for monitoring Redux state. + +To install it, open Flipper and click on the Plugin Manager on the top left sidebar. Type in “redux-debugger“ in the Install Plugins search bar and install the Flipper plugin with that name. + +## idb + +Flipper relies on Facebook’s idb tool to debug iOS apps running on your device. We’ll need to install it: + +``` +brew tap facebook/fb +brew install idb-companion +pip3 install --user --upgrade fb-idb +``` + +Since we run `pip3 install` with `--user` instead of running it with `sudo`, the `idb` executable gets installed in your userdir. For me, running MacOS with Python 3.9, it got installed in `~/Library/Python/3.9/bin/idb`. For Flipper to be able to talk to `idb`, you’ll need to set the IDB Binary Location in the Flipper Settings. + +If you have trouble getting Flipper to work with a physical iOS device, it may be due to Python weirdness. The above steps have been tested with Python 3.9 sourced from Homebrew. Let @ashoat know if you have any trouble! + +## Reactotron + +Reactotron is an event tracker and logger that can be used to aid in debugging on React Native. + +``` +brew install reactotron && brew upgrade reactotron +``` + +## Android Emulator + +In order to test the Android app on your computer you’ll need to set up an Android Emulator. To do this we’ll need to open up the Virtual Device Manager in Android Studio. You can access the Virtual Device Manager from the “Welcome to Android Studio” screen that pops up when you first open the application, from More Actions → Virtual Device Manager. If you already have a project open, you can access it from Tools → Virtual Device Manager. + +With the Virtual Device Manager open, select “Create device” on the top left. Feel free to select any “device definition” that includes Play Store support. + +On the next screen you’ll be asked to select a system image. Go for the latest version of Android that’s been released. + +From there you can just hit Next and then Finish. You should then be able to start your new Android Virtual Device from the Virtual Device Manager. diff --git a/docs/nix_mobile_workflows.md b/docs/nix_mobile_workflows.md new file mode 100644 --- /dev/null +++ b/docs/nix_mobile_workflows.md @@ -0,0 +1,97 @@ +# Mobile workflows + +## Running mobile app on iOS Simulator + +First, make sure that the keyserver is running. If you haven’t already, run: + +``` +cd keyserver +yarn dev +``` + +Next, make sure that the Metro bundler is running. If you haven’t already, open a new terminal and run: + +``` +cd native +yarn dev +``` + +This command runs two processes. The first is the Metro bundler, which handles bundling our app’s JavaScript code and communicating with the debug build of the app running on either a physical or virtual device. The second is the `remotedev-server` for Redux, which is a proxy of sorts through which the Redux monitor (running in the Chrome extension) can communicate with the debug build of the app. + +Finally, open `native/ios/Comm.xcworkspace` in Xcode. Select a Simulator from the Scheme menu in the Workspace Toolbar. Then hit the Run button to build and run the project. + +## Running mobile app on Android Emulator + +First, make sure that the keyserver is running. If you haven’t already, run: + +``` +cd keyserver +yarn dev +``` + +Next, make sure that the Metro bundler is running. If you haven’t already, open a new terminal and run: + +``` +cd native +yarn dev +``` + +This command runs two processes (see previous section for details). + +Next, boot up an Android Emulator using Android Studio’s Virtual Device Manager. You should have a single Android Emulator (or plugged-in device) running at one time. + +Finally, use this command to build and run the Android app: + +``` +cd native +yarn react-native run-android +``` + +## Running mobile app on physical iOS devices + +There are a few things you’ll need to do before you can deploy the app to a physical iOS device. + +### Xcode settings + +First, in Xcode, open the Comm workspace `native/ios/Comm.xcworkspace`. Make sure that you’re signed into Xcode with an Apple Developer account (either the Comm developer team’s or your own). You can see any accounts currently associated with Xcode by navigating to Xcode → Preferences → Accounts. + +Next, you’ll want to ensure that the Comm project is configured with a valid Team, Bundle Identifier, and Provisioning Profile. To access these settings, navigate to View → Navigators → Project, and select the “Comm” project in the left sidebar. Then, select “Comm” from the “TARGETS” list, and navigate to the “Signing & Capabilities” tab. You should verify the following settings: + +### Connecting to local keyserver + +If you want your custom build of the app to connect to your local instance of the Node.js server (the `keyserver` subdirectory of the repo), you’ll need to do some additional work. First, confirm that your computer and physical iOS device are on the same network. If you’re running a local keyserver instance, you’ll need to be able to reach it with your device. Local keyservers run on the local IP address at port 8043. + +To find your machine’s local IP address, navigate to System Preferences → Network, and select the hardware interface you’re currently using to connect to the internet (Wi-Fi, or potentially a Thunderbolt port for ethernet connections). Next, click “Advanced” and go to the “TCP/IP” tab. Your local IP address is listed as the “IPv4 Address”. Try visiting this IP address using a browser on your device. It should display an “It works!” message if your iOS device can reach your machine. + +Finally, we need to direct the mobile app to use your local keyserver instance. There are a few different ways to do this, depending on your situation: + +- As long as you’re deploying a debug build, this strategy should work for you. You can create a `network.json` file in `native/facts` that will override the default. + + ``` + mkdir -p native/facts + vim native/facts/network.json + ``` + + Your `network.json` file should look like the following, where `w.x.y.z` is the local IP address you found earlier: + + ```json + { + "natDevHostname": "w.x.y.z" + } + ``` + + You’ll need to delete and reinstall the app for changes to `native/facts/network.json` to take effect, as the default production URL is persisted in Redux. + +- If you’re deploying a release build, the above strategy won’t work. Your best bet to override the server URL is to get to the secret “Developer tools” menu option in the app. + + 1. You may need to use a real production account for this, since the server address will default to the production server if this is the first build you’ve deployed to the target device. + 2. Next, in order for the “Developer tools” menu option to appear, you’ll need to add your user ID to [the list of user IDs in `staff.json`](https://github.com/CommE2E/comm/blob/master/lib/facts/staff.json). A good way to figure out your user ID is to use the Chrome Redux debugger to inspect the `currentUserInfo` property when logged into the web app. + 3. Finally, you should be able to navigate to Profile → Developer tools in the app and set the address of the local server. It should look something like this: + + ``` + http://w.x.y.z/comm + ``` + + Where `w.x.y.z` is the local IP address you found earlier. + +- Alternately, if you’re on a release build and option 2 above seems like too much of a hassle, you should be able to simply change the value of [`productionNodeServerURL` in the code](https://github.com/CommE2E/comm/blob/9e6a13f1569787b498a72c890b12ce0dd8323804/native/utils/url-utils.js#L12). Note that you’ll need to delete and reinstall the app for this change to take effect, as the default production URL is persisted in Redux. diff --git a/docs/nix_shared_workflows.md b/docs/nix_shared_workflows.md new file mode 100644 --- /dev/null +++ b/docs/nix_shared_workflows.md @@ -0,0 +1,67 @@ +# Shared workflows + +## Codegen + +We use a couple of tools that automatically generate code. There is always a source of truth – usually some file(s) with schemas. + +### Codegen for JSI + +JSI is a framework in React Native that allows C++ and JS to communicate synchronously and directly. The codegen for JSI takes a Flow schema and generates C++ files that enable communication between JS and C++ in `react-native` apps. + +The script to generate this code is written in JavaScript and is included as a npm package so no additional software is needed to use it. The schema has to be defined in Flow as an interface, and that interface must inherit from react-native’s `TurboModule` interface. + +To run the JSI codegen, just run: + +``` +cd native +yarn codegen-jsi +``` + +The input Flow schemas are located in `native/schema`. + +### Codegen for gRPC + +In order to generate the codegen files for gRPC, you should run the following command: + +``` +cd native +yarn codegen-grpc +``` + +# Working with Phabricator + +## Creating a new diff + +The biggest difference between GitHub PRs and Phabricator diffs is that a PR corresponds to a branch, whereas a diff corresponds to a commit. + +When you have a commit ready and want to submit it for code review, just run `arc diff` from within the Comm Git repo. `arc diff` will look at the most recent commit in `git log` and create a new diff for it. + +## Updating a diff + +With GitHub PRs, updates are usually performed by adding on more commits. In contrast, in Phabricator a diff is updated by simply amending the existing commit and running `arc diff` again. + +When you run `arc diff`, it looks for a `Differential Revision: ` line in the commit text of the most recent commit. If Arcanist finds that line, it will assume you want to update the existing diff rather than create a new one. Other Arcanist commands such as `arc amend` (which amends commit text to match a diff on Phabricator) also look for the `Differential Revision: ` line. + +## Working with a stack + +One of the advantages of Phabricator’s approach is that larger, multi-part changes can be split up into smaller pieces for review. These multi-part changes are usually referred to as a “stack” of diffs. + +When creating a diff that depends on another, you should make sure to create a dependency relationship between those two diffs, so that your reviewers can see the stack on Phabricator. The easiest way to do that is to include `Depends on D123` in the commit text of the child commit, but the dependency relationship can also be specified using the Phabricator web UI. + +You’ll find that mastering Git’s interactive rebase feature (`git rebase -i`) will help you a lot when working with stacks. Interactive rebases make it easy to “diff up” multiple commits at once, or to amend a specific commit in the middle of a stack in response to a review. + +## Committing a diff + +After your diff has been accepted, you should be able to land it. To land a diff just run `arc land` from within the repository. + +If you have a stack of unlanded commits in your Git branch, `arc land` will attempt to land all of those diffs. If some of the diffs in your stack haven’t been accepted yet, you’ll need to create a new, separate branch that contains just the commits you want to land before running `arc land`. + +Note that you need commit rights to the repository in order to run `arc land`. If you don’t have commit rights, reach out to @ashoat for assistance. + +## Final notes + +When developing, I usually just pop up three terminal windows, one for `yarn dev` in each of keyserver, web, and native. + +Note that it’s currently only possible to create a user account using the iOS or Android apps. The website supports logging in, but does not support account creation. + +Good luck, and let @ashoat know if you have any questions! diff --git a/docs/nix_web_setup.md b/docs/nix_web_setup.md new file mode 100644 --- /dev/null +++ b/docs/nix_web_setup.md @@ -0,0 +1,9 @@ +# Nix web prerequisites + +## React Dev Tools Chrome extension + +The React Dev Tools Chrome extension lets you inspect the React component tree for web applications in Chrome. You can install it by navigating [here](https://chrome.google.com/webstore/detail/react-developer-tools/fmkadmapgofadopljbjfkapdkoienihi) on Chrome. + +## Redux Dev Tools Chrome extension + +The Redux Dev Tools Chrome extension lets you watch for Redux actions and inspect the Redux store state, both for web applications in Chrome, but also for our native applications using the “Remote DevTools” functionality. To install it, navigate [here](https://chrome.google.com/webstore/detail/redux-devtools/lmhkpmbekcpmknklioeibfkpmmfibljd) on Chrome. diff --git a/docs/nix_web_workflows.md b/docs/nix_web_workflows.md new file mode 100644 --- /dev/null +++ b/docs/nix_web_workflows.md @@ -0,0 +1,84 @@ +# Development + +## Flow typechecker + +It’s good to run the `flow` typechecker frequently to make sure you’re not introducing any type errors. Flow treats each Yarn Workspace as a separate environment, and as such runs a separate type-checking server for each. + +You should now be able to run `flow` in any of the Yarn workspaces: + +``` +cd lib +flow +``` + +## Running keyserver + +To run the web app, landing page, or the mobile app on the iOS Simulator or Android Emulator, you’ll need to run the keyserver. + +Open a new terminal and run: + +``` +cd keyserver +yarn dev +``` + +This command runs three processes. The first two are to keep the `dist` folder updated whenever the `src` folder changes. They are “watch” versions of the same Babel and `rsync` commands we used to initially create the `dist` folder (before running the `generate-olm-config.js` script above). The final process is `nodemon`, which is similar to `node` except that it restarts whenever any of its source files (in the `dist` directory) changes. + +Note that if you run `yarn dev` in `keyserver` right after `yarn cleaninstall`, before Webpack is given a chance to build `app.build.cjs`/`landing.build.cjs` files, then Node will crash when it attempts to import those files. Just make sure to run `yarn dev` (or `yarn prod`) in `web` or `landing` before attempting to load the corresponding webpages. + +## Running web app + +First, make sure that the keyserver is running. If you haven’t already, run: + +``` +cd keyserver +yarn dev +``` + +Next, open a new terminal and run: + +``` +cd web +yarn dev +``` + +You should now be able to load the web app in your web browser at http://localhost/comm/. + +This command will start two processes. One is `webpack-dev-server`, which will serve the JS files. `webpack-dev-server` also makes sure the website automatically hot-reloads whenever any of the source files change. The other process is `webpack --watch`, which will build the `app.build.cjs` file, as well as rebuilding it whenever any of the source files change. The `app.build.cjs` file is consumed by the Node server in order to pre-render the initial HTML from the web source (“Server-Side Rendering”). + +## Running landing page + +First, make sure that the keyserver is running. If you haven’t already, run: + +``` +cd keyserver +yarn dev +``` + +Next, open a new terminal and run: + +``` +cd landing +yarn dev +``` + +You should now be able to load the landing page in your web browser at http://localhost/commlanding/. + +This runs the same two processes as the web app, but for the landing page. Note that the `landing.build.cjs` file (similar to the web app’s `app.build.cjs` file) is consumed by the Node server. + +# Debugging + +## React Developer Tools + +- For web, you can access the React Developer Tools through the Chrome extension by opening the Chrome Developer Tools and selecting the “Components” or “Profiler” tabs. This should work in both our development environment and in production. +- For iOS and Android, you can access the React Developer Tools through Flipper. First start a debug build of a React Native app. Next, just open up Flipper and you should be able to see an option for “React DevTools”. Flipper communicates with the app through the Metro bundler that gets started when you run `cd native && yarn dev`. + +## Redux Developer Tools + +- For web, you can access the Redux Developer Tools through the Chrome extension by opening the Chrome Developer Tools and selecting the “Redux” tab. This should work in both our development environment and in production, although in production you won’t be able to see Redux actions from before you opened up the Redux dev tools. +- For iOS and Android, you can access the Redux Developer Tools through the Chrome extension’s Remote DevTools functionality. First, to open the Remote DevTools, right click on any webpage, go into the “Redux DevTools” menu, and select “Open Remote DevTools”. Then hit “Settings”, select “Use custom (local) server”, and configure it to connect to `localhost` on port 8043. This will connect to the `remotedev-server` instance that you started when you ran `cd native && yarn dev`. + +## Debugging JavaScript + +- For web, you can just use your browser of choice’s dev tools. +- For iOS and Android, you should use Flipper. First start a debug build of a React Native app. Next, just open up Flipper and you should be able to see an option for “Hermes Debugger (RN)”. Flipper communicates with the app through the Metro bundler that gets started when you run `cd native && yarn dev`.