Page MenuHomePhabricator

[services] Clean up `run_service.sh` using ShellCheck
ClosedPublic

Authored by abosh on Aug 2 2022, 12:47 PM.
Tags
None
Referenced Files
F2847243: D4723.id15239.diff
Mon, Sep 30, 2:30 AM
F2847242: D4723.id15237.diff
Mon, Sep 30, 2:29 AM
F2846827: D4723.diff
Mon, Sep 30, 12:28 AM
Unknown Object (File)
Tue, Sep 24, 11:24 PM
Unknown Object (File)
Tue, Sep 10, 7:01 PM
Unknown Object (File)
Tue, Sep 10, 7:01 PM
Unknown Object (File)
Tue, Sep 10, 7:01 PM
Unknown Object (File)
Tue, Sep 10, 7:01 PM

Details

Summary

Related Linear issue here. This is part of a set of diffs that will allow ShellCheck to be added to the CI. See inline comments for specific details of the ShellCheck error/warning output.

Test Plan

Diff Detail

Repository
rCOMM Comm
Branch
master
Lint
No Lint Coverage
Unit
No Test Coverage

Event Timeline

Exclude hidden files and directories using -not -path find flag instead of piping to grep

Additionally, this script is run during the Backup, Blob, and Tunnelbroker builds so if those workflows continue to pass CI we can reasonably assume that the script continues to work as expected.

I don't think that's true, we just build the image, but never run it.

'cd services && docker-compose build --no-cache blob-server'

This script is part of the CMD layer, so it's what is going to be ran if we do run it, but we don't

Taking a step back, this can eventually be replaced by just installing the service (e.g. make install), and running tunnelbroker or the related command without using this

This revision is now accepted and ready to land.Aug 2 2022, 1:24 PM
abosh marked 3 inline comments as not done.

Remove cut -d/ -f2- command because we want to keep the leading ./ on line 15

services/lib/docker/run_service.sh
7 ↗(On Diff #15239)

This code can look intimidating at first glance, but find is much better than ls so here's my explanation at making it easier to parse!

Overall, the code produces the same output as ls EXCEPT it preprends EXE_PATH to the front of each file/directory ls would output. Additionally, ls prints in human-readable format (so multiple files can be printed on the same line), but this find command will print each file/directory on a newline. This is OK, however, since the count of EXE (shown on line 8) is checked against 1 (from line 10):

if $EXES -ne 1; then

And if it is not 1, the code will exit. Thus, the find and ls commands are equivalent in the case where there is only 1 non-hidden file/directory in EXE_PATH with regards to the count of the number of files.

Note that like ls, this find command excludes hidden files and directories and is not recursive.

Here's the technical details of this command:

-mindepth 1

This is included to exclude the . directory in the current directory. Read more: https://stackoverflow.com/a/13525005/13079675

-maxdepth 1

This is included to make the find command not recursive, i.e., only search for files/directories in the current directory and no deeper.

-not -path '*/.*'

This flag filters out hidden files and directories. Read more: https://askubuntu.com/a/318211

15 ↗(On Diff #15239)

We no longer need the $EXE_PATH since find prepends the EXE_PATH to the front of each file it outputs.

7–8
8
15

This script is part of the CMD layer, so it's what is going to be ran if we do run it, but we don't

I see! That makes sense, I saw it in the Dockerfiles and assumed it was run.

Taking a step back, this can eventually be replaced by just installing the service (e.g. make install), and running tunnelbroker or the related command without using this

Is that something we want to do? I don't have much context on this script's usage.

Adding @atul and @tomek as blocking to get at least one more pair of eyes on this.

This revision now requires review to proceed.Aug 2 2022, 1:34 PM
In D4723#135736, @jon wrote:

Additionally, this script is run during the Backup, Blob, and Tunnelbroker builds so if those workflows continue to pass CI we can reasonably assume that the script continues to work as expected.

I don't think that's true, we just build the image, but never run it.

So should the test plan be updated?

atul added inline comments.
services/lib/docker/run_service.sh
7 ↗(On Diff #15239)

Appreciate the super thorough explanation here!

This revision is now accepted and ready to land.Aug 3 2022, 10:51 AM
abosh added 1 blocking reviewer(s): jon.EditedAug 3 2022, 11:13 AM

Taking a step back, this can eventually be replaced by just installing the service (e.g. make install), and running tunnelbroker or the related command without using this

@jon, is this something you'd like to do? I've made a Linear issue here. Feel free to cancel/backlog/whatever if it's not something we need to do.

So should the test plan be updated?

Updated it, but just by removing the current one. I still think we should add a more thorough test plan, so I'm adding @jon as blocking for his feedback, since this was confusing to me:

This script is part of the CMD layer, so it's what is going to be ran if we do run it, but we don't

How do we get it to run so I can test it? Sorry, not very experienced with Docker.

This revision now requires review to proceed.Aug 3 2022, 11:13 AM

How do we get it to run so I can test it? Sorry, not very experienced with Docker.

This would be a question for @karol or @max

@jon, is this something you'd like to do? I've made a Linear issue here. Feel free to cancel/backlog/whatever if it's not something we need to do.

I think we can do that conversation on linear. But the diff here should still go in

This revision is now accepted and ready to land.Aug 3 2022, 11:43 AM
In D4723#136124, @jon wrote:

@jon, is this something you'd like to do? I've made a Linear issue here. Feel free to cancel/backlog/whatever if it's not something we need to do.

I think we can do that conversation on linear. But the diff here should still go in

Ok, landing this!