[Chore] Only checkout single-branch for grpc

Summary: Minimize how much we checkout for grpc

Test Plan:

cd services/base-image
docker build .

Reviewers: atul

Reviewed By: atul

Subscribers: ashoat, tomek, adrian, abosh

Differential Revision: https://phab.comm.dev/D4686