All of the job and core nonsense will be significantly mitigated by D4816 where it will be unlikely for anyone to build anything locally.

But to respond:

What are the consequences of this? I'm not too familiar with threaded or parallel computing, but to me this sentence sounds sort of worrying

The consequences is that there's a potential that every vCPU on the computer could be doing 2 things, which means 100% CPU load for extended periods of time. When a computer is maxed out like this, it means that the computer's usability could be diminished (think of cleaning and running an xcode project).

When a computer tries to do a 1000 different things, then you have to worry about things like contexting switching overhead between threads causing a lot of lost productivity, but limiting the cores to 4 puts a low upper bound to this.

From what I got, if we over-subscribe cores, if most builds are single threaded it won't matter since only one core will do the work anyways?

Correct, if you have (n/2) single-threaded builds going concurrently, then you will only be at 50% cpu load. If you have 8 cores, and one of those jobs starts using 4 cores for the build, then you will jump up to 7/8 cores being used, which is ~87%. Which means the computer should still be pretty usable while the build is going.

But what happens when something is massively parallel?

Then they will go up to a max of 4 cores to attempt that job/build.

I'm just failing to see how this is a good compromise

On one end, you can do many single core jobs, or a single many core job (which is likely bound by the available cpu count, as there's no benefit to providing more than what the cpu is capable of).

The compromise here is that most builds are either single threaded, or have long single-threaded critical sections (e.g. cmake .. or linking).

For example, in our current dev shell, aws-sdk-cpp, protobuf, and nodejs use more than a single core for their build and take longer than a few seconds to build. The aws-sdk-cpp is only building 3 apis, so it's finished in a minute or so. Protobuf may take a few minutes. However, nodejs has a very large build, and will like sit on 4 cores for 10-20 mins even after all other builds have completed. In this example, a 6 or 8 core machine will have a potential few minute window where they are maxed out, but for most of it, they are just under 100%, which allows for people to do other things and still use the machine.

I've also learned from experience that some C++ builds can't have incredibly high ram usage if they have a lot going on (e.g. 1-4 GBs per thread). So having a set core limit of 4 is also to prevent paging in ram (using swap) which makes the build significantly slower.

It's a header to the settings set provide below. It is a bit odd to read as it's providing context to something which hasn't happened yet. I could set the values, then describe them in a comment below it. But we've been doing comments before code everywhere else and felt that comment after code would be even more jarring.