Page MenuHomePhabricator

[services] Rust Integration - c++ - Add worker
ClosedPublic

Authored by karol on Sep 6 2022, 7:21 AM.
Tags
None
Referenced Files
F1746511: D5068.id16798.diff
Mon, May 13, 2:56 PM
F1746509: D5068.id16790.diff
Mon, May 13, 2:56 PM
F1746508: D5068.id16586.diff
Mon, May 13, 2:56 PM
F1746507: D5068.id16584.diff
Mon, May 13, 2:56 PM
F1746505: D5068.id16371.diff
Mon, May 13, 2:56 PM
Unknown Object (File)
Thu, May 9, 5:21 PM
Unknown Object (File)
Thu, May 9, 5:21 PM
Unknown Object (File)
Thu, May 9, 5:21 PM

Details

Summary

Depends on D5112

In response to this issue, we should implement the code that will delegate the work away from the reactors to separate threads. For this, I created a tool named Worker.

Test Plan

Services build as usual.

Diff Detail

Repository
rCOMM Comm
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

tomek requested changes to this revision.Sep 7 2022, 6:24 AM
tomek added inline comments.
services/lib/src/Worker.h
22 ↗(On Diff #16371)

This might throw and we should never throw in destructors

This revision now requires changes to proceed.Sep 7 2022, 6:24 AM

The Worker class is not that well-thought-through and I realize that. The first problem is that there's no limit to schedules, another one is that we don't have that much control over the threads - we join them but in the destructor, and this is basically a singleton with a static instance; that means it's going to be released when the app's about to terminate. We could maybe detach the threads... But all in all, I think that we should think about what is happening here and pursue more control over this "worker" aka "thread manager"/"thread pool".

I know I do this too often but let's do a follow-up here (I guess you know what the circumstances are and why I keep creating the follow-up): https://linear.app/comm/issue/ENG-1760/implement-a-correct-thread-pool-in-c-for-reactors

tomek requested changes to this revision.Sep 9 2022, 5:21 AM
In D5068#148377, @karol wrote:

The Worker class is not that well-thought-through and I realize that. The first problem is that there's no limit to schedules, another one is that we don't have that much control over the threads - we join them but in the destructor, and this is basically a singleton with a static instance; that means it's going to be released when the app's about to terminate. We could maybe detach the threads... But all in all, I think that we should think about what is happening here and pursue more control over this "worker" aka "thread manager"/"thread pool".

I know I do this too often but let's do a follow-up here (I guess you know what the circumstances are and why I keep creating the follow-up): https://linear.app/comm/issue/ENG-1760/implement-a-correct-thread-pool-in-c-for-reactors

I'm not sure if this is the right approach. I would rather not have something that isn't well-though in the codebase - especially something which is in the core and touches threading stuff. The reason is that it's a lot cheaper to build something than to modify it, and if we decide to commit to this solution, which we know has a lot of issues, we're going to regret it. Additionally, if we know that this isn't good enough, we're not going to put it to production without the improvements, so there isn't too much point in pushing this.

First of all, have you checked what are community's best practices in handling thread pooling in C++? Are there any libraries which are considered good? Can we use them directly or as an inspiration for our solution?

This revision now requires changes to proceed.Sep 9 2022, 5:21 AM

I understand, you're right. The goal was to provide the POC asap so the performance tests would work. I'm going to look into the c++ thread pooling.

address feedback - use boost's thread pool

tomek requested changes to this revision.Sep 13 2022, 11:13 AM

I'm glad that you have decided to use a well recognized library and its thread pool implementation!

services/lib/src/Worker.h
17 ↗(On Diff #16586)

As I mentioned in one of the other diffs, this is not a Worker. The workers are used by thread pool to execute scheduled jobs. This class is a ThreadPool.

18 ↗(On Diff #16586)

Why do we need to use a pointer?

26 ↗(On Diff #16586)

Have you checked if there's no way for this call to throw?

Have you checked if calling this is needed?

35–45 ↗(On Diff #16586)

I'm still not convinced that complicating the API by introducing a callback that is executed just after the task, on the same thread, without any scheduling is a good idea. We should keep things simple and we can do that by having a simple API with schedule(Task). We can then can have a function that creates a task by wrapping one with try-catch and adding a callback.

The main problem with the current API is that thread pool is something which should be really generic and we're creating a contract that is convenient only for particular task type.

If you really think this method is essential and beneficial, we can keep it, but we should also expose schedule(Task) and use it internally in this method.

This revision now requires changes to proceed.Sep 13 2022, 11:13 AM
services/lib/src/Worker.h
17 ↗(On Diff #16586)

Yes, you're right, sorry if I missed that.

18 ↗(On Diff #16586)

We don't actually. I initially couldn't get this working without it for some reason but now I see it works just alright.

26 ↗(On Diff #16586)

Honestly, it's hard to say whether it throws or not, there's no information in the docs, I tried to dig into the implementation but there are a lot of system calls, etc. there and I'm not sure if it's worth it to reverse engineer this to the bottom of it, especially that...

I think we can just delete this line. This is a singleton, which means it has just one single, static instance. This means that this instance is going to be released when the whole program goes down. And the server basically loops forever and the only case that it terminates is when something goes really bad (that shouldn't occur). I don't see a scenario where this gets released gracefully, therefore join doesn't seem to be really necessary here.

35–45 ↗(On Diff #16586)

I was thinking about this for a bit and I don't think this is a good idea, sorry.

The problem with calling one method from the other is that passing arguments would be problematic as we should receive an unknown number of unknown arguments that should be passed to the callback.

I just don't think it makes sense to write functionalities that are not going to be used - and that would be a function schedule(Task) - we're not going to use it anywhere. And if anyone ever needs something like this, it can easily be introduced like this:

void schedule(Task task) {
  boost::asio::post(this->pool, [task]() {
    try {
      task();
    } catch (std::exception &e) {
      LOG(ERROR) << "error occured in a scheduled task: " << e.what();
    }
  });
}

I'm ok though with renaming the current function to something like scheduleWithCallback.

tomek requested changes to this revision.Sep 15 2022, 7:42 AM
tomek added inline comments.
services/lib/src/Worker.h
24–25 ↗(On Diff #16654)

Is there a reason behind introducing an empty private constructor? Are there any downsides of just deleting it?

33–34 ↗(On Diff #16654)

We're copying task and callback twice. Can we avoid this?

17 ↗(On Diff #16586)

Now I'm confused. Your comment indicates that you agreed that the class should be named ThreadPool, but in the current revision it is still a Worker.

26 ↗(On Diff #16586)

According to the docs https://www.boost.org/doc/libs/1_66_0/doc/html/boost_asio/reference/thread_pool/_thread_pool.html the destructor

Automatically stops and joins the pool, if not explicitly done beforehand.

Even when a class is used as a singleton, we still should destruct it gracefully so that all the work can be finished, regardless the reason.

We're definitely overusing singleton pattern. In this case the pool should be just an ordinary class, which can have multiple instances. What's wrong with having two or more pools? Then we could have a singleton that holds an instance of this class. Paying up this debt isn't urgent, but overall we should design classes so that they work in any context, not just as singletons.

35–45 ↗(On Diff #16586)

Ok, let's keep it.

This revision now requires changes to proceed.Sep 15 2022, 7:42 AM
services/lib/src/Worker.h
24–25 ↗(On Diff #16654)

It's a part of the singleton pattern. By doing it, we forbid creating additional instances and have just one.

33–34 ↗(On Diff #16654)

We might, but I think that's okay. Alternatives we have:

  • pass them as references - this is most dangerous and would probably crash right away. If the originals get out of scope (deallocated), we'd end up with an invalid access crash.
  • move them - this is tricky in c++ and would probably require wrapping them into unique_ptrs.

Having these options, I still lean towards the current solution. I don't think copying them is that expensive.

17 ↗(On Diff #16586)

Oh crap, forgot to rename it, sorry about this.

26 ↗(On Diff #16586)

I disagree, sorry.

we still should destruct it gracefully so that all the work can be finished, regardless the reason

I already stated that this is impossible. The destructor of this singleton's never going to be called. The only scenario in which it stops working is when the service crashes. And in this case, all the threads are going to be stopped forcibly anyway,

What's wrong with having two or more pools?

I think this is unacceptable. I think we should never have multiple pools (https://stackoverflow.com/a/35328044/15854120). Maybe if there were a couple of different "components" doing different jobs, but having multiple pools for one purpose? I think it looks more like an anti-pattern. We can always extend the thread limit, but overall, I'd go for more control of what's going on.

Then we could have a singleton that holds an instance of this class

This doesn't make sense at all to me. If we have class A that is a singleton, why would we ever come up with a solution where A is not a singleton and we create an additional class B that is a singleton and holds an instance of A? It's equivalent, we just have an additional layer of abstraction that doesn't do anything really. did I miss something here?

I think the current solution will work just alright.

services/lib/src/Worker.h
26 ↗(On Diff #16586)

I just don't understand why we would want to have many instances of thread pools flying around. This introduces a lot of complexity and becomes error-prone (we really need to be in control of their state etc.)

tomek added inline comments.
services/lib/src/Worker.h
26 ↗(On Diff #16586)

Finding one random stack overflow post doesn't necessarily mean that this is the best and only solution to the problem. You can find a lot of posts that state otherwise, e.g. https://stackoverflow.com/questions/26243422/is-having-a-single-threadpool-better-design-than-multiple-threadpools with some real advantages of having multiple pools.

Having a single thread pool is NOT a good design because in case of 1 thread pool, if one part of the application becomes slower, threads will concentrate there.

or

OS Threads are a limited resource. If you have an application that uses threads for different purposes, some of them might become busy and keep a lot of threads working for them, or some service might have a bug where in some circumstance a thread isn't returned to the pool. If that can happen to one thread, the same circumstance may be applicable to all of the threads, and a whole thread pool can be drained this way.

This doesn't make sense at all to me. If we have class A that is a singleton, why would we ever come up with a solution where A is not a singleton and we create an additional class B that is a singleton and holds an instance of A? It's equivalent, we just have an additional layer of abstraction that doesn't do anything really. did I miss something here?

Layers of abstraction sometimes doesn't introduce any new logic, but instead improve readability, maintainability, and overall design of the system.

I just don't understand why we would want to have many instances of thread pools flying around. This introduces a lot of complexity and becomes error-prone (we really need to be in control of their state etc.)

E.g. from a cited post

The purpose of having separate dedicated threadpools is so that an activity doesn't get starved for threads because other activities took all the threads. If some service has its own threadpool then it is assured of having a certain number of threads at its disposal and it's not as sensitive to demands made by other services. I think you could figure other examples of scenarios in which multiple thread pools are beneficial.

But overall, if we design classes, we should try to assume as little as possible. So it's always better to have a class that can be used in more different contexts.

Let's keep the current design for now. We will improve this when necessary.

This revision is now accepted and ready to land.Sep 16 2022, 10:07 AM
services/lib/src/Worker.h
26 ↗(On Diff #16586)

The SO post you provided doesn't really get anything new to the table I think. It's all true, the point is that I perceive the grpc server as an activity/module/component that does exactly one job. Let's say we had another component that performs file i/o, then creating another thread pool for this would totally make sense. But here, there are the same tasks basically but for different clients.

A good question would be: how would you specify what the exact number of thread pools should be? On what that would be based?

Layers of abstraction sometimes doesn't introduce any new logic, but instead improve readability, maintainability, and overall design of the system.

I understand, but in this case, I just don't see a profit at all. It just brings more complexity and some redundancy. What exactly is the advantage of this approach?

Let's keep the current design for now. We will improve this when necessary.

Let's do it, the discussion itself is interesting though.

services/lib/src/Worker.h
26 ↗(On Diff #16586)

It's all true, the point is that I perceive the grpc server as an activity/module/component that does exactly one job. Let's say we had another component that performs file i/o, then creating another thread pool for this would totally make sense. But here, there are the same tasks basically but for different clients.

I think we agree but weren't communicating effectively. I wasn't proposing that we should have a couple of pools in this module. I was advising that the class should be written in a reusable way.

I understand, but in this case, I just don't see a profit at all. It just brings more complexity and some redundancy. What exactly is the advantage of this approach?

The class would be more reusable, so using it in a different context would be cheap and would not cause any risk of breaking existing usages.

services/lib/src/Worker.h
26 ↗(On Diff #16586)

I now fully understand, and that makes sense (this class resides in fact in the lib directory). I think this can be done on demand as you said earlier.