Working with threads in C++

Working with threads in C++

Multithread workers

preface

I had some comments already on this article. The posts are saying that I will run into race conditions and other multi-threading issues. But please, read this article as a tutorial. It is NOT wrong. It just does not cover everything.

If I had gone all the way to the deep ends of multithread-programming and all the caveats and issues we will face. DO you think anyone would try it? The post I made here was to encourage people to consider parallel programming, CPU; GPU, whatever we now have. This article was just a start, and it was my take on it. No not discourage people, let the people learn like I did. Like you did, and many do! That is the preface! Learn! And tell me if I am wrong.

Intro

Like in my previous blog posts, I try to explain things in an understandable but yet also useful way. I try to provide components that are extendable.

Also, first. This is more longer post than my previous posts here. But it also updates the things I have done. You will see.

I also try to explain the problem, bit of a design and the solution with my understanding around the problem area. If you have read my previous post about Multi State Management in Games you will perhaps see what I mean. Read more about me myself and I in the bottom of this article.

In this article I will go in to the "complex" world of multi-threading. In quite simple and easy steps and with and example, and hopefully also some design behind of that example.

In computer programs and in situations you are trying to solve with computers often require you to process huge amounts of data. It being graphics related, 3-D, data manipulation, procedural generation, mandelbrot?, cellular automata?, what ever. There are places where you do things in serial, meaning that you do one thing then do the other thing. Then there are places where you can do things in parallel, do this and this thing together to get the expected result. Doing things at the same time in computer science is called as parallel-programming or multi-threading.

I won't go in to the details of operating systems and how processes relate to threads but to open up some basics I say that each process can have multiple threads (tasks) running in parallel (remember multi-threading). A thread is a unit of execution, a thread completes things you requested in your program. Also each thread in the process share the same memory space of the process as the process is the one who controls the memory space for the process. Sharing the same memory space with multiple simultanously running tasks accessing the same memory gives you tons of advantages but it can (and will) lead in to problems if unaware. Gladly there are mechanics to help with that. In this article I will NOT go to thread-safety. This is about thread basics. That's a short intro on processes and threads. But, to go more towards the subject of this article...

singethread.png

Earlier, even using multi threading, things being processed by your lovely computer did not happen in parallel, because the hardware could not do that. The CPU under the hood only could run one thing at the time. In multi-threading OS (Like AmigaOS for example) the operating system interrupted the running thread and gave time for the next thread in the queue to do its job and so on. Then came the multi threading CPUs, a CPU that can run 2 threads at once, 4 threads, 8 threads, 16 threads, ... parallel programming and multi-threading. So the operating systems started using the features, making it possible to run things in parallel. You could do many calculations at the same time, awesome!

multithread1.png

Multi-threading is used everywhere, in every internet server you use, in every game you play, everywhere. People think it's hard to understand but the basics are actually pretty easy to grasp, you just have heard it's hard (It can be, believe me).

Multi-threading is by far not a new concept. It has been around for tens and tens of years and still we all struggle with it. Me too, even with tens of years with programming experience, so lets do this together.

Single-Threading vs. Multi-Threading

Consider a following problem

You have a big set of data, lets say, 10 files containing an arbitrary number of text lines you need to sort in certain order.

Traditionally you would just process each file sequentially, one by one:

  1. open a file
  2. sort the file
  3. open second file
  4. sort it
  5. and so on

multithread3-single.png

But as I stated earlier the modern CPUs can process many things in parallel. So by doing each file one by one you would waste precious cycles of computing power. Instead of processing files one by one sequentially you can do many of them at the same time in parallel. What if you could open 8 files simultaneously and process them without loosing any CPU power doing that?

multithread2.png (Image courtesy of here )

For example I have a pretty old and faithfull CPU (i7-4770) which has 4 CPU cores and each of them can run 2 threads. Meaning the CPU can run 8 things in parallel. If you would do the file processing in the old way, only one core would be processing the files, one-by-one.

Of course, in the heart, the operating system (OS) handles how it shares threads for the CPU to execute so you can have 8 threads running at the same time but you if you have more and your CPU can't support more the OS will handle the rest. The OS will interrupt your other running threads periodically and make your other threads to run. But still, you will save time because you can run 8 threads at the same time.

Multi-threading in real life

Common task in current solutions could be for example the following

  1. A Web server serving multiple clients (your browser) asking for a Web page
  2. A multi-player game where multiple players interact with each other and the world
  3. A task that can be split into parts. Like processing a set of files, or rendering a mandelbrot series of images.
  4. An artificial intelligence (AI) making decisions on multiple factors
  5. ... imagine, what you can do in parallel....

By the way, may say that men can't multitask. Well, I am a man and believe me, I can, like can you.

The example in this article

I wanted to implement a simple C++ class to make multiple thread workers doing certain things in parallel. For example if you have a set of similar things you can split them in to parts which can be done simultaneously to accomplish the result.

The example does not really do anything it just calculates some numbers and has delays. But it shows how this can be done with the basic C++ and lambdas. If you do not know what a lambda expression is, check this article for example . They are temporary funtions and a very important part of a (any) programming language like C++.

At this point I will mention that the whole source code for this example is here in my GitHub in the ThreadWorker project .

Just a side note. There are a LOT of C++ libraries that help you with threads, for example one of the popular ones is boost::thread availabe here . But that is just one popular example. In this article I will use standard std threading features.

I wanted to forget the semantics and innerworkings of threads outside of my Worker class. I wanted to offer an easy to use, yet extendable, API which goes over the threading issues. So at first I do this, create the workers:

    // Create requested number of workers
    std::vector<std::shared_ptr<Worker>> my_workers;
    for (uint16_t i = 0; i < workerCount; ++i) {
        my_workers.emplace_back(std::make_shared<Worker>(i));
    }

I will introduce the worker class later. But remember, you already have the link to my GitHub repo where it's available so if you feel hasty, go for it. <3

The above code just creates a vector of requested amount of workers (which are threads).

Later in the main demo application I find the first free worker for a new task as follows:

for (auto w = my_workers.begin(); w != my_workers.end(); ++w) {
    // Find first idle worker
    if (w->get()->state() == Worker::WorkerState::EIdle) {
        Worker* worker = w->get();
        auto lfunc = [worker, steps]() {
            for (auto i = 0; i < steps && worker->state()==Worker::WorkerState::EWorking; ++i) {
                LOG_INFO() << "Lambda worker: " << worker->id() << " at work, step: " << i;
                std::this_thread::sleep_for(std::chrono::seconds(1));
             }
         };
         w->get()->work(lfunc);
         break;
     }
 }

The people who know modern C++ immediately notice that I first find a worker with a free state (EIdle) and then give it a new task to process with a lambda, a temporary local function. I described it above in this article. Search google if you don't know what a lambda function is.

understand.png Image courtesy of here .

With w->get()->work(lfunc);the worker begins executing the given task, sets its state to EWorkingand starts executing the task it has been given to. When finished it sets its state back as free (EIdle) again being available for a next task.

As the design allows passing any lambda function to the worker you can make the worker do almost anything you want. You might need (will need to) change the signature of the Worker::work() to suit your needs but remember, this is just an example to give you a hint.

Check the cpp file for ThreadWorkerDemo.cpp on how I use the Worker class.

I tried to encapsulate the thread handling to the Worker class. It has start, stop and work methods. I will give you an explanation in the following chapters.

Remember, your main thread is working independently of the workers and can control them as you wish. So, to stop your thread, in this example when the main thread receives a quit command I tell all the workers to stop.

std::cout << "Terminating\n";

// Stop the workers, this will do thread join()
for (auto w = my_workers.begin(); w != my_workers.end(); ++w) {
    w->get()->stop();
}

Every worker of course has to call thread::join() to get everything synced up. But the Worker class handles this.

So basically, I have three public methods in my worker which you need to use:

  1. start() -> start the worker to listen for more work. Infinite loop until ended
  2. work(lambda) -> start working with a new task, if free
  3. stop() -> stop the loop and do a thread::join()

The Worker class

So what is special in the implementation and design of the Worker class which encapsulates the thread and the worker? Nothing. Let me give few insights. But I think the best thing for you now would be to go and check the implementation of my Worker class here . But...

After creating a worker instance, you start it with this:

    void start() {
        LOG_INFO() << "Starting worker: " << m_id;
        m_running = true;
        m_state = WorkerState::EIdle;
        m_thread = std::thread([this]() { doRun(); });
    };

Remember, we are using Lambdas everywhere, you see one above: m_thread = std::thread([this]() { doRun(); }); See the earlier sections for links to lambda in C++.

But what happens. As you see, it sets the state to EIdle, meaning the worker is ready to accept new task. It also does start the thread and makes the doRun() method of the Worker class as the thread main method.

Now, in this design and implementation doRun() is a private method of the Worker class and it does the following:

    void doRun() {
        while(m_running) {    
            auto timep1 = std::chrono::system_clock::now();
            if (m_state == WorkerState::EWorking) {
                LOG_INFO() << "Worker " << m_id << " starting task\n";
                if(m_usefunction) {
                    m_task();
                } else {
                    doWork();
                }
                m_state = WorkerState::EIdle;
                auto timep2 = std::chrono::system_clock::now();
                std::chrono::duration<float> elapsed(timep2 - timep1);
                LOG_INFO() << "Worker [" << m_id << "] task complete, time spent: " << std::to_string(elapsed.count());
            }
            else {
                LOG_INFO() << "Worker: " << m_id << " @" << Worker::WorkerStateToString(m_state);
                std::this_thread::sleep_for(std::chrono::seconds(KThreadWaitSeconds));
            }
        }
    }

130993-the-mummy-returns-the-mummy-returns-oconnells-son.jpg

If given a task (lambda) it calls the m_task() otherwise it will call the default task which just waits. After the task it will set the state again to EIdle making the worker to accept another task to work on.

This is just how you start the worker and how the worker waits for new things to do. but how do you schedule a task for the worker? Well, you call work(lambda)

    void work(std::function<void(void)> task) {
        const std::lock_guard<std::mutex> lock(m_mutex);
        if (m_state == WorkerState::EIdle) {
            m_task = task;
            m_usefunction = true;
            m_state = WorkerState::EWorking;
        }
    }

As you see the work function takes a std::function() (You will need to change the signature to suit your purposes sadly) and before changing the state of the worker it synchronizes it with a mutex for thread safe execution. This is another topic which I will not go in to in this article. Afterh the work() has been called, the Worker will begin to work on the given task as soon as it can. You remember from the above that the worker is running all the time, waiting for a new task to do.

To stop a worker you have to call stop() on the Worker instance.

The implementation in this example is as follows:

void stop() {
        LOG_INFO() << "Stopping worker: " << m_id;
        if (m_running) {
            m_running = false;
            m_state = WorkerState::EExiting;
            m_thread.join();
        }
    };

It just checks if the worker is running and changes the states as requested. The thread will stop after thread::join().

This is just bare bones of threading

Multi-threading is a complex issue. There are tons of issues to tackle. For example the following:

  1. Thread synchronization and lockups
  2. Memory safety

Just to name few.

Remember what I said in the beginning of this post. Threads belong to a process and the process handless the memory which can be accessed. Meaning every thread in the process can access the same memory. If multiple threads access the same memory at the same time you will have a UB situation (Undefined Behaviour) where you do not know which thread got the information and which thread updated the information. This comes up to thread synchronization.

If you have multiple threads working on same components/objects, you need to synchronize.

Thankfully there are methods to do this. There are Mutexes and Critical Sections. You already see these being used in the code I provided here. For example my logger class DebugLogger shares the std::out with multiple threads, I synchronize it with a mutex, check the code. It's in the github. Also the Worker class uses a mutex. I will cover on these kinds of issues later in this blog.

Last words, before the next ones <3

I know, this is just bare bones on how to tackle with threads in C++. But I tried to provide you a good example on how to do it with standard C++ without any libraries.

Of course all practical projects will have different requirements, this was just an example and a article to explain this thing to everyone. Hopefully in understandable way.

This project barely scratches the surface of the subject, but I think it might give you a good heads up on where to go. Especially for the people who are starting to tackle these issues.

With modern CPUs there are a ton of concepts to cover for parallel computing, and faster calculation. For example current GPUs can do a lot of diffent kind of calculation with their units with compute sharers, and all that happens in parallel with the CPU. Also CPUs have tons of optimisations like intrinsics and such. These are advanced topics which I might cover later. Leave a comment if you want such articles!

About me

I am a programmer who started doing assembly programming as a hobby when I was a teen in 1982. I wanted to know how the games I played were made. Since then I began playing with C and later with C++. I have been doing other languages as well like C#, Java, Ruby, PHP, Python, Javascript, Perl, Rust, ... once you have gained a knowledge of certain language and you have the understanding of building a computer program and the semantics the language does not matter.

I have worked with few big companies for example like Nokia, Kone and Metos. Currently working in a company I can't mention at this point.

I started as a engineer, a programmer and a designer, but I have also worked as a product and project manager in agile projects for over ten years. So I can also lead projects and I know how to lead designing projects. This article however is just to show my technical knowledge <3

I have a great respect to everyone in this area. Currently I work in a HW Embedded sector doing low level software for embedded systems. But I love games and game programming, by the way, I am currently playing Mass Effect Legendary Edition ! (Non-paid advertisement, I love the series!)

Disclaimer

This article by far provides a perfect design or perfect practices to work in the area of threads or threading or software design. It is just my view on how I wanted to present this issue to you all. I am not a scientific person, I am far from it. I am a practical person who always does things to learn new things. That is something I have done for almost 40 years in the area of programming and software design.

I will not be liable for any damages caused by this article, or the practices introduced here, to you, your business or to your mental health <3

Also, about the demo for this article. Read the README please. The UI sucks, it just reads user input in the main thread and gives commands to the Workers. The output is not synchronized. The UI was not a point of this article.

Have a nice summer!

Edit

English language is not my native language. I had to edit, and I still have to edit a lot of spelling mistakes.

ThreadWorker (this) in my GitHub My github My Blog My Knights who say RPG gaming Discord Boost library for C++ Multi-threading in general) Lambda specs

For lambda, please do a Google search. It is a new thing for most old school c++ programmers but you can find a lot of info on it in Google. As an experienced programmer, or one who wants to learn you will find a way how to use it and learn the syntax I used in these examples. "Force be with you"

Please leave a comment and share a link if you find this insightful!