Tag: Worker thread

Multithreaded Programming

32nd Friday Fun Session – 25th Aug 2017

Multithreading is a very important aspect of programming. Here we discuss some fundamental concepts and issues around it.

Process vs. Thread

A thread is a sequence of execution, a feature of operating system (OS). OS allocates processor time to a thread. Every process must have at least one thread. After all, it is a thread that executes the instructions.

Operating system gives each process some memory and it makes sure memory of one process is not accessible by other processes. However, all threads of a process would share its memory space, as the threads are parts of it.

Why do we need more than one thread?

As stated earlier, every process has at least one thread. Why do we need more than one thread? There are several reasons for that:

  1. Better UI experience: we can leave an exclusive thread to get user input, giving user a smooth interactive experience while having other threads to do other works.
  2. Do more things: with more than one processor available, using more than one thread, we can get more work done simultaneously.
  3. Use more things: different parts of software might be using difference resources of the computer. More than one thread can be used to simultaneously work with those resources.
  4. Doing slow things: sometimes we need to do some time consuming or slow thing, like writing to disk. We can create a separate thread to do that.

Concepts and Issues

We will focus on the following using VC++ concepts/terminologies/semantics:

  1. Types of threads
  2. Creating threads
  3. Memory issues
  4. Synchronizing threads
  5. Coordinating threads
  6. Exiting threads

Types of threads

There are two kinds of threads: user-interface (UI) thread and worker thread.

UI thread

UI thread has a message pump/loop that is nothing but an infinite loop that keeps on reading a message directed to the thread. Usually, PostThreadMessage is used to post (there is no corresponding send function) message to it. All these messages are queued in message queue, retrieved one by one and processed.

Since UI thread is in an infinite loop, it does not exit on its own. It has to be explicitly exited using PostQuitMessage.

UI thread can create UI components like showing a message box or other UI etc.

Worker thread

Worker thread does not have any message pump. That also means, it cannot receive any message. It cannot show any UI. It executes a set of instructions that is part of it and exits once execution finishes, meaning the thread does not exist any more after that.

Creating threads

AfxBeginThread function can be used to create both kinds of threads. However, a thread can be created in two modes: one that starts executing immediately; another that gets created in suspended mode and can be resumed later.

Each thread’s stack takes a 1 megabyte. 64 bit applications would fare better but keeping the number of threads low should be a priority.

Memory Issues

The thread can work on its own, allocating some (heap) memory that it will exclusively use. However, many a times, main process allocates some (heap) memory and passes it to a thread. This memory is usually freed up (to avoid memory leaks) by the process itself, as it allocated the memory at the first place. However, the process has to be careful not to free up memory when it is still being used by the thread. If the memory is deleted / freed and the thread is still accessing it, application/process/thread would crash (thread is part of the process).

That is one of the reasons we need to track the thread as to when it exits.

When we know that all threads that are using a certain memory have exited, we can free the memory. One way of doing this is again sharing/passing certain process (memory) flag to thread that it can use/set to indicate when it exits.

Synchronizing threads

In last section, we saw that process shares some memory with threads. At times, more than one thread might be using the same piece of memory. If multiple threads, running concurrently, access/modify common memory simultaneously, there would be data inconsistency and access violation crash. To deal with this we need to make those code segments as critical section. Access to critical section is to be controlled by synchronization techniques. There are a number of them. One light weight (compare to mutex etc.) intra-process technique (processor-specific test and set instruction) would be critical section (InitializeCriticalSection, EnterCriticalSection, LeaveCriticalSection, DeleteCriticalSection).

However, common memory object is not the only thing that we might need to synchronize.

Coordinating threads

One of the reasons, we create threads is to do some additional work. At times, one thread (main thread or a created one), let’s call it waiter, is waiting somewhere for another thread (main thread or a created one), let’s call it doer, to finish certain work. How would that be achieved?

We can use event based signalling mechanism. We can create an event (CreateEvent). A waiter can wait on it, usually using a blocking call (WaitForMultipleObjectsEx). Once doer is done, it can signal (SetEvent) using the event handle that it is done. Waiter would come out of the wait (blocking call) and it would know that the work for which it was waiting for is completed.

Waiter thread can again put the event to nonsignaled state (ResetEvent) and start waiting for the doer to signal it, and the cycle can go on and on.

An event can be either named or unnamed.

An event can also be created in two modes: manual-reset that requires an explicit (ResetEvent) call to put that event into nonsignaled state; another is auto-reset that creates the event with nonsignaled state.

Once we are done with an event, it should be closed (CloseHandle) else it would cause resource (handle) leak.

Exiting threads

When a process needs to exit, it should make sure that all the created threads are gracefully shut down first.

UI thread can be instructed to exit using PostQuitMessage function. However, it might take a while as there might be other messages in the queue. In that case, application should wait for the sake of a graceful shutdown. Otherwise, depending on what this thread is designed to do, the system might go to an unstable state, say, due to an inconsistent state of an inter-process synchronization object.

Since worker thread cannot be posted a message, we can use TerminateThread. However, once again this is a dangerous thing to do, for the same reason (can result in unstable state) as mentioned earlier.

So we need to be patient during shutdown. We can use the shared variable approach as discussed earlier, or an event based mechanism so that the process knows when all threads have exited.

Index

Interpreting IIS Internals

22nd Friday Fun Session (Part 2) – 16th Jun 2017

We are trying to understand how an HTTP request is processed by .NET web application, hosted in IIS in various scenarios with a focus on synchronization of the processing of that request. To be precise, we are interested in the thread and process contexts involved while serving an HTTP request.

Client request

Multiple clients across the globe, using their respective browsers, sending HTTP request to the web server.

Web server

All these http requests are ending up in IIS web server, hosting the application.

IIS kernel mode

In web server, HTTP listener, a kernel mode device driver, part of network subsystem, and part of the IIS – kernel mode of IIS to be precise, called http protocol stack (Http.sys), listens for http requests.

HTTPS.sys, as a forwarder, might directly pass the request to the right worker process, or as a request queuer, queues it unless a worker process picks it up. Once the response of that request reaches to it, it returns that back to client browser. Also as a kernel level cacher, it does some kernel level caching and if possible, returns the cached output directly, without involving any user level processing.

Worker process

Worker process, w3wp.exe, an executable, a process to OS, runs inside IIS user mode, is little different than other processes in the operating system (OS), in the sense that it can contain multiple application domains.

Application domain

Application domain represented by AppDomain object, a .NET concept, is a virtual process within a process. As said, it runs within a worker process. One application domain does not share its static variables etc. with another application domain running within the same worker process.

Usually, all static variables etc. of a process are visible to all within a process. It is not the case for worker process. This is how worker process is a special process, where one or more application domains are running inside it, as if each of them is a separate process, providing isolation. So what usually we are used to thinking as per process, in the world of IIS, inside worker process, it is actually per application domain.

Why Application domain, you might ask. Well, a web server can have many applications. Creating one worker process for each of them will end up creating many processes that is quite expensive. Creating an application domain for each of them and putting them together inside a single process is much cheaper.

Note that, one of the application domains can be stopped without affecting other application domains inside a worker process.

Worker thread

When worker process receives a request, it uses worker thread to process that request. If two users send two requests to the same web application, both of them can be simultaneously executed by two worker threads and served.

At any point of time, a worker thread can only be executed within a single application domain. An application domain can have multiple worker threads. But these worker threads are not confined to a single application domain. Rather they belong to worker process. When a thread is done with serving a request for a particular application domain, it is freed. At a later point of time, the same thread can be used to serve another request, belonging to a different application domain.

Web application

We develop web application. We are interested to know how this ending up running in IIS environment. Application domains are associated with web application. One web application has typically, one application domain running inside a worker process. One worker process might be running many application domains, each supporting a separate web application.

Application pool

Application pool is the container for (web) applications. Every application has to be assigned to a single application pool. A number of web applications can be assigned to a single application pool. But as mentioned earlier, a single application cannot be assigned to multiple application pools. All applications belonging to an application pool share the same configuration.

Worker process runs inside the application pool.

Application pool can be recycled, restarted. Applications belonging to other application pool are not affected by this. Thus application pool provides isolation.

So we see that a number of applications are contained within an application pool. And then a worker process running inside an application pool is again running a number of application domains, each application domain serving a different web application. Thus, we have two different levels of isolation.

Web garden

How many worker processes can be there inside an application pool? Well, it is configurable. By default, it is only one worker process running inside an application pool, supporting multiple web applications, by creating one application domain for each of the applications. And all these application domains are running as separate processes within that single worker process.

However, application pool can be configured to run more than one worker processes. In that case, it is called a web garden. In this situation, multiple worker processes can be running in a single application pool. Each of these worker processes, once again running multiple application domains, each belonging to one application.

In this scenario, each of the worker processes can have its own application domain for the same application. In other words, for a certain web application, we can have multiple application domains, each running in a separate worker process, all in the same application pool. To be precise, one application or web site can have multiple instances running in a single web server, if web garden is enabled.

This is important as it renders uses of static variables, application-wide variables etc. problematic in web application.

Web farm

When one web server is not enough to serve the clients requests, we need more of them to host the same application/web site. We call it web farm.

A load balancer would sit in front of the web servers and its IP will be exposed to external world. HTTP requests will come to load balancer first and it will then distribute the load to different web servers.

Individual web server can share the same database or replicated/partitioned database.

In a nutshell

Single server, application pool running one worker process

So we see that, multiple https requests for the same web application would be simultaneously served by multiple threads. Those threads can be executed within a single application domain belonging to a single worker process. This happens when only one worker process is set to run for an application pool.

Simple.png

In the above image, we see IIS having two parts – system and user mode. HTTP.sys is in kernel mode, forwarding HTTP request to 3rd application pool, belonging to application X. We further see that a single worker process inside that 3rd application pool is running two application domains X and Y. Two threads within application domain X – Thread 1 and Thread 2 are serving the two requests, respectively. The response will go back to client browser through HTTP.sys.

Single web server, application pool running more than one worker process, called web garden

Or the threads can come from different application domains associated with the same web application or web site, running inside different worker processes, all contained within the same application pool. This can happen in web garden configuration, where multiple worker processes are allowed to execute within a single application pool. We can understand any locking mechanism that works within a single process would not work in this setup. We would need to implement an inter-process synchronization mechanism, if our application is deployed in web garden.

Web Garden

In the above image, showing web garden, two requests are being served by two worker threads, belonging to two application domains (both associated with same web application), each running in a separate worker process, both of them (worker processes) contained within the same application pool.

Multiple web servers behind load balancer, called web farm

Or the threads can come from different physical web servers. This can happen in a web farm scenario, where multiple web servers sit behind a load balancer. We can understand that an inter-process synchronization mechanism, which works across the processes within an OS, would not work here. Since we have multiple web servers here, each running its own OS, inter-process synchronization mechanism would not work for application-wide synchronization.

Web Farm.png

In the above image, showing web farm, two requests are being served by two worker threads, each running in a separate web server.

Index