Go Simplified: Scheduling and Context Switching

Source: content.techgig.com

So you recently got introduced to this marvelous language or have been using it for a while. You got your hands greasy with incredible goroutines and channels and the other shebang. However you would like some more information on just how it all works underneath those APIs. Well say no more cause I got you with this Simplified series article explaining just what the scheduler is and how does Go manage all the context switches.

What is the Scheduler ?

The Go scheduler is part of the Go runtime which is part of the executable. It’s called an M:N scheduler as well. [We’ll see why this M:N, down the road]

Go runtime will create a number of OS threads = GOMAXPROCS. GOMAXPROCS is the default number of processors on the machine. As of Go 1.14, Go allows for asynchronous preemption routines. [We’ll get to this later in this article]

The scheduler will also pre-empt a goroutine based on a time slice in case you thought that idle routines will keep on hogging memory. [Leaks can happen still but hey, Go is trying it’s best. Okay!?]

Okay, but can you make it a more graphic ?

Sure, below diagram shows the states a Go routine can be in.

1. When the routine, is created, it achieves the Runnable state.
2. When it gets scheduled, it goes to the Executing state.
3. If the routine while in Executing state, run through it’s time slice, it is pre-empted by the scheduler and goes back to Runnable state.
4. If the routine gets blocked, eg. I/O event, blocked on channel etc. It moves to Waiting state.
5. Once the block no longer exists, it goes back to Runnable state.

States achieved by goroutines

Let’s try a full rundown of all elements involved in routine scheduling.

Goroutine context switch

1. For a system core, an OS thread is created M.
2. For that thread, a logical processor P is created which attached itself to the OS thread and then schedules goroutines on that OS thread context.
3. R1 here is a goroutine currently executing on the thread M.
4. R2, R3, R4 form the Local Running Queue on the processor P.
5. GR1, GR2 form the Global Running Queue.
6. Once the local running queue is empty, the processor will pull threads from the global running queue to execute them.
7. Whenever a new goroutine is created, it gets added to the end of the global running queue.

So, I can say that at any given moment, I can schedule N routines on M OS threads which runs at most GOMAXPROCS number of processors.

Congratulations! You now know why we called it the M:N scheduler.

Okay looks fairly straight forward, how does the blocking work ?

Glad you asked. Let’s take a look at Context Switching due to Synchronous call first :

Context Switching due to synchronous system call

1. Let’s assume G1 starts to read a file synchronously which then blocks M1.
2. Runtime then asks for another thread from the pool cache/OS. Let’s call this M2.
3. Now processor P is detached from thread M1 and attached to thread M2. R1 is still attached to M1.
4. P now schedules R2. R3 remains in local running queue.
5. Once R1 has finished, it’s moved back to local running queue and M1 is put to sleep and put to thread pool cache.

Tell me about asynchronous system calls context switching now !!

Asynchronous system calls happen when the file descriptor that is used to do the I/O operation is set to non blocking mode.

Now if that file descriptor is not ready, the process is not blocked but an error is returned instead. Application now retries the operation at a later time. Application now needs to create an event loop and create call backs or create a table to maintain these mappings and states.

Go uses netpoller viz. an abstraction built in syscall package. When a goroutine makes an async call and the file-descriptor is not ready, netpoller is used to park the routine.

netpoller uses interfaces provided by system to check on the status of the file descriptor. Once it gets notification for the file descriptor, it in turn notifies the go routine.

Lets see this in a diagram :

Context Switching due to Asynchronous System Calls

1. Let’s imagine that R1 is making an async call which has now returned and error due to the file descriptor not being ready.
2. The scheduler upon encountering the error, it moves the goroutine to the netpoller thread.
3. netpoller will not keep checking and notify R1 when the process can be resumed.
4. P on the side can start working on R2 while R1 is watched by netpoller.
5. Once R1 is finished, it goes back to the local running queue.

So here, we did not need to ask for another thread from the OS. We used the netpoller thread to park the time taking routine.

Cool, but what happens when a processor runs out of routines to work on ?

Here is where the concept of work stealing comes into picture. Unlike most humans, processors just love executing tasks. So below are the rules followed for work stealing :

1. If no routines are present in Local Running Queue(LRQ), try to steal from another Local Running Queue.
2. If no routines are found in the other LRQ, check Global Running Queue(GRQ).
3. If no routines are found in the GRQ, check the netpoller.

This provides a good way to make the best use of time at hand.

Hope you liked this article. Show some love in the form of comments, critiques and applauds!

If you like the idea of this series, please let me know that as well in the comment section so that I can keep bringing you more on the Simplified series !!