Ruby provides a mixed bag of tools for building asynchronous systems. While CRuby wraps all Ruby code in a Global Virtual Machine Lock (GVL), JRuby and TruffleRuby provide truly parallel threads. Code analysis reveals that thread synchronization primitives are often used incorrectly, and while results may be okay on CRuby, running the same code on JRuby or TruffleRuby can expose race conditions with unexpected consequences. We present a light weight per-thread fiber scheduler which improves the concurrency of existing code with minimal changes. We discuss it’s implementation within Ruby and evaluate its performance using Net :: HTTP
.
This is the final report required by the Ruby Association Grant. There are two previous progress reports: December 3647 and January .
Introduction
In , Yukihiro Matsumoto (Matz) introduced an optional Thread primitive to MRI for producer-consumer style programs, inspired by OCaml’s implementation of green threads . At the time, multi-core processors were not popular, and so interleaved execution of Ruby code (concurrency) was acceptable performance trade off.
In , Koichi Sasada introduced native threads into YARV, so that blocking system calls would not stall the entire virtual machine (VM). In order to retain compatibility with existing C code which expected non-reentrant execution, the “Giant VM Lock” (GVL) was also introduced. This lock prevents the parallel execution of many parts of the Ruby VM, including Ruby code and C extensions. In , YARV was merged into CRuby . It was assumed that fine-grained locking would eventually be implemented, however it turned out to be difficult due to thread safety and performance issues, and thus CRuby threads are unable to provide true parallelism.
In many situations, C extensions can release the GVL in order to improve parallelism. This can improve the scalability of a multi-threaded program. However, even this is not guaranteed . Thus, it can be challenging building truly scalable systems using threads with CRuby.
In addition, the non-determinism of threads creates a combinatorial explosion of program states. We have analysed existing Ruby code and identified a significant number of thread-safety issues . Even within Ruby itself, non-determinism leads to unpredictable bugs, including garbage collection of weak references , and deadlocks during signal handling .
, Threads, as a model of computation, are wildly nondeterministic, and the job of the programmer becomes one of pruning that nondeterminism. Although many research techniques improve the model by offering more effective pruning, I argue that this is approaching the problem backwards. Rather than pruning nondeterminism, we should build from essentially deterministic, composable components. Nondeterminism should be explicitly and judiciously introduced where needed, rather than removed where not needed. Edward A. Lee EECS Department University of California, Berkeley Technical Report No. UCB / EECS – – 1
January , 2019
Modern languages have tackled these problems in a variety of ways. Node.js uses a single-threaded event-driven design, with explicit callbacks. Using async / await style syntax helps to alleviate the need for heavily nested code, but it also adds a new semantic dimension which can be cumbersome in practice.
Go uses a multi -threaded scheduler, where multiple “goroutines” are scheduled across multiple system threads. Some thread-safe constructs are provided by default, but the developer may still be required to understand and manage shared mutable state, and unfortunately, In practice, this model is still not as good as a well optimized event loop .
JRuby and TruffleRuby both provide Thread-level parallelism and as a consequence, programs that worked correctly in CRuby due to the GVL, may work incorrectly on these implementations. While true parallelism is a welcome addition, existing thread safety issues are exacerbated with additional nondeterminsm in the form of preemptive scheduling and simultaneous execution which can lead to data corruption .
Existing work shows that event loops can be used to build scalable systems , however existing implementations must wrap around or replace Ruby constructs. We investigated how to use Fibers to expose non-blocking operations directly within CRuby. In order to work within existing event loops, we implemented a per-thread fiber scheduler which provides hooks for these non-blocking operations. This report presents an overview of the design, and shows how the approach improves the performance of existing real-world programs.
Implementation
You never change things by fighting the existing reality. To change something, build a new model that makes the existing model obsolete. R. Buckminster Fuller
Our proposed design requires four minor changes. First, we introduce a thread scheduler interface which provides low level hooks for blocking operations. Then, we introduce non-blocking fibers. We extend threads to track whether the current execution context allows non-blocking operations, and if so, invoke the scheduler hooks. Finally, we change all I / Os to be (internally) non-blocking by default, so that they invoke these hooks when they would otherwise block.
The per-thread fiber scheduler interface is used to intercept blocking operations. A typical implementation would be a wrapper for a gem like
EventMachine or Async . This design provides separation of concerns between the event loop implementation and application code. It also allows for layered schedulers which can perform instrumentation.
class Scheduler # Wait for the given file descriptor to become readable. def wait_readable (fd) end # Wait for the given file descriptor to become writable. def wait_writable (fd) end # Wait for the given file descriptor to match the specified events within the specified timeout. def wait_for_single_fd (fd, events, timeout) end # Sleep the current task for the specified duration, or forever if not specified. def wait_sleep (duration=nil) end # The Ruby virtual machine is going to enter a system level blocking operation. def enter_blocking_region end # The Ruby virtual machine has completed the system level blocking operation. def exit_blocking_region end # Intercept the creation of a non-blocking fiber. def fiber (& block) Fiber.new (blocking: false, & block) end # Invoked when the thread exits. def run # Implement event loop here. end end
GIPHY App Key not set. Please check settings