Download it here.

Async provides a simple process pool that allocates a number of worker threads
that may then be utilized collectively without having to deal directly with
threads. Results of jobs sent to the pool are “future” objects, called Asyncs.
Asyncs’ values are then acquired by calling Async.sync, which blocks until the
value has been set by the pool. A supervisor thread monitors the worker threads
and restarts any that terminate abnormally.

Usage is simple:

// create a pool of four processes
Pool pool = Pool(4)
 
// send a job to the pool
Async result = Pool->send(exp, 10);
 
// get the result
result->sync();
// => 22026.46484375
 
// alternately, you can call an async
result();
// => 22026.46484375
 
// pools can conveniently map across a sequence
Async mapped_result = pool->map(exp, enumerate(5));
mapped_result->sync();
// => ({ 1.0, 2.718281745910645, 7.389056205749512, 20.08553695678711, 54.59814834594727 })
 
// or iterate without collecting return values
int x = 0;
pool->iter(lambda (int n) { x += n; }, enumerate(100));
write("%d", x);
// => 4950
 
// close the pool, blocking until all workers have terminated
pool->close();

Submit article

The module came out of my original locks module, which contained only the Lock and RLock classes. The new library contains the following classes:

• Semaphore: simplifies use of semaphores
• Shared: simplifies use of shared memory
• Synchronized: shared memory made safe for use between processes
• Lock: a binary semaphore, or mutex
• RLock: a recursive Lock
• Event: a simple mechanism to signal multiple processes
• Pipe: simplifies use of pipes
• Channel: a two-way communications channel made with pipes
• Queue: a synchronized first in, first out class

MP additionally includes various utilities to deal with common tasks:

• get-pid: gets the current processes’ pid
• with-lock-held: a macro that evaluates its body while holding a lock
• wait: waits for a condition with variable length polling to reduce processor load
• map and iter: versions of map and dolist evaluated asynchronously using the Cilk API

Perhaps the most useful function in the module is with-lock-held, which simplifies the most common locking case: assuring that a block of code is executed atomically.

(setf lock (RLock))
(with-lock-held lock
  (access-restricted-resource))

In this case, no more than one process can evaluate (access-restricted-resource) at a time.

You can download the MP module from my newLisp module repository.

Submit article

The Thread module in the standard library provides a complete system for multi-threading. Initially, a thread is created by instantiating a new instance of the Thread.Thread class with a function that will be executed in a new thread and any arguments to that function:

void say_hello(string name)
{
    write("Hello, " + name + ".\n");
}
 
int main()
{
    Thread.Thread thread;
    thread = Thread.Thread(say_hello, "Jeff");
 
    return -1; // return -1 so execution does not terminate
}

Pike’s interpreter is completely thread-safe. All execution is protected by a global interpreter lock, similar to that of Python. Unlike Python, however, Pike’s locking is finely grained and released often, making threads quite useful in Pike.

Thread.Mutex and Thread.MutexKey

Mutexes are required to protect critical sections that must be executed atomically. Thanks to the interpreter lock, no two threads will ever access the same variable concurrently. However, if a variable is modified or accessed multiple times in a thread, a mutex must be used to guard it:

Thread.Mutex m = Thread.Mutex(); // create a mutex
void func()
{
    Thread.MutexKey k = m->lock(); // acquire the lock
 
    ... // do stuff
    ... // do more stuff
 
    return;
}

Notice that the mutex was never released. Once the MutexKey (returned by Mutex->lock()) goes out of scope, the mutex is automatically released. This means that a method can be made to be synchronized simply by acquiring a lock at the beginning of the function.

Thread.Condition

Condition variables are used to synchronize events across multiple threads. A thread controlling a resource holds a condition variable, with other threads waiting for the condition to be signaled. The controlling thread may chose to signal just one or all waiting threads at once.

A MutexKey must be provided by the waitin thread to guarantee synchronicity with the signaling thread.

Thread.Mutex mutex = Thread.Mutex();
 
void consumer(Thread.Condition c, Thread.Condition c2)
{
    while (1) {
        Thread.MutexKey key = mutex->lock();
        c->wait(key);
        key = 0;
        write("Pong!\n");
        c2->signal();
    }
}
 
void producer(Thread.Condition c, Thread.Condition c2)
{
    while (1) {
        Thread.MutexKey key = mutex->lock();
        c2->wait(key);
        key = 0;
        write("Ping!\n");
        c->signal();
    }
}
 
int main()
{
    Thread.Condition condition1, condition2;
    Thread.Thread prod, cons;
    condition1 = Thread.Condition();
    condition2 = Thread.Condition();
    prod = Thread.Thread(producer, condition1, condition2);
    cons = Thread.Thread(consumer, condition1, condition2);
    condition2->signal(); // get the producer started
    return -1; // keeps script running
}

In this way, a condition variable is akin to a simplified counting semaphore, which may be incremented n times to signal n waiting threads.

Thread.Local

Thread-local storage may be created with Thread.Local:

string thread_fn()
{
    Thread.Local data = Thread.Local();
 
    data->set("foo");
    return data->get();
}

Thread.Queue

Queues are first in, first out containers.
Thread.Queue is expandable, so pushing new elements into the queue will not block:

void consumer(Thread.Queue q)
{
    while (1) {
        string value = q->read(); // block until value available
        do_something_with(value);
    }
}

Another interesting method is read_array, which reads as many values as are available. This is extremely useful. Reading a batch of work results in less time spend blocking, making active queues much more efficient.

Thread.Fifo

The primary difference between the Queue and Fifo classes is that the Fifo class is bounded and will block on writes when full.

void producer(Thread.Fifo q)
{
    string value;
    while (1) {
        value = get_value_from_somewhere();
        q->write(value); // blocks until space is available
    }
}

Conclusion

Pike makes threading easy on programmers. Pike’s internals are thread-safe; a programmer never needs to worry about corrupting memory. Mutually exclusive locks are only used to protect blocks of code that must be executed without interruption, so that only the concurrency of the overall technique must be considered.

Submit article

However, for long-running processes that intermittently access shared data (for example, using the (share) function), semaphores are used to lock that data. Technically, newLISP semaphores are counting semaphores (semaphores that may be incremented above 1), rather than binary semaphores (which are the equivalent of mutually exclusive locks).

These functions that wrap the semaphore command. They function as binary semaphores. Included is a macro with-lock-held, which executes its body with the passed lock acquired.

(constant 'ACQUIRE -1 'RELEASE 1)
 
;;; Convenience functions, macros
 
(define-macro (with-lock-held)
  (letex ((lock (args 0)) (body (cons 'begin (rest (args)))))
    (:acquire lock)
    (let ((res body))
      (:release lock)
      res)))
 
(define (getpid) (sys-info 6))
 
(global 'with-lock-held 'getpid)
 
;;; Locks are generic binary-style semaphores like mutexes.
 
(define (Lock:Lock)
  (let ((sem (semaphore)) (pid (share)))
    (semaphore sem RELEASE)
    (share pid nil)
    (list (context) sem pid)))
 
(define (Lock:acquire lock)
  (semaphore (lock 1) ACQUIRE)
  (share (lock 2) (getpid)))
 
(define (Lock:release lock)
  (if (= (getpid) (share (lock 2)))
    (begin
      (semaphore (lock 1) RELEASE)
      (share (lock 2) nil))
    (throw-error "Cannot release lock that is locked by another process.")))
 
;;; Recursive locks (RLocks) are locks that "owned" by a
;;; locking process and are not re-locked by subsequent code
;;; in the same process.
 
(define (RLock:RLock)
  (let ((sem (semaphore)) (depth (semaphore)) (pid (share)))
    (semaphore sem RELEASE)
    (semaphore depth 0)
    (share pid nil)
    (list (context) sem depth pid)))
 
(define (RLock:acquire lock)
  (let ((sem (lock 1)) (depth (lock 2)) (pid (lock 3)))
    (if (= (share pid) (getpid))
      (semaphore depth 1)
      (begin
        (semaphore sem ACQUIRE)
        (semaphore depth 1)
        (share pid (getpid))))))
 
(define (RLock:release lock)
  (let ((sem (lock 1)) (depth (lock 2)) (pid (lock 3)))
    (if (= (share pid) (getpid))
      (begin
        (semaphore depth -1)
        (when (= 0 (semaphore depth))
          (share pid nil)
          (semaphore sem RELEASE)))
      (throw-error "Cannot release rlock that is locked by another process."))))

Here is a quick sample of how to use the locks to synchronize access to a page of shared memory:

;;; create a shared resource that to be protected by a lock
(setq mem (share))
(setq mem-lock (Lock))
 
(:acquire mem-lock)
(share mem "foo")
(spawn 's
  (begin
    (:acquire mem-lock)
    (share mem "bar")
    (:release mem-lock)))
;;; at this point, the spawned process is blocking, waiting for us
;;; to release mem-lock. mem still holds "foo"
(:release mem-lock)
;;; now, the spawned process continues, and sets mem to "bar"

The same may be done more concisely using with-lock-held:

;;; create a shared resource that to be protected by a lock
(setq mem (share))
(setq mem-lock (Lock))
 
(with-lock-held mem-lock
  (share mem "foo")
  (spawn 's
    (with-lock-held mem-lock
      (share mem "bar"))))

download

Submit article

The global interpreter lock (GIL) is the method that Python uses to maintain internal thread-safety when not all Python objects are thread-safe. The GIL ensures that no Python object may be modified by multiple threads at the same time. This is transparent to the Python programmer; the GIL is locking internal structures. A list that is modified by multiple threads without explicit locking will still behave erratically.

This makes threading less useful in web development. Because Python can not directly maintain state between reqeusts (indirectly, via the client, state may be maintained using session cookies or other similar strategies), each request is a new thread:

 1. Determine what data is being requested
 2. Get the data
 3. Format the data
 4. Send the data

Each of these steps depends on the state information from the previous step. There is not a lot of room for asynchronicity. The usefulness of threading is pushed back to the web server. That is where integrated server/framework solutions, like Ocsigen, have an advantage.

Many web-based applications, however, are not simple database front ends. There are a few common situations where threading becomes useful; specifically, when there are side effects of a request.

On a side note, experts will tell you that only POST request should result in side effects. This is misleading. POST should certainly be used for user-intended side effects. However, what happends when there is an error loading a page?

Error messages

In the event of a coding or environment error, Django sends me an email. This is a wonderful (and an occassionally annoying) feature. But what if the problem is caused by the data we input, rather than the code? I can detect this in my code and present the user with a nice error page apologizing for the inconvenience, but now I want to notify the content team that an entry in the database is invalid in some way.

Sending a message, especially if Django is connecting to a remote mail server, can take a while. This is a good use for threading. The thread should deal with as little data processing as possible, concentrating instead on the side effect.

import threading
from django.core.mail import send_mail
 
def foo(request, identifier):
    data = get_some_data(identifier)
    try:
        error_check(data) # raise exception if data invalid
    except InvalidData, e:
        # Format our information here, in the main thread
        subject = "Invalid data in entry %d" % identifier
        message = """
            There is an error in entry %d.  Please check this
            data at http://path/to/django/admin/app/table/%d.
        """ % (identifier, identifier)
        recipients = ['someone@somewere.com', 'someoneelse@somewhere.com']
        from = 'root@server.com'
 
        # Create a new thread in Daemon mode to send message
        t = threading.Thread(target=send_mail,
                             args=[subject, message, from, recipients],
                             kwargs={'fail_silently': True})
        t.setDaemon(True)
        t.start()
    return HttpResponseServerError(some_error_page)

Note particularly that we use t.setDaemon(True) on the thread before starting it. This tells Python not to wait for the thread to exit before returning data to the client.

Writing files

Many of our web-based programs write data to static files. Some applications maintain logs. Others publish static HTML files to the enterprise server. These are great uses for Python threads, since the GIL is released during file IO operations.

Caching remote data

We are a large business and applications are developed by various (often isolated) units. I often find myself in a situation where my application depends on mutable data extracted from another system but does not have direct access to the data’s database.

Rather than depend on low internal network latencies and download the data extract on each page load, I cache the data and keep a local copy, using a script to synchronize the data. For simple data, this can be stored using Django’s low level cache. But if some state needs to be maintained for that data, a database table is a better method.

When there are a large number of such objects, a scheduled update process becomes burdensome, especially if the number of objects increases regularly. Therefore, the solution is to trigger the event when the data is accessed (via a page load).

Each cached database entry gets a timestamp field to note its last update. The model defines an update method and the model manager’s get() method checks the entry’s timestamp to see if the data needs to be freshened. The update routine is called in a separate thread.

Warning: in Django, when a model instance is found via a relationship, the manager’s get() method is not called! This is because these objects are accessed via a django.db.models.fields.related.ManyRelatedManager, rather than the model’s own manager. If you wish to solve this without duplicating code, look into Django signals.

The problem is that this is not a simple side effect and the GIL will get in the way. The solution is to do all of the necessary logic to determine if the object needs to be updated in the main thread. Just before returning the response to the user, the update thread is started in Daemon mode. That minimizes competition for the GIL. The user will not get the updated version of the object on this page load, but the next user will.

The other alternative is to use os.fork or the subprocess module to launch the update routine in a separate process (with its own interpreter instance and its own GIL). This sort of solution gets used a lot in PHP because of its lack of threads. In Python, threads tend to be more useful and much less resource hungry.

Submit article

Python’s thread and threading libraries use POSIX threads. The threading library is the higher level of the two and is therefore the one to use in your typical programming tasks. The Queue module provides a thread-safe mechanism for communicating between threads, like a combination list and semaphore.

POSIX threads are expensive, so it takes a little planning to know when to use them. Generally, the best uses of threads are for multiple tasks that cause side effects but do not depend on the state of other threads, such as output. An example of this is a program that writes out a large number of files with data from a database or a large data migration. Here is the general template for a threading class that encapsulates the actions to be taken.

Assume we have around 500 XML documents to download off of a remote server (via HTTP). Each is a large enough file that it warrants downloading several at a time. The server and the network can take a fair load, but we don’t want to simulate a botnet attack or overload our local network connection, which would slow down each download to a crawl and drastically increase collisions and errors while downloading. Let’s limit the number of files we download at once to 4. We start off with a function to download the files:

import urllib
 
def get_file(url):
    try:
        f = urllib.urlopen(url)
        contents = f.read()
        f.close()
        return contents
     except IOError:
        print "Could not open document: %s" % url

So much for error handling, but you get the idea. Assuming our url is stored in a variable with the name url, to execute this function in another thread, we run:

import threading
 
thread = threading.Thread(target=get_file, args=(url,))

We can make that a little simpler in two ways. The object oriented way is to implement the threading.Thread class. We would then put the code get_file(url) in our run() method. This is useful for instances when the result of the function is required for later processing. If the results are not needed, we can simplify using the functional programming method and utilize a partial application:

from functools import partial, threading
 
thread = threading.Thread(target=partial(get_file, url))

While that method is more fun, let’s use the OO method (no pun intended) since we want to do something with this data. Remember, we are downloading the file and storing it as a string, rather than simply downloading the file to the local file system. That implies we have more work to do after the download.

import urllib, threading
 
class FileGetter(threading.Thread):
    def __init__(self, url):
        self.url = url
        self.result = None
        threading.Thread.__init__(self)
 
    def get_result(self):
        return self.result
 
    def run(self):
        try:
            f = urllib.urlopen(url)
            contents = f.read()
            f.close()
            self.result = contents
         except IOError:
            print "Could not open document: %s" % url

Now we have our Thread implementation. Note that instantiating an instance of FileGetter does not cause the thread to start. That is done with the start() method. However, we don’t want all of the threads running at the same time, so we need to use the Queue module and a couple of helper functions to manage our list of files.

import threading
from Queue import Queue
 
def get_files(files):
    def producer(q, files):
        for file in files:
            thread = FileGetter(file)
            thread.start()
            q.put(thread, True)
 
    finished = []
    def consumer(q, total_files):
        while len(finished) < total_files:
            thread = q.get(True)
            thread.join()
            finished.append(thread.get_result())
 
    q = Queue(3)
    prod_thread = threading.Thread(target=producer, args=(q, files))
    cons_thread = threading.Thread(target=consumer, args=(q, len(files))
    prod_thread.start()
    cons_thread.start()
    prod_thread.join()
    cons_thread.join()

Let’s take a look at what we did here. The first function, producer, accepts the queue and the list of files. For each file, it starts a new FileGetter thread. The last line is significant. We add the thread to the queue. The second parameter, boolean True, tells the put() method to block until a slot is available. Note that the thread stores before the blocking does. This means that even if the queue is full, the thread will have started. Because of this, we reduce our queue size to 3.

The second function, the consumer, reads items out of the queue, blocking until an item is available in the queue. Then comes the important part, thread.join(). This causes the consumer to block until the thread completes its execution. This line is what keeps the queue from emptying before the next thread has complete execution (and therefore starting more threads). The consumer uses the module-level variable, finished, to store the results of each thread’s execution.

Last, we begin a thread for the producer and the consumer, start them, and then block until they have completed. Here is the complete code:

import urllib, threading
from Queue import Queue
 
class FileGetter(threading.Thread):
    def __init__(self, url):
        self.url = url
        self.result = None
        threading.Thread.__init__(self)
 
    def get_result(self):
        return self.result
 
    def run(self):
        try:
            f = urllib.urlopen(url)
            contents = f.read()
            f.close()
            self.result = contents
         except IOError:
            print "Could not open document: %s" % url
 
def get_files(files):
    def producer(q, files):
        for file in files:
            thread = FileGetter(file)
            thread.start()
            q.put(thread, True)
 
    finished = []
    def consumer(q, total_files):
        while len(finished) < total_files:
            thread = q.get(True)
            thread.join()
            finished.append(thread.get_result())
 
    q = Queue(3)
    prod_thread = threading.Thread(target=producer, args=(q, files))
    cons_thread = threading.Thread(target=consumer, args=(q, len(files))
    prod_thread.start()
    cons_thread.start()
    prod_thread.join()
    cons_thread.join()

Of course, this approach is not perfect. A queue is FIFO – first in, first out. If one of the threads currently executing finishes before the thread ahead of it, we lose efficiency in that now we only have three files downloading at a time. However, the solution to that is a complex one and outside the scope of this article.

Edit 02/19/2009: Fixed: FileGetter was not setting self.result in its run method. Thanks to tgray for pointing out the problem.

Submit article

On unix-based systems, newLISP uses the operating system’s fork to create a child process. State may be shared between two processes using either pipes or semaphores and shared memory. It is slightly faster to use shared memory and semaphores, but more difficult to track and coordinate. The process of forking the evaluation of an expression and accessing the result can be made easier with a couple functions to handle the messy business of coordinating access to shared memory. The first, which we will call spawn (since I like Erlang), will fork a child process that will evaluate an expression, allocate a shared memory address, and create a semaphore to signal the process.

(define-macro (spawn)
  (letn ((expr (args 0))
         (sem (semaphore))
         (res (share))
         (pid (fork (begin
                      (let ((res-val (eval expr)))
                        (semaphore sem -1)
                        (share res (source 'res-val))
                        (exit))))))
    (list pid sem res)))

Notice that block that the function is actually forking. After evaluating the expression passed to spawn, it blocks by setting the semaphore to -1. A positive value will cause it to unblock and evaluate (share res (source ‘res-val)). (source) serializes the value of ‘res-val into a string (which will actually assign res-val when it is evaluated). It then exits. The function itself returns a list of the pid of the forked process, the semaphore and the shared memory address.

The spawn process can be used like so: (setq proc (spawn (+ 3 6)))

Then we need a function that will read in the result from the spawned process. It must block until the process is complete, and then attempt to read in the value from the shared memory. Since we are using fork (which is unix-only, sorry!), we may as well clean up the shared memory afterward (which also only works on unix).

(define (receive proc , res-val serialized-result)
  (let ((pid (proc 0)) (sem (proc 1)) (res (proc 2)))
       (semaphore sem 1)
       (wait-pid pid)
       (setq serialized-result (share res))
       (share nil res) ; release shared memory
       (semaphore sem 0) ; release semaphore
       (if (catch (eval-string serialized-result) 'res-val)
           res-val 'invalid-result)))

Here, we pass receive the result of the spawn function. The value returned will be either the result of evaluating the expression passed to spawn or the symbol, ‘invalid-result. This function first unblocks the spawned process, allowing it to store its result in shared memory, blocking until it completes. It access the shared memory and then releases the shared memory address and semaphore.

The last statement is slightly tricky (if you are unfamiliar with newLISP error handling). It attempts to evaluate the serialized result pulled from shared memory. In the case of our example above, (setq proc (spawn (+ 3 6))), this would expand to something like (set 'res-val 9). Remember that our statement above used res-val to store the result for serialization. To avoid side effects, we declare res-val locally in our arguments (placing local variables after a comma in a lambda list is a newLISP convention to create local, nil variables; it is no different than declaring them as nil in a let statement or using the function local) and catch the result in it. This is a bit redundant. It would expand like, (catch (eval-string (set 'res-val 9)) 'res-val), but the effect is the same and the overhead of an extra set is negligible.

These functions do not provide the power of Erlang’s concurrent syntax (especially since they use OS threads, which are inefficient, although this is mitigated somewhat by the size and speed of newLISP). Instead, they achieve the desired result of simplifying the spawning of a child process (non-blocking) and accessing the result. For many purposes, the complexity of formulating expressions in a concurrent fashion outweighs the gains (which sometimes are not so great) of concurrent programming. By simplifying and automating the process, we negate that particular difficulty and can use the method that provides the most efficiency.

Well, on unix, anyway.

Submit article