Download it here.

Async provides a simple process pool that allocates a number of worker threads
that may then be utilized collectively without having to deal directly with
threads. Results of jobs sent to the pool are “future” objects, called Asyncs.
Asyncs’ values are then acquired by calling Async.sync, which blocks until the
value has been set by the pool. A supervisor thread monitors the worker threads
and restarts any that terminate abnormally.

Usage is simple:

// create a pool of four processes
Pool pool = Pool(4)
 
// send a job to the pool
Async result = Pool->send(exp, 10);
 
// get the result
result->sync();
// => 22026.46484375
 
// alternately, you can call an async
result();
// => 22026.46484375
 
// pools can conveniently map across a sequence
Async mapped_result = pool->map(exp, enumerate(5));
mapped_result->sync();
// => ({ 1.0, 2.718281745910645, 7.389056205749512, 20.08553695678711, 54.59814834594727 })
 
// or iterate without collecting return values
int x = 0;
pool->iter(lambda (int n) { x += n; }, enumerate(100));
write("%d", x);
// => 4950
 
// close the pool, blocking until all workers have terminated
pool->close();

Submit article

The module came out of my original locks module, which contained only the Lock and RLock classes. The new library contains the following classes:

• Semaphore: simplifies use of semaphores
• Shared: simplifies use of shared memory
• Synchronized: shared memory made safe for use between processes
• Lock: a binary semaphore, or mutex
• RLock: a recursive Lock
• Event: a simple mechanism to signal multiple processes
• Pipe: simplifies use of pipes
• Channel: a two-way communications channel made with pipes
• Queue: a synchronized first in, first out class

MP additionally includes various utilities to deal with common tasks:

• get-pid: gets the current processes’ pid
• with-lock-held: a macro that evaluates its body while holding a lock
• wait: waits for a condition with variable length polling to reduce processor load
• map and iter: versions of map and dolist evaluated asynchronously using the Cilk API

Perhaps the most useful function in the module is with-lock-held, which simplifies the most common locking case: assuring that a block of code is executed atomically.

(setf lock (RLock))
(with-lock-held lock
  (access-restricted-resource))

In this case, no more than one process can evaluate (access-restricted-resource) at a time.

You can download the MP module from my newLisp module repository.

Submit article

The Thread module in the standard library provides a complete system for multi-threading. Initially, a thread is created by instantiating a new instance of the Thread.Thread class with a function that will be executed in a new thread and any arguments to that function:

void say_hello(string name)
{
    write("Hello, " + name + ".\n");
}
 
int main()
{
    Thread.Thread thread;
    thread = Thread.Thread(say_hello, "Jeff");
 
    return -1; // return -1 so execution does not terminate
}

Pike’s interpreter is completely thread-safe. All execution is protected by a global interpreter lock, similar to that of Python. Unlike Python, however, Pike’s locking is finely grained and released often, making threads quite useful in Pike.

Thread.Mutex and Thread.MutexKey

Mutexes are required to protect critical sections that must be executed atomically. Thanks to the interpreter lock, no two threads will ever access the same variable concurrently. However, if a variable is modified or accessed multiple times in a thread, a mutex must be used to guard it:

Thread.Mutex m = Thread.Mutex(); // create a mutex
void func()
{
    Thread.MutexKey k = m->lock(); // acquire the lock
 
    ... // do stuff
    ... // do more stuff
 
    return;
}

Notice that the mutex was never released. Once the MutexKey (returned by Mutex->lock()) goes out of scope, the mutex is automatically released. This means that a method can be made to be synchronized simply by acquiring a lock at the beginning of the function.

Thread.Condition

Condition variables are used to synchronize events across multiple threads. A thread controlling a resource holds a condition variable, with other threads waiting for the condition to be signaled. The controlling thread may chose to signal just one or all waiting threads at once.

A MutexKey must be provided by the waitin thread to guarantee synchronicity with the signaling thread.

Thread.Mutex mutex = Thread.Mutex();
 
void consumer(Thread.Condition c, Thread.Condition c2)
{
    while (1) {
        Thread.MutexKey key = mutex->lock();
        c->wait(key);
        key = 0;
        write("Pong!\n");
        c2->signal();
    }
}
 
void producer(Thread.Condition c, Thread.Condition c2)
{
    while (1) {
        Thread.MutexKey key = mutex->lock();
        c2->wait(key);
        key = 0;
        write("Ping!\n");
        c->signal();
    }
}
 
int main()
{
    Thread.Condition condition1, condition2;
    Thread.Thread prod, cons;
    condition1 = Thread.Condition();
    condition2 = Thread.Condition();
    prod = Thread.Thread(producer, condition1, condition2);
    cons = Thread.Thread(consumer, condition1, condition2);
    condition2->signal(); // get the producer started
    return -1; // keeps script running
}

In this way, a condition variable is akin to a simplified counting semaphore, which may be incremented n times to signal n waiting threads.

Thread.Local

Thread-local storage may be created with Thread.Local:

string thread_fn()
{
    Thread.Local data = Thread.Local();
 
    data->set("foo");
    return data->get();
}

Thread.Queue

Queues are first in, first out containers.
Thread.Queue is expandable, so pushing new elements into the queue will not block:

void consumer(Thread.Queue q)
{
    while (1) {
        string value = q->read(); // block until value available
        do_something_with(value);
    }
}

Another interesting method is read_array, which reads as many values as are available. This is extremely useful. Reading a batch of work results in less time spend blocking, making active queues much more efficient.

Thread.Fifo

The primary difference between the Queue and Fifo classes is that the Fifo class is bounded and will block on writes when full.

void producer(Thread.Fifo q)
{
    string value;
    while (1) {
        value = get_value_from_somewhere();
        q->write(value); // blocks until space is available
    }
}

Conclusion

Pike makes threading easy on programmers. Pike’s internals are thread-safe; a programmer never needs to worry about corrupting memory. Mutually exclusive locks are only used to protect blocks of code that must be executed without interruption, so that only the concurrency of the overall technique must be considered.

Submit article

However, for long-running processes that intermittently access shared data (for example, using the (share) function), semaphores are used to lock that data. Technically, newLISP semaphores are counting semaphores (semaphores that may be incremented above 1), rather than binary semaphores (which are the equivalent of mutually exclusive locks).

These functions that wrap the semaphore command. They function as binary semaphores. Included is a macro with-lock-held, which executes its body with the passed lock acquired.

(constant 'ACQUIRE -1 'RELEASE 1)
 
;;; Convenience functions, macros
 
(define-macro (with-lock-held)
  (letex ((lock (args 0)) (body (cons 'begin (rest (args)))))
    (:acquire lock)
    (let ((res body))
      (:release lock)
      res)))
 
(define (getpid) (sys-info 6))
 
(global 'with-lock-held 'getpid)
 
;;; Locks are generic binary-style semaphores like mutexes.
 
(define (Lock:Lock)
  (let ((sem (semaphore)) (pid (share)))
    (semaphore sem RELEASE)
    (share pid nil)
    (list (context) sem pid)))
 
(define (Lock:acquire lock)
  (semaphore (lock 1) ACQUIRE)
  (share (lock 2) (getpid)))
 
(define (Lock:release lock)
  (if (= (getpid) (share (lock 2)))
    (begin
      (semaphore (lock 1) RELEASE)
      (share (lock 2) nil))
    (throw-error "Cannot release lock that is locked by another process.")))
 
;;; Recursive locks (RLocks) are locks that "owned" by a
;;; locking process and are not re-locked by subsequent code
;;; in the same process.
 
(define (RLock:RLock)
  (let ((sem (semaphore)) (depth (semaphore)) (pid (share)))
    (semaphore sem RELEASE)
    (semaphore depth 0)
    (share pid nil)
    (list (context) sem depth pid)))
 
(define (RLock:acquire lock)
  (let ((sem (lock 1)) (depth (lock 2)) (pid (lock 3)))
    (if (= (share pid) (getpid))
      (semaphore depth 1)
      (begin
        (semaphore sem ACQUIRE)
        (semaphore depth 1)
        (share pid (getpid))))))
 
(define (RLock:release lock)
  (let ((sem (lock 1)) (depth (lock 2)) (pid (lock 3)))
    (if (= (share pid) (getpid))
      (begin
        (semaphore depth -1)
        (when (= 0 (semaphore depth))
          (share pid nil)
          (semaphore sem RELEASE)))
      (throw-error "Cannot release rlock that is locked by another process."))))

Here is a quick sample of how to use the locks to synchronize access to a page of shared memory:

;;; create a shared resource that to be protected by a lock
(setq mem (share))
(setq mem-lock (Lock))
 
(:acquire mem-lock)
(share mem "foo")
(spawn 's
  (begin
    (:acquire mem-lock)
    (share mem "bar")
    (:release mem-lock)))
;;; at this point, the spawned process is blocking, waiting for us
;;; to release mem-lock. mem still holds "foo"
(:release mem-lock)
;;; now, the spawned process continues, and sets mem to "bar"

The same may be done more concisely using with-lock-held:

;;; create a shared resource that to be protected by a lock
(setq mem (share))
(setq mem-lock (Lock))
 
(with-lock-held mem-lock
  (share mem "foo")
  (spawn 's
    (with-lock-held mem-lock
      (share mem "bar"))))

download

Submit article

The Cilk project is a parallel programming language based on C. Its goal is to radically simplify concurrency in C applications. Cilk applications are first compiled to C. Cilk adds simple concurrency syntax and a scheduler to determine whether or not to launch a new thread or simply execute the code block. Here is a sample from the Cilk website:

cilk int fib (int n)
{
    if (n > 2) return n;
    else
    {
        int x, y;
 
        x = spawn fib (n-1);
        y = spawn fib (n-2);
 
        sync;
 
        return (x+y);
    }
}

The next version of newLISP will have the spawn, sync, and abort functions. Here is the same function, written in newLISP:

(define (fib n)
  (if (> n 2) 1
    (begin
      (spawn 'x (fib (- n 1)))
      (spawn 'y (fib (- n 2)))
      (sync 10000)
      (+ x y))))

There are a few notable differences.
spawn not only spawns a new process, but it also sets the target symbol to be assigned when sync is run.
sync itself has a millisecond timeout. There is currently no scheduler and sync cannot block indefinitely, but hopefully these will be added in the full release.

spawn returns the pid of the launched child process, which may be cancelled later using (abort pid). On its own, abort cancels all child processes.

sync causes the current process to block until all children have completed or the timeout (in milliseconds) is reached. An interesting addition is that sync can also be used as a predicate to determine if all child processes have quit:

(until (sync 10000)
    (println "Waiting... (foot-tapping)")
    (sleep 100))

By itself, it returns a list of pids of child processes:

(spawn 'foo (+ 1 1))
(spawn 'bar (+ 2 2))
(sync) ; =>(12345 12346)
(sync 100) ; => true
foo ; => 2
bar ; => 4

Underneath the hood, the new functionality is not using Cilk – it simply aims to add similar behavior to newLISP. Because of newLISP’s small size (~500 KB in memory), forked processes (in the typical sense, not the Erlang one) are cheap. With sync functionality, newLISP will have a real edge on languages using OS threads.

In other news

Another recent development release of newLISP (9.3.5) added another important feature – the ability to pre-compile regular expressions:

(set 're (regex-comp "(foo|bar)"))
(regex re "raising the bar" 0x10000)

The new regular expression int-optin, 0x10000, alerts the regex function to expect a precompiled pattern. This works in all regex functions.

Submit article

Python’s thread and threading libraries use POSIX threads. The threading library is the higher level of the two and is therefore the one to use in your typical programming tasks. The Queue module provides a thread-safe mechanism for communicating between threads, like a combination list and semaphore.

POSIX threads are expensive, so it takes a little planning to know when to use them. Generally, the best uses of threads are for multiple tasks that cause side effects but do not depend on the state of other threads, such as output. An example of this is a program that writes out a large number of files with data from a database or a large data migration. Here is the general template for a threading class that encapsulates the actions to be taken.

Assume we have around 500 XML documents to download off of a remote server (via HTTP). Each is a large enough file that it warrants downloading several at a time. The server and the network can take a fair load, but we don’t want to simulate a botnet attack or overload our local network connection, which would slow down each download to a crawl and drastically increase collisions and errors while downloading. Let’s limit the number of files we download at once to 4. We start off with a function to download the files:

import urllib
 
def get_file(url):
    try:
        f = urllib.urlopen(url)
        contents = f.read()
        f.close()
        return contents
     except IOError:
        print "Could not open document: %s" % url

So much for error handling, but you get the idea. Assuming our url is stored in a variable with the name url, to execute this function in another thread, we run:

import threading
 
thread = threading.Thread(target=get_file, args=(url,))

We can make that a little simpler in two ways. The object oriented way is to implement the threading.Thread class. We would then put the code get_file(url) in our run() method. This is useful for instances when the result of the function is required for later processing. If the results are not needed, we can simplify using the functional programming method and utilize a partial application:

from functools import partial, threading
 
thread = threading.Thread(target=partial(get_file, url))

While that method is more fun, let’s use the OO method (no pun intended) since we want to do something with this data. Remember, we are downloading the file and storing it as a string, rather than simply downloading the file to the local file system. That implies we have more work to do after the download.

import urllib, threading
 
class FileGetter(threading.Thread):
    def __init__(self, url):
        self.url = url
        self.result = None
        threading.Thread.__init__(self)
 
    def get_result(self):
        return self.result
 
    def run(self):
        try:
            f = urllib.urlopen(url)
            contents = f.read()
            f.close()
            self.result = contents
         except IOError:
            print "Could not open document: %s" % url

Now we have our Thread implementation. Note that instantiating an instance of FileGetter does not cause the thread to start. That is done with the start() method. However, we don’t want all of the threads running at the same time, so we need to use the Queue module and a couple of helper functions to manage our list of files.

import threading
from Queue import Queue
 
def get_files(files):
    def producer(q, files):
        for file in files:
            thread = FileGetter(file)
            thread.start()
            q.put(thread, True)
 
    finished = []
    def consumer(q, total_files):
        while len(finished) < total_files:
            thread = q.get(True)
            thread.join()
            finished.append(thread.get_result())
 
    q = Queue(3)
    prod_thread = threading.Thread(target=producer, args=(q, files))
    cons_thread = threading.Thread(target=consumer, args=(q, len(files))
    prod_thread.start()
    cons_thread.start()
    prod_thread.join()
    cons_thread.join()

Let’s take a look at what we did here. The first function, producer, accepts the queue and the list of files. For each file, it starts a new FileGetter thread. The last line is significant. We add the thread to the queue. The second parameter, boolean True, tells the put() method to block until a slot is available. Note that the thread stores before the blocking does. This means that even if the queue is full, the thread will have started. Because of this, we reduce our queue size to 3.

The second function, the consumer, reads items out of the queue, blocking until an item is available in the queue. Then comes the important part, thread.join(). This causes the consumer to block until the thread completes its execution. This line is what keeps the queue from emptying before the next thread has complete execution (and therefore starting more threads). The consumer uses the module-level variable, finished, to store the results of each thread’s execution.

Last, we begin a thread for the producer and the consumer, start them, and then block until they have completed. Here is the complete code:

import urllib, threading
from Queue import Queue
 
class FileGetter(threading.Thread):
    def __init__(self, url):
        self.url = url
        self.result = None
        threading.Thread.__init__(self)
 
    def get_result(self):
        return self.result
 
    def run(self):
        try:
            f = urllib.urlopen(url)
            contents = f.read()
            f.close()
            self.result = contents
         except IOError:
            print "Could not open document: %s" % url
 
def get_files(files):
    def producer(q, files):
        for file in files:
            thread = FileGetter(file)
            thread.start()
            q.put(thread, True)
 
    finished = []
    def consumer(q, total_files):
        while len(finished) < total_files:
            thread = q.get(True)
            thread.join()
            finished.append(thread.get_result())
 
    q = Queue(3)
    prod_thread = threading.Thread(target=producer, args=(q, files))
    cons_thread = threading.Thread(target=consumer, args=(q, len(files))
    prod_thread.start()
    cons_thread.start()
    prod_thread.join()
    cons_thread.join()

Of course, this approach is not perfect. A queue is FIFO – first in, first out. If one of the threads currently executing finishes before the thread ahead of it, we lose efficiency in that now we only have three files downloading at a time. However, the solution to that is a complex one and outside the scope of this article.

Edit 02/19/2009: Fixed: FileGetter was not setting self.result in its run method. Thanks to tgray for pointing out the problem.

Submit article

On unix-based systems, newLISP uses the operating system’s fork to create a child process. State may be shared between two processes using either pipes or semaphores and shared memory. It is slightly faster to use shared memory and semaphores, but more difficult to track and coordinate. The process of forking the evaluation of an expression and accessing the result can be made easier with a couple functions to handle the messy business of coordinating access to shared memory. The first, which we will call spawn (since I like Erlang), will fork a child process that will evaluate an expression, allocate a shared memory address, and create a semaphore to signal the process.

(define-macro (spawn)
  (letn ((expr (args 0))
         (sem (semaphore))
         (res (share))
         (pid (fork (begin
                      (let ((res-val (eval expr)))
                        (semaphore sem -1)
                        (share res (source 'res-val))
                        (exit))))))
    (list pid sem res)))

Notice that block that the function is actually forking. After evaluating the expression passed to spawn, it blocks by setting the semaphore to -1. A positive value will cause it to unblock and evaluate (share res (source ‘res-val)). (source) serializes the value of ‘res-val into a string (which will actually assign res-val when it is evaluated). It then exits. The function itself returns a list of the pid of the forked process, the semaphore and the shared memory address.

The spawn process can be used like so: (setq proc (spawn (+ 3 6)))

Then we need a function that will read in the result from the spawned process. It must block until the process is complete, and then attempt to read in the value from the shared memory. Since we are using fork (which is unix-only, sorry!), we may as well clean up the shared memory afterward (which also only works on unix).

(define (receive proc , res-val serialized-result)
  (let ((pid (proc 0)) (sem (proc 1)) (res (proc 2)))
       (semaphore sem 1)
       (wait-pid pid)
       (setq serialized-result (share res))
       (share nil res) ; release shared memory
       (semaphore sem 0) ; release semaphore
       (if (catch (eval-string serialized-result) 'res-val)
           res-val 'invalid-result)))

Here, we pass receive the result of the spawn function. The value returned will be either the result of evaluating the expression passed to spawn or the symbol, ‘invalid-result. This function first unblocks the spawned process, allowing it to store its result in shared memory, blocking until it completes. It access the shared memory and then releases the shared memory address and semaphore.

The last statement is slightly tricky (if you are unfamiliar with newLISP error handling). It attempts to evaluate the serialized result pulled from shared memory. In the case of our example above, (setq proc (spawn (+ 3 6))), this would expand to something like (set 'res-val 9). Remember that our statement above used res-val to store the result for serialization. To avoid side effects, we declare res-val locally in our arguments (placing local variables after a comma in a lambda list is a newLISP convention to create local, nil variables; it is no different than declaring them as nil in a let statement or using the function local) and catch the result in it. This is a bit redundant. It would expand like, (catch (eval-string (set 'res-val 9)) 'res-val), but the effect is the same and the overhead of an extra set is negligible.

These functions do not provide the power of Erlang’s concurrent syntax (especially since they use OS threads, which are inefficient, although this is mitigated somewhat by the size and speed of newLISP). Instead, they achieve the desired result of simplifying the spawning of a child process (non-blocking) and accessing the result. For many purposes, the complexity of formulating expressions in a concurrent fashion outweighs the gains (which sometimes are not so great) of concurrent programming. By simplifying and automating the process, we negate that particular difficulty and can use the method that provides the most efficiency.

Well, on unix, anyway.

Submit article