Hacking Vim bills itself as a recipe book. While it provides quite a few good ones, it seems to be organized more like a tutorial, using the examples to walk through some of the cooler features of vim. That said, the recipes it provides are very useful and a good number have made it into my ~/.vimrc.

I found some examples to be overly contrived. The author could have done a better job relating samples to real-world situations. I also wish it had gone into more depth when explaining some examples early on. If you don’t skip ahead and read the section on vim scripting (or know it already), there are a few traps in the examples.

For all that, though, my now heavily dog-eared and ruffled copy was well worth the money. Like other books from the same publisher, Hacking Vim stays on subject, maintaining its focus and pragmatism. Definitely a good source of worthwhile tips.

Submit article

newLISP modules

A Google Code project has been created for my newLISP modules. Anyone who would like to contribute is welcome. It can be found at http://code.google.com/p/actools-newlisp/.

Phorms

The Phorms library also has a Google Project page, located at http://code.google.com/p/php-phorms/. Contributions, as always, are welcome.

PHP Pipeline

PHP Pipeline is an asynchronous HTTP pool. It is very simple to use and extremely efficient on resources. Code and samples available at http://code.google.com/p/php-pipeline/.

JoQ

JoQ is a simple message broker written in Java. Its design goals are efficiency, speed, and ease of use. It is optimized for use as a massive job queue for multiple work flows. One neat feature is the ability to create multiple queues on the fly and add jobs to multiple queues; retrieved jobs are removed from all queues in which they appear.

This is a fairly new project and there is not much code yet written, but if anyone has any suggestions, comments, or contributions, I am happy to hear them.

Submit article

The official CGI module has some real issues. First, it combines POST and GET variables into a single structure. This has two serious consequences: 1) the application has no way to determine the method by which a parameter is passed (that information is completely lost), and 2) name clashes between GET and POST result in the loss of one or the other parameter (in the case of the CGI module, GET information would be overwritten.)

Another issue in the CGI module is with the put-page function, which breaks if “%>” is used inside of a code island, even legitimately, such as in a string.

Web fixes both of these problems and provides a number of other features, including:

• ASP/PHP-style templates
• Getting/setting cookies, GET, and POST parameters
• Entity encoding and decoding
• HTTP header control
• Sessions
• Custom session storage
• URL building and parsing
• URL encoding and decoding
• Query string building and parsing

Additionally, Web does not suffer from the GET/POST issues that the CGI module does, nor does it mishandle tags inside of code.

HTTP Headers

Headers are set using Web:header, which accepts two parameters – the header name and the header value. By default, one header is already set: Content-type: text/html. Headers are output using Web:send-headers, which is called before any other output. The convenience function Web:redir redirects the browser to the passed URL.

GET and POST

GET and POST variables are accessed using Web:get and Web:post, which may be called in two ways. When called with a single parameter, these functions act like lookup and return the GET/POST parameter with the named key. When called with no parameters, they return an association list of the corresponding query.

Cookies

Cookies must be set before headers are sent. They are set using the Web:cookie function, which accepts up to six parameters.

;; Set a basic cookie
(cookie "foo" "bar")
;; Access a cookie value
(cookie "foo")
;; Delete a cookie by setting expires to now
(cookie "foo" nil (date-value))
;; Set a cookie that expires in an hour
(cookie "foo" "bar" (+ (date-value) (* 60 60)))

See the documentation for a full list of accepted parameters.

Sessions

By default, sessions use file-based storage (located at /tmp). They are controlled with a few simple functions. The default exit function is wrapped to ensure that Web:close-session is called at the end of a script (at least, a script that calls exit at its end.)

(Web:open-session)
(Web:session "foo" "bar") ; store "foo" as "bar"
(Web:send-headers) ; start output
(println "<p><strong>foo is:</strong> " (Web:session "foo") "</p>")
(exit 0)

It is a simple matter to design and use custom storage handlers. The function Web:define-session-handlers allows customization of which functions are called to begin/load, close/write, delete a session, and clear old sessions. See the documentation for a list of helpful functions and variables for custom storage handlers.

Templates

Templates work almost identically to the official CGI module, with a few differences. First, Web:eval-template is called with a string, rather than a file, to permit other storage methods and programmatic building of templates. Second, <%= .. %> may be substituted for the default opening tag as a shortcut for <% (print … ) %>. Last, the opening and closing tags may be customized by setting values of Web:OPEN_TAG and Web:CLOSE_TAG. The shortcut tag will always be Web:OPEN_TAG appended with and equal sign.

Encoding and decoding

There are several functions to make encoding, decoding, and escaping strings easier for dealing with URLs, javascript, and HTML entities.

Web:escape takes a string and encodes the basic HTML character entities (apostrophe, quote, ampersand, and left and right angle brackets.) Web:unescape provides the reverse. The function Web:escape-js does no encoding of entities, but instead ensures that a string may be safely output in javascript string without causing syntax errors.

Web:encode-entities encodes all HTML entities, including a number of entities that are not fully supported by all browser. The full list of entities is derived from Wikipedia. Its reverse, Web:decode-entities, translates entities back into their character equivalents.

The functions Web:url-encode and Web:url-decode deal with hex-encoding/decoding strings for use in URLs.

Query strings are easily created using Web:build-query, modeled after the PHP function http_build_query. Its counterpart, Web:parse-query, takes a query string and turns it into an association list.

For URLs, it is simpler to use Web:build-url, which takes a URL and any number of association lists which it uses to build a complete URL. Each alist may overwrite parameters from the previous, including any parameters on the passed URL.

(let ((url "http://www.artfulcode.net/?s=newlisp"))
  (println (Web:build-url url '(("s" "newlisp web module") ("foo" "bar")))))
;; => http://www.artfulcode.net/?s=newlisp+web+module&foo=bar

It also has a counterpart, Web:parse-url, which breaks a URL up into an association list of its component parts.

Download

You can find it in the repository or download it directly here.

Submit article

Forms are created by extending the abstract Phorm class. Validation is simple and easy to implement. Django users especially will find Phorms’ validation familiar:

require_once('lib/phorms/phorms.php');
 
function required($value)
{
  if ($value == '' || is_null($value))
    throw new ValidationError('This field is required.');
}
 
class CommentForm extends Phorm
{
  protected function define_fields()
  {
    // Define form fields
    $this->post_id = new HiddenField(255);
    $this->first_name = new TextField("First name", 25, 255, array('required'));
    $this->last_name = new TextField("Last name", 25, 255, array('required'));
    $this->email = new EmailField("Email address", 25, 255, array('required'));
    $this->url = new URLField("Home page", 25, 255);
    $this->number = new IntegerField("Favorite number", 7, array('required'));
    $this->message = new LargeTextField('Message', 5, 40, array('required'));
    $this->notify = new BooleanField('Reply notification');
 
    // Add some help text
    $this->notify->set_help_text('Email me when my comment receives a response.');
    $this->email->set_help_text('We will never give out your email address.');
  }
}

Setting default values is simple:

$form = new CommentForm(Phorm::POST, true, array('post_id'=>42, 'notify'=>true));

And form validation is a breeze:

$valid = $form->is_valid();

Phorm instances serialize to an HTML table and provide methods to open and close the form:

echo $form->open($_SERVER['PHP_SELF']);
echo $form;
echo $form->close();

The Phorms library:

• supports file and image uploads
• has easy to use validation
• is extensible by subclassing the build-in PhormField and PhormWidget classes
• is completely documented with phpdoc

You can download it from my projects page or directly.

Update: fixed a couple bugs and put API docs online.

Submit article

There are not a huge number of features at the moment, but here is the list:

• install modules from multiple repositories
• install/uninstall module dependencies
• automatic upgrade of modules
• newlisp version checking
• file clash checking

Basic setup

Download the archive nl-install from http://static.artfulcode.net/nl-install/. Extract and edit the nl-install file to set up your target installation directory (/usr/share/newlips/site-lisp by default) and any (future) extra repositories you would like to add. Make the file executable and place in your path, then update:

chmod +x nl-install
sudo cp nl-install /usr/local/bin
sudo nl-install update

Usage

To see the list of all available modules or just the ones installed:

nl-install list available
nl-install list installed

To install one or more modules:

sudo nl-install install acode:csv
sudo nl-install install acode:matching acode:util

If there are other modules that must be installed first, you will be prompted for their installation.

To uninstall a module:

sudo nl-install uninstall acode:csv

Re-install a module to get the latest version.

Repositories

Setting up your own repository is easy. Just write a package listing and put it online somewhere. A sample is included in the tarball:

(repository
  (title "My repository")
  (identifier "foo") ; once chosen, this should *not* change
  (contact "My Name ")
  (version 1000) ; repository version number (1.0)
 
  (module
    (title "module1")
    (description "A short description of module1")
    (url "http://location/of/module1.lsp")
    (qwerty "http://location/of/module1.qwerty")
    (version 1010) ; 1.1
    (newlisp-min-version 10000) ; 10
    (newlisp-max-version 10001) ; 10.0.1
    (depends nil)) ; no dependencies
 
  (module
    (title "module2")
    (description "A short description of module2")
    (url "http://location/of/module2.lsp")
    (qwerty "http://location/of/module2.qwerty")
    (version 2011) ; 2.1.1
    (newlisp-min-version 10000) ; 10
    (newlisp-max-version 10001) ; 10.0.1
    (depends "foo:module1")))

Let me know about it and I will put it in the repository list.

Submit article

The module came out of my original locks module, which contained only the Lock and RLock classes. The new library contains the following classes:

• Semaphore: simplifies use of semaphores
• Shared: simplifies use of shared memory
• Synchronized: shared memory made safe for use between processes
• Lock: a binary semaphore, or mutex
• RLock: a recursive Lock
• Event: a simple mechanism to signal multiple processes
• Pipe: simplifies use of pipes
• Channel: a two-way communications channel made with pipes
• Queue: a synchronized first in, first out class

MP additionally includes various utilities to deal with common tasks:

• get-pid: gets the current processes’ pid
• with-lock-held: a macro that evaluates its body while holding a lock
• wait: waits for a condition with variable length polling to reduce processor load
• map and iter: versions of map and dolist evaluated asynchronously using the Cilk API

Perhaps the most useful function in the module is with-lock-held, which simplifies the most common locking case: assuring that a block of code is executed atomically.

(setf lock (RLock))
(with-lock-held lock
  (access-restricted-resource))

In this case, no more than one process can evaluate (access-restricted-resource) at a time.

You can download the MP module from my newLisp module repository.

Submit article

The Mysql module differs from the distribution standard module in several important ways. Most obviously, it uses FOOP wrappers for MySQL types. It also requires clients to free results instances; in the standard module, only the base MYSQL instance itself must be freed (using MySQL:close-db).

The significance of this is that it is much simpler to create multiple connections (without having to duplicate the entire context at compile time). Result sets are completely independent of each other, and several may be maintained in any state at once. This also means that a spawned process may be given its own Mysql instance to use without having to worry about other processes’ instances interfering. Using the standard module, the entire context would need to be cloned at compile time and given a static symbol reference (e.g., (new 'MySQL 'db)) in order to run multiple instances or connections to a server.

Moreover, because this module uses unpack and MySQL C API accessor functions, there is no need for the client to calculate member offsets in MySQL compound types. So long as newLisp was compiled for the same target as the libmysqlclient library (both are 32 bit or both are 64 bit), everything should work out of the box. Additionally, MySQL errors are now checked in the connect and query functions and re-thrown as interpreter errors. Instead of checking for nil returns and a using MySQL:error to get the error message, standard error  handling with the catch function may be used.

Several convenience functions and macros have been defined in the mysql context for common operations, including connecting to the database and iterating over results without having to worry about catching errors and managing memory. SQL statements may be passed as strings or a list containing a format string and its parameters, which will type-checked and automatically be escaped as needed.

(mysql:on-connect '("localhost" "user" "secret" "my_database")
  (lambda (db err)
    (if err
      (println "Error! " err)
      (mysql:row-iter db "SELECT * FROM some_table" true
        (lambda (row)
          (println row))))))

This module has been tested with MySQL version 5 and 5.1 and newLisp version 10.0.1. It requires newLisp 10.0 or later. You can download it from the Artful Code newLisp module repository.

Submit article