Samuel Sjöberg's weblog

Skip to navigation

Dynamic lists with data iterators

NTK's CMS that I am currently developing is based on pages, not much different from for example a blog post. These pages are coupled with advertisement, access restrictions and are organized in a dynamic hierarchy that can be rearranged via the CMS.

In addition to these generic pages, other more specific modules are needed. For example, a document archive, a news section and an organization handler is present. But, how should a page be handled that should be a generic page, but also needs to list dynamic content from the database? If a separate module needs to be created for simple listings of data (e.g., all public mailing lists), the modules can soon become unmanageable, cluttering the mod_rewrite rules and resulting in a bunch of simple PHP files that performs the additional SQL queries.

My solution to this is something I call data iterators (in the absence of something more descriptive). The idea is to load an SQL query, coupled with a template, from the database and inject the result into the generic page. To use an iterator, a specific keyword is used. After the result is returned from the query, it replaces the keyword and the content is fed to the user.

This particular description assumes that some kind of template engine is used to separate the business and presentation logic. If you are new to templates, you might want to read this article that discuss the use of templates in PHP. The template class that I will present here is based on the one in the article.

To implement this functionality, the following things must be done.

The iterator template

To be useful, the iterator template should be able to add content before and after the iteration of the query result. At the same time, the template should be kept as simple as possible to make it easy to create new iterators.

As an example, let us specify an iterator named maillists. The query that is used to populate the iterator template is:

SELECT title, description FROM maillists
WHERE type = 'public'
ORDER BY list_name ASC

To keep things as simple as possible, the columns in the result is accessed by their respective name. So, for each iteration (i.e., row in the result) the template should access the title of the maillist with the variable $title. The {BEGIN} and {END} keywords determine the boundaries of the loop. Below is the template used for the query above.

<div class="maillists">

Notice that no PHP start tags are needed in the iterator templates.

Identify and execute iterators

I support multiply types of variable substitution in NTK's system, so I am using an array of keywords that defines the PHP function that should be executed when a keyword is found.

$keywords = array(
   'list' => 'iterator'

Wherever a keyword is found in the content, it should be replaced with the corresponding iterator. Below is a formatted content string with a iterator keyword that exemplifies how keywords are used. When the example is run through the parser, it will replace the keyword {list=maillists} with the maillists iterator.

<h1>Our mailing lists</h1>
<p>To join a mailing list, use the 
form to the right</p>
<p>If you have any questions, please 
contact us</p>

The functions to parse the content for keywords and look up and execute iterators are listed below. They are fairly straightforward for persons familiar with regular expressions and the use of call_user_func in PHP. The hard part is actually to insert the data into the template. This is covered in the next section.

The replaceKeyWords function searches the content for any keywords. For each keyword-value pair that is found, the iterator function is executed once.

 * Replace the keywords in the content with an iterator.
 * @param content the content with iterator keywords
function replaceKeywords(&$content) {
   global $keywords;
   $processed = array();
      $content, $matches);
   list($key, $value) = $matches;
   for ($i = 0, $l = count($key); $i < $l; $i++) {
      $cacheKey = $key[$i] .'='. $value[$i];
      // Only look-up each keyword once.
      if (array_key_exists($key[$i], $keywords)
         && !array_key_exists(
            $cacheKey, $processed)) {
         $data = @call_user_func(
            $keywords[$key[$i]], $value[$i]);
         $processed[$cacheKey] = true;			
         $content = str_replace("{$cacheKey}", $data, 

Below is the iterator function. It first gets the query and template of the iterator, then it executes the SQL query and inserts the result in the template. This example uses PEAR's DB package for database abstraction and assumes there exists an active connection.

 * This function executes the iterator.
 * @param iteratorName the name to look up in the db
 * @return result of iterator as formatted string
function iterator($iteratorName) {
   global $db;	
   $res = $db->query("SELECT query, template 
      FROM iterators 
      WHERE name = '$iteratorName'");
   if (PEAR::isError($res)) die($res->getMessage());
   if (!count($iterator)) 
      return "";
   $res = $db->query($iterator['query']);
   if (PEAR::isError($res)) die($res->getMessage());
   $list = array();
   while ($res->fetchInto($row))
   $list[] = $row;
   $tpl = &new Template($iterator['template']);
   $tpl->add('iterator', $list);
   return $tpl->fetch();

For less obstructive indentations and color-coding, see these examples in source format.

Working with string templates

Now we have a way to find and execute iterators, but the big mystery is still how to insert the result from the iterator's query into the template. The problem is that the template is a string and not a file. When PHP files are used as templates, one can simply use PHP-tags to print a variable. In the same way, loops can be constructed. With a string, that approach will not work. Enters eval...

To view the complete Template class, take a look at the PHP source. The most important function is the db_template that is called to evaluate the template if it turns out to be a string and not a file.

function db_template() {
   preg_match('/{BEGIN}(.*){END}/sm', $this->tpl, $loop);
   $loop = addslashes($loop[1]);
   $data = "";
   foreach ($this->vars['iterator'] as $item) {
      eval("\$data .= \"$loop\";");
   return preg_replace('/{BEGIN}(.*){END}/sm', 
      $data, $this->tpl);

The function above starts by extracting the loop from the template. Then, the content of the loop is escaped to be sure that variables are preserved and that the string is not erroneous on evaluation.

The foreach block loops over each row in the result (stored in the template variable named iterator). In the loop, extract is used to expand each $item, which is an associative array with keys corresponding to the variables used in the template. In each iteration, the $loop template is evaluated with the variables extracted from $item being inserted into the $data variable.

Finally, the extracted loop, together with the keywords, are replaced with the content in the $data variable.

Putting it together

Now, with all the pieces in place, I hope you have enough information to implement your very own data iterators. I hope I managed to explain the concept and that the code excerpts are sufficient enough. I chose to show code-snippets instead of a full-blown implementation because the integration into a specific CMS probably dictates how the implementation can be formed.

I would also like to add that the code-snippets are from a OO solution and re-factored to stand alone functions. This being said, I cannot guarantee that the examples run without parse errors.

Pages linking to this entry

Pingback is enabled on all archived entries. Read more about pingback in the Pingback 1.0 Specification.

About this post

Created 10th February 2006 20:29 CET. Filed under PHP.

0 Pingbacks