So templating is one of those other subjects that can lead to a bar brawl at a developer conference. The mechanism I’m discussing here is something I have proposed be added to the DNN framework, but on both occasions that I presented it, it led to very passionate discussions with two sides dug in deep hurling verbal grenades at the other. Boy. I thought it’d be a pretty quick and boring discussion with hopefully a quick acceptance. Instead I’ve had to retract it and now it remains implemented in a few modules of mine (Document Exchange, Yet Another Gallery, and the new DNN Blog module).
Goals
So why do we need templating? Well, the overarching goal is to make it easy for “someone” to adjust the “look and feel” of your module to suit their needs. There is no controversy here. We deliver modules as generic solutions and I have no illusion that it will fit each and every situation. So it pays to have something in place that allows the module to be adjusted. Now, the reason I added quotes to the statement just before, is because the controversy unfolds when we look more closely at these aspects: for whom and how adjustable?
For whom?
There are, in my opinion, two things to consider here. First we must consider security. Templating solutions are akin to programming in that you are telling the server how to display stuff on the page. This is basically what the whole web application is all about. So in some respects you’ve entered the domain of web programming. And one thing we need to consider when we offer someone the ability to “roll their own” is the potential of screwing things up and/or getting at sensitive data. So in terms of screwing things up we are assuming no malicious intent on behalf of the “programmer”, but just his/her lack of understanding of the mechanism. I think we have an imperative to “protect users from themselves” as module programmers, so in this case we must have a vision of who will be doing templates and if the person with that level of security clearance is allowed to screw things up. In DNN we distinguish between host (Superuser) and admin, for instance. The host is supposed to know what he/she is doing and has more access to things that can screw up the system. The admin is more protected. Users with “edit” level access to modules are supposed to be even more protected.
This leads to the second aspect of the target audience: Technical ability. We make tacit assumptions about the people working with our modules. As mentioned above there is a ladder of technical ability starting at the user with view permission, to the users with edit permission, to admin, to host. We assume that the end users have little or no technical background and we assume that the host user knows what a connection string is, for instance. So when we’re looking at a templating solution we must connect the technical ability of the user that we’re targeting with the solution.
What should be “templateable”?
This is also not cast in stone. The baseline is obviously that you have some control over the HTML that is being pumped out from the server. And that we can have bits of information from objects (i.e. from the database) be injected into that HTML stream. But the next level would allow us to add interaction. I.e. buttons, forms, etc. We must realize, however, that adding interactive elements potentially introduces “holes” in our application that the ignorant or the malicious may traverse. Look at it this way: if your templating solution just displays data and has no API entry points to manipulate data, there’s not much anyone can do except make a mess of the presentation. As soon as you start opening up to data manipulation you really need to keep an eye on security.
A (very) brief history of templating in DNN
Let’s look at where we’ve come from with DotNetNuke. I remember Joe exclaimed during one of our debate sessions: “we already have a templating solution and it’s called asp.net”. And of course he’s completely right. Asp.net (and asp before it) are in essence templating solutions. With classic asp you were scripting a page and quite literally steering the HTML output. Asp.net added the notion of “controls” and the event driven model of web forms. This is the foundation of DNN. And Joe is quite right in pointing out that we already have this. But we must keep in mind that asp.net does allow us to do *anything* we want. I.e. we can add “script runat=server” blocks and basically do anything we wish to the DNN installation. The limits are exactly the same as for me as module programmer. And this is, of course, where it will differentiate with other solutions.
Then Token Replace came along. Token replace is a mechanism that was added to the DNN core back in 2007 by the Germans (Sebastian Leupold, Stefan Cullman among others). It standardized a way of making HTML templates with “tokens” that, at runtime, would be replaced with data. So “<b>Hello </b>” would print “Hello Peter” in bold if I were logged in. Having this mechanism in the core meant that modules could leverage the mechanism by implementing an interface on their objects (IPropertyAccess) which allowed the templating engine to retrieve data values. The typical use case was for (1) display of module content and (2) emails. Token replace has remained mostly untouched since its introduction in 2007. As far as API stability it concerned, they don’t come more stable than this.
Finally a word or two about Razor, knockout js, and more recent developments. I am not an expert on the full scala of .net programming paradigms, but I have seen them appear on the scene in DNN over the years. Those familiar with the framework will know that the newer modules (Journal, MemberDirectory) in DNN are programmed using this mechanism. It also qualifies as a templating system even though the intent of these modules is maybe not necessarily to allow people to change their appearance. This mechanism is targeted more as a replacement for web forms and is heavily influenced by MVC, the web programming paradigm that has come in vogue about 4-5 years ago and is now seen by many as “best practice”.
The direction I took
So who was I targeting with my templating solution? I feel very strongly about the need for a templating solution that targets users at “admin” level that have a basic understanding of HTML, are not daunted by seeing an HTML blurb, but do not necessarily have a programming background. It is one step up from being able to edit a text in Word. Knowing that “<p>Lorum Ipsum</p>” creates a paragraph of text. I also feel that at the very least the templating solution should cater for “mildly advanced” templating scenarios, meaning I want to be able to display data as I wish and use repetition and conditionals. Repetition comes down to “display this set of (uniform) data using that template”. Think of a table of data where each row is an element of data. Conditionals are the “if .. then” constructs. Testing the value of a field and then optionally rendering some HTML blurb. E.g. *if* the user is logged in *then* display “hello “ plus the user’s first name.
As you can probably deduce by now, token replace is still closest to what I’m looking for. All other templating solutions either grant way too much access and/or are too technical to explain to someone without any programming experience. But you must realize that this relates to the criteria. I’m not saying the other templating solutions are useless, just that their use case is different.
Plus: Token replace is incredibly robust. If the token is wrong then the site does not crash. It will just display the non-replaced token. Or a space or a broken token. But your HTML remains in place and you can quickly analyze where things went wrong. The fact that square brackets are used, reduces the amount of HTML errors that can be introduced and improves legibility as HTML and tokens can be well distinguished. In all, token replace makes it easier to provide a mechanism that “protects users from themselves”. But token replace has its limitations.
Extending Token Replace
The current token replace feature in DNN is great but lacks support for repetition and conditionals. It’s really just one template and a couple of objects fed into the engine and HTML comes out. This is where I asked myself: could token replace be taught to do repetition and conditionals? This is where we’re going to examine how token replace actually works. At the heart of token replace is a simple regex replace. It checks to find combinations. It then looks up the object from the list of objects it has been fed. Once found it calls the GetProperty method on the object with the property name and the format string. This is not rocket science, and it has worked very well over the years. Here’s the central “ReplaceTokens” method from BaseTokeReplace.cs.
protected virtual string ReplaceTokens(string strSourceText)
{
if (strSourceText == null)
{
return string.Empty;
}
var Result = new StringBuilder();
foreach (Match currentMatch in TokenizerRegex.Matches(strSourceText))
{
string strObjectName = currentMatch.Result("${object}");
if (!String.IsNullOrEmpty(strObjectName))
{
if (strObjectName == "[")
{
strObjectName = ObjectLessToken;
}
string strPropertyName = currentMatch.Result("${property}");
string strFormat = currentMatch.Result("${format}");
string strIfEmptyReplacment = currentMatch.Result("${ifEmpty}");
string strConversion = replacedTokenValue(strObjectName, strPropertyName, strFormat);
if (!String.IsNullOrEmpty(strIfEmptyReplacment) && String.IsNullOrEmpty(strConversion))
{
strConversion = strIfEmptyReplacment;
}
Result.Append(strConversion);
}
else
{
Result.Append(currentMatch.Result("${text}"));
}
}
return Result.ToString();
}
Now. Where could be intervene? The approach I’ve taken is to stay with the regex mechanism and add new kinds of tokens. Tokens that depart from the pattern and that can be picked up by a new regex that will do the repetition magic. Specifically I’ve opted for a pattern where the token begins with “[subtemplate|”. This indicates to the templating engine that what follows defines a set of conditions, properties, etc for a sub template. What is a subtemplate? Well, it’s an HTML blurb just like the main template blurb. And it gets used as many times as needed and is injected at the place where the subtemplate token was put in the main template. The beauty of this solution is that this can be done indefinitely and that it keeps things separate. I.e. I can clearly see the HTML blurb that will be repeated, vs that which will appear only once.
What does this look like? Well, concretely a repetition token could look like this:
This tells the engine to repeat template “Blog.html” over the collection “Blogs” with a parameters “pagesize=5”. The regex bit to capture this is trivial. What is more interesting is how we get the data. Given the fact that we’re setting this up to be recursive/nestable, we need to come up with some model to retrieve the “Blogs” collection. It does this by raising an event which gets captured by the control hosting the template. The template class has an event called GetData:
Public Event GetData(DataSource As String, Parameters As Dictionary(Of String, String), ByRef Replacers As List(Of GenericTokenReplace), ByRef Arguments As List(Of String()), callingObject As Object)
And it calls this event itself whenever its raised by one of its children. The hosting control (in the blog module v 6.0.0, this is the Blog.ascx) will contain the code to capture this event and hand back what is necessary to the engine:
Private Sub vtContents_GetData(DataSource As String, Parameters As Dictionary(Of String, String), ByRef Replacers As System.Collections.Generic.List(Of GenericTokenReplace), ByRef Arguments As System.Collections.Generic.List(Of String()), callingObject As Object) Handles vtContents.GetData
Select Case DataSource.ToLower
Case "blogs"
Dim blogList As IEnumerable(Of BlogInfo) = BlogsController.GetBlogsByModule(Settings.ModuleId, UserId, BlogContext.Locale).Values.Where(Function(b)
Return b.Published = True
End Function).OrderBy(Function(b) b.Title)
Parameters.ReadValue("pagesize", _pageSize)
If _pageSize > 0 Then
_usePaging = True
Dim startRec As Integer = ((_reqPage - 1) * _pageSize) + 1
Dim endRec As Integer = _reqPage * _pageSize
Dim i As Integer = 1
For Each b As BlogInfo In blogList
If i >= startRec And i <= endRec Then
Replacers.Add(New BlogTokenReplace(Me, b))
End If
i += 1
Next
Else
For Each b As BlogInfo In blogList
Replacers.Add(New BlogTokenReplace(Me, b))
Next
End If
...
As you can see the “DataSource” parameter is responsible for telling us what needs to be retrieved. The hosting control will define a whole bunch of these collections that the engine can iterate through. In the case of the blog module you’d have “blogs” for a list of blogs of this module, “posts” for a list of posts, “categories”, “tags”, “comments”, “authors”, etc. And some variations: “allcategories”, for instance, will list all categories regardless of the selected blog, whereas “categories” will limit itself to the current selection. Note this kind of fine tuning could also have been done using the parameters. In the end, the goal is to make these templates as intuitive as possible to those that will work with them.
The parameters is a dictionary of name/value pairs extracted from the last bit of the token. The replacers are a group of TokenReplace objects. This is more than a single token replace object. It depends on your application. Here I add not just the blog into the token replace, but also things like the user that is the owner of the blog. That allows us to access several objects in the template. The arguments are used to pass back values into the token replace that can arise from processing. A common use case for this is the “page is current” that you may need for a pager. This actually comes from the context and is determined by the code in GetData. The data from this resurfaces as the “custom” object in the template. Finally, the callingObject is the object that is the “parent” that made the call. If you’re getting the categories belonging to a post, then the callingObject is the post.
The above takes care of pretty much any repetition scenario. It is even possible to implement paging and not have to retrieve every record from the database. Now let’s look at conditionals. We can apply the subtemplate technique for conditional processing as well.
Here the template PostDetails.html should be loaded if the value for “postselected” from the “query” object evaluates to “True”. Again, the regex to parse this is not rocket science. Before conditional template generation one might be tempted to use regular token replace to hide data from view using css or javascript. But that technique is only useful if the information is public. If you want to really hide bits from certain users, you can’t send it over as html and then hide it with css. The data will still be in the page. Instead, the processing needs to happen at the server and the data must really not go over the wire. Conditional subtemplates do this. The second big benefit here in comparison with regular Token Replace, is that it avoids needless processing. A conditional template will not be loaded for processing if the condition fails.
Because conditional subtemplates can lead to an explosion of the number of templates you’re going to manage, I did include a mechanism that you see in various Token Replace enhancements. This is to do the conditional “inline” in the template. E.g.
Here the span will be rendered if query:t equals “1”. Although it adds some flexibility, I believe legibility is greatly reduced and it should only be used in small snippets IMO. I’ve added the 1 to “if|1” to help avoid nesting errors.
Wrapping this up into an engine
Having extended Token Replace, the next step is to wrap this up into several useful classes and design a pattern in which to store the templates. The final set of classes is larger than you probably would think:
A number of classes (TokenReplace, BaseCustomTokenReplace, BaseTokenReplace) are copies of the core Token Replace. This was done because the regex used by the core’s token replace was too hungry. This would require a whole new bog post to discuss, so I’m not going in to that right here. Other than the regex statement, the classes are identical to the core token replace engine.
TemplateController and TemplateManager
The TemplateController class contains generic methods for the engine like loading the template file. The TemplateManager is designed to make sure all paths are set correctly based on a single setting of which template to use. One challenge is to relate the absolute and relative paths. The engine needs both the absolute paths to load files (e.g. the templates) and relative paths to send to the browser (i.e. paths to additional css files). A complete template would look something like this:
And these templates can be either “system” templates (i.e. rolled out with the module and not meant to be edited) or “local” (in the portal home directory). This still needs to be extended with “host level” templates (i.e. in the “portals/_default” folder).
ViewTemplate and Template
So drilling down from the displaying control (Blog.ascx), the first thing we find is the ViewTemplate control. The control is initialized with properties like where the template is found on the server’s disk. It will look for “Template.html” as its default entry point and begin processing from there. It is also responsible for telling the page renderer to include any css or javascript files that are part of the template.
The ViewTemplate control instantiates a Template object. This is the heart of the engine. It is responsible for parsing and adding subtemplates. Both this class and the ViewTemplate control pass through the GetData event so it can be caught by the containing control.
GenericTokeReplace and BlogTokenReplace
These classes stack up (BlogTokenReplace inherits from GenericTokenReplace) to provide a layer over the core token replace logic to add the specific objects used in Blog’s token replace. The GenericTokenReplace class takes care of adding the “custom” object and the resources file (“resx” object). BlogTokenReplace does the adding of blog, post etc objects. E.g.
Public Sub New(actModule As DotNetNuke.Entities.Modules.ModuleInfo, security As Modules.Blog.Security.ContextSecurity, blog As BlogInfo, Post As PostInfo, settings As Common.ModuleSettings, viewSettings As ViewSettings, comment As CommentInfo)
MyBase.new(Scope.DefaultSettings)
Me.PrimaryObject = comment
Me.ModuleInfo = actModule
Me.UseObjectLessExpression = False
Me.PropertySource("security") = security
Me.PropertySource("settings") = settings
Me.PropertySource("viewsettings") = viewSettings
If Post IsNot Nothing Then
Me.PropertySource("post") = Post
Me.PropertySource("author") = New LazyLoadingUser(PortalSettings.PortalId, Post.CreatedByUserID, Post.Username)
Me.PropertySource("blog") = Post.Blog
Me.PropertySource("owner") = New LazyLoadingUser(PortalSettings.PortalId, Post.Blog.OwnerUserId, Post.Blog.Username)
ElseIf blog IsNot Nothing Then
Me.PropertySource("blog") = blog
Me.PropertySource("owner") = New LazyLoadingUser(PortalSettings.PortalId, blog.OwnerUserId, blog.Username)
End If
Me.PropertySource("comment") = comment
Me.PropertySource("commenter") = New LazyLoadingUser(PortalSettings.PortalId, comment.CreatedByUserID, comment.Username)
End Sub
Here the token replace engine gets primed with several objects from the context.
CustomParameters, Resources, LazyLoadingUser and TemplateRepeaterItem
These classes are helpers in the token replace engine. I.e. they implement IPropertyAccess and provide us with objects we can use in the template. Resources is a resource file (providing fallback as in regular DNN usage). The LazyLoadingUser takes care of UserInfo objects. It does two things. First it implements IPropertyAccess to the UserInfo and its Profile in one go. Secondly it waits to load the userinfo until it actually needs it. The module has many objects that relate to a user. A blog has an owner, a post has an author, etc. And it’d be a waste if we’d retrieve these userinfo objects for every single item being displayed over and over again. When most of the time we’re not even going to use it. So instead I’ve created this class which will load the user once the GetProperty is hit for a property it doesn’t already have.
The TemplateRepeaterItem provides an item to relate the current item in a list to the list. I.e. IsFirst, IsLast etc.
TemplateSettings
There is a way to hook in specific template settings. You can see this in action in the Yet Another Gallery module. The goal of that module is to provide as many nice jQuery image display components to a user. Many of these jQuery components have settings (e.g. speed of transition). Since these settings, for the most part, fall into the category “string”, “int”, “bool” etc. it is not hard to create a little editor that will take the description of the settings and can then be accessed through a “templatesettings” object in the template. Currently it’s not used in the Blog module, but it could be in upcoming versions.
Results
Here are a couple of screenshots of what I’ve been able to do with this templating technique which both show the same data set but in complete different renderings:
The solution is not yet completely finished. One thing still lacking is a template editor. Currently, you can set up your own templates under “Portals/[id]/Blog/Templates/” and they will appear in the dropdown for template selection in the module. Using the existing templates as an example you can create your own templates with any text editor.
Critique
Obviously this is not the templating mechanism that will “rule them all”. I’m confident it merits its own place in templating, but I want to touch on some comments I’ve received on this technique.
Why not use XML?
The square bracket syntax is somewhat arbitrary (it was chosen for token replace and I’ve stuck with it). Shifting to XML processing means you get a clear nested syntax that can be natively interpreted by .net. The passing of parameters and other arguments becomes a lot simpler as you’d use attributes and elements to do this. And I can most definitely see the advantage here. In fact, I’ve hesitated a very long time whether or not to go down that path.
The main reason I did not choose to move to XML is the level of competence I was requiring of the template creator. HTML, as most of you know, is cr*ppy XML (HTML is actually derived from SGML, not XML). In HTML you don’t need to close certain tags. You don’t need to put quotes around attribute values, etc. I hate this fact, but I have to live with it. I’m a big fan of XML and I think that the fact that HTML 5 did not opt to go XML is just a missed opportunity. But such is life.
With regard the XML templates it would mean that you couldn’t rely on the fact that the XML template would actually be valid XML and hence you wouldn’t be able to use the .net XML processing engine (it would throw exceptions even trying to load the templates). For me, this pretty much removed the biggest advantage of using XML. It means that I’d still needed to build a regex parsing engine to get to the right XML bits. And this engine would be way more difficult to write because you’d need to find XML nesting despite the fact that it is embedded in SGML. This is a gargantuan task.
I have too many templates to keep track of. Why not do everything in one file?
This is a matter of taste. I don’t really mind the fact that I’m jumping between files. Yes, there are moments when you can get a bit lost (for the default template in the blog module I have 20 template html files). But having everything in the same file means you need to keep track of nesting visually yourself. Having end users take care of nesting, leaves them much more open to errors than subtemplates, in my opinion.
Conclusions
All in all I’ve been able to cater for quite a few scenarios with this templating technique. It has its drawbacks, but I think it is the best to date given the goals I set myself. That is: accessible/comprehensible to “admin” level users, robust, and safe. Feel free to use this code or enhance it. If you feel part of this mechanism should find its way into the core, then make yourself heard in DNN’s Community Voice.