Javascript Optimization Programming Projects startup web-design

Javascript Element Creator

Some time ago I was working on optimizing the client side code of my website,, an online trip planner.
This website does automatic trip planning, and the problem was that recalculating trips was slow. After profiling, I found out that most of the time wasn’t actually taken up by the algorithm, but by the UI. Rendering the trip to html was the costly part. The process was like so:

Client-side Javascript code generates new trip prefs -> application calculates new trip -> Client-side Javascript gets the new trip, and creates new html.

It’s important to note that the app is “ajax based”, so the actual trip html was generated by the Javascript code, and not the server. At the time I was using Mochikit to generate the new html. Mochikit has a pretty nifty API for generating html, but it’s quite a bit on the slow side. Basically, this API is a wrapper around createElement.

Well, first I did a little test, and found out that generating html with cloneNode and innerHTML is much faster than createElement. Still, there was a problem – I needed to generate many similar elements – similar but not identical. Consider entries on a trip itinerary – they all look the same, yet each one has a different name, a different time string, and a different onclick event.

What I needed was a Javascript based html template library. My requirements:
1. Speed. Html had to be generated quickly.
2. Expressiveness. It had to be able to create pretty arbitrary html with a given context. For example, an anchor element (<a> tag) with a given href property, and a given text content.
3. References to inner elements: Many elements inside the generated html need various events attached to them, or various code processing. This should be easy to achieve.
4. The library has to allow the template html to be written as html, and not only as javascript strings.

So, I sat down with Benny, and we wrote the Javascript Element Creator, which we are now releasing under the BSD license. I originally wrote it to work with Mochikit and the Sizzle library, and Benny changed his version to worked with jquery.

After adding the code to my project, I got two things: first, everything worked much, much faster. Second, it was much easier changing the generated html when it was generated according to a template, and not directly in code.


1. Write your html somewhere visible to the javascript code. Add the “template” class to the upper node, and the id will be the name of the template. For example:

    <div id="some_div" class="template">

2. Similarly to other template engines, add double brackets to signify where text should be inserted:

    <div id="some_div" class="template">
        <a href="[[link_url]]">[[link_text]]</a>

3. Create a creator object. It will “collect” your template, and will make it available to your code.

var creator = new ElementCreator();

4. Generate your DOM object, and add it to the document;

var obj = creator.generate("some_div",
                           {link_url: '/url/',
                            link_text: 'hello world'});
appendChildNodes(foo, obj);

The code

We decided to publish for now only the jquery version. I might publish the mochikit version as well at a later date. Since Benny wrote the jquery version, he also wrote the tests for that version.

All in all, the final code is pretty short, and could probably be even shorter. Still, it’s good enough, and gave me a very real performance boost.

Here is the code, have fun with it.


My Startup –

Plnnr logoI’ve been waiting for this blog post for more than a year now.
This is my startup, If you want, go there first, form an opinion, and then get back here, I’ll wait :)
As you can tell, is an automatic trip planner – it saves you a lot of the trouble and hard work of planning a trip, and leaves you to decide just on the interesting things – what you plan to do on your trip.

I’ve been working on this with my partner, a product manager, for about 15 months now. We opened up the website for public access on July 18. Since then we showed it to people to get feedback but we didn’t publicize it.
Now, we’re starting to publicize it “for real”, with the goal of getting people to really use it.

We’ve been working on this project with no funding all this time, as our part time job, and as a result – there’s still a lot of work to be done. We have a lot of amazing features planned, and we have a great vision ahead of us, and we hope that we can achieve it.

This is the point where you come in – tell us what you think – here, or on our blog, and soon on the feedback system. We need your help!
Of course, if you find it in your heart to tell your friends who are planning a trip about our website, please do :)

There’s a lot more to be said about this project, and I intend to say it in future blog posts, so stay tuned!

Databases Design Programming startup

Actual Data Always Needs To Be Explicit

This might seem obvious, but it wasn’t to me it first. Now I consider it a database design rule of thumb, or even a patten.
I’ll explain using an example. Consider an application where you also need automatic tagging of text. (As in generating keywords.) So you’ll have a table for objects that have textual fields (for instance, blog posts), and a table for tags.
Now, you would need a many-to-many mapping between these two tables. Various ORMs might do this automatically for you, or you might add a PostTag table yourself, with foreign keys to the other tables.

You think this might be enough, as your smart tagging algorithm can add tags and attach tags to blog posts. If you want to change it manually, then no problem, you just modify any of these tables. For example, if the algorithm makes a mistake, you just erase the mapping and/or the tag.

The problems start when you want to run the algorithm more than once.
First, the algorithm must not create duplicates on the second run. This is very easy to implement and doesn’t require any change to the DB. Now, let’s say that a taggable object (our blog post) has changed, and we want to update the tags accordingly. We might want to erase all the mappings we created for this object. No problem, also easy to do.

What about manual changes? Should these be erased as well? Probably not, at least not without alerting their creator. So we need to record the source of these mappings in an extra column of the mapping table, and use it to mark manually and algorithmically generated mappings differently.

How about deletions? What if the first time around, the algorithm made a mistake, and added a wrong tag, which was manually removed? Running the algorithm again will cause the tag to be added again. We need some way to mark “negative tags” , which are also pieces of information. The easiest way I found of doing this is adding a boolean “valid” column to the mapping table.

It’s important to note that this also applies to all mapping types and not just to many-to-many. So even when you don’t naturally need a separate table for a mapping, you should consider adding one, if the mapping is part of the actual data you keep. Also, if you need to keep extra data about the mapping itself, for example “relationship type” in a social network or “tag weight” as in our example, you would already have a separate table anyway.

I encountered this issue when I implemented my multiple source db-design. A reminder: I had data collected from various sources, and then combined together to a final merged record. The combining was done automatically.
My mistake was that I only considered the records as pieces of data, and didn’t consider that the actual grouping of raw data records is also part of the information I keep. As such, I should have represented the groupings in a separate table, with the added columns, as I outlined in this blog post.

Javascript Programming startup web-design

Mochikit Drag&Drop Corner Case

I found myself working again on the UI for my startup. As my Javascript library, I use Mochikit. One of the reasons for that is that it’s the Turbogears builtin, and I came to like it. The other is that it’s really easy to create DOM objects with it.

In any case, Mochikit has really easy support for good looking drag&drop. However, as usual, my requirements were strange enough to fall upon the following corner case:
I wanted to add a “tool tip popup” for some text, where I would display pertinent information to said text. To make the tool tip popup thingy work, I used the following css “on mouse over” visibility trick:
.tooltip {
display: none;

.parent_object_class:hover .tooltip {
display: block;

This works beautifully, and with a little bit of positioning, and maybe an event here and there, you can make it appear where you want.

Cue the drag&drop. I wanted to add some drag&drop based slider to that tool tip. Since I wanted to limit the “draggability” of the slider’s selector, I used the snap argument for Mochikit’s Draggable object so that if you move the mouse too far, the dragged selector stays at the limit of a predefined area.
This was all very well, and both of the tricks described worked pretty fine separately, until I tried to put them together.
When dragging and leaving the allowed area for the drag, because of the snap argument, the dragged object stays back, and mouse is no longer over a child element of the original tooltip and tooltipped text. This means that the css trick no longer applies, and the tooltip loses visibility. This would have been fine if the drag ended there. However, the drag was not ended, and at each move of the mouse, the coordinates would grow more. Since I use the drag coordinates to compute the result of the drag, I got some pretty strange results.

To work around this behavior, I used Draggable’s starteffect and endeffect optional arguments to make sure the tooltip remained visible, thus avoiding this issue.

Still, there were many other issues with all this drag&drop going around, and I decided to go for a simpler design, and not put in more time on this.
Issue sealed with a Keep It Simple Stupid.

Databases Design startup

Database Design Problem

A few weeks ago, I had to work out a database design for my startup. I had a bit of a hard time deciding on a design direction, but after thinking about it, I settled on a design I was happy with.

While I was still making up my mind, I discussed the problem with a couple of friends, and to better describe the problem and the proposed solutions I wrote up a short document describing them. I decided to publish this document along with my choice and considerations. Maybe someone else will benefit from my choice, or at least from the alternatives I listed.

Problem description:
We want to to have a table with collected information from various sources.

For example, let’s say we want to collect information about paintings. We’d want to have a database holding for each painting we know about its dimensions, painter, description, link to an image file, etc. Since we collect this information from various sources (maybe harvest information from multiple websites), we would like our application to display each field either from all sources, or from the best source available.
(Note: in my original formulation, being able to display the value from the best source was enough).

CSS Javascript startup web-design

Website development and not supporting Internet Explorer 6

My partner and I started to work on a website a few months ago. We have a working prototype, and we are always improving it. My work is mostly concentrated on a smart Python backend, and on a Javascript front-end, while a thin controller acts as a messenger between the two.
Lately, I’ve worked on improving the UI. As expected, I rely heavily on CSS. I generate a lot of html elements using Mochikit and format them with CSS classes. While obviously better than the old alternatives, I still don’t like CSS. Maybe it’s because I don’t understand it deeply enough, but for me, there is still a lot of voodoo involved. An example I found, which luckily I didn’t run into yet, is collapsing margins.

Still, even with all its voodoo, CSS is bearable. At least until you get to IE. My latest run in with it was a scrolling bug, and I ran into many other issues. However, as much as I complain, I’m probably getting it easy, as when we started work, we decided not to support IE 6, at least until required.
Our reasoning was:

  • Developing for IE6 both independently and consistently with other browsers has a high cost attached to it.
  • IE6’s use rates are declining, and will decline even more by the time we launch (See these statistics for example).
  • Our first versions were mostly required as a prototype to prove our technology to potential investors.
  • As a two-men team, and a one man programming team, we are very low on development resources.

Given my latest bout of UI programming, this choice made me just a little bit happier.