Most popular entries in the last 365 days

Much of my previous JavaScript experience was highly coupled with the PrototypeJS framework. This operates by extends many of the built in prototypes so that you can code beautifully in an object oriented way by chaining. Unfortunately, extending built in prototypes is very bad behaviour, so I have since jumped the jQuery bandwagon, but applying UnderscoreJS for getting chain like helper methods. So far it is less error prone than what PrototypeJS was, and animations actually queue up in a sensible way (something I could never get right for all cases with Prototype's animation framework Scriptaculous - as can be seen by hovering/unhovering the clux.org logo intensely).

Additionally, UnderscoreJS includes a template helper function (source looks identical to this). Which means that I could theoretically use it both client side and server side. I have, however, decided to take it to the next level. I will only render templates client side.

It's remarkable that I can even consider this, but browser progression has indeed now reached levels that mean developers can simply discount users without JavaScript. But while it can be dangerous (one missing comma, and your page could be unusable), with proper testing the benefits far outweigh the dangers.

1. Everything except the first response can be done in low-bandwidth JSON. The first response requires just a little more. Prepare a 10 line, one-size-fits-all template with a script tag that dumps a stringified object right inside. With proper handling this can be used to render it on the client for the first response as well.
2. With a REST interface, you can simplify your code so much that creating a mobile version of your site is a matter of just changing some templates/JavaScript. I use Spine for interacting seamlessly with my backend (because it is essentially a lighter version of BackboneJS).
3. With good use of script loader/minifiers/compilers, you could theoretically serve only one highly compressed file, and load in others pieces of JavaScript as you would need it. Minimizing data transferred and number of HTTP requests. RequireJS combined with UglifyJS seems the way to go.
4. With HTML5 history, you can make function like a normal website where urls are bookmarkable as well. The implication is of course that every url you create must map onto the one you are getting JSON requests from. But this should be trivial. Simply serve the short template described in point 1, and embed the normal JSON response in the script tag. Catching it is as easy as (req.xhr) ? res.send(c) : res.render('layout',{c:c}); with Node.

a few years ago, this would have been horrible, now it's amazing.

I very recently concluded, that as everything was very suboptimal with the Django, and especially the MongoDB usage combo, I should go for something else. This was not a very hard decision, as thankfully most of my code from my Django project was client-side JavaScript. I could turn to newer and hipper python frameworks like Twisted to encroach upon some of the newer web frameworks' performance, but with no asynchronous Python drivers for MongoDB, it seems I would have to learn yet another scripting language - or return to PHP.

Turns out there's another way now. JavaScript.

The reason for why DOM scripting is so good with JavaScript, is that it's completely event driven and asynchronous. One thread running from top to bottom, simply attaching callbacks to events, and with a sensible hierarchy of events so that events bubble up, i.e. a click on a ul > li > h1 will trigger the most specific match first, then eventually any click listeners on li, ul, and body unless cancelled in the innermost matches.

NodeJS maintains exactly this event loop in its JavaScript interface, but uses it to actually implements a socket thread pool underneath (in C), for you to build a web server upon. It generates a lot of buzz these days, and with no thread per connection overhead from Apache, it gives you a remarkably simple way to create a web page with amazing performance. I mean, it's not quite C or Erlang like yet, but for an easy to write scripting language, it is astonishing. Take it from me, who am now actually coding multi-threaded C++ professionally (albeit not very well); this shit is hard, very error-prone, and very slowly progressing. It should only be done if you have plenty of time to do it, or you are an expert. Thankfully, I have plenty of time for it, with concentration inducing Grooveshark playing in the background.

This puts me in a peculiar situation. Everything I do in my own time, is done in JavaScript. NodeJS runs Google's highly efficient V8 JavaScript engine, MongoDB runs Firefox's SpiderMonkey engine, and the result is that to me, everything runs smoothly. The native MongoDB drivers runs problem-free, asynchronously, with the same syntax as the mongo shell. Most importantly, however, the rise of JavaScript compatibility in browsers is now so high that developers can justify complete reliance on it. Goodbye to the days of replicating validation and rendering twice. Better browser support plus universal JavaScript allows for code reuse!

My server code is, at the moment, shorter than what I had in Django, but already I have more functionality. Clearly, the challenges lie elsewhere. NodeJS is easy, and brilliant. That said, perhaps the most useful thing I have learnt recently came as a result of this interactive JavaScript tutorial. Don't miss out.

*waves goodbye to Apache*

One thing you notice pretty quickly with MongoDB is that most object relational/document mappers (ORM) tend to be useless. Django's internal one was attempted patched for NoSQL, but when you cannot even embed documents into others, you have essentially downgraded to a poor-man's SQL. MongoEngine, the most popular ORM for Python immitates Django's model style, and allows you to define base Documents subclasses which can reference (contain) EmbeddedDocument subclasses. It works the way you should expect for simple queries, albeit a bit weird for .save() where when only a partial query dictionary is supplied, it will actually overwrite that document destroying the missing keys.

This can be be bypassed by other methods, and overall MongoEngine works reasonably well. I was able to add queries, alias my table field names to shorter ones to save disk space (but get them back to understandable ones after mapping), but for every embedded document, MongoEngine added 3 extra fields to that corresponding JSON object. From what I gathered, it is some backwards effort to resolve the internal MongoEngine document subclasses from these fields. It worked, but it meant that every element in the base collection, and every sub element which was defined with an EmbeddedDocument, would have 3 useless fields containing the exact same info everywhere. If you have N embedded documents in your base documents, that means 3xN fields of waste per query. That's a fuckload of waste.

Additionally, getting manual connection to do a full-out manual query (definitely needed for advanced sites/optimization) feels extremely hacky. Thankfully, I am not dealing with mediocre Python ORM anymore. If anyone is considering that, they should really use something lighter lest they have to figure out what horrors lurk beneath. But keep in mind that there are no non-blocking MongoDB drivers for Python.

Enter Mongoose. Much less hidden code execution, transparent class like schema's with self-defined getters/setters and validators. No extra fields in your database, almost same syntax as the mongo shell, and manual queries a lot easier to inject. I wish there was more to say, but it just seems to work without side-effects, which is what an ORM's supposed to do. In fact, it's the only thing it's supposed to do. It's a helper library!

tl;dr: MongoEngine bad, Mongoose good.