Post-Theory

Recently I’v been working on writing some “back end” code for a Tumblr blog that wants to do some non-standard things using Tags.

I’m all about trying to hack the Tumblr platform: hacking with the Tumblr API & doing unorthodox things in Tumblr themes.

The project is called Lensblr (http://lensblr.com)  I’m not directly involved with creating the site. I’m just writing some devious back-end stuff to create added functionality.

The project caught my attention because navigation & view of the Tumblr is done exclusively through use of the /tagged/foobar pages. There is no view of the “main post feed”.  It’s an interesting idea towards creating a custom Tumblr website.

Magical Post Tags

One goal of the designer was to automatically add/remove specific tags from posts, based on set criteria.  In this case, the criteria is rather simple:

• Based on the number of “notes” a post gets, add/remove tags that “move” the post to/from different pages on the Tumblr.

  For example, after a post reaches 50 notes, we want to remove the tag the displays the post on one page, and move it to another page called “Featured Posts”

This is easily implemented using cron job that runs a script to analyze the posts on the blog, and use the Tumblr API to “retag” the posts as needed.

Of course, this kind of tag modification for changing how a site is presented could be extended and extrapolated to do lots of interesting things, based on any number of factors. It’s a promising idea.

Magical Post Shuffling and Randomization

A second goal of the designer behind Lensblr was to implement a way so that posts on a particular /tagged/ Page could be randomly shuffled, such that older posts would sometimes reach the first page.

• The principle is more equal exposure of content, regardless or whether or not that content was posted 2 months ago or an hour ago.
  
  
• Not all content is Timely; and most people never make it past the 1st or 2nd page of a blog.

Fortunately, you can modify the Published Date on posts using the Tumblr API.  And when viewing posts through the standard “Posts” interface (mounted at / ), this works rather well. You are able to shuffle posts around so that older and newer posts get equal chance of exposure on the 1st or 2nd page of the blog.

Unfortunately, Tumblr seems to have a bug/limitation with sorting posts by “Time” on the /tagged/foobar pages:

• If a post is on page 3 of /tagged/foobar, even modifying the post’s Publish Date is not enough to ever bounce it up to page 1.  The post is “stuck” on page 3 forever. (or page 4, 5… etc as more posts are added).

Interestingly, on the individual pages  /tagged/foobar/page/X  the posts *are* sorted chronologically — but only relative to the other posts on that page.

Based on this, I’ve concluded that while Tumblr “sorts by Time” it only does so per page; the actual posts that are put on a given page comes from “sorting by Post ID”.  It’s a bit inconsistent.

So although we had a sound principle and strategy, implemented the code to “shuffle” the posts around — it just doesn’t work within Tumblr’s buggy / limited system.

Working within Tumblr’s Limitations

Tumblr does not provide Developers with many good means for representing blog Content with different “views” or in different ways. Using Tags seems to be the only practical way to achieve these ends, for now.

Things that are not particularly useful:

• The “Static Pages”

  While these can contain arbitrary HTML completely separate from the main “Theme” they are rather useless from the perspective of creating / loading dynamic content, or content that changes with high frequency (say once a day).

  “Pages” are not accessible through the Tumblr API, so making automatic changes to these Pages with code is not possible.

• The Tumblr API

  The API is OK if you are developing an application for the desktop or a mobile device. It lets you do *most* things you might want to do to create a “Tumblr Experience” on a mobile device, or say a “Tumblr Posting / Editing” program on the Desktop.

  Where the API is not particularly useful is in trying to create dynamically generated content on web pages.

  1. Tumblr only supports OAuth1.0 (they should really upgrade to an OAuth2.0 interface)

  2. Tumblr does not support CORS, a technology that allows controlled “cross-site” AJAX calls.  Without CORS, as a developer you are limited to using rather ugly JSONP callbacks, and then only HTTP “GET” commands.

  3. The available methods in the Tumblr API in many cases do not provide sufficient means to request the specific information you want:  first, there are not enough “filters” to select the data you want, no literally no means to control the “output” received from the API.  For example, if all I want is the list of “Tags” from posts on a blog - I still have to download *all that other data* which has no use for me. That adds up rather quickly to a *lot* of transferred data.

     The API needs better methods of “selecting” the types of posts to return, and then needs to implement a way to filter down the data returned to only what you require in order to make the API useful in the context of creating dynamic web pages.

  4. The API Server is just plain slow. Response times I observe are often in the range of 600-800 ms for simple requests (say “grab 50 posts”), and up to 2000-3000 ms for larger requests (say, “grab 200 posts”).
     
     Part of the problem is the before-mentioned lack of ability to “filter down” to just the data you need.
     
     Another part of the problem is that the API Servers refuse to use HTTP Keep-Alive if you will be requesting multiple pieces of data in quick succession.  The API Server should enable Keep-Alive.

So Why Bother?

Because Tumblr is a great social platform. You bother trying to create custom/specialized Tumblr pages to create a unique “Web Experience” while at the same time being able to take advantage of the socialaspects that Tumblr can provide to your Page and your Content.

As a Developer however — this is all rather frustrating.

Anyway, Tumblr just put up a Job Listing for API Lead. Perhaps that is a sign that things will get better in the near future.

But for now, it’s all pretty foobar.

I’m putting up a beta preview of my new Pretty Tag Cloud maker for Tumblr.

Design your own Pretty Tag Cloud: choose your fonts, colors, etc.

Then Save it, share a link, embed it on your Tumblr Theme or in a Tumblr Post.

w00t !

Part 3 in my series on “Tumblr Theme Limitations”

AKA, Why Tumblr’s Theme Template System is Incredibly Lame,
and Hacks You Can Use to Get Around Its Limitations & Lameness

Introduction

In my previous Article, I pondered whether or not URL’s supplied by Tumblr’s Theme Templater were in general safe to include directly in JavaScript strings: i.e., that they had at least been properly Escaped for inclusion inside a Double (or single) quoted String.

I decided that, No: Tumblr is lame. So I should not assume that:

1. URL’s have been properly escaped for inclusion in Strings
   
   
2. URL’s have been properly URL Encoded to replace URL-unsafe characters

But at least there is the {JS…} Variable Transform, which does seem to function properly — so far as I can tell. And it’s useful too — we’ll get to that more in this article.

The Question

This led me to wonder if in general, URL’s supplied by Tumblr’s Theme Templater were even safe to include directly in your HTML — e.g.

<a href="{SomeURLVariable}"> ... 

The Question: Answered

The Answer is NO: You cannot assume that Every URL that Tumblr passes you is Safe to include in your HTML.

Case Study: {LinkURL}

{LinkURL} is the “Click-through Link” rendered in Photo Posts. I also discussed this very same variable last time

Let us suppose that somebody enters a rather naughty “Click-through” URL:

http: //naughty.urls/break” onclick=”javascript: alert(‘Tumblr is lame!’); return false;

Now, you being the good Theme Developer you are, write your Theme Code like this:

<a href="{LinkURL}"><img .... ></a>

Looks fine, right? Wrong. This is what actually gets rendered on the page:

<a href=”http://naughty.urls/break” onclick=”javascript: alert(‘Tumblr is lame’)”><img …. ></a>

Well - you can guess what happens when someone clicks that Image!

Go see for yourself on this page. Really, go try it. There are two examples. One contains a “naughty” URL, and the other a “nice” URL which just happens to include a “ character in it.

The (hacky) JavaScript Solution

Since I’ve already established that the {JS…} Variable Transformation seems to function properly, I’m going to solve this problem with some hacky JavaScript. Once again.

For the sake of brevity, let’s just assume that {LinkURL} exists — I already showed you how to handle those cases when it does not.

And here’s the basic code:

{block:Photo}
    <script>
        document.write('<a href="' + encodeURI( {JSLinkURL} ) + '">');
    </script>
    <img src="{PhotoURL-500}">
    </a>
    ...
{/block:Photo}

Wrapping {JSLinkURL} with the encodeURI JavaScript function makes sure that the link is actually safe to include inside the href=”…”.

You can see now, on this new page, the results.  The “naughty” URL no longer does anything bad, and the “nice” URL works properly.

If {LinkURL} is broken, then what else is Broken?

Honestly, I don’t know yet. I haven’t tried to test every single type of {URL} Variable on Tumblr’s Theme Templater.

Here are things I know think are OK:

1. In {block:ContentSource} Blocks, the {SourceURL} seems to be OK.

2. URLs to pages in your own blog [eg, {Permalink} ] all seem to be OK.

Things I have observed to be broken in the same way as {LinkURL}:

1. Links to a User’s own “Pages”:

   Inside the {block:HasPages} and then {block:Pages}, the {URL} variable is not properly escaped / URL encoded

   What could happen is a user sets up one of their pages with a URL like:

   htp://my-blog.tumblr.com/”About Me”

   That will break, if you try to include the {URL} directly in your HTML.

As I find more bugs, I will update this post to let you know!

In Part 1, I discussed an approach to handling Tumblr Theme Variables that do not have corresponding {block:Foobar} Variables to wrap code to only render when the variable {Foobar} exists.

I used the {PlayCount} variable (rendered in Audio and Video blocks) as my example. In the end, I came up with the following solution:

<script>
    if ( "{PlayCount}" ) {
        document.write("<span>played {PlayCount} times!</span>");
    }
</script>

… which works fine (in this case), even though it is an ugly solution.

But Sometimes Things are More Complicated

Now I would like to talk about the {LinkURL} variable, which is rendered inside of Photo posts. This is the so-called “Click-through URL” for a Photo.

Now, Tumblr has the absolute strangest feature for this variable:  a corresponding {LinkOpenTag} and {LinkCloseTag}, which renders an HTML anchor tag  (<a href=”…… “>) for the {LinkURL} only if it exists.

Well, that might be fine for some people. But what if I want to modify that <a> tag? Add a class name to it? A title attribute? Etc.

Basically - I find {LinkOpenTag} and {LinkCloseTag} completely bizarre and completely out of place with the rest of the Tumblr Theme Template system.

And again - we have the same problem: There is no {block:LinkURL} variable to tell you when {LinkURL} exists.

Trying to Solve the Same Problem the Same Way

So, of course I tried to use the same approach as before:

<script>
   if ( "{LinkURL}" ) {
      document.write('<a class="foo" href="{LinkURL}"> .. </a>');
   }
</script>

And yes, this probably works 99% of the time.

But I don’t really trust a URL can be inserted directly into a Javascript chunk in this way. A URL could possibly break the Javascript String in this example, if it has not been escaped properly.

Does Tumblr escape the {LinkURL} properly? I don’t know — but seeing how many other things they manage to mess up, I’m just going to guess and say No, just to be on the safe side. Maybe it’s escaped fine for HTML, but not JavaScript. Who knows.

So…Tumblr’s Variable Transformations to the Rescue!

Wait - again you ask? I thought you dismissed that in Part 1.

I did dismiss it.

I showed how {JSPlayCount} does not do what the Tumblr Documentation says it should - namely return a Javascript-safe string representation of {PlayCount} wrapped in quotes. [verbatim from the documention]

Instead it renders nothing, and not wrapped in quotes. Fine.

But it Gets Weirder.

But I really want to make sure my {LinkURL} is safe to include in my script.

So I tried playing around with {JSLinkURL}. Now, I figured it would behave like {PlayCount} — namely that if it didn’t exist, then {JSLinkURL} should render nothing.

So I had a stroke of genius:

<script>
    if ( [ {JSLinkURL} ].length) {
        document.write('.....');
    }
</script>

Notice the brackets create a list. Now, an empty list [] is a valid JavaScript expression, so if {JSLinkURL} does not render anything at all, this yields:

if ( [ ].length ) { .... }

Which is valid Javascript, and will return False, if {LinkURL} does not exist.

Problem Solved — Right?

Oh, wrong, wrong again.

I thought it was working.

And then I came upon some curious posts, which for some reason or other, have no {LinkURL}but where {JSLinkURL} actually does render an empty JavaScript string “” !

But then again, it is Tumblr. Sometimes it works, sometimes it doesn’t.

In these weird cases, where there really is no LinkURL, but {JSLinkURL} renders an empty string, you end up with this:

if ( [ '' ].length ) { ... }

Which unfortunately, returns True in Javascript.   @#$!

So, revising the code a little:

if ( [ {JSLinkUrl} ].length && [ {JSLinkUrl} ][0] ) {
    ....
}

This works in both cases. If {JSLinkUrl} renders nothing, it yields:

if ( [].length && [][0] )  { ... }

Which is fine because JavaScript has short-circuit evaluation with the && operator.

If {JSLinkURL} renders an empty string, it yields:

if ( [ '' ].length && [ '' ][0] )   { ... }

Which is also fine, because although [ ” ].length is True,  [ ” ][0] is False in Javascript, since ” evaluates to False.

Putting it all Together:

<script>
    if ( [ {JSLinkUrl} ].length && [ {JSLinkUrl} ][0] ) {
        document.write('<a class="foo" title="bar" href="' + 
                        encodeURI( {JSLinkUrl} ) + '"> ... </a>');
    }
</script>

Edit 2/24/2012:

Using what I uncovered in Part 3, I now recommend wrapping {JSLinkURL} with encodeURI because this variable is not properly escaped / URL encoded!

This is a fairly robust way to emulate a {block:LinkURL}, while ensuring that:

1. The JavaScript raises no errors
2. The LinkURL you insert is safe to insert directly into the JavaScript code.

It’s ugly. It’s a hack. But it works.

Sometimes it seems like Tumblr actually had developers in mind when they created their Theme Template system.

Other times  — well, a lot of things are simply a f**ing cludge.

Let me give you an example I recently encountered, and how I solved it (in an awful, hacky way.)

For some theme Variables, Tumblr was nice enough to include a {block:Foobar} variable to let you know whether or not some piece of data {Foobar} exists or not.

eg, The Post Title: {Title} and the corresponding {block:Title}:

Anything wrapped with {block:Title} … {/block:Title} is only rendered IF {Title} exists.

That’s super nice. I like that.

And it sure would be nice if a bunch of over Variables also had those “if this exists” wrapper blocks.

Here’s the specific case I’m talking about. I was working on the HTML to render Video posts. One of the neato variables in the Video block is {PlayCount}.

OK, cool. I would like my theme to show how many plays a video got.

So, I very naively wrote something like this:

<span>played {PlayCount} times!</span>

And no matter how many times I played that damn video, {PlayCount} never rendered anything.

Well, it turns out that {PlayCount} only exists if a video was uploaded to Tumblr, not if you embed a video from say YouTube.

OK, fair enough Tumblr - I understand you can’t count the number of plays of an embedded video.

But then, you should at least let us know that you can’t tell us the {PlayCount}, with a nice {block:HasPlayCount} or something. Otherwise my theme / blog looks stupid.

Steps Toward a (hacky) Solution:

1. Let’s try some Javascript.

   Also, I happen to know that Tumblr is so cool, that it has variable transformations.

   So I know that I can “Prefix any theme variable with JS to output a Javascript string (wrapped in quotes)“  [straight from the docs]

   Oh, cool. So, if {PlayCount} doesn’t exist, then {JSPlayCount} should yield an empty Javascript string: ””   Right?
   

   <script>
        if ({JSPlayCount}) {
             document.write(“<span>played {PlayCount} times!</span>”);
        }
   </script>

   There we go! Perfect!

2. Wrong.

   It turns out that if a Variable like {PlayCount} doesn’t exist, then {JSPlayCount} doesn’t exist either - at all. And no, it does not get wrapped in quotes. You do not get an empty string. You get nothing.

   So you get a Javascript error with the statement:

   if ()

   …because you need something inside that if() statement.

3. OK, fine. I know {PlayCount} should be a number, so it’s probably safe NOT to use the JS Variable Transform:

   <script>
        if (“{PlayCount}”) {
             document.write(“<span>played {PlayCount} times!</span>”);
        }
   </script>

   And that works just fine. It’s ugly, but it works, and does not generate Javascript errors.

But wait, There’s More!

    … continued in Part 2!