Twitter, meet Pub Sub, or How to be the next Twitter

I've been experimenting with the Twitter API lately, getting a feel for not only its published capabilities but the nuances of what you can achieve with it.  If you want to see where I've gone with that, feel free to drop in on socialseer.com from time to time -- as I learn new tricks or come up with new ideas, the web site may go in a completely different direction.

Most of the time, I think about the humans at the other end of the tweet.  Where are they? Are they male or female? Are they happy or sad? Who are they talking about? What are they talking about?  These are all interesting questions, and interesting questions to try to get a computer to answer.  I have code that can answer some of these precisely, guess at some of them, and some are still beyond my grasp.

But even with incomplete and sometimes questionably accurate information, a wealth of analysis can be done.  You can see a blog post here about the reverberations of Rush Limbaugh's "slut" comment and, again, my socialseer.com website has many other examples.

Sometimes, one of my posts will get a lot of hits -- mostly when the topic is on something controversial or in the news.  And perhaps it's a bit of vanity, but I try to cater to that in the posts.  Deep down, though, I'm really looking at how I can extract more interesting information from Twitter and how that information can be used. That probably puts me in a very distinct and small minority -- I don't care about the events as much as how the events can be observed through the Twitter lens.

If you're with me so far, we're about to go a bit deeper and nerdier...

The starting point for this analysis is a mechanical access to the information in Twitter.  Twitter offers a reasonably decent API (reasonably, because it's full of quirks, inconsistencies, strange failures, and limits you have to deal with).  And for the most part, they offer it to any or all to play with.  That's kind and generous of them, and I am grateful to them for that.

At the lowest level, Twitter turns out to be a (quirky, etc.) implementation of what software engineers call "publish and subscribe" -- just normally done by humans.  I write a Tweet and send it, and those of you who follow me on Twitter will see it in your timeline.

It turns in this case, Bianca is not interested in just any Tom, Dick, or Harry

Whether you read the Tweet or not is another question, if you have, say, Tweetdeck running my Tweet shows up on in your open application; message received as far as the software is concerned.

Twitter also allows me to publish messages and to tag them (via hashcodes) for others to subscribe to, although that mechanism is much weaker (there is no formal following of hashcodes in Twitter for example, but must applications allow you to open a persistent and updating search).

Net net, Twitter is using classic approach to integration between software applications and adapting it for communication between humans.

You're saying, at this point, OK I concede you're right, but so what?

One so what is that it gives us a different way to look at Twitter and describe its function a bit more rigorously.  That, I realize, is not a great benefit.

The other so what is that it gives an insight into what's missing from Twitter or what else you could do with it.  So what ( :-) ) are those insights?

Let's start with a kind of hack which is being used to enrich the content of Tweets, such as how a lot of Twitter clients show pictures that are referenced by URLs in the tweet.  The picture is not really part of the Tweet, but it appears to the recipient to be part of it.

Twitter on the Mac opens a small window with
the photo  from a Tweet without bothering you with the mechanics of where the photo's stored

The problem with this kind of solution is two-fold.  First, it is implemented in an informal and ad-hoc way.  Twitpic works, but if I want to build my own sort of Twitpic, it probably won't work.  Second, the picture is completely independent of the core Twitter system, so it could be deleted or modified in some fashion after the Tweet is sent.  Probably with Twitpic you cannot do that, but I bet some enterprising hacker is looking for a way to send a Tweet out with a lolcat, get it retweeted a million times, and then replace the picture with an advert for credit cards.

Still with me? Wondering why?  Ah, well, I'm finally getting around to my real point now...

Even with the current Twitter  we could -- albeit in a way that violates the TOS -- use it to communicate things other than (im)pure English (or German or Chinese or).  We could, for example, send complete messages structured for computers to consume, not humans, at least not directly.

Let me give you a simple example.  Suppose you're a band and you want to tell your fans where and when your next concert is.  You could send a snippet of <140 character prose "we'll be at Bob's Bar in Orlando next Friday at 8pm".  You could send something with greater detail: "we'll be at Bob's Bar, 1234 John Young Pkwy, Orlando 39999 at 8:00pm through 10:00pm".  Or you could send the kind of calendar invite many of us send around in email.  Twitter could display the invite in human readable format, a map of the location, and the date and time in a way that when we click it, it could go right into our calendar.

More examples beyond that -- well, start thinking -- there's a business model for a start up in a lot of them!

But even beyond that, you could (well, I could) imagine a new Twitter, one which is not meant for direct human consumption at all.  One that is purely meant for communication between applications.  You might still limit the message size, but might allow something a bit more reasonable -- say 1K -- that would allow for a decent sized JSON payload.  As a business model, you could sell access -- say a small per message fee to publish, but none to subscribe -- with perhaps a free tier of a limited amount of messages (100 per day?) per user to encourage adoption.  Or charge people for messages above a certain size.  Some messages could be ultimately intended for direct human consumption -- activity stream kind of things -- but some could be just for communication between applications.

The key thing would be that if we all agreed to use this system, it wouldn't matter what our underlying applications were.  And so long as each message type had a uniquely identifying marker, a million different message types could coexist without problem.  In sense, it's like the notion of routing web service calls over messaging middleware, something that was incorporated in things like the SOAP specifications but just about never seen in the wild.

Some obvious categories of messages:

• Document update notifications: I would always have the most recent version of a presentation, or a contract, or a PDF.  This would be a much more refined way of doing sharing than, for example, shared folders in Dropbox, and a much more independent mechanism than Google Docs.
• Schedule updates: It's just painful to receive an email every time a meeting gets changed, especially when it's a useless message like "adding Bob".  If we're all on the same calendaring system, we can do this now, but when we aren't, the routing of meeting invites and updates via email gets messy.
• Bridging activity streams: People are trying to build walled gardens around activity streams, like Salesforce's Chatter.  Either you're in (and pay) or your out.  But if I don't know my subscribers, how would I know what system they are using and reach them there?
• Software updates: you could have a generic demon that subscribes to update notifications from your application vendors, and then launches the application to update.  It's annoying, on the Mac, that Apple has its updater, Microsoft has it updater, Adobe has its -- and they all want to run all the time on their own.

What else? Again, it's up to your imagination to decide.  And your business accumen to bring to market.

The key is to have a single pub-sub service like a Twitter that's tuned for mechanical consumption.  If you like the idea, and think we should give it a go, maybe something small & open source, let me know at charles@mcguinness.us.  If you want to just take this idea and run with it, go for it.  All I ask is for friends and family shares when you go public :-) You never know, something like this could be ten times larger than Twitter in the apps world we live in.