Twitter, its power users, and its third party developers, 11 months later

Remember this?



This was an interview that was conducted on May 30, 2008 by Robert Scoble and Jesse Stay, who interviewed Evan Williams and Biz Stone. Stay recently referenced the interview in a tweet that compared this interview to Oprah's interview of Williams (which I still haven't seen).

In the ensuing FriendFeed discussion, I noted that I couldn't really remember the specifics of the interview, and that I wanted to look it up again. But to understand the interview, you have to look at some of the circumstances surrounding the interview.

The obvious place to start is a post in the Twitter developer blog, but that blog apparently no longer exists; when I tried to go to the post, I ended up at a wiki instead. So I'm going to rely on MG Siegler's summary of the blog post, which included a question.

charles asks if there’s anything users can do to lighten our load.

In a comment on one of my own blog posts, Charles clarified what he was asking.

For clarification, my original question was intended to elicit some indication of which methods of accessing Twitter caused the heaviest impact and, by extension, suggest that those features be throttled -- and not necessarily by user/community voluntary behavior. I was just being polite; as in, "we" (the users) might be willing to live with occasionally reduced functionality, if it means the service would stay up while "they" (Twitter's developers) continue to repair and re-build it. I don't know if my question indeed prompted them to consider throttling the service, but clearly that's what they have done and it didn't happen until after I posed the question. And, with some user grumbling, it does appear to be working. Mission accomplished.

That was written afterwards, but the real brouhaha started when Twitter's Alex Payne responded to Charles' question:

The events that hit our system the hardest are generally when “popular” users - that is, users with large numbers of followers and people they’re following - perform a number of actions in rapid succession. This usually results in a number of big queries that pile up in our database(s). Not running scripts to follow thousands of users at a time would be a help, but that’s behavior we have to limit on our side.

So Siegler wrote the post Twitter: Don’t blame Ruby, blame Scoble, Scoble vented and wrote, and eventually Scoble was invited to come to the Twitter offices to hash things out. Scoble brought along Jesse Stay, so when he ended up videotaping the session, Stay appeared also.

Why was Jesse Stay there? Well, aside from the fact that he was supposed to have lunch with Scoble before the Twitter call intervened, Stay had some concerns of his own - concerns that he expressed in a blog post a couple of weeks after the interview:

I’ve been following various development mailing lists lately, and I’m seeing a trend of developers starting to bail on Twitter. This is a scary thought, because when the developers bail, so will the users....

I’m very worried for Twitter. As more developers jump ship and work on other platforms such as Plurk and FriendFeed (which really isn’t a direct competitor to Twitter), this great tool is going to be left in the dust with no new development and large networks of people moving elsewhere. Twitter’s largest traffic comes from the API itself, and as that traffic dies down, so will Twitter. Imagine, for instance, if Seesmic were to stop development on Twhirl due to the costs associated with keeping up with API flaws? That would be quite a chunk of Twitter’s users being forced over to the other Twhirl clients, FriendFeed and Seesmic itself - it’s such an easy transition were Twitter support to be dropped! What happens when Twhirl begins supporting Plurk?

Twitter needs to do something, and they need to do it fast. I agree they need to get their infrastructure in place, but before even doing that they really need to put every hack possible in place to keep the API up, keep it working, and work with the developers to ensure they are staying happy. A large revolution is about to take place, and I’m afraid it won’t be pretty.

OK, that's ancient history. And if you want to see more, see Stay's post-interview post, another Jesse Stay post on the API, and (if you're a real glutton for punishment) my four various posts from that period.

But where are we now?

First, let's look at the issues surrounding users with large numbers of followers. Twitter has made improvements to its system, which is good because some of the follower counts today dwarf anything that was prevalent 11 months ago. On that same Oprah interview, Oprah talked to Ashton Kutcher, who now has over 1 million followers. People are having follower races to save lives - what would have happened if that had been staged in the spring of 2008 instead of the spring of 2009?

But while Twitter is now robust enough to handle people with over a million followers, the API has - well, let's let Jesse Stay tell the tale:

[About a year ago] I started SocialToo, a service that originally we built around the auto-follow concept....Twitter, at the time, was the easiest solution to build around, and made the most sense for where we had started so I figured we had to make what we did with it perfect. Here we are, one year later, and I’m still trying to make it work perfect, but not because our code sucks - it’s because Twitter keeps changing their system, and the rules that go with it!

[On April 21] Twitter pulled the rug out from under its developers once more by, with absolutely no notice, announcing that (paraphrased, in my words) since their way was the right way, they were discouraging auto-following, and would only allow a user to follow 1,000 people per day. What Twitter neglected was that, while not many, myself and others were building business plans around the users that would need this. A little notice would have been helpful, but is very consistent with the way developers have been treated over the past year or more by Twitter.

Louis Gray looked at people with large numbers of followers and the auto-follow concept:

Assuming Ashton Kutcher and others were to follow Twitter's rule to only follow 1,000 new accounts a day, it would take Ashton 3 years to follow all that follow him, assuming no more new users found his account interesting. It seems Twitter would prefer that these celebrity accounts only follow, say... 93 as Ashton does, rather than the nearly 400,000 Britney Spears follows, which I would guess would be even higher if it weren't for Twitter's API troubles.

Now I realize that I advanced the view that things are better in an early April comment on Stay's blog, but I think I'll have to pull a Jim Bakker on that one - I was wrong.

Assume the worst-case scenario in which all the developers bail on Twitter, the auto follow concept continues to be denigrated, and famous people continue to get million-plus follower counts. In effect, this means that Twitter becomes more of a one-way broadcast medium rather than a two-way communication platform.

Or maybe it's always been that way, and my perceptions have colored me. When you tweet your response to the question "What are you doing?" there's no guarantee that anyone's listening. And why talk about two-way communication when most people prefer one-way reading rather than two-way discussion anyway?