About Me


Connect

Monday, October 12, 2009

Will Twitter Go Down With The Waves

Posted by: Danish 12:47 PM

Twitter has been down for more than 2 hours,the last tweets on my time-line reflect panic as one would expect from the passengers of a ship as it goes down with the waves! Well, i haven't got a Google Wave account yet, the analogy is partly coincidental :p

Was i surprised? Not at all! This is not the first time that the micro-blogging site has gone down, since past few months,outages have become a regular affair and most likely will continue for a while. No! I don't mean to say that the guys at Twitter are not smart enough, indeed, it would be foolish to say that for guys who came up with such a brilliant concept such as Twitter. The only speculation that can be made is the probability that the issue lies deeper than it seems to us.

Outages are not specific to Twitter alone, most popular services including Google, Gmail and Facebook have had their blues too. It is an irony of fate that the very participatory nature of the Social Web is also the factor that takes it down often. For instance, the Internet traffic generated by Michael Jackson's death was much more than the servers were prepared for; it seems that most of the people in the world decided to use the Web to find more information about MJ, unprecedented numbers of requests coming in was interpreted as a DDoS attack by Google which then went into emergency mode. However, other Social Web sites such as Facebook, Friendfeed, MySpace and Twitter of course are far more open, consequently they could not balance the load and crashed.

So couldn't they build the service keeping load balancing in mind? This question is complex too, the Web is evolving at such a rapid pace that it is no longer possible to follow the traditional model where even the tiniest detail has gone through analysis and validation before the development has actually begun. Most of the popular service providers of the day are widely used for features some of which had not even been foreseen when the service was built for the first time. Adding newer features to existing applications is a continuous process that is conceived of and implemented in response to changing paradigms. Quick adaptation and innovation-on-demand is what drove their growth and has enabled them to change the way we use the Internet now.

Keeping the above scenario in mind, it is easier to understand why Web 2.0 websites often face scalability issues. Most of them had started up targeting a small segment of users, with plans to upscale operations on a future date. But as the service becomes popular and the user base increases exponentially, 
 too many requests lead to bottlenecks that make the system defunct for a period of time. There are technical reasons of course, I have come across quite a large number of papers blaming Ruby on Rails, the framework on which Twitter was built. This aspect cannot be refuted absolutely when one considers the fact that this framework has not been proven on large-scale application development.  Facebook has addressed this issue by migrating the background code, it also has a limit on the number of friends a user can add. But there has to be something more than the weakness of the development framework.

One of the reasons that i love Twitter for is the fact that there is no restrictions apart from the 140 characters per tweet limit and the model itself. Unlike IMs or Social Networks, Twitter delivers messages from multiple users to multiple users. This makes every user a center of information propagation or many-to-many messaging system yet the server remains centralized, so as the number of users increase the processes increase exponentially since every user follows an unspecified number of users and is in turn followed by other users. i am not aware of the business logic or algorithm used by it but can understand that it must be efficient enough to handle all those transaction most of the time. However, the frequent outages suggest that something at the fundamental layer needs a re-look at.

As i said in the beginning i do not have a Google Wave account yet but from what i have read and seen in the Developer Conference preview video, the architecture is more inclined towards being a distributed one rather than being centralized. i wonder if the folks at Twitter have looked in that direction, if the server is unable to handle the load now, waves of users are just starting to migrate to Twitter. Read: Has Cattlegate Opened The Floodgate?

Also read : http://www.techcrunch.com/2008/05/22/twitter-at-scale-will-it-work/

P.S.: Inviting my friend netgenre to write a guest article on data marshaling in the above context

Update
 Recently i came across (through Twitter, of course) a excellent article at High Scalability which offers to explain the different factors responsible for the scalability issue of most social media sites. i found the part on Pull-on-Demand vs Push-on-Demand approaches quite interesting. In the first approach, which  is followed by Facebook ( according to the same article), the service queries all your friends,fetches their updates and changes and provides them to you at one place. So if you have 1000 friends, it makes 1000 queries to show your friends update. So with every new user or even connection the number of queries to execute rises exponentially.
 In contrast, Push-on-Demand approach deals with updates in a different manner. Rather than waiting for queries,.it pushes the data to friends right when it is changed! The user no longer needs to pull data, since it is already there when s/he logs in. Of course this model is not without drawbacks but that is beyond the current scope.
But more importantly, which model does Twitter use ? 
i am not sure, but most likely it uses a hybrid of both model...Just a guess right now ;)

Posted By Danish 12:47 PM

Saturday, October 3, 2009

The N00B's Guide To Twitterverse

Picking up the thread from the last post, I will try to demystify the basics of Twitter for the benefit of the complete N00B. Here is how twitter works and the meanings of the terms that baffle you the first time you start using it.

Twitter Jargon

Tweet - The only question that drives Twitter is "What are you doing?" and a Tweet is a response to that question. If you are into social networking sites like Facebook and Orkut or even a Skype or GTalk user, you must already know what the status message means. But unlike the other services, the status message on Twitter i.e. the Tweet is not a passive message but the medium of communication or the counter-part of the IM you send to your chat buddy. However, a Tweet is generally visible universally unless, of course you have blocked a person. A Tweet can be of 140 character length only, which could be one of the reasons why Tweets travel faster than anything else i know of.


Handle - Username, this is important as it is likely to become your identity on Twitter, for example my Twitter handle is danishctc. You can find my profile on twitter by simply typing http://twitter.com/danishctc in the location bar or your browser. You can also reply me or mention it by simply prefixing my handle with the @ (at) symbol : @danishctc

DP - Stands for display picture or avatar. It is highly recommended that you put up your own picture as DP.


RT: Retweet is nothing but the same tweet being posted again by another user. There could be many reasons for doing this but the most common ones are- the person liked the particular tweet and would like to share it with her/his followers  and the original Tweet is kept to make sure that the response is seen in the context. To retweet, you can copy the whole tweet along with the original tweeterer's handle (prefix the @ (at) symbol to the handle ) and type RT right at the beginning. (Most clients make this easy, explained here just to make sure)

Screenshot of my convo with de3p , pushkarbhatt
Replies/Mentions: these are tweets by other users who have either replied,addressed or mentioned your handle. A handle prefixed by @(at) symbol is treated by Twitter as a tweet meant for that particular person and shows up on his timeline; for example- "danishctc: @anandmisra hi"


DM: Direct message, visible only to the recipient. Normally, all clients have a functionality to let you send direct messages to your friends but you can send dm by simply typing the character D before the handle.
Example:
"danishctc: D anandmisra this a dm"

Friends - Twitter is people-driven, if you want to harness its power then you have to follow people whose updates you would want see. You could start following people in the suggested list that Twitter displays when you sign up or celebrities you already know ( for example Shashi Tharoor, Celina Jaitly, Gul Panag) but it is important that you find other like-minded users to make your tweeting productive. As with so many services, Twitter can also import your contact list from other accounts and suggest your pre-existing contacts that are already on Twitter. To follow them you can go to their profile page and click on the link "Follow" (Note:  Following a large number of people in a very short time is seen suspiciously by Twitter since spam bots seem to follow a similar pattern, trying interacting as much as possible )

Followers - These are people who are following your updates. They are important from your point of view since they are your main audience as well as customer-base.

Favorites- If you find a particular tweet interesting and dont want to lose it in the melee, you can add it to your favorites. To add a Tweet to favorites, click on the star icon, the Tweet can now be found in the  favorite section of your Twitter profile.


 One of the major advantages of Twitter is the fact that it is not limited to a single client, single domain or even single device. I prefer tweeting from mobile rather than the desktop.

The original interface for interaction was Twitter.com and continues to be so since that is where you manage your account and set preferences. This is also the site you should be using if you are a new user.However, this interface lacks certain features for more advanced users but this is aptly compensated by numerous third-party clients.

Clients- There are plenty of clients avaible, download the one which meets your expection. On desktop/laptop, i find Tweetdeck the best, followed by Twhirl and Seesmic. Firefox add-on, Twitter Fox is another client that i find myself using a lot. On mobile (i use Nokia N95 8GB), Gravity comes across as the client with best features but i have also used Fring and Twibble for tweeting.
There also some very cool web-based clients that you should check out: http://dabr.co.uk and http://slandr.com 

Update - Thanks to Softykid for reminding me about Snaptu, a mobile client that i had heard a lot about yet  never got to installing it. But now that i have done it, it is strongly recommended that you try it out :)

P.S.: Thats it for now, hope someone finds it useful coz i didn't get much pleasure compiling it! not my kinda thing ;)

The Twitter Song 

Posted By Danish 8:05 PM