Design Question - A scalable chat application on phone browsing


  • 0

    Design a scalable chat application on phone.

    • Parental Control - Parents being able to read the chat.

  • 1

    For chat application, usually you want to maintain persistent network connection between the client and the server. We can use Socket to provide a bi-directional communication channel between a client and a server. This means the server can push messages to clients. That way, client don't have to poll the server frequently for changes which can be a burden especially when there are many connected clients at one time.

    When a person sends a message in the chat application, the server will get it and push it to all other connected clients. Since this is a mobile app, a mobile app is just like a web client but with a different end point.

    Am I on the right track? For parental control you need to design user roles privileges. Any other requirements?


  • 0

    @1337c0d3r Yeah, push messages are great!

    But are you gonna store the chat data on your servers?
    Since most of the chat applications, whatsapp, wechat etc. they store the chat data on user's phone ex : SQLite for android.
    Whenever user clicks on another user to chat, and starts to scroll to the top, it'll have to load the chat, and loading this data from server is a good thing?


  • 0

    @andyreadsall said in Design Question - A scalable chat application on phone browsing:

    @1337c0d3r Yeah, push messages are great!

    But are you gonna store the chat data on your servers?
    Since most of the chat applications, whatsapp, wechat etc. they store the chat data on user's phone ex : SQLite for android.
    Whenever user clicks on another user to chat, and starts to scroll to the top, it'll have to load the chat, and loading this data from server is a good thing?

    Ohh, I see what you meant. If there are billions of users, the number of chat messages can be overwhelming and storing all of them on the server seems to require tons of storage. I know some chat applications do not store data on the server side, because your chat history is gone once you lost the phone.

    So storing the chat data on the client side seems to make sense. Depending on the type of phone, there might be limitations on the client side, for example space/memory limitation. So if you chat a lot on the phone, the old messages might have to be purged (or uploaded to the server) to free up some space.

    Usually what I noticed in the chat application you get shown the recent conversation. When you initiate a chat session, you don't have to load the entire chat history.


  • 0

    So, assume users don't care about chat history (aka lost your phone and your entire chat history is gone), we can store all chat data entirely on client side.

    But according to the requirements parents need to be able to read the chat. Does the parent read the chat using the same client or from a different client? If it is the latter, the chat history need to be stored on server side too.


  • 0

    @1337c0d3r I've noticed that the chat get's synced every night at 3 AM on to Google Drive on WhatsApp. So in that case, even if the local data is lost, it can be reloaded. Using this approach, If the chat history is really tons of data for us to store, then we could guarantee the user that the history is available for a day, after which it synced on user's google drive and deleted from our servers. Again, this just another way for us to save storage space. On whatsapp, the sync happens from the phone, but this can be achieved at the application backend servers too, since user has given access the drive, and we have the data. This would avoid problems like client not being connected to the network or phone being turned off during the sync.

    I'm not sure how exactly the interviewer wanted it to be designed when it comes to accounts. But I would say think that Parents have their own accounts, and would be checking the message from their own phone.


  • 0

    @andyreadsall Interesting approach to use Google Drive as a backup. This saves them storage cost, so why not?

    Are you required to write code for this question? Or designing the database schema?


  • 0

    @1337c0d3r This was more of a discussion, and talk about how you would approach. It was asked to one of my friends recently. I'll check with him again and update the question.


  • 1

    Actually thinking in terms of storage space, it is actually not a lot, even in the unlikely event your app reaches the scale of Facebook.

    According to this post, Facebook surpassed 1 billion messages sent in 2009. Assuming each message's average length is 100 characters (probably on the higher end), the minimum storage needed per day is:

    1 billion x 100bytes = 100GB.

    Now, let us compare to Facebook photos, which takes 2-3 Terabytes (1 Terabyte = 1000GB) being uploaded per day according to Facebook.

    So chat messages consumes around 3% of the storage needed to store photos.

    Just for fun, let's compare it to video. According to this post, 300 hours of videos are being uploaded to YouTube every minute. The video size depends on the bitrate of the video when the user uploaded it. Assuming most videos are in 720p, storage takes around 1GB per hour. That means per day the minimum storage is (not counting replica):

    300 x 60 x 24 x 1 = 432000GB.

    As you can see, storage of chat messages take only 0.02% of what video needs. And photos uploaded to Facebook is less than 1% (0.7%) in terms of size compared to videos being uploaded to YouTube.


  • 0

    @1337c0d3r Amazing analysis! I think this is exactly what they expect from the candidate to talk about. Nice!


  • 0

    @andyreadsall Haha, actually I made a mistake. It is 300 hours of videos uploaded to Youtube every minute, so my estimation was way lower... No wonder, I am surprised that videos not taking much more space than photos? I am correcting it now.


  • 0

    Very interesting discussion. What about scalability ? How could we solve a problem when we have a lot of friends (I am a celebrity) and they update their chat status very often ( offline, online, visible, unvisible). Is it worth to transfer all their status to me via opened websockets, when they change their status very often? Thank you


  • 0

    @elmirap This is a very interesting follow up. Assuming you were to support group chat functionality with one million concurrent users, this will be a huge design challenge. Do you have any good ideas?


    EDIT: Seems like maintaining one million web sockets is not an issue on a single server, according to this SO post: http://stackoverflow.com/a/17451928/490463

    So yeah, I would still go with web sockets.


  • 0

    Actually I think that it is impossible to convey to the user all friends information. We have to split friends on circles, which has a rating , and we will update the friends status of the first circle very often. These are friends, with which user has a pernament communication and cares about them the most. The other users could be updated on longer period with some delay. When i log on the system, i will pull all friends infromation. Ofcource we have to store all users interaction in some memory distributed cache.


  • 0

    @elmirap I think you are talking about a social network :D

    Designing a social network that scales is a totally different beast, and is worth a design discussion topic on its own.

    This is a chat application on mobile. The main use case is to have a 1-1 conversation between two people.


  • 0

    @1337c0d3r I was thinking when you have a lot of friends and you need to see their status infromation. How would this be designed? How would you communicate when you don't know the status of the friend


  • 0

    @elmirap Yes, sure. This is a valid requirement for chat apps. If you have lots of friends that's not a problem with websockets. It will only publish your status change to all your online friends.


  • 0

    yes, shall we store these updates in RDBS or memory?


  • 0

    @elmirap You mean you have a giant friend list and many friends are changing their statuses frequently? XD

    This is still fine I think. The server can do batching, so multiple statuses change are batched together in one push to the client. Saves phone's battery and also avoid communication overhead. :)


  • 0

    How shall we avoid a server overload, when each mobile user get its updates from the server?


Log in to reply
 

Looks like your connection to LeetCode Discuss was lost, please wait while we try to reconnect.