I think this is a pretty basic publisher-subscriber problem, but don't know if I have the right answer.
publishers, a list of publishers and
queues, a list of queues. Publishers publish "posts" to queues.
Question: Given an incoming post, how do you determine which queue to send it to?
I thought of two solutions:
For each post, send it to the queue numbered
randint(0,len(queues)). I think on average, this distributes the posts pretty well among all queues.
Assign each publisher to a specific queue(s). A post could contain a
publisher_id, and it would be sent to the queue numbered
hash(publisher_id) % len(queues)). A publisher could be assigned multiple queues by adding a random alphanumberic string with restricted number of digits to the hash. However this approach isn't distributed as well because busy publishers will overload their queues, while tiny publishers will underuse their queues. Only benefit of this is if we needed posts of the same publishers to be on the same queues.
I feel Solution #1 is the best but it seems too simple. What do you guys think? Any other solutions?