The current state of the room directory
For now, when we are searching for rooms with the “Default Servers” option, we are requesting 10 rooms from the homeserver for each protocol (by “protocol”, I mean non-Matrix protocols that are bridged to the user’s home servers, like IRC, Gitter, Slack, etc…) that is bridged to the homeserver. This can be quite slow. For instance, we are fetching about 100 rooms from the homeserver “matrix.org” even if we would need to show to the user only 20 of them. This is really bad regarding the performance of the application, furthermore because we have to download/generate the avatar of each room loaded.
Further more, we are building the initial list of the rooms by merging the lists of rooms of the different protocol. So you can already say “What if one room (that wasn’t fetched because we already had 10 rooms from this protocol) of a protocol A have more members than one of the rooms (that *are* listed in the directory) of a protocol B?” and that’s a big problem, because we will miss certain rooms that are actually more “popular” than the ones listed with this method. A similar problem is when we fetch more rooms from the homeserver: we can’t just append the freshly fetched rooms without risking to mess up with the ordering of the rooms as some of them may turns out to have more members than some of the rooms already in the list.
That’s why we have to find a solution to these issues.
A first (easy) solution
The first and easy solution would be to bring back the combo selector we had for selecting which protocol we want to search rooms for, by adding another radio button in the server chooser popover with this selector. However, it would be less good for the user experience as the reason of removing the combo selector was because the user would want to only search for rooms (and not for a room on a certain protocol) whether they are on the homeserver or accessible through a bridge. (And another reason would be that I would have spent time to implement this feature for nothing :P)
Another (less easy) solution
Another solution would be to implement a data structure that would abstract the choice of which requests should be done to fetch a room so that we would only need to use a method on it to get the next room and this room would be the most popular among all the other rooms for all the protocols.
This data structure (let’s call it the “RoomPool”) would be based on a `HashMap<Protocol, VecDeque<Room>>` type. When initializing the RoomPool, we would fetch one (or more?) room for each protocol and store them in the queues.
Each time we would want to get a room from the RoomPool, we would peek in each queue to know which protocol have the most popular room and then pop this room. If one of the queue is empty, we would make a request to the homeserver in order to fetch 5 (or more?) rooms for this protocol and then append them to this queue.
It seems as if it is a rather simple algorithm to solve this issue. There are parameters that could be tuned in order to optimize it: how many rooms do we fetch for each protocol when initializing the RoomPool? Or when we fetch again rooms for an emptied queue. Do we fetch new rooms right away when the last room of a queue has been removed or do we wait the next time we try to peek on the queue?
Please, tell me what do you think about this solution, what cold be improved, etc…
[UPDATE] I forgot to talk about it: the API used by Fractal to discover which other protocols are supported by the HS is Riot.im/Synapse specific so we should simply ignore all of this multi-protocol thing if it isn’t supported by the homeserver.