Improving the performance of the room directory

The current state of the room directory

For now, when we are searching for rooms with the “Default Servers” option, we are requesting 10 rooms from the homeserver for each protocol (by “protocol”, I mean non-Matrix protocols that are bridged to the user’s home servers, like IRC, Gitter, Slack, etc…) that is bridged to the homeserver. This can be quite slow. For instance, we are fetching about 100 rooms from the homeserver “matrix.org” even if we would need to show to the user only 20 of them. This is really bad regarding the performance of the application, furthermore because we have to download/generate the avatar of each room loaded.

Further more, we are building the initial list of the rooms by merging the lists of rooms of the different protocol. So you can already say “What if one room (that wasn’t fetched because we already had 10 rooms from this protocol) of a protocol A have more members than one of the rooms (that *are* listed in the directory) of a protocol B?” and that’s a big problem, because we will miss certain rooms that are actually more “popular” than the ones listed with this method. A similar problem is when we fetch more rooms from the homeserver: we can’t just append the freshly fetched rooms without risking to mess up with the ordering of the rooms as some of them may turns out to have more members than some of the rooms already in the list.

That’s why we have to find a solution to these issues.

A first (easy) solution

The first and easy solution would be to bring back the combo selector we had for selecting which protocol we want to search rooms for, by adding another radio button in the server chooser popover with this selector. However, it would be less good for the user experience as the reason of removing the combo selector was because the user would want to only search for rooms (and not for a room on a certain protocol) whether they are on the homeserver or accessible through a bridge. (And another reason would be that I would have spent time to implement this feature for nothing :P)

Another (less easy) solution

Another solution would be to implement a data structure that would abstract the choice of which requests should be done to fetch a room so that we would only need to use a method on it to get the next room and this room would be the most popular among all the other rooms for all the protocols.

This data structure (let’s call it the “​RoomPool”) would be based on a `HashMap<Protocol, VecDeque<Room>>` type. When initializing the RoomPool, we would fetch one (or more?) room for each protocol and store them in the queues.

Each time we would want to get a room from the RoomPool, we would peek in each queue to know which protocol have the most popular room and then pop this room. If one of the queue is empty, we would make a request to the homeserver in order to fetch 5 (or more?) rooms for this protocol and then append them to this queue.

It seems as if it is a rather simple algorithm to solve this issue. There are parameters that could be tuned in order to optimize it: how many rooms do we fetch for each protocol when initializing the RoomPool? Or when we fetch again rooms for an emptied queue. Do we fetch new rooms right away when the last room of a queue has been removed or do we wait the next time we try to peek on the queue?

Please, tell me what do you think about this solution, what cold be improved, etc…

[UPDATE] I forgot to talk about it: the API used by Fractal to discover which other protocols are supported by the HS is Riot.im/Synapse specific so we should simply ignore all of this multi-protocol thing if it isn’t supported by the homeserver.

Advertisements

3 thoughts on “Improving the performance of the room directory

  1. I often encourage/nag posters I come across in Planet Gnome to always say in the second sentence what their , software does. You never know who will come across your post, you never get a second chance to make a first impression, you’re the marketing department for project XYZ, etc. But here I’m not even sure what the project is called 😉 ! Gnome Rooms? Nope. Is it “Fractal is a Matrix messaging app for GNOME written in Rust”? (But then what is Matrix?)

    Good luck with your project.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s