[#827] Serious performance issues for user with many projects

Description

Summary:

Detailed description

Hi all,

my name is Andrea and I work for an italian company as a software architect. My company has a customer who is using FusionForge 6.0.4 and who's experimenting some major performance issue. The platform I am talking has about 600 active projects and more than 700 active users. 3 of these users are some kind of super users. This means they are associated to the majority of these projects (a few more than 400) with a custom role who grants them many privileges (but they are far from having administrative rights). For these 3 users it takes from 12 to 20 seconds to fulfill any request sent to the platform (even the login action). After digging into the code and sniffing the communication channel to the db, we found out that every request send THOUSANDS of query to the database (about 3.500). Anyway, probably because of MySQL cache, this phase tooks no more than 3 seconds. The real problem stands after this: after the data is retrieved the cpu usage of the core that is fulfilling the request goes immediately up to 100% until the elaboration is over (usually from 9 to 17 seconds).

Almost all queries are something like "SELECT * FROM pforole WHERE roleid=...", which makes me think this issue is related to the new RBAC features you recently added. Maybe the software is not intended or designed to manage a situation where a single user can have many projects associated to himself, but this is our use case and this is causing severe problems. NOTE: we did not have this problem with older versions of FusionForge.

We would like to ask you if you know some solution we did not think about. If there are no solutions we would like to ask you to please think about one and then implement it.

An idea for an evolution could be extending the permissions of global roles, for instance adding rights for docman and file release system (we actually would need only these two). Though this would be a patch and not a true solution to the problem, this would allow us to remove projects associations from this users and give them global roles for their needings.

I would be very thankful if you kindly give me some support.

Thank you very much! Andrea

General Information
Submitted by:
Andrea Mallegni
Date Submitted: 2016-08-12 14:53
Last Modified by: Nobody
Last Modified: 2017-11-02 20:00
Date Closed: 2017-12-21 09:57
Permalink: https://fusionforge.org/tracker/a_follow.php/827
Actions
Monitor
Votes: 0/1 (0%)
Internal Fields
Data Type: Support requests
Assigned to: Nobody (None)
State: Closed
Priority: 3
Extra Fields
Version:
6.0
Follow-up tabs
Message  ↓
Date: 2016-09-27 12:33
Sender: Franck Villaume

Hi Andrea,

point 1: any idea of how many trackers ? how many FRS packages? Some feature has permission per object (tracker, FRS, Task, Forum ...) That may be one lead. point 2: good news! point 3: do not hesitate to contact me directly. I'm part of TrivialDev Company which offers support & development around FusionForge. We have some good experience to add new features on request.

Date: 2016-09-27 08:42
Sender: Andrea Mallegni

ERRATA CORRIGE:

  1. The linked role work as expected. I was absolutely wrong, I made mistakes while doing my tests. I started from scratch and had the results I expected. I will let you know if this approach is a solution for our problem.

Thank you

Date: 2016-09-27 08:19
Sender: Andrea Mallegni

Hello Franck,

sorry but I've been involved into a major project that stole all of my time, so that only now I am able again to answer you.

Well: 1. Usually 5, not considering the global roles "Anonymous" and "Logged In".

  1. Yes I had a look but either I did not understand how they should work or they do not work as expected. For instance, I added my user to project A with a role that allows Doc Manager Admin privileges, I marked that role as shared. Later I linked that role in project B, but if I browse project B's Doc Manager with my user I get a permission denied message and not only I do not have docman admin privileges, I don't even have docman read access.

  2. Thank you, I'm looking forward to see the feature request. Our customer is willing to pay anybody who will takeover this request and accelerate this process, is it something we can talk about?

Thank you, Andrea

Date: 2016-08-31 18:06
Sender: Franck Villaume

Hi,

point 1: about performance how many roles you have per project?

point 2: global roles did you looked at creating roles in a project, make it "Shared role (can be referenced by other projects)", then reference it into other project to see if users with this role can get access to feature such as "docman"?

point 3: I will create a feature request then the community will look about how to implement it. I have no idea about when this feature will be released from a community point of view.

Date: 2016-08-29 07:28
Sender: Andrea Mallegni

Hello Franck,

if I may I would like to ask you if you already scheduled the issue and if you have any idea about when the feature will be released.

Thank you very much! Andrea

Date: 2016-08-16 07:18
Sender: Andrea Mallegni

very very sorry, I said MySQL but I intended PostreSQL obviously :) I ask our DBAs if they took all the necessary decisions to perfectly tune the database. Anyway our analysis says that the time consumed querying the database is not the critical point, the elaboration of the retrieved data is the critical point instead. The customer I am talking about used version 4.8, we took care of the migration following all the migration steps from 4.8 down to 6.0.4.

Thank you very much for your very fast answer and support!

Date: 2016-08-14 11:05
Sender: Franck Villaume

Hi,

yes RBAC feature still need some improvement :-)

Did you activate the PostgreSQL prepared transaction for your PostgreSQL tuning? (There is no MySQL database ...)

RBAC feature is available since 5.1 version. What version were you using before?

Agreed with your proposition to extend global roles.

No attached documents

No related commits.

Field Old Value Date By
status_idOpen2017-12-21 09:57
Franck Villaume
close_dateNone2017-12-21 09:57
Franck Villaume

No relations found.