Home My Page Projects FusionForge
Summary Activity Forums Tracker Lists News SCM Files Mediawiki Hudson/Jenkins

[#427] docman wastes a lot of memory

Date:
2012-05-22 12:48
Priority:
3
State:
Closed
Submitted by:
Michael Kluge (mkluge)
Assigned to:
Franck Villaume (nerville)
Target Release:
5.3
Found in Version:
5.1
Severity:
major
Resolution:
Fixed
Summary:
docman wastes a lot of memory

Detailed description
If a file is uploaded, the file content is loaded into a PHP variable. Then a function is called that creates a Document object. This function call creates the first internal copy of this variable. Within this function Document->create there are at leat two more internal copies (data1 and base64_encode(data)) of this data created. The the DB is call, which creates the next copy. So to upload a 100 MB file. Docman needs at least 500M main memory. This should be optimized.
Message  ↓
Date: 2012-08-26 10:19
Sender: Franck Villaume

test done with current trunk : memory limit set to 128M. Max upload size is 80M.
I close the case.

Date: 2012-06-26 19:31
Sender: Michael Kluge

No, not really. I can't test the new code within my 5.1 installation because I also had to replace some files from the 'common' dir which would break it ... The testcase was a php memory limit of 128 MB and a 70 MB pdf that needed to be put into docman. If that works for you in your test environment, I think the case can be closed.

Date: 2012-06-26 18:08
Sender: Franck Villaume

any update on this topic ?

Date: 2012-06-03 16:01
Sender: Franck Villaume

in trunk, docman now stores files in the filesystem. no more base64_encode(data)
lot of memory saved.
take a look at r15660

Date: 2012-05-23 05:51
Sender: Michael Kluge

Hard to give a valueable feedback. I copied the new code into my 5.1 infrastructure and the script still goes to nirvana if I upload a 70M file. The php memory limit is set to 256M. So it is either still using a lot on memory or something is wrong with the 5.1/trunk mix I used for the test. I have'nt much done with php in terms of debugging the memory consumption, so it might take some time until I can give a better feedback.

Date: 2012-05-22 22:32
Sender: Franck Villaume

please take a look at r15547 in trunk.

Date: 2012-05-22 20:37
Sender: Franck Villaume

agreed. some enhancements are in their way but targeted to trunk.

Field Old Value Date By
Severitynormal2012-08-26 10:19nerville
ResolutionNone2012-08-26 10:19nerville
status_idOpen2012-08-26 10:19nerville
close_dateNone2012-08-26 10:19nerville
Target Releasetrunk2012-05-22 22:32nerville
assigned_tonone2012-05-22 22:32nerville
Target ReleaseNone2012-05-22 22:32nerville