Bug 94858 - FILEOPEN: .ODS - very slow, .XLSX - very fast
Summary: FILEOPEN: .ODS - very slow, .XLSX - very fast
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
3.6.0.4 release
Hardware: x86-64 (AMD64) All
: medium normal
Assignee: Markus Mohrhard
URL:
Whiteboard: target:5.2.0
Keywords: bibisected, filter:ods, perf, regression
Depends on:
Blocks:
 
Reported: 2015-10-07 12:51 UTC by Igor
Modified: 2016-10-25 19:11 UTC (History)
6 users (show)

See Also:
Crash report or crash signature:


Attachments
slow opened document (2.96 MB, application/vnd.oasis.opendocument.spreadsheet)
2015-10-07 12:51 UTC, Igor
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Igor 2015-10-07 12:51:57 UTC
Created attachment 119386 [details]
slow opened  document

LibreOffice Calc very slow opens "ods" document in the attachment. Tested at next versions:
Product  Version      Time        Format
Apache	4.1.1.6	      0:40	  ods
Libre	3.4.6.2	      0:46	  ods
Libre	3.3.0.4	      0:55	  ods
Libre	4.4.5.2	      5:40	  ods
Libre	3.6.7.2	      6:30	  ods
Libre	5.0.2.2	      7:20	  ods
Libre	5.0.2.2	      0:28	  xlsx
As you can see on new versions of Calc opens the document for a long time.
BUT if i "save as xlsx" this document - file is opened very quickly. Also if I open xlsx file I see what works more than 1 CPU.

Ubuntu 14.04.3 LTS
12GB ddr3 1333Mhz
AMD FX(tm)-8150 Eight-Core Processor
Comment 1 tommy27 2015-10-08 04:53:38 UTC
tested under Win8.1 x64 AMD A8-6410, 8GB RAM

on my system both 3.4.6.2 and 3.5.7.2 take 2 minutes to load that file and 3.6.0.4 shows an important performance drop-off with 13 minutes and 40 seconds!!!

performance is better with 5.1.0.0 alpha daily build installed yesterday which takes 7 minutes and 25 seconds but it's still far from the original loading time

It's a performance regression and needs bibisecting
I put a Calc expert in CC list.
Comment 2 Markus Mohrhard 2015-10-10 00:03:33 UTC
The content.xml contains about 1 million lines if the file is formatted by xmllint --format.

That file is just horribly huge. Of course we need like forever to import the file.
Comment 3 tommy27 2015-10-10 03:41:56 UTC
but why the different loading speed loading the same file in different LibO releases? what caused the performance drop?
Comment 4 Igor 2015-10-14 07:12:59 UTC
 d202b17d88ecb0b608d81518624021c832c7dfdb is the first bad commit
commit d202b17d88ecb0b608d81518624021c832c7dfdb
Author: Bjoern Michaelsen <bjoern.michaelsen@canonical.com>
Date:   Wed Apr 25 07:28:24 2012 +0200

    source-hash-ce97851773a06103504972eb2771eecd7dd81e36

    commit ce97851773a06103504972eb2771eecd7dd81e36
    Author:     David Tardon <dtardon@redhat.com>
    AuthorDate: Mon Feb 6 19:12:02 2012 +0100
    Commit:     David Tardon <dtardon@redhat.com>
    CommitDate: Mon Feb 6 19:12:02 2012 +0100

        fix typo

:100644 100644 e1c1d62aa980fee004430f920cdbe3fd1ce79bf0 9acf11b8f6f5e26b03649767813ac42f72c38e1b M	autogen.log
:100644 100644 c14237a7b6ebde67a83585c9b057c78710e08ea2 db4232175b715b6c7f322b17041f56a9145e1622 M	ccache.log
:100644 100644 c407d12366338584cbcebf2197cd7fcdcf1c522b 1b83a94159f8aa22e004b5dc2ebe1895b32a2724 M	commitmsg
:100644 100644 3be616510b5296b5ae2f5c154a6c51f7ba49bf24 cc9f341a09ba536bb41d4219c5b7f5dd219d7cc6 M	dev-install.log
:100644 100644 637e789a93608b99c13fec9e598c2e7a4c454c6d 08ab33c46c34b7b9b0f8b7f21161ad1c1a2ed59a M	make.log
:040000 040000 c47ba9e6977c3c8a957b11ec3f8b85cfa50362af f87831ea583aaccb888e681ce264cc1e4e44d3aa M	opt







git bisect log
# bad: [423a84c4f7068853974887d98442bc2a2d0cc91b] source-hash-c15927f20d4727c3b8de68497b6949e72f9e6e9e
# good: [65fd30f5cb4cdd37995a33420ed8273c0a29bf00] source-hash-d6cde02dbce8c28c6af836e2dc1120f8a6ef9932
git bisect start 'latest' 'oldest'
# bad: [e02439a3d6297a1f5334fa558ddec5ef4212c574] source-hash-6b8393474974d2af7a2cb3c47b3d5c081b550bdb
git bisect bad e02439a3d6297a1f5334fa558ddec5ef4212c574
# bad: [8f4aeaad2f65d656328a451154142bb82efa4327] source-hash-1885266f274575327cdeee9852945a3e91f32f15
git bisect bad 8f4aeaad2f65d656328a451154142bb82efa4327
# bad: [369369915d3582924b3d01c9b01167268ed38f3b] source-hash-45295f3cdceb4c289553791071b5d7f4962d2ec4
git bisect bad 369369915d3582924b3d01c9b01167268ed38f3b
# good: [351622aec2dff3cc3bbbb020ad0097c4322d2a21] source-hash-2c4537471c932b65e6f72e41881b505c4bbad12c
git bisect good 351622aec2dff3cc3bbbb020ad0097c4322d2a21
# bad: [378efb6e51212a05d1bd4b85c916eec5753c1744] source-hash-d453788ac0476cc02b929b0907718ca771d6d956
git bisect bad 378efb6e51212a05d1bd4b85c916eec5753c1744
# bad: [1a3c4b54a8782fe0f4bdba221e87012a92e4d323] source-hash-a330f38093e2643a26239557050561afae9ff23d
git bisect bad 1a3c4b54a8782fe0f4bdba221e87012a92e4d323
# good: [cf86b7f14a98d2d81a5cd93507acb35ff6775d8b] source-hash-85c6244b85b29c1d2bb9d89b62e9512dd65378b5
git bisect good cf86b7f14a98d2d81a5cd93507acb35ff6775d8b
# good: [bc87fae0fc661b44769d71e41a0e8ce3dac3e857] source-hash-f176c9ba7be7f3051a52b9f57b56124038c0cfd6
git bisect good bc87fae0fc661b44769d71e41a0e8ce3dac3e857
# bad: [d202b17d88ecb0b608d81518624021c832c7dfdb] source-hash-ce97851773a06103504972eb2771eecd7dd81e36
git bisect bad d202b17d88ecb0b608d81518624021c832c7dfdb
# good: [9300cbe83880d09cc6d581eb73a92f35f3456b31] source-hash-43c7830b03d141ae11d8617c0fdabefa32dd243c
git bisect good 9300cbe83880d09cc6d581eb73a92f35f3456b31
# first bad commit: [d202b17d88ecb0b608d81518624021c832c7dfdb] source-hash-ce97851773a06103504972eb2771eecd7dd81e36
Comment 5 Igor 2015-10-14 07:37:23 UTC
also provides additional information about time to open document

Version 3.7.0.0.alpha0+ (Build ID: 188526) -                               5:26
LibreOffice 3.6.0alpha0+ Build ID: 45295f -                                4:40
LibreOffice 3.5.0 Build ID: 2c45374 -                                      0:48
LibreOffice 3.6.0alpha0+ Build ID: d45378 -                                4:47
LibreOffice 3.6.0alpha0+ Build ID: a330f38-libreoffice-3-5-branch-point	-  4:55
LibreOffice 3.5.0 Build ID: 85c6244 -                                      0:48
LibreOffice 3.6.0 Build ID: f176c9b -                                      0:39
LibreOffice 3.6.0alpha0+ Build ID: ce97851-libreoffice-3-5-branch-point -  4:53
LibreOffice 3.6.0 Build ID: 43c7830 -                                      0:37
Comment 6 Robinson Tryon (qubit) 2015-11-19 00:43:51 UTC Comment hidden (obsolete)
Comment 7 Robinson Tryon (qubit) 2015-12-10 01:53:41 UTC Comment hidden (obsolete)
Comment 8 Markus Mohrhard 2016-02-09 03:43:45 UTC
Nasty O(n^2) algorithm in the outline code.

I have a solution for master.
Comment 9 Commit Notification 2016-02-15 03:01:16 UTC
Markus Mohrhard committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=a8232b30687879f31768b89f4ff0bcf9457a7e77

tdf#94858, avoid O(n^2) algorithm during outline import

It will be available in 5.2.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.