Bug 66209 - inserting/deleting rows: data integrity problem when data for "non-active" charts are updated
Summary: inserting/deleting rows: data integrity problem when data for "non-active" ch...
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Chart (show other bugs)
Version:
(earliest affected)
4.0.0.3 release
Hardware: Other All
: highest major
Assignee: Not Assigned
URL:
Whiteboard:
Keywords: bibisected, regression
Depends on:
Blocks: Chart-Data Memory
  Show dependency treegraph
 
Reported: 2013-06-26 14:14 UTC by László Németh
Modified: 2024-05-02 03:18 UTC (History)
12 users (show)

See Also:
Crash report or crash signature:


Attachments
test data (107.45 KB, application/vnd.oasis.opendocument.spreadsheet)
2013-06-26 14:14 UTC, László Németh
Details
Screenshots of sheet 2 after row insertion in sheet 1 (37.87 KB, image/png)
2013-08-01 12:29 UTC, László Németh
Details
Simple test case with just one chart on each sheet (18.39 KB, application/vnd.oasis.opendocument.spreadsheet)
2016-11-11 07:51 UTC, Lars Jødal
Details

Note You need to log in before you can comment on or make changes to this bug.
Description László Németh 2013-06-26 14:14:16 UTC
Created attachment 81477 [details]
test data

The attached test file contains the same diagrams on the sheets. Inserting an empty row before the data rows results data integrity problem: the diagram on the second sheet will have bad data range (missing data range update). Undo doesn't help to restore the original data range of the charts. Checked with LO 4.0.4, too. The phenomenon depends from the size/empty charts of the second sheet, and other operations, but it is significant on the newly loaded test file.

There is a workaround for this bug: renaming the first sheet after opening the file will keep data integrity of the diagrams during row insertion/deletion.
Comment 1 László Németh 2013-06-26 14:16:01 UTC
(detected in LO 4.0.3.3, maybe wrong in older LO 4 versions, too.)
Comment 2 Cor Nouws 2013-06-26 17:49:43 UTC
Hi Lásló,

(In reply to comment #0)

> The attached test file contains the same diagrams on the sheets. Inserting
> an empty row before the data rows results data integrity problem: the
> [...]

Is there any inteference with the setting in Calc that extends references when rows/columns are inserted at the edges of ranges?
 ( Tools > Options > Calc > General ..)

Ciao,
Cor
Comment 3 László Németh 2013-06-26 20:25:11 UTC
(In reply to comment #2)
Hi Cor,

There is no interference. This is a malfunction, because it occurs only in special circumstances, eg. on a sheet with many (~20) diagrams, also it is a regression, because LibreOffice 3 has no problem with it.
Comment 4 László Németh 2013-07-08 14:07:04 UTC
It is a problem of work sheets with multiple charts (not only multiple empty charts, the original document contains more data and non-empty charts).
Comment 5 Michael Meeks 2013-07-30 12:42:36 UTC
Muthu - a chart regression :-) no idea if you have any insight off-hand into that ?
Comment 6 Muthu 2013-07-31 04:27:06 UTC
No idea...let me check...
Comment 7 Muthu 2013-08-01 10:45:24 UTC
I tried to find the issue for quite some time now.
I checked on different versions of LO and all looked alike :(
(And the language in the sheet is not helping as well :( )

Could you provide a detailed step wise info about how to reproduce the problem, please?

I also didn't understand what you mean by "Inserting an empty row before the data rows results data integrity problem"

It would help if you could attach screenshots as well, please?

Really sorry to be troubling you...

Thanks!
Comment 8 László Németh 2013-08-01 12:29:08 UTC
Created attachment 83429 [details]
Screenshots of sheet 2 after row insertion in sheet 1
Comment 9 László Németh 2013-08-01 12:32:08 UTC
Muthu : I have downloaded the test file, and inserted an empty row in the sheet 1 in the row 3 (before the data). In LibreOffice 3, all chars show the correct data range, in LibreOffice 4, the second chart on the sheet 2 will be bad (see the attached screenshots). Thanks for your checking!
Comment 10 Muthu 2013-08-05 11:45:29 UTC
Thank you, for the information. It took some more time to understand the problem :)

One more important point is:
The row needs to be inserted before opening Sheet2.

i.e. if we click on Sheet2 and back to Sheet1_2 and then insert the row - the chart data ranges are updated properly.
Comment 11 Björn Michaelsen 2014-01-17 09:58:41 UTC Comment hidden (obsolete)
Comment 12 Stéphane Guillou (stragu) 2014-02-13 23:18:47 UTC
Could someone please test this to see if it is still an issue in 4.1 and/or 4.2? We are trying to close mab4.0 as it reached its end of life (EOL). Cheers.
Comment 13 László Németh 2014-02-14 09:06:34 UTC
(In reply to comment #12)
> Could someone please test this to see if it is still an issue in 4.1 and/or
> 4.2? We are trying to close mab4.0 as it reached its end of life (EOL).
> Cheers.

It is still an issue in 4.2, I have tested it.
Comment 14 Stéphane Guillou (stragu) 2014-02-14 14:56:43 UTC
Thanks for testing! Moving this to mab4.1 as 4.0 reached end of life a while ago.
Comment 15 tommy27 2014-05-12 19:50:23 UTC
(In reply to comment #13)
> It is still an issue in 4.2, I have tested it.

@Lazlo
your message was from february 2014, would you please give us an update with current 4.2.4.2?
if issue persists I'll move it to mab4.2 list
Comment 16 m_a_riosv 2014-05-17 15:15:19 UTC
Works for me with:
Win7x64Ultimate
Version: 4.0.6.2 Build ID: 2e2573268451a50806fcd60ae2d9fe01dd0ce24)
Version: 4.1.6.2 Build ID: 40ff705089295be5be0aae9b15123f687c05b0a
Version: 4.2.0.4 Build ID: 05dceb5d363845f2cf968344d7adab8dcfb2ba71
Version: 4.2.1.1 Build ID: d7dbbd7842e6a58b0f521599204e827654e1fb8b
Version: 4.2.2.1 Build ID: 3be8cda0bddd8e430d8cda1ebfd581265cca5a0f
Version: 4.2.3.3 Build ID: 882f8a0a489bc99a9e60c7905a60226254cb6ff0
Version: 4.2.4.2 Build ID: 63150712c6d317d27ce2db16eb94c2f3d7b699f8
Version: 4.2.5.0.0+ Build ID: da62ab5783e548c5785b3950c03d078b1f3e849d
   TinderBox: Win-x86@42, Branch:libreoffice-4-2, Time: 2014-05-16_08:13:28
Version: 4.3.0.0.alpha1+ Build ID: 48eccfb812284f43ba24c3be3903537ce954944d
   TinderBox: Win-x86@39, Branch:master, Time: 2014-05-16_00:35:19

Inserting a couple of rows in Sheet1_2.A2 or A10 updates properly the chart in Sheet2.C4, as first action without go to Sheet2, following what Muthu explains in comment#10, and no matter if the option about expand references explained by Cor in comment#2 is activated or not.

Perhaps an issue in relation with the user profile.
https://wiki.documentfoundation.org/UserProfile
Comment 17 Joel Madero 2014-05-23 02:51:54 UTC
Ubuntu 14.04
LibreOffice 4.2.4.2 release

Confirmed

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ f91a107c097e5932ba1ed1d88e574e900ebfda00 is the first bad commit
commit f91a107c097e5932ba1ed1d88e574e900ebfda00
Author: Bjoern Michaelsen <bjoern.michaelsen@canonical.com>
Date:   Tue Dec 11 10:20:39 2012 +0000

    source-hash-2e053cf5ea4d93a2e1845e795a9c7fe1e08c84af
    
    commit 2e053cf5ea4d93a2e1845e795a9c7fe1e08c84af
    Author:     Takeshi Abe <tabe@fixedpoint.jp>
    AuthorDate: Wed Dec 5 08:33:07 2012 +0900
    Commit:     Takeshi Abe <tabe@fixedpoint.jp>
    CommitDate: Wed Dec 5 10:42:17 2012 +0900
    
        sal_Bool to bool
    
        Change-Id: Ic1cc3312e61e602c33be7444f8d4bbad9268ae77

:100644 100644 42f18f15fb3370ca3f85d6a637b17ae05b10996f 4e4564a1257d79f9fd86b264270d348bf06d23e3 M	autogen.log
:100644 100644 b4973c1d86cfd5c0d550e84e7e483802d979829e 1224d320eac5e10b39e8481456547368fe09c748 M	ccache.log
:100644 100644 a0e5b02e554537a0bed70a3203ca3995bb6972bd ab62cc407b67c0ca8a1a96f877dc86a92f01e1ef M	commitmsg
:100644 100644 6ba1538977f3fb5b8f0ee54bbd04be28afdd85e3 575f5b911fb36490ff12ae85452ce424b34a3382 M	dev-install.log
:100644 100644 dc4842a6e74513bbe562bb0d78a1ed545ead53e7 3f76084c0057251a1332a9be8b51e31a06e5590e M	make.log
:040000 040000 e70233c1668fc1da017a420ac2ed5ee366e253ad 9457873d1fe1b1108dc20777c8f0180cefe48cdd M	opt

# bad: [793dbf6f80f497dfe587d560d6257f42a24273f6] source-hash-1581b1fc3ac82a7bd62df968226e98604a4ca52d
# good: [65fd30f5cb4cdd37995a33420ed8273c0a29bf00] source-hash-d6cde02dbce8c28c6af836e2dc1120f8a6ef9932
git bisect start 'latest' 'oldest'
# bad: [8092559c5013969ebda017d79200463b9b975038] source-hash-fd84daf696a368c2c7561b5253b32a63ecdeca4a
git bisect bad 8092559c5013969ebda017d79200463b9b975038
# good: [2bdbcaf93e4616633a733de7eb88ba19571929ac] source-hash-cf04745f7a027594fd64a493c276a8280dbccfe1
git bisect good 2bdbcaf93e4616633a733de7eb88ba19571929ac
# good: [f823dd19086ed09fb8e2072050e963cf1bfcd5fe] source-hash-b679a2a02180c017bd8b596fb2e4f283bad93b75
git bisect good f823dd19086ed09fb8e2072050e963cf1bfcd5fe
# good: [8cf46b05e525ac3602433613f227f0dcacc44035] source-hash-1692cf6854ff7adbb2bd47f2f7ec2b3de51864f3
git bisect good 8cf46b05e525ac3602433613f227f0dcacc44035
# good: [3cb56a12a43a7c2a43954dc7694a6f6bde544f98] source-hash-41c2b0375773b2d2945d75e255ea6bb6c7fd378d
git bisect good 3cb56a12a43a7c2a43954dc7694a6f6bde544f98
# bad: [f91a107c097e5932ba1ed1d88e574e900ebfda00] source-hash-2e053cf5ea4d93a2e1845e795a9c7fe1e08c84af
git bisect bad f91a107c097e5932ba1ed1d88e574e900ebfda00
# good: [6590d64c096bc4ea15e3ae66f07a21392e8ba11c] source-hash-b67a51b40a4876f4bd97a2917103112006710b0c
git bisect good 6590d64c096bc4ea15e3ae66f07a21392e8ba11c
# good: [1bac3450cfb9ea0163e761f3c3751ed0b1340af3] source-hash-4026e1824de8ff9b5d006ae6eba491f91bc4e599
git bisect good 1bac3450cfb9ea0163e761f3c3751ed0b1340af3
# good: [a2876016251884845497fc1905d94e469a999204] source-hash-ce90f99a2d66c2b998ad3f9f028e2ea623a757f5
git bisect good a2876016251884845497fc1905d94e469a999204
# first bad commit: [f91a107c097e5932ba1ed1d88e574e900ebfda00] source-hash-2e053cf5ea4d93a2e1845e795a9c7fe1e08c84af
Comment 18 Kohei Yoshida 2014-07-09 23:57:36 UTC
Go to Tools - Options - LibreOffice - Memory, and check your setting for "Number of objects".  The default is 20.  Increase that to something like 100.  Restart LibreOffice and give this document another try.  Will that improve that situation a bit?
Comment 19 Kohei Yoshida 2014-07-10 02:00:24 UTC
See if this commit

http://cgit.freedesktop.org/libreoffice/core/commit/?id=b023565d4f064cd0312e8c1fcc23a9f552112935

helps the situation.  That change makes chart objects exempt from automatic unloading per this configuration setting.
Comment 20 László Németh 2014-07-15 08:22:21 UTC
(In reply to comment #18)
> Go to Tools - Options - LibreOffice - Memory, and check your setting for
> "Number of objects".  The default is 20.  Increase that to something like
> 100.  Restart LibreOffice and give this document another try.  Will that
> improve that situation a bit?

Yes, it solves the problem in this case. Thanks for this great workaround! But the result depends from the sum of the objects of all opened documents, and it needs to restart LibreOffice after the modified memory settings.
Comment 21 Michael Meeks 2014-07-15 08:34:44 UTC
So - if changing that limit improves things, then Kohei's commit(s) on master should fix this. Can you re-try with a recent master build to verify ?
Comment 22 László Németh 2014-07-15 16:00:42 UTC
(In reply to comment #21)
> So - if changing that limit improves things, then Kohei's commit(s) on
> master should fix this. Can you re-try with a recent master build to verify ?

I tried with a recent master yesterday, so the commits have not solved this regression yet (but I haven't checked it without the second commit).

Changing the memory settings works with older versions, too.

I will check it without the second commit.
Comment 23 László Németh 2014-07-16 20:54:58 UTC
Only with the first commit [From the log: Revert "bnc#883684: Better fix for this." This reverts commit a0bd5587a5ac62974bdb10731d3fd21584521a72.], it works well. It seems, the second commit doesn't fix this issue, but I will check it again.
Comment 24 László Németh 2014-07-17 07:53:21 UTC
I have checked it again, first commit solves this issue, the second one doesn't!
Comment 25 Kohei Yoshida 2014-07-18 15:18:18 UTC
We need a new solution for this.  The first commit is a brute-force, non-elegant way of making the problem go away.  Not the right fix.

We need a way to either reload objects to memory from persistent representation in case it gets notified of data updates, or a way to make such chart objects with active listening exempt from unloading.  The first commit blatantly make all chart objects unloadable, which is not elegant.
Comment 26 Kohei Yoshida 2014-07-18 15:20:02 UTC
BTW, it may not just be chart objects that have listener capability, so we need to fix it correctly so that it would apply to other types of OLE objects with listener capability.
Comment 27 Joel Madero 2014-11-02 15:59:19 UTC
REOPENED is wrong status for this bug as it's not assigned to a developer. That being said, it looks like it's been confirmed so marking as NEW.
Comment 28 tommy27 2014-12-08 08:34:17 UTC
please give an update of the bug status with current LibO 4.3.4.1 or 4.4.0.0 beta.
we are in the process of migrating all 4.2.x MABs which are still reproducible to the MAB 4.3 list
Comment 29 Robinson Tryon (qubit) 2014-12-08 13:39:48 UTC
(In reply to László Németh from comment #0)

Repro steps:
>

Open 'test data' ODS

Click on "Sheet2" to view diagram (and (optionally) compare to screenshot attached to this bug). See that bars are increasing.

> Inserting an empty row before the data rows

Click on the "4" of row 4 and Insert -> Rows

> diagram on the second sheet

Click on "Sheet2" to view diagram again (bars should remain the same, but if the bug is present, will be decreasing)

RESULT:

> diagram on the second sheet will have bad data range (missing data range
> update).

CONFIRMED on Ubuntu 14.04
Testing with LO 4.3.5.1 and 4.4.0.0.beta2

> Undo doesn't help to restore the original data range of the charts.

CONFIRMED as well.

Moving mab4.2 -> mab4.3
Comment 30 tommy27 2015-06-23 10:27:15 UTC
still reproducible with LibO 4.4.3.2 under Win7x64
Comment 31 Robinson Tryon (qubit) 2015-12-13 11:09:27 UTC Comment hidden (obsolete)
Comment 32 Lars Jødal 2016-11-11 07:51:08 UTC
Created attachment 128654 [details]
Simple test case with just one chart on each sheet

Still a problem with LibO v. 5.2.2.2 on both Win10 and WinVista.

The headline and some of the comments relate the problem to having a high number of objects. However, this simple test example with just two charts (on Sheet1 and copy on Sheet2) gives similar problems.

To reproduce:
1) Open the Calc file.
2) Check that the charts on Sheet1 and Sheet2 are identical. They both show the data from Sheet1.
3) Close Calc.
4) Re-open the file.
5) Insert a new row in Sheet1 as the first row (i.e. "above" the data). The chart on Sheet1 is (correctly) not changed.
6) Check the chart of Sheet2. This is now (incorrectly) changed. More precisely, the chart on Sheet2 refers to the original row numbers, not the original data.

In line with comment #10, the chart on Sheet2 _is_ updated if the chart has been viewed (or perhaps only if it has been viewed/updated recently?). This is the reason for step 3-4 above.

It seems from the comments that people agree that this is a serious bug. Yet, it is unsolved after 3 years and development from LibO 4.0 to present 5.2. Is it a very difficult problem? (I know that bugs do not disappear by themselves, and I have a high regard for everybody working on eliminating bugs and developing LibO.)
Comment 34 Lars Jødal 2018-01-09 13:42:01 UTC
Problem still present in the upcoming version 6.0.0 (from test of 6.0.0.1). As mentioned in comment #1, a workaround is to rename the charts before making the change. (This is not very logical, but seems to work.)
Comment 35 Xisco Faulí 2019-04-30 10:08:27 UTC
Dear DHARA BHAVSAR,
This bug has been in ASSIGNED status for more than 3 months without any
activity. Resetting it to NEW.
Please assigned it back to yourself if you're still working on this.
Comment 36 Lars Jødal 2019-04-30 13:19:54 UTC
Changed the headline - as demonstrated by comment 32, the problem is not the number of objects. The word "non-active" in the new headline is a qualified guess - again, see comment 32.

And yes, the bug is stil present - I just tested with the current fresh version 6.2.3

Version: 6.2.3.2 (x64)
Build ID: aecc05fe267cc68dde00352a451aa867b3b546ac
CPU tråde: 4; Styresystem: Windows 10.0; Gengiver af brugergrænseflade: GL; VCL: win; 
Lokalisering: da-DK (da_DK); Sprog for brugergrænseflade: da-DK
Calc: threaded
Comment 37 Lars Jødal 2020-10-09 07:35:09 UTC
Tested in LO 7.0.2.2 according to recipe in comment 32. Bug is still present.

Version: 7.0.2.2 (x64)
Build ID: 8349ace3c3162073abd90d81fd06dcfb6b36b994
CPU threads: 8; OS: Windows 10.0 Build 19041; UI render: Skia/Vulkan; VCL: win
Locale: da-DK (da_DK); UI: en-GB
Calc: threaded
Comment 38 QA Administrators 2022-10-10 03:32:02 UTC Comment hidden (obsolete)
Comment 39 Lars Jødal 2022-12-27 09:36:54 UTC
Some version of this bug is still present in LO 7.4.3.2.

Using the original test data from comment 1, I fail to reproduce the bug. However, using my own test sample from comment 32, I can still reproduce the bug as described in that comment.

(Admission: I have realized that the diagram on Sheet1 is faulty - it has an internal data table rather than referring to the data in the sheet. However, the bug represents itself in the diagram in Sheet2, which DOES refer to the data in Sheet1. I.e., the test case is still relevant as described in comment 32.)

Thus, it is still the case that a diagram that has not been viewed, may fail to update its data references when these data moves, e.g. because of insertion of a row. And it is still the case that viewing the diagram before the insertion somehow "activates" that diagram, causing correct update of the data references in the "activated" diagram.

Version: 7.4.3.2 (x64) / LibreOffice Community
Build ID: 1048a8393ae2eeec98dff31b5c133c5f1d08b890
CPU threads: 4; OS: Windows 10.0 Build 19044; UI render: Skia/Vulkan; VCL: win
Locale: da-DK (da_DK); UI: da-DK
Calc: threaded
Comment 40 banerpunep 2023-04-05 08:19:09 UTC Comment hidden (spam)
Comment 41 emerald 2023-07-10 09:09:11 UTC Comment hidden (spam)
Comment 42 Matt K 2023-07-28 02:04:28 UTC
I'm unable to repro by following the steps in comment 32 using the following version:

Version: 24.2.0.0.alpha0+ (X86_64) / LibreOffice Community
Build ID: 38fc7ddc2eb9cab52d55d233e5049bc39f18d3bd
CPU threads: 4; OS: Windows 10.0 Build 19045; UI render: Skia/Raster; VCL: win
Locale: en-US (en_US); UI: en-US
Calc: threaded

In particular, the chart on both sheets doesn't change when inserting new rows of data above the original data.

Setting to NEEDINFO.
Comment 43 Lars Jødal 2023-08-21 05:21:53 UTC
(In reply to Matt K from comment #42)
> I'm unable to repro by following the steps in comment 32 using the following
> version:
> 
> Version: 24.2.0.0.alpha0+ (X86_64) / LibreOffice Community
> Build ID: 38fc7ddc2eb9cab52d55d233e5049bc39f18d3bd
> CPU threads: 4; OS: Windows 10.0 Build 19045; UI render: Skia/Raster; VCL:
> win
> Locale: en-US (en_US); UI: en-US
> Calc: threaded
> 
> In particular, the chart on both sheets doesn't change when inserting new
> rows of data above the original data.
> 
> Setting to NEEDINFO.

I am mystified by the version number in this report. LO 7.6 is just about the be officially release, so how can it be tested with version 24.2?

Testing the steps in comment 32 with LO 7.6.0.3, I (unfortunately) still reproduce the bug.

Version: 7.6.0.3 (X86_64) / LibreOffice Community
Build ID: 69edd8b8ebc41d00b4de3915dc82f8f0fc3b6265
CPU threads: 8; OS: Windows 10.0 Build 19045; UI render: Skia/Raster; VCL: win
Locale: da-DK (da_DK); UI: da-DK
Calc: CL threaded
Comment 44 Lars Jødal 2024-01-04 10:59:37 UTC
Unfortunately, the bug is still reproduced in LO 24.2 beta (due to be released in February 2024, got it):

Version: 24.2.0.0.beta1 (X86_64) / LibreOffice Community
Build ID: 5f390384195b7264c6e52add9e90a39790285249
CPU threads: 4; OS: Windows 10.0 Build 19045; UI render: Skia/Raster; VCL: win
Locale: da-DK (da_DK); UI: da-DK
Calc: threaded

I am reproducing as in comment 32. As noted there, if Sheet2 is viewed then the bug is NOT seen (until after a re-opening of the document). Is there perhaps some optimization on chart updates focusing on recently used charts, with the unwanted side effects that not-recently used charts are not updated at all, even when the update changes the data area of the chart?

Setting the status back to NEW instead of NEEDINFO.
Comment 45 Steve271 2024-05-01 23:24:57 UTC
Tested following the steps in comment 32, bug is still being reproduced.

Version: 7.3.7.2 / LibreOffice Community
Build ID: 30(Build:2)
CPU threads: 8; OS: Linux 6.5; UI render: default; VCL: gtk3
Locale: en-US (en_US.UTF-8); UI: en-US
Ubuntu package version: 1:7.3.7-0ubuntu0.22.04.4
Calc: threaded