Bug 133764 (Calc-large-spreadsheets)

Summary:	[META] Very large spreadsheets problem
Product:	LibreOffice	Reporter:	Roman Kuznetsov <79045_79045>
Component:	Calc	Assignee:	Not Assigned <libreoffice-bugs>
Status:	NEW ---
Severity:	normal	CC:	aron.budea, gang65, gerry.treppel, gmolleda, mark.g.stewart, michael.meeks, oliver.brinzing, thomas.lendo
Priority:	medium
Version:	7.0.0.0.alpha1+
Hardware:	All
OS:	All
See Also:	https://bugs.documentfoundation.org/show_bug.cgi?id=133619
Whiteboard:
Crash report or crash signature:		Regression By:
Bug Depends on:	91424, 133619, 138426, 148038, 150505, 151049, 50916, 105036, 109061, 133033, 133649, 133730, 133748, 133749, 133751, 133830, 133853, 133880, 134362, 134392, 134550, 135420, 135507, 136156, 136546, 136735, 136818, 137228, 141647, 143214, 144363, 144380, 145839, 146162, 146815, 147451, 147458, 147509, 147830, 147894, 148143, 148147, 149112, 149764, 150430, 150648, 151046, 160399
Bug Blocks:	108252, 109530

Description Roman Kuznetsov 2020-06-07 17:12:31 UTC

Description:
[META] Very large spreadsheets problem

After bug 50916 we have experimental feature that allows have more than 1024 columns and more than 1048576 rows. 

I suggest to store here all bugs/enhs about this (great!) feature

Steps to Reproduce:
-

Actual Results:
-

Expected Results:
-


Reproducible: Always


User Profile Reset: No



Additional Info:
-

Comment 1 Thomas Lendo 2020-06-07 19:35:47 UTC

Roman, Aron Budea already created Bug 133619 a few days ago.

Comment 2 Aron Budea 2020-06-07 20:02:44 UTC

That's fine, bug 133619 is specifically for regressions, ie. that occur with normal-sized spreadsheets after the very-large-spreadsheet-work.
This bug can be for issues that involve very large spreadsheets. These aren't regressions, since such sheets weren't supported in the past.

Comment 3 Seweryn Niemiec 2020-12-07 13:38:24 UTC

Some reproducible crashes on simple columns manipulations:

1. Selecting whole column gives:

Fatal exception: Signal 6
Stack:
/app/libreoffice/program/libuno_sal.so.3(+0x3f513)[0x7f461e7e9513]
/app/libreoffice/program/libuno_sal.so.3(+0x3f773)[0x7f461e7e9773]
/usr/lib/x86_64-linux-gnu/libc.so.6(+0x39800)[0x7f461e517800]
/usr/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x145)[0x7f461e517775]
/usr/lib/x86_64-linux-gnu/libc.so.6(abort+0x127)[0x7f461e500855]
/app/libreoffice/program/libvcllo.so(+0x38ca7b)[0x7f461ade2a7b]
/app/libreoffice/program/libvclplug_gtk3lo.so(+0xe2124)[0x7f4615172124]
/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0(g_main_context_dispatch+0x15f)[0x7f4616f0375f]
/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0(+0x55b08)[0x7f4616f03b08]
/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0(g_main_loop_run+0x83)[0x7f4616f03e23]
/app/libreoffice/program/libvclplug_gtk3lo.so(+0x12de8d)[0x7f46151bde8d]
/app/libreoffice/program/libsvxlo.so(+0x359334)[0x7f461cd1e334]
/app/libreoffice/program/libsofficeapp.so(+0x36333)[0x7f461e6e9333]
/app/libreoffice/program/libsofficeapp.so(+0x36639)[0x7f461e6e9639]
/app/libreoffice/program/libvcllo.so(+0x791cc9)[0x7f461b1e7cc9]
/app/libreoffice/program/libuno_sal.so.3(+0x18992)[0x7f461e7c2992]
/app/libreoffice/program/libuno_sal.so.3(+0x3f6da)[0x7f461e7e96da]
/usr/lib/x86_64-linux-gnu/libc.so.6(+0x39800)[0x7f461e517800]
/usr/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x145)[0x7f461e517775]
/usr/lib/x86_64-linux-gnu/libc.so.6(abort+0x127)[0x7f461e500855]
/app/libreoffice/program/libvcllo.so(+0x38ca7b)[0x7f461ade2a7b]
/app/libreoffice/program/libvclplug_gtk3lo.so(+0xe2124)[0x7f4615172124]
/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0(g_main_context_dispatch+0x15f)[0x7f4616f0375f]
/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0(+0x55b08)[0x7f4616f03b08]
/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0(g_main_context_iteration+0x33)[0x7f4616f03bd3]
/app/libreoffice/program/libvclplug_gtk3lo.so(+0xe3a8c)[0x7f4615173a8c]
/app/libreoffice/program/libvcllo.so(+0x788f72)[0x7f461b1def72]
/app/libreoffice/program/libvcllo.so(_ZN11Application7ExecuteEv+0x45)[0x7f461b1e1595]
/app/libreoffice/program/libsofficeapp.so(+0x3ceee)[0x7f461e6efeee]
/app/libreoffice/program/libvcllo.so(_Z10ImplSVMainv+0x51)[0x7f461b1e98e1]
/app/libreoffice/program/libsofficeapp.so(soffice_main+0xa3)[0x7f461e717773]
/app/libreoffice/program/soffice.bin(+0x10b0)[0x561214dca0b0]
/usr/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf2)[0x7f461e502022]
/app/libreoffice/program/soffice.bin(+0x10ee)[0x561214dca0ee]

and crash.

2. Changing column width gives:

LibreOffice 7.0 - Fatal Error: multi_type_vector::position#1750: block position not found! (logical pos=15999999, block size=2, logical size=1048576)

and crash.

Document format: xlsx (loading .ods converted by Gnumeric gives the same results)
Document stats: 887459 cells, 407 rows
LibreOffice version: 7.0.3.1
Fresh installation from flatpack.

Unfortunately the document contains sensitive medical information and I can't share it.
In Gnumeric it works properly.

Comment 4 Roman Kuznetsov 2020-12-07 16:37:03 UTC

(In reply to Seweryn Niemiec from comment #3)
> Some reproducible crashes on simple columns manipulations:
> 
> 1. Selecting whole column gives:
> 
> Fatal exception: Signal 6
> Stack:
> /app/libreoffice/program/libuno_sal.so.3(+0x3f513)[0x7f461e7e9513]
> /app/libreoffice/program/libuno_sal.so.3(+0x3f773)[0x7f461e7e9773]
> /usr/lib/x86_64-linux-gnu/libc.so.6(+0x39800)[0x7f461e517800]
> /usr/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x145)[0x7f461e517775]
> /usr/lib/x86_64-linux-gnu/libc.so.6(abort+0x127)[0x7f461e500855]
> /app/libreoffice/program/libvcllo.so(+0x38ca7b)[0x7f461ade2a7b]
> /app/libreoffice/program/libvclplug_gtk3lo.so(+0xe2124)[0x7f4615172124]
> /usr/lib/x86_64-linux-gnu/libglib-2.0.so.
> 0(g_main_context_dispatch+0x15f)[0x7f4616f0375f]
> /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0(+0x55b08)[0x7f4616f03b08]
> /usr/lib/x86_64-linux-gnu/libglib-2.0.so.
> 0(g_main_loop_run+0x83)[0x7f4616f03e23]
> /app/libreoffice/program/libvclplug_gtk3lo.so(+0x12de8d)[0x7f46151bde8d]
> /app/libreoffice/program/libsvxlo.so(+0x359334)[0x7f461cd1e334]
> /app/libreoffice/program/libsofficeapp.so(+0x36333)[0x7f461e6e9333]
> /app/libreoffice/program/libsofficeapp.so(+0x36639)[0x7f461e6e9639]
> /app/libreoffice/program/libvcllo.so(+0x791cc9)[0x7f461b1e7cc9]
> /app/libreoffice/program/libuno_sal.so.3(+0x18992)[0x7f461e7c2992]
> /app/libreoffice/program/libuno_sal.so.3(+0x3f6da)[0x7f461e7e96da]
> /usr/lib/x86_64-linux-gnu/libc.so.6(+0x39800)[0x7f461e517800]
> /usr/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x145)[0x7f461e517775]
> /usr/lib/x86_64-linux-gnu/libc.so.6(abort+0x127)[0x7f461e500855]
> /app/libreoffice/program/libvcllo.so(+0x38ca7b)[0x7f461ade2a7b]
> /app/libreoffice/program/libvclplug_gtk3lo.so(+0xe2124)[0x7f4615172124]
> /usr/lib/x86_64-linux-gnu/libglib-2.0.so.
> 0(g_main_context_dispatch+0x15f)[0x7f4616f0375f]
> /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0(+0x55b08)[0x7f4616f03b08]
> /usr/lib/x86_64-linux-gnu/libglib-2.0.so.
> 0(g_main_context_iteration+0x33)[0x7f4616f03bd3]
> /app/libreoffice/program/libvclplug_gtk3lo.so(+0xe3a8c)[0x7f4615173a8c]
> /app/libreoffice/program/libvcllo.so(+0x788f72)[0x7f461b1def72]
> /app/libreoffice/program/libvcllo.
> so(_ZN11Application7ExecuteEv+0x45)[0x7f461b1e1595]
> /app/libreoffice/program/libsofficeapp.so(+0x3ceee)[0x7f461e6efeee]
> /app/libreoffice/program/libvcllo.so(_Z10ImplSVMainv+0x51)[0x7f461b1e98e1]
> /app/libreoffice/program/libsofficeapp.so(soffice_main+0xa3)[0x7f461e717773]
> /app/libreoffice/program/soffice.bin(+0x10b0)[0x561214dca0b0]
> /usr/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf2)[0x7f461e502022]
> /app/libreoffice/program/soffice.bin(+0x10ee)[0x561214dca0ee]
> 
> and crash.
> 
> 2. Changing column width gives:
> 
> LibreOffice 7.0 - Fatal Error: multi_type_vector::position#1750: block
> position not found! (logical pos=15999999, block size=2, logical
> size=1048576)
> 
> and crash.
> 
> Document format: xlsx (loading .ods converted by Gnumeric gives the same
> results)
> Document stats: 887459 cells, 407 rows
> LibreOffice version: 7.0.3.1
> Fresh installation from flatpack.
> 
> Unfortunately the document contains sensitive medical information and I
> can't share it.
> In Gnumeric it works properly.

please create a separate bug report, this META is for catalogize all bugs about very large table

Comment 5 Gerry 2021-05-26 10:15:31 UTC

Wouldn't it make sense to put this meta-bug on the agenda of the weekly ESC call to discuss/coordinate it? The meta bug "Very large spreadsheets problem" seems to touch different sub-systems of LibreOffice (at least: filters, Calc data structure, UI), which might need more coordination. This meta-bug is a prerequisite to finally enable >1024 columns by default.

Comment 6 Justin L 2021-12-08 08:47:00 UTC

> How to enable support for very large spreadsheets in LO7 RC2 Calc?
Via the following steps in Tools -> Options...:
- First under LibreOffice -> Advanced, check Enable experimental features (may be unstable),
- After a restart, under LibreOffice Calc -> Defaults, check Enable very large spreadsheets (16m rows, 16384 cols).

Comment 7 Miklos Vajna 2022-03-30 14:50:01 UTC

Just to avoid outdated info, 16k cols are now the default on master towards 7.4, see <http://llunak.blogspot.com/2022/03/enabling-calc-support-for-16384-columns.html>. (No change for rows.)