Bug 33393 - After autosave image shows read-error and will be lost on save
Summary: After autosave image shows read-error and will be lost on save
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
3.3.0 release
Hardware: All All
: medium critical
Assignee: Michael Stahl (allotropia)
URL:
Whiteboard: target:3.6.0 target:3.5.1
Keywords:
: 35631 45167 (view as bug list)
Depends on: Image-Caching
Blocks: mab3.5
  Show dependency treegraph
 
Reported: 2011-01-23 21:24 UTC by Bryan Quigley
Modified: 2012-10-30 14:15 UTC (History)
15 users (show)

See Also:
Crash report or crash signature:


Attachments
Read_Error_file (37.13 KB, application/vnd.oasis.opendocument.text)
2011-01-23 21:24 UTC, Bryan Quigley
Details
File at the start of editing. (583.92 KB, application/vnd.oasis.opendocument.text)
2012-02-16 18:48 UTC, Andrew Pitonyak
Details
File saved the moment the bug appeared. (212.56 KB, application/vnd.oasis.opendocument.text)
2012-02-16 18:49 UTC, Andrew Pitonyak
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Bryan Quigley 2011-01-23 21:24:00 UTC
Created attachment 42354 [details]
Read_Error_file

Attaching file that exhibits the issue.  I have manually remove over 22
graphics from the file (removing from content.xml, manifest.xml and pictures
folder when unzipped).  I had to do this manually due to the issue that when I
resave the file, it no longer will exhibit this behavior.  This behavior occurred originally without any manual editing of odt files, but is much more difficult to reproduce.

This is a long-standing regression compared to OpenOffice.org

Steps to reproduce:
Download the file to the desktop (DO NOT OPEN DIRECTLY)
Open the file
Set autorecovery on and to 1 minute.
Change/Add text on the first page (delete 09)
Wait at least one minute, maybe two
Note the Read-Error Message

For the data loss part, simply save the file, then reopen.

This bug has been reported elsewhere:
https://bugs.launchpad.net/openoffice/+bug/157249
https://bugzilla.novell.com/show_bug.cgi?id=440795
Comment 1 Rainer Bielefeld Retired 2011-01-23 22:10:16 UTC
I see the effect with "LibreOffice 3.3.0 RC4 - WIN7  Home Premium (64bit) German UI  [OOO330m19 (build 6 / tag 3.3.0.4)]". The question is what we can do, currently it seems to be a damaged document causing some problems. The effect seems to disappear when I copy / paste all contents to a new WRITER document, but I also was not able to reproduce the problem with the original document in further tests.

@Bryan Quigley:
Please contribute information concerning your Platform and OS!
Any chance to reproduce
Comment 2 Bryan Quigley 2011-01-23 22:17:06 UTC
I believe the file was originally created on Ubuntu 9.04 or 9.10 and the corresponding version of OpenOffice, but I am not entirely sure.
Comment 3 Bryan Quigley 2011-01-23 22:18:33 UTC
Once you save the file, the issue will no longer reappear reliably.   The method I used to make this file, was the best I could do at making it easy to reproduce the effect.
Comment 4 Steve Edmonds 2011-03-17 00:38:07 UTC
Hi.
I had this in OO a year or so ago, think I filed a bug, can't remember.
I have a feeling it related to EPS images in my case and then images just kept disappearing. What was happening was that the linking was damaged. 
I.e after the the save when the image was read-error I could go into the odt file and the image was still there, what had broken was the reference to the image.
Comment 5 Frédéric Buclin 2011-05-30 14:12:18 UTC
I can reproduce with LibO 3.4rc2. Why is the severity "normal" only? Dataloss is pretty critical.
Comment 6 Rainer Bielefeld Retired 2011-05-30 21:45:49 UTC
(In reply to comment #5)
> I can reproduce 

With what documents?
Comment 7 Don't use this account, use tml@iki.fi 2011-05-31 01:41:21 UTC
> Why is the severity "normal" only?

Because the severity field is fairly meaningless here, as anybody can change it.
Comment 8 Frédéric Buclin 2011-05-31 15:03:32 UTC
(In reply to comment #6)
> With what documents?

With the document attached to this bug, as well as some of my own documents used at work.
Comment 9 Andreas 2011-09-07 05:33:18 UTC
I have this also - and have had it with several versions of OOo / LO.

I can't say exactly how it happens, but I did most of my work on a linux box, exchanging documents with Windows users I begged and finnally convinced to download LO. Yey very often when email a version back and forth - this would happen.

Currently I am on a mac with LibreOffice 3.4.1 (Build:103) and it still happens. My collaborators does not complain about it, so I wonder if it is a *nix thing.

I really hope this will get some attention. Please tell me how I can help (technical bugfixing skillz = zero)
Comment 10 Eugene Bodrero 2011-11-17 15:51:10 UTC
I am having the same problem with Read Error replacing my graphics in LO Writer 3.4.4 (build 402) since switching to LO. I am using native ODT format and occurs with several of my ODT docs. The images are actually lost from the data, 8 MB file becomes 1 MB file. 

Documents were originally created in OO. We did not have the problem until moving to LO 3.3. I had hoped that 3.4.4 would resolve this significant issue.

Initially we thought this was related to errors caused by MAC users. We did some testing and opened the file under Mac LO 3.4.4, saved, closed, then opened in the same version with Windows 7 LO. The images were present, then at some point they were replaced with Read Error. Mac user does not have the same problem. This error was not seen when alternately opening OO files between Mac and Windows.

Error occurs whether files are on my local Windows 7 machine or on the network drive.

My testing shows that corruption occurs during autosave. Manually saving the file first does not eliminate the error. I am able to reproduce the problem easily.

Eugene
Comment 11 Eugene Bodrero 2011-11-18 10:29:35 UTC
We spent a few hours going back and forth between Windows and Mac, opening vulnerable LO files in Mac and Windows, pasting the content into new files, saving, using network and local file copies, and editing with autosave enabled and disabled. The Read Error for the images is associated with autosave and occurs under Windows and Mac regardless of the other variables.

All of the images are png. We did find that a small png button that we placed during this testing was not lost, only the larger images.

We edited a file in Lotus Symphony and brought it back into LO in Windows and the Read Error still occurred with Autosave. Older files from OO also experienced the Read Error with autosave.

We have used OO for years to create user manuals, which contain many images and extensive use of styles. In addition to the image read error bug, we also found that when an image is inserted, the document view jumps to the bottom of the document. This is an undesirable characteristic that was introduced with LO, and not present in OO.

IBM Lotus Symphony opens the ODT docs without error, but under Mac it only opens the files on our server as read-only. Symphony was stable; autosave worked correctly, and image insertion did not change the document view. However, Symphony does not seem to have the math component which we used for formula objects, although it did allow editing of existing math objects. Nevertheless, it is a very good replacement candidate for those looking to avoid the mentioned LO bugs.

Corel does a poor job of opening ODT files, the formatting was completely lost.

For now, we are reverting to OO 3.3.0 until compelled to make a change or these LO bugs are fixed.

Eugene
Comment 12 Cor Nouws 2011-11-19 02:03:24 UTC
Never seen this problem with whatever old files I use. 

So to be able to track down to the source of this partictular issue (some damaged file, as suggested in an earlier comment?), having a simple test document would be useful.
Comment 13 Daniel 2011-12-15 02:34:54 UTC
We experience the same problem in a mixed network environment. It seems that the error appears mostly (maybe only) when our Mac users edit the documents. We are using LO 3.3.2 on the Mac which connects by SMB to a Linux server. The documents are in ODT format and created with LO.

This bug is very critical for us since it leads to data loss. However, it is difficult to give a file that reproduces the problem since you never know which image will disappear.
Saving the document after getting the "Read Error" will lead to reduced file size as reported previously, corresponding to the amount of data in the lost image.
Comment 14 Rainer Bielefeld Retired 2011-12-15 11:02:40 UTC
I also observed picture loss with an own document and Parallel Dev-Installation of  "LibreOffice 3.5.0 Beta1 - WIN7 Home Premium (64bit) German UI [Build-ID: 7362ca8-b5a8e65-af86909-d471f98-61464c4] Windows_Release_Configuration  11-Dec-2011 06:51". Not 100% sure that autosave is the reason.
Pictures will not reappear when I close and reopen the document.
I will try to get a sample document from the confidential test document.

"Bug 35631 - FILESAVE Images fail to show and are removed after autosave" might be a DUP
Comment 15 Steve Edmonds 2011-12-15 23:10:49 UTC
*** Bug 35631 has been marked as a duplicate of this bug. ***
Comment 16 Steve Edmonds 2011-12-15 23:15:15 UTC
Since disabling autosave and backup early this year I have not experienced this problem.
Comment 17 Rainer Bielefeld Retired 2011-12-16 23:36:10 UTC
I sometimes see the effect, but until now I can't reproduce it in a reliable way. Simply opening  a document and waiting (with 1 minute autosave interval) does not show the problem.
Comment 18 Björn Michaelsen 2011-12-23 11:49:48 UTC
[This is an automated message.]
This bug was filed before the changes to Bugzilla on 2011-10-16. Thus it
started right out as NEW without ever being explicitly confirmed. The bug is
changed to state NEEDINFO for this reason. To move this bug from NEEDINFO back
to NEW please check if the bug still persists with the 3.5.0 beta1 or beta2 prereleases.
Details on how to test the 3.5.0 beta1 can be found at:
http://wiki.documentfoundation.org/QA/BugHunting_Session_3.5.0.-1

more detail on this bulk operation: http://nabble.documentfoundation.org/RFC-Operation-Spamzilla-tp3607474p3607474.html
Comment 19 Rainer Bielefeld Retired 2011-12-25 00:21:50 UTC
Related to "Bug 32461 - Graphics disappear from document"

Inappropriate NEEDINFO reverted to NEW
Comment 20 Giuseppe Zizza 2011-12-31 04:46:09 UTC
I can confirm this bug too for both Windows and OSX version and it's also a quite annoing one.
Comment 21 Cor Nouws 2012-01-24 04:57:45 UTC
*** Bug 45167 has been marked as a duplicate of this bug. ***
Comment 22 Cor Nouws 2012-01-24 05:18:07 UTC
I hope that the description in Bug 45167 is helpful to reproduce the problem.
Comment 23 Cor Nouws 2012-02-06 11:27:12 UTC
@ someone:
pls try the description here:  
  https://bugs.freedesktop.org/show_bug.cgi?id=45167#c0

I tried again, and after some various work and returning to the document, the images are los: read-error!
Comment 24 Jakub Cerny 2012-02-14 08:44:39 UTC
We are experiencing this bug very severely on OS X Lion and 3.5 version. All our templates are unusable because we are losing all logos and graphic elements put into them. This happens both on local files and files on SMB shares.
Comment 25 Rainer Bielefeld Retired 2012-02-14 08:50:08 UTC
@Jakub Cerny
Are you able to create a test kit that reliably reproduces the bug with definitively proper documents? 

@Michael:
May be you can try to get some progress here? The bug is not easy to reproduce, but for those who suffer from it it's worse than hell.
Please feel free to reassign (or reset Assignee to default) if it’s not your area or if provided information is not sufficient. Please set Status to ASSIGNED if you accept this Bug.
Comment 26 Cor Nouws 2012-02-14 11:29:37 UTC
(In reply to comment #25)

> Are you able to create a test kit that reliably reproduces the bug with definitively proper documents? 

here (striked because duplicate, not because it's solved.)
  https://bugs.freedesktop.org/show_bug.cgi?id=45167#c0
Also see comment #23 in this issue.
Comment 27 Andrew Pitonyak 2012-02-16 18:46:04 UTC
Trying to update documentation while recording changes on 64-bit Fedora 16 using LO 3.5 release version. Some of the graphics showed the "read error". 

when I checked the new file, 12 of the 19 contained PNGs are simply gone. I mean they are missing from the internal picture directory.... and not too surprising, they are also not in the manifest. I looked at content.xml and found the frame should contain one of the missing images and it was simply not there.... completely gone. I will see if I can reproduce. On the plus side, I was tracking changes, so you can see everything that I did right up until the graphics went away!
Comment 28 Andrew Pitonyak 2012-02-16 18:48:09 UTC
Created attachment 57180 [details]
File at the start of editing.

I started editing this file, enabled change tracking..... And then started making changes.
Comment 29 Andrew Pitonyak 2012-02-16 18:49:30 UTC
Created attachment 57181 [details]
File saved the moment the bug appeared.
Comment 30 Steve Edmonds 2012-02-16 21:35:11 UTC
I don't know if it is any help, but way back when I had this problem in OOO I looked into the files after the read error. First I noticed that the image file names in the zip file had changed, then a few saves later they disappeared from the zip file. It was as if the image file names got changed breaking the internal linking. I am pretty much over that bug now that I turn all auto-save and backup off.
Comment 31 Andrew Pitonyak 2012-02-18 13:26:57 UTC
Using Bryan's document, I am able to reproduce at will. Now if only I knew the code well enough to find the problem. I asked for help on the dev mailing list for some pointers as to where to look.
Comment 32 Michael Meeks 2012-02-20 12:11:08 UTC
I reproduced the issue here - nasty. Reading the utterly badly designed GraphicManager for inspiration ... :-)
Comment 33 Michael Meeks 2012-02-20 12:45:11 UTC
XML delta is:
- <draw:image xlink:href="Pictures/200004AD0000475F000033B381B9C98F.svm" xlink:type="simple" xlink:show="embed" xlink:actuate="onLoad"/>
+ <draw:image/>
Comment 34 Michael Meeks 2012-02-20 13:03:59 UTC
with a dbgutil build I get this on autosave:

warn:legacy.osl:32088:1:/data/opt/libreoffice/master/sw/source/core/graphic/ndgrf.cxx:810: <SwGrfNode::_GetStreamForEmbedGrf(..)> - embedded graphic file not found!
warn:legacy.osl:32088:1:/data/opt/libreoffice/master/sw/source/core/graphic/ndgrf.cxx:475: Grafik kann nicht eingeswapt werden
warn:legacy.osl:32088:1:/data/opt/libreoffice/master/sw/source/core/graphic/ndgrf.cxx:810: <SwGrfNode::_GetStreamForEmbedGrf(..)> - embedded graphic file not found!
warn:legacy.osl:32088:1:/data/opt/libreoffice/master/sw/source/core/graphic/ndgrf.cxx:475: Grafik kann nicht eingeswapt werden

Breakpoint 1, SwGrfNode::_GetStreamForEmbedGrf (this=0x9042af0, _refPics=..., _aStrmName="200004AD0000475F000033B367F3281F.svm")
    at /data/opt/libreoffice/master/sw/source/core/graphic/ndgrf.cxx:810
810	            OSL_FAIL( "<SwGrfNode::_GetStreamForEmbedGrf(..)> - embedded graphic file not found!" );
(gdb) l
805	            uno::Reference < io::XStream > refStrm = _refPics->openStreamElement( _aStrmName, embed::ElementModes::READ );
806	            pStrm = utl::UcbStreamHelper::CreateStream( refStrm );
807	        }
808	        else
809	        {
810	            OSL_FAIL( "<SwGrfNode::_GetStreamForEmbedGrf(..)> - embedded graphic file not found!" );
811	        }
812	    }
813	
814	    return pStrm;
(gdb) p _aStrmName
$1 = "200004AD0000475F000033B367F3281F.svm"

#0  SwGrfNode::_GetStreamForEmbedGrf (this=0x9042af0, _refPics=..., _aStrmName="200004AD0000475F000033B367F3281F.svm")
    at /data/opt/libreoffice/master/sw/source/core/graphic/ndgrf.cxx:810
#1  0xad85ae45 in SwGrfNode::SwapIn (this=0x9042af0, bWaitForData=0 '\000') at /data/opt/libreoffice/master/sw/source/core/graphic/ndgrf.cxx:452
#2  0xad72a7f2 in SwNoTxtFrm::PaintPicture (this=0x90525b8, pOut=0x8b18c0c, rGrfArea=...)
    at /data/opt/libreoffice/master/sw/source/core/doc/notxtfrm.cxx:869
#3  0xad728ccc in SwNoTxtFrm::Paint (this=0x90525b8, rRect=...) at /data/opt/libreoffice/master/sw/source/core/doc/notxtfrm.cxx:319
#4  0xad8deee1 in SwLayoutFrm::Paint (this=0x9052448, rRect=...) at /data/opt/libreoffice/master/sw/source/core/layout/paintfrm.cxx:3255
#5  0xad8e154e in SwFlyFrm::Paint (this=0x9052448, rRect=...) at /data/opt/libreoffice/master/sw/source/core/layout/paintfrm.cxx:3928
#6  0xad7bd326 in SwVirtFlyDrawObj::wrap_DoPaintObject (this=0x9052520) at /data/opt/libreoffice/master/sw/source/core/draw/dflyobj.cxx:533
#7  0xad7bc869 in drawinglayer::primitive2d::SwVirtFlyDrawObjPrimitive::get2DDecomposition (this=0x9400198, rViewInformation=...)
    at /data/opt/libreoffice/master/sw/source/core/draw/dflyobj.cxx:274
#8  0xb5c77067 in drawinglayer::processor2d::VclPixelProcessor2D::processBasePrimitive2D(drawinglayer::primitive2d::BasePrimitive2D const&) ()
   from /data/opt/OOInstall/program/libmergedlo.so
#9  0xb5c64d8c in drawinglayer::processor2d::BaseProcessor2D::process(com::sun::star::uno::Sequence<com::sun::star::uno::Reference<com::sun::star::graphic::XPrimitive2D> > const&) () from /data/opt/OOInstall/program/libmergedlo.so
#10 0xb6654af5 in sdr::contact::ObjectContactOfPageView::DoProcessDisplay(sdr::contact::DisplayInfo&) ()
   from /data/opt/OOInstall/program/libmergedlo.so
#11 0xb6654c63 in sdr::contact::ObjectContactOfPageView::ProcessDisplay(sdr::contact::DisplayInfo&) ()
   from /data/opt/OOInstall/program/libmergedlo.so
#12 0xb667408c in SdrPageWindow::RedrawLayer(unsigned char const*, sdr::contact::ViewObjectContactRedirector*) const ()
   from /data/opt/OOInstall/program/libmergedlo.so
#13 0xb6715a94 in SdrPageView::DrawLayer(unsigned char, OutputDevice*, sdr::contact::ViewObjectContactRedirector*) const ()
   from /data/opt/OOInstall/program/libmergedlo.so
#14 0xadbb469f in SwViewImp::PaintLayer (this=0x9058548, _nLayerID=1 '\001', pPrintData=0x0, _pPageBackgrdColor=0xbfc465b4, _bIsPageRightToLeft=
    false, pRedirector=0xbfc465d0) at /data/opt/libreoffice/master/sw/source/core/view/vdraw.cxx:148
#15 0xad8de187 in SwRootFrm::Paint (this=0x9059920, rRect=..., pPrintData=0x0)
    at /data/opt/libreoffice/master/sw/source/core/layout/paintfrm.cxx:2992
#16 0xadbbd0e0 in ViewShell::Paint (this=0x9058390, rRect=...) at /data/opt/libreoffice/master/sw/source/core/view/viewsh.cxx:1678
#17 0xad6201da in SwCrsrShell::Paint (this=0x9058390, rRect=...) at /data/opt/libreoffice/master/sw/source/core/crsr/crsrsh.cxx:1165
#18 0xadd9fcbf in SwEditWin::Paint (this=0x903d7b8, rRect=...) at /data/opt/libreoffice/master/sw/source/ui/docvw/edtwin2.cxx:534

Seems the image ID got corrupted: some change in the SVM stream perhaps:

(gdb) p GetGrfObj().GetUniqueID()
$6 = "200004AD0000475F000033B 367F3281F"
vs.
Pictures/200004AD0000475F000033B3 81B9C98F.svm

I guess [ spaces added to separate the image checksum at end of id ].
Comment 35 Michael Meeks 2012-02-20 13:17:45 UTC
adding:

--- a/sw/source/core/graphic/ndgrf.cxx
+++ b/sw/source/core/graphic/ndgrf.cxx
@@ -798,6 +798,13 @@ SvStream* SwGrfNode::_GetStreamForEmbedGrf(
             }
         }
 
+        fprintf( stderr, "look for '%s' %d\n",
+                 rtl::OUStringToOString( _aStrmName, RTL_TEXTENCODING_UTF8 ).getStr(),
+                 _refPics->hasByName( _aStrmName ) );
+
+        fprintf( stderr, "look for [200004AD0000475F000033B381B9C98F.svm] %d\n",
+                 _refPics->hasByName( rtl::OUString::createFromAscii("200004AD0000475F000033B381B9C98F.svm" ) ) );
+
         // assure that graphic file exist in the storage.
         if ( _refPics->hasByName( _aStrmName ) &&
              _refPics->isStreamElement( _aStrmName ) )

Shows that it is indeed the 2nd autosave that fails, and/or the first autosave that busts things:

look for '200004AD0000475F000033B381B9C98F.svm' 1
look for [200004AD0000475F000033B381B9C98F.svm] 1
look for '200004AD0000475F000033B381B9C98F.svm' 1
look for [200004AD0000475F000033B381B9C98F.svm] 1
...
look for '200004AD0000475F000033B381B9C98F.svm' 1
look for [200004AD0000475F000033B381B9C98F.svm] 1
look for '200004AD0000475F000033B381B9C98F.svm' 1
look for [200004AD0000475F000033B381B9C98F.svm] 1
...
look for '200004AD0000475F000033B381B9C98F.svm' 1
look for [200004AD0000475F000033B381B9C98F.svm] 1
look for '200006B1000048A900003B26E43FDA1F.svm' 1
look for [200004AD0000475F000033B381B9C98F.svm] 1
look for '200006B1000048A900003B26AFABBAD6.svm' 0
look for [200004AD0000475F000033B381B9C98F.svm] 1
warn:legacy.osl:32664:1:/data/opt/libreoffice/master/sw/source/core/graphic/ndgrf.cxx:817: <SwGrfNode::_GetStreamForEmbedGrf(..)> - embedded graphic file not found!
warn:legacy.osl:32664:1:/data/opt/libreoffice/master/sw/source/core/graphic/ndgrf.cxx:475: Grafik kann nicht eingeswapt werden
look for '200004AD0000475F000033B367F3281F.svm' 0
look for [200004AD0000475F000033B381B9C98F.svm] 1
warn:legacy.osl:32664:1:/data/opt/libreoffice/master/sw/source/core/graphic/ndgrf.cxx:817: <SwGrfNode::_GetStreamForEmbedGrf(..)> - embedded graphic file not found!
warn:legacy.osl:32664:1:/data/opt/libreoffice/master/sw/source/core/graphic/ndgrf.cxx:475: Grafik kann nicht eingeswapt werden

And the cockup is all related to the last checksum field of the ID: fun ! :-)
Comment 36 Michael Meeks 2012-02-20 13:23:13 UTC
Interestingly, if I load and immediately save I get a document with the images included, -but- this:

look for '200004AD0000475F000033B367F3281F.svm' 1
look for [200004AD0000475F000033B381B9C98F.svm] 0
look for '200004AD0000475F000033B367F3281F.svm' 1
look for [200004AD0000475F000033B381B9C98F.svm] 0

So - the new checksums in this case are being found; a quick check in the .zip file shows the svms have been renamed Pictures/ in the .odt:

before:
200004AD0000475F000033B381B9C98F.svm  200006B1000048A900003B26E43FDA1F.svm
after:
200004AD0000475F000033B367F3281F.svm  200006B1000048A900003B26AFABBAD6.svm

So - I guess, the act of saving is updating these URLs to point to new names that are allocated for the same svm in the backup file, but not renaming the local files in the document that shares these GraphicObjects.

I guess this is the ultimate joy from the $#%#$%ing decision to use random string names for all of this lot - which (in turn) I would argue comes from the decision to use UNO - which makes string / id passing 'obvious' ;-)

Wunderbar !
Comment 37 Michael Meeks 2012-02-20 13:42:41 UTC
vcl/source/gdi/gdimtf.cxx (GDIMetaFile::GetChecksum) was altered:

commit d1da85e8f5d3b6a0c5d1fdcc4fedb0eff5e90b65
Author: ka <kai.ahrens@oracle.com>
Date:   Fri Feb 4 14:49:25 2011 +0100

    ka102: SVG import implementation

But - *surely* we can't be fragile on this algorithm changing; it should be fine to have any name for a file in Pictures/ surely !?
Comment 38 Michael Meeks 2012-02-20 13:49:44 UTC
The difference between the serialized versions before and after appears to be an upgrade of the meta-file format:

before:
00000000  56 43 4c 4d 54 46 01 00  31 00 00 00 01 00 00 00  |VCLMTF..1.......|
00000010  01 00 1b 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000020  01 00 00 00 01 00 00 00  01 00 00 00 01 00 00 00  |................|
00000030  00 5f 47 00 00 b3 33 00  00 ad 04 00 00 84 00 01  |._G...3.........|
after:
00000000  56 43 4c 4d 54 46 02 00  32 00 00 00 01 00 00 00  |VCLMTF..2.......|
00000010  01 00 1b 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000020  01 00 00 00 01 00 00 00  01 00 00 00 01 00 00 00  |................|
00000030  00 5f 47 00 00 b3 33 00  00 ad 04 00 00 01 84 00  |._G...3.........|

I assume we upgrade it on the auto-save. The files are rather different after the first four lines; diff -u shows little similarity after then.
Comment 39 Michael Meeks 2012-02-21 08:17:46 UTC
With:

--- a/svx/source/xml/xmlgrhlp.cxx
+++ b/svx/source/xml/xmlgrhlp.cxx
@@ -621,6 +621,8 @@ sal_Bool SvXMLGraphicHelper::ImplWriteGraphic( const ::rtl::OUString& rPictureSt
                 }
                 else if( aGraphic.GetType() == GRAPHIC_GDIMETAFILE )
                 {
+                    fprintf (stderr, "xmlgrhlp.cxx - write meta-file ! 0x%lx\n",
+                             (long)aGraphic.GetChecksum() );
                     pStream->SetVersion( SOFFICE_FILEFORMAT_8 );
                     pStream->SetCompressMode( COMPRESSMODE_ZBITMAP );
 
@@ -643,6 +645,9 @@ sal_Bool SvXMLGraphicHelper::ImplWriteGraphic( const ::rtl::OUString& rPictureSt
                         rMtf.Write( *pStream, GDIMETAFILE_WRITE_REPLACEMENT_RENDERGRAPHIC );
 
                     bRet = ( pStream->GetError() == 0 );
+
+                    fprintf (stderr, "xmlgrhlp.cxx - done write meta-file ! 0x%lx\n",
+                             (long)aGraphic.GetChecksum() );


I get:

xmlgrhlp.cxx - write meta-file ! 0x67f3281f
xmlgrhlp.cxx - done write meta-file ! 0x67f3281f
xmlgrhlp.cxx - write meta-file ! 0xafabbad6
xmlgrhlp.cxx - done write meta-file ! 0xafabbad6

Which seems to suggest that the id change is over & done by the time we get to writing the graphic out; and/or is not immediately caused by the writeout.
Comment 40 Michael Meeks 2012-02-21 09:48:57 UTC
@@ -387,6 +398,9 @@ sal_Bool SwGrfNode::ImportGraphic( SvStream& rStrm )
     const String aGraphicURL( aGrfObj.GetUserData() );
     if( !GraphicFilter::GetGraphicFilter().ImportGraphic( aGraphic, aGraphicURL, rStrm ) )
     {
+        fprintf (stderr, "Very curious set User Data of '%s' vs 0x%lx\n",
+                 rtl::OUStringToOString(aGraphicURL, RTL_TEXTENCODING_UTF8).getStr(),
+                 aGraphic.GetChecksum());
         aGrfObj.SetGraphic( aGraphic );
         aGrfObj.SetUserData( aGraphicURL );
         return sal_True;

Shows:

Very curious set User Data of 'vnd.sun.star.Package:Pictures/200004AD0000475F000033B381B9C98F.svm' vs 0x67f3281f

which we would expect I guess, the URL name is set to the user data of the graphic there.
Comment 41 Michael Meeks 2012-02-21 10:16:51 UTC
By now, I'm highly suspicious of:

void SwXMLTextParagraphExport::setTextEmbeddedGraphicURL(
    const Reference < XPropertySet >& rPropSet,
    OUString& rURL) const
{
    if( rURL.isEmpty() )
        return;

    SwGrfNode *pGrfNd = GetNoTxtNode( rPropSet )->GetGrfNode();
    if( !pGrfNd->IsGrfLink() )
    {
        String aNewURL( RTL_CONSTASCII_USTRINGPARAM("vnd.sun.star.Package:") );
        aNewURL += String(rURL);
        pGrfNd->SetNewStreamName( aNewURL );

        // #i15411# save-as will swap all graphics in; we need to swap
        // them out again, to prevent excessive memory use
        pGrfNd->SwapOut();
    }
}

Which sets a new stream name for these images:

@@ -827,6 +848,9 @@ void SwGrfNode::_GetStreamStorageNames( String& rStrmName,
     if( !aUserData.Len() )
         return;
 
+    fprintf (stderr, "UserData '%s' NewStrmName '%s'\n",
+             rtl::OUStringToOString(aUserData, RTL_TEXTENCODING_UTF8).getStr(),
+             rtl::OUStringToOString(aNewStrmName, RTL_TEXTENCODING_UTF8).getStr());

shows wonders like:

UserData 'vnd.sun.star.Package:Pictures/200004AD0000475F000033B381B9C98F.svm' NewStrmName 'vnd.sun.star.Package:Pictures/200004AD0000475F000033B367F3281F.svm'

just before things go completely pear-shaped :-)

So - I guess, the ambiguity of what a vnd.sun.star.Package: URL points at: presumably not into the autosave package ;-) bites.

I hadn't realised that on top of the poor choice of string naming of images, we have the poor choice of ambiguous URLs.
Comment 42 Michael Meeks 2012-02-22 07:22:29 UTC
the txtparae.cxx code that does the export does:

    // xlink:href
    OUString sOrigURL;
    rPropSet->getPropertyValue( sGraphicURL ) >>= sOrigURL;

This calls into /data/opt/libreoffice/master/sw/source/core/unocore/unoframe.cxx:1402ish that in a nutshell returns:
          pGrfNode->GetGrfObj().GetUniqueID()
Which is:
(gdb) p sOrigURL
$9 = "vnd.sun.star.GraphicObject:200004AD0000475F000033B367F3281F"

    OUString sURL(GetExport().AddEmbeddedGraphicObject( sOrigURL ));

This saves the embedded object into the output stream giving it a new name:

(gdb) p sURL
$11 = "Pictures/200004AD0000475F000033B367F3281F.svm"

    setTextEmbeddedGraphicURL( rPropSet, sURL );

And this then calls the 'SetNewName' on the graphic:

        String aNewURL( RTL_CONSTASCII_USTRINGPARAM("vnd.sun.star.Package:") );
        aNewURL += String(rURL);
        pGrfNd->SetNewStreamName( aNewURL );

Which then causes the grief.
Comment 43 Michael Meeks 2012-02-22 07:48:27 UTC
Nice to know this is a re-hash of 2005 - one 'thb' even advised on the topic back then ;-) I can't reproduce this bug with my moronic patch applied either. cf: https://issues.apache.org/ooo/show_bug.cgi?id=48434
Comment 44 Michael Meeks 2012-02-22 07:50:58 UTC
patch is:

diff --git a/sw/source/filter/xml/xmltexte.cxx b/sw/source/filter/xml/xmltexte.cxx
index c62bce3..5c89944 100644
--- a/sw/source/filter/xml/xmltexte.cxx
+++ b/sw/source/filter/xml/xmltexte.cxx
@@ -221,7 +221,9 @@ void SwXMLTextParagraphExport::setTextEmbeddedGraphicURL(
     {
         String aNewURL( RTL_CONSTASCII_USTRINGPARAM("vnd.sun.star.Package:") );
         aNewURL += String(rURL);
-        pGrfNd->SetNewStreamName( aNewURL );
+
+// This is nonsensical.
+//        pGrfNd->SetNewStreamName( aNewURL );
 
         // #i15411# save-as will swap all graphics in; we need to swap
         // them out again, to prevent excessive memory use
Comment 45 Not Assigned 2012-02-22 08:10:00 UTC
Michael Meeks committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=0389b77a3cbea09ddbae238d7934d4c6349a8d37

fdo#33393 - tentative workaround for autosave image loss
Comment 46 Michael Meeks 2012-02-22 08:15:55 UTC
Divining a sane and consistent purpose in Michael Brauer's original work here is somewhat difficult. Suffice it to say that -no-other- image save goes and tries to rename the stream name that we get the GraphicObject from.

Then again - other components don't wrap their own versions of SwapIn / SwapOut either - which looks extremely curious in ndgrf.cxx -
Comment 47 Not Assigned 2012-02-22 08:45:17 UTC
Michael Meeks committed a patch related to this issue.
It has been pushed to "libreoffice-3-5":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=8a8eb870b8037801f81c680441d10a1ef95bfb06&g=libreoffice-3-5

fdo#33393 - tentative workaround for autosave image loss


It will be available in LibreOffice 3.5.1.
Comment 48 zeitlinie 2012-02-22 13:08:19 UTC
Today I've submitted Bug 46447
"Images disappear and corrupt Impress presentation"
to https://bugs.freedesktop.org/show_bug.cgi?id=46447

After that I was advised at users@de.libreoffice.org that this bug, although it happens in the Impress module, might actually be related to the bug here.
Any comments?
Comment 49 Florian Reisinger 2012-03-05 23:05:58 UTC
Once had the problem with 3,5,1rc1 Win7 x64 SP 1
Comment 50 Rainer Bielefeld Retired 2012-03-24 09:52:40 UTC
Fix is in 3.5, 3.6, there will be no 3.4.7, so I closed tis one. Please feel free to reopen if I did wrong.
Comment 51 Alex Thurgood 2012-03-26 08:39:20 UTC
(In reply to comment #50)
> Fix is in 3.5, 3.6, there will be no 3.4.7, so I closed tis one. Please feel
> free to reopen if I did wrong.

Hi Rainer,

I still have this problem in 3.5.1 on Mac OSX. It seems like the problem is present for others too :

https://bugs.freedesktop.org/show_bug.cgi?id=47832

where it was tested on 3.5.2 RC1. Consequently, I'm re-opening.


Alex
Comment 52 colin.mercer 2012-04-09 10:55:28 UTC
This problem appears to be predicated on the auto-save combined with changing the picture.  It appeared to be more noticeable when saving in/restoring from a .docx document.

Because it is related to auto-save, which by default is 15 minutes, then it will be as serious issue with long documents.  it is a significant bug as the author will have little or no indication it has happened.  Sending a document with missing drawings could affect the author's credibility.
Comment 53 Cor Nouws 2012-04-09 12:06:03 UTC
Hi Alex,

(In reply to comment #51)

> I still have this problem in 3.5.1 on Mac OSX. It seems like the problem is
> present for others too :
> 
> https://bugs.freedesktop.org/show_bug.cgi?id=47832
> 
> where it was tested on 3.5.2 RC1. Consequently, I'm re-opening.

That bug says specifically 'Buttons on forms' thus is far more specific then this bug. So I doubt if one could see it as a duplicate. Maybe a related corner case?

Anyway, I'm testing 3.5.2 now with the steps I used previously to reproduce that bug ...
Comment 54 Cor Nouws 2012-04-09 12:39:07 UTC
(In reply to comment #53)

> Anyway, I'm testing 3.5.2 now with the steps I used previously to reproduce
> that bug ...

I cannot reproduce this bug anymore.
So the other one must be different / special case.
Comment 55 Chris Peñalver 2012-04-09 14:51:40 UTC
colin.mercer@botley.com, please do not toggle the status. For more on this please see: http://wiki.documentfoundation.org/BugReport_Details#Version
Comment 56 Geoff King 2012-06-12 09:40:21 UTC
Hello - I just encountered this bug or something like it.  Images show as read-error in LO and the xlsx file would not open in Office 2010. I'm using version 3.5.4 using Mac Lion. This is what I did:

1- started docx document in Office 2010 on Windows 7, added text, a few pictures, and table of contents.
2- opened docx in LO 3.5.4 on Mac Lion.  Made various edits and added about 13 jpeg photos using Insert > Picture > From File in various places in the document. I did crop one of the images in LO and edited a few with "Edit with External Tool" (Mac Preview rotation) and all seemed okay from those. 
3- re-Saved the file as docx and dropboxed it
4- re-opened the docx file at work today and got the following results. 

MS Office 2010 will not open it now and says it has an error.  Libreoffice 3.5.4 on Windows 7 will open it, but there are read-error notices where the pictures should be.  

I also notice that the file is 4MB, which would be small considering that many pictures. They would have been a few MB each.  

Also, the few images that were added originally in step 1 are still present and okay.  All other images are missing with read-error.

Based on the file-size I'm assuming that the file was not saved correctly on the Mac at step 3 and the images were lost at that point.  They were visible in the document upon closeing the file on the Mac at step 3.

I'm not sure if auto-save is turned on the Mac or not as I'm not at that computer right now, but assume it would be on as default.  I assume dropbox did not mangle the file since the other original pictures added at step 1 are okay and the text is readable. 

Please let me know what other info I can provide.
Comment 57 Geoff King 2012-06-12 09:51:30 UTC
Looking at the file again I wanted to note that 1 of the 13 pictures I added in step 2 was retained.  I think it was the last one added and it was the one that was cropped (although it doesn't appear to be cropped now).
Comment 58 Geoff King 2012-06-12 12:39:42 UTC
Turns out that this is easy for me to reproduce on Windows 7 with LO3.5.4. using .docx format and autosave (Save AutoRecovery Information Every) turned on. Both docx and autosave are key aspects. 

Here are my approximate steps.  
1-opened LO and made sure autosave was on, and it is by default
2-changed autosave to 1 minutes (as noted in other comments) although it doesn't work at other intervals either.  I figured it would be more likely to have problems if it autosaved more often
3-started new file and saved as docx
4-Added some text and formatting and a few images here and there, while waiting a few minutes between.  Also clicked the save button once or twice to save the file between waits and adding images.
5-After about 10 minutes and minimal additions, I saved the file, close it, and closed LO.  
6-Re-opened the file in LO and the first image I added was read-error.  The other images were present, but some in slightly wrong places. 

Hope this helps.
Comment 59 okko7 2012-07-17 19:52:59 UTC
Fresh install of Linux Mint 13 Mate with LibreOffice 3.5.3.2 and the bug is still present !
Comment 60 tommy27 2012-07-18 09:00:21 UTC
@okko7

please try installing brand new LibO 3.5.5 and tell if the bug is still reproducible. It's always better to use the latest available release.
Comment 61 Roman Eisele 2012-07-18 09:01:58 UTC
Well, I would consider the problem(s) mentioned from comment #56 on as a
separate bug, because unlike the original problem they are limited to .docx and
.xlsx (and maybe other MSO2007+) file formats -- or am I missing something?

Therefore, I suggest to open a new bug report for the new problems, to copy
comment #56 et seqq. to the new bug report, and to close this (present) bug
report again. What do other people think?
Comment 62 okko7 2012-07-18 09:31:01 UTC
Opened a new bug for the problem appearing with .xlsx and .docx files:
https://bugs.freedesktop.org/show_bug.cgi?id=52226 and closed the present bug report.