Bug 33365

Summary: EDITING, Rand() Function
Product: LibreOffice Reporter: Jörgen <hyperapatisk>
Component: CalcAssignee: Kohei Yoshida <kohei>
Status: RESOLVED FIXED    
Severity: normal CC: erack, LibreOffice, michael.stahl, robinson.libreoffice
Priority: medium Keywords: easyHack, skillCpp
Version: 3.3.0 RC4   
Hardware: x86-64 (AMD64)   
OS: Windows (All)   
See Also: https://issues.apache.org/ooo/show_bug.cgi?id=78165
https://bugs.freedesktop.org/show_bug.cgi?id=70474
Whiteboard: target:4.1.0 target:4.0.0.0.beta0
Crash report or crash signature: Regression By:
Attachments: Calc document with Rand() (mal)function as described.
Document for test

Description Jörgen 2011-01-22 08:07:11 UTC
Created attachment 42308 [details]
Calc document with Rand() (mal)function as described.

Rand() uses a bad Pseudo-random code for generating "random" numbers.

Even with as few records as 1000, the "random" numbers repeats itself several times.

(Version 3.2 of OpenOffice has the same "feature", so it's not a new one.)
Comment 1 Kohei Yoshida 2011-01-22 08:23:11 UTC
We call system's rand() call to make random numbers.  So, if you want to see real improvement, then you need to talk to the system developers.

Here is the code that we use:

(double)rand() / ((double)RAND_MAX+1.0)
Comment 2 Kohei Yoshida 2011-01-22 08:28:32 UTC
BTW, if you know of a good way to generate good quality random numbers, please share with us.
Comment 3 Kohei Yoshida 2011-01-22 08:29:06 UTC
I'll take this meanwhile.
Comment 4 Kohei Yoshida 2011-01-22 08:54:41 UTC
Ok.  I found a nice page summarizing various rng algorithms, both good and bad.

http://www001.upp.so-net.ne.jp/isaku/rand.html (Japanese language only)

I couldn't find an English page that contains as good info about random number generators....
Comment 5 Kohei Yoshida 2011-01-22 09:05:03 UTC
http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/SFMT/index-jp.html

This generator appears to be most statistically accurate according to the article.

The source code is released under modified BSD license.
Comment 6 tester8 2011-01-22 10:42:05 UTC
Created attachment 42313 [details]
Document for test
Comment 7 tester8 2011-01-22 10:43:34 UTC
Not reproduced on LO RC4 Ubuntu 10.04 x86.
I have generate 1000 numbers about 20 times. Every time all numbers was different (see cell D1 in attached document). Every time standart deviation was in range 0.27-0.30 when the theoretical one is equal to sqrt(3)/6=0.2887...

Average also was about theoretical value 0.5
I think it is good pseudorandom (random) numbers set.

May be late I will test this on XP.
Comment 8 tester8 2011-01-24 08:19:22 UTC
Hm. Reproduced on XP x86 LO RC4.
Every time it was 12-25 numbers that have twin.

May be Windows version of LO should has it's own random generator if standard one doesn't get expected results? Especially if MSO doesn't has this bug.
Comment 9 Rainer Bielefeld Retired 2011-04-28 10:34:19 UTC
[Reproducible] with WIN!
I am not familiar with all details of randomized numbers generators, but with tester8's sample document and "LibreOffice 3.3.2  – WIN7  Home Premium  (64bit) German UI [OOO330m19 (Build:202 / tag 3.3.2.2)]" I got in 20 refreshes an average of 27,6 duplicate Numbers in the 1000 values. That seems to be quite a lot and is very similar to tester8's results. 

It would really be interesting to know why tat works so much worse under WIN.
I doubt, but can such a problem also be related to the processor? I use 64 bit AMD Phenom II X4 955 Processor 3.2 GHz, 4GB RAM

I believe we do not need additional information from reporter?

Modified Status due to facts.
Comment 10 Michael Stahl (allotropia) 2012-11-28 15:56:03 UTC
seriously, Calc uses libc rand() for random numbers???

there is already rtl/random.h and Writer uses that even for boomark names...
Comment 11 Kohei Yoshida 2012-11-28 16:39:07 UTC
(In reply to comment #10)
> seriously, Calc uses libc rand() for random numbers???
> 
> there is already rtl/random.h and Writer uses that even for boomark names...

This smells like an easy hack....
Comment 12 Eike Rathke 2012-11-29 11:24:08 UTC
(In reply to comment #10)
> seriously, Calc uses libc rand() for random numbers???
> there is already rtl/random.h and Writer uses that even for boomark names...

Calc and rand() existed before rtl/random.h ;-)


(In reply to comment #11)
> This smells like an easy hack....

I just made this one..


Code pointer: replace rand() with something more sensible in sc/source/core/tool/interpr1.cxx ScInterpreter::ScRandom()
Comment 13 Eike Rathke 2012-11-29 12:14:51 UTC
Btw, MS-Excel up to 97 used this random generator: http://support.microsoft.com/kb/86523

For 2003, 2007 and 2010 it claims an algorithm developed by B.A. Wichman and I.D. Hill, see http://support.microsoft.com/kb/828795

See also:

"A Comparison of Random Number Generators Used in Business - 2004 Update" http://academic.research.microsoft.com/Paper/5668932.aspx

"Microsoft Excel’s ‘Not The Wichmann–Hill’ random number generators" http://www.sciencedirect.com/science/article/pii/S016794730800162X
Comment 14 Eike Rathke 2012-12-05 16:13:04 UTC
Apache OO just re-implemented RAND() with a B.A Wichmann and I.D Hill generator.. http://svn.apache.org/viewvc?view=revision&revision=1416271
Comment 15 Not Assigned 2012-12-08 01:34:25 UTC
tino committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=8450a99c744e9005f19173e4df35d65640bcf5c4

fdo#33365 added wrapper for boost random, use that in RAND()



The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 16 Not Assigned 2012-12-10 15:36:17 UTC
Tino Kluge committed a patch related to this issue.
It has been pushed to "libreoffice-4-0":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=d421e1d25393416708f7c3fba625bd057deb9e08&g=libreoffice-4-0

fdo#33365 added wrapper for boost random, use that in RAND()


It will be available in LibreOffice 4.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 17 Johnny_M 2013-10-15 06:40:12 UTC
There seems to be a similar issue with the "Rnd" function of BASIC. Submitted bug 70474.
Comment 18 Robinson Tryon (qubit) 2015-12-16 00:43:41 UTC
Migrating Whiteboard tags to Keywords: (EasyHack,SkillCpp )
[NinjaEdit]