Great Expectations: A Critique of Current Approaches to Random Number Generation Testing & Certification

Darren Hurley-Smith, Julio Hernandez-Castro

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Random number generators are a critical component of security systems. They also find use in a variety of other applications from lotteries to scientific simulations. Randomness tests, such as the NIST’s STS battery (documented in SP800-22), Marsaglia’s Diehard, and L’Ecuyer et al.’s TestU01 seek to find whether a generator exhibits any signs of non-random behaviour. However, many statistical test batteries are unable to reliably detect certain issues present in poor generators. Severe mistakes when determining whether a given generator passes the tests are common. Irregularities in sample size selection and a lack of granularity in test result interpretation contribute to this. This work provides evidence of these and other issues in several statistical test batteries. We identify problems with current practices and recommend improvements. The novel concept of suitable randomness is presented, precisely defining two bias bounds for a TRNG, instead of a simple binary pass/fail outcome. Randomness naivety is also introduced, outlining how binary pass/fail analysis cannot express the complexities of RNG output in a manner that is useful to determine whether a generator is suitable for a given range of applications.
Original languageEnglish
Title of host publication4th International Conference on Research in Security Standardisation
Subtitle of host publicationSSR 2018
EditorsCas Cremers, Anja Lehmann
Number of pages21
ISBN (Electronic)978-3-030-04762-7
ISBN (Print)978-3-030-04761-0
Publication statusE-pub ahead of print - 21 Nov 2018

Publication series

NameLecture Notes in Computer Science

Cite this