Exploration of Low Precision Numerical Hardware and Stochastic Rounding

In 1985 a floating-point standard has been introduced, which, among multiple things, defined a set of required and recommended arithmetic operations and mathematical functions. It is arguably the most important standard in the history of computing. Most computer processors and mathematical libraries adopted the standard, which in turn improved reproducibility between different versions of hardware or even between hardware from different vendors. Two subsequent revisions of the standard have been released, in 2008 and in 2019.<br /> <br /> For decades the behaviour of 64-bit (double precision) and 32-bit (single precision) arithmetic has been relatively stable and predictable, with software developers being confident in achieving bit reproducibility in most cases. The latest TOP500 supercomputer list contains 158 machines with NVIDIA or AMD vector and matrix arithmetic operations, which diverge from the IEEE 754. Furthermore, rounding methods that are not standard, such as stochastic rounding, are being included in hardware: Graphcore IPU, Amazon Trainium, and Tesla Dojo chips. Most of the low precision arithmetic, despite being introduced for machine learning, are used in scientific computing in general for mixed precision algorithms.<br /> <br /> This sudden change in numerical hardware (at the time of writing still ongoing), a fundamental feature of computers, presents new challenges. We need to develop methods to understand or test the numerical behaviour in order to be able to document it for each new device. It is important to understand the behaviour of software when creating new devices as well as when simulating the current ones. We also need to consider how to adapt our software to effectively use the low precision hardware, minimizing the effects of non-standard numerical operations. Finally, we should study how to combine these results to drive the standardization of low precision hardware in order to obtain consistent behaviour in the future.<br /> <br /> PhD projects in this space can go into three main directions: software, hardware, or mathematical study, with possibilities for overlap.

<p>Potential areas of focus include:</p> <ul> <li>developing theory and software for determining numerical properties through testing;</li> <li>efficient and numerically accurate use of low precision arithmetic [1];</li> <li>development and simulation of custom arithmetic formats for various applications;</li> <li>standardization of reduction operations (see [2, p. 66]) for the next revision of the IEEE 754 standard due to appear around 2030 (new revision every 10 years);</li> <li>standardization of numerical hardware for machine learning (see IEEE P3109);</li> <li>applications, hardware implementation, and theoretical properties of stochastic rounding [3];</li> <li>rounding error analysis of numerical hardware;</li> <li>investigation of alternative representations, such as fixed-point, logarithmic or level-index.</li> </ul> <p>Wider active research topics in computer arithmetic and related areas can also be considered (see [4] and references therein as well as the latest papers in&nbsp;<a href="https://arith2022.arithsymposium.org/program.html">ARITH</a>).</p> <p>Good programming skills in C, MATLAB or other languages, and previous experience in developing mathematical software or hardware are essential.</p> <h2><strong>References</strong></h2> <p>[1]&nbsp;N. J. Higham and T. Mary,&nbsp;Mixed precision algorithms in numerical linear algebra, Acta Numerica, 31 (2022), pp. 347&ndash;414.</p> <p>[2] &nbsp;IEEE Standard for Floating-Point Arithmetic, IEEE Std 754-2019 (revision of IEEE Std 754- 2008), Institute of Electrical and Electronics Engineers, Piscataway, NJ, USA, July 2019.</p> <p>[3]&nbsp;M. Croci, M. Fasi, N. J. Higham, T. Mary, and M. Mikaitis,&nbsp;Stochastic rounding: Implementation, error analysis and applications, Roy. Soc. Open Sci., 9 (2022), pp. 1&ndash;25.</p> <p>[4] &nbsp;J.-M. Muller, N. Brunie, F. de Dinechin, C.-P. Jeannerod, M. Joldes, V. Lef&egrave;vre, G. Melquiond, N. Revol, and S. Torres,&nbsp;Handbook of Floating-Point Arithmetic, Birkh&auml;user, Boston, MA, USA, second ed., 2018.</p>

<p>Formal applications for research degree study should be made online through the&nbsp;<a href="https://www.leeds.ac.uk/research-applying/doc/applying-research-degrees">University&#39;s website</a>. Please state clearly in the Planned Course of Study section that you are applying for <em><strong>PHD Computing</strong></em> and in the research information section&nbsp;that the research degree you wish to be considered for is <strong>Exploration of Low Precision Numerical Hardware and Stochastic Rounding</strong>&nbsp;as well as <a href="https://eps.leeds.ac.uk/computing/staff/12197/dr-mantas-mikaitis">Dr Mantas Mikaitis</a>&nbsp;as your proposed supervisor.</p> <p>If English is not your first language, you must provide evidence that you meet the University&#39;s minimum English language requirements (below).</p> <p><em>As an international research-intensive university, we welcome students from all walks of life and from across the world. We foster an inclusive environment where all can flourish and prosper, and we are proud of our strong commitment to student education. Across all Faculties we are dedicated to diversifying our community and we welcome the unique contributions that individuals can bring, and particularly encourage applications from, but not limited to Black, Asian, people who belong to a minority ethnic community, people who identify as LGBT+ and people with disabilities. Applicants will always be selected based on merit and ability.</em></p> <p class="MsoNoSpacing">Applications will be considered on an ongoing basis. &nbsp;Potential applicants are strongly encouraged to contact the supervisors for an informal discussion before making a formal application. &nbsp;We also advise that you apply at the earliest opportunity as the application and selection process may close early, should we receive a sufficient number of applications or that a suitable candidate is appointed.</p> <p>Please note that you must provide the following documents in support of your application by the closing date of 31 May 2023:</p> <ul> <li>Full Transcripts of all degree study or if in final year of study, full transcripts to date</li> <li>Personal Statement outlining your interest in the project</li> <li>CV</li> <li>Funding information:&nbsp; School of Computing Scholarship</li> </ul>

Applicants to research degree programmes should normally have at least a first class or an upper second class British Bachelors Honours degree (or equivalent) in an appropriate discipline. The criteria for entry for some research degrees may be higher, for example, several faculties, also require a Masters degree. Applicants are advised to check with the relevant School prior to making an application. Applicants who are uncertain about the requirements for a particular research degree are advised to contact the School or Graduate School prior to making an application.

The minimum English language entry requirement for research postgraduate research study is an IELTS of 6.5 overall with at least 6.5 in writing and 6.0 in reading, listening and speaking)or equivalent. The test must be dated within two years of the start date of the course in order to be valid. Some schools and faculties have a higher requirement.

<p class="MsoNoSpacing">A highly competitive School of Computing Studentship consisting of the award of fees at the UK fee rate with a maintenance grant (currently &pound;17,668 for session 2022/23) for 3.5 years.</p> <p>This opportunity is open to UK applicants only. All candidates will be placed into the School of Computing Studentship Competition and selection is based on academic merit.<br /> <br /> Please refer to the&nbsp;<a href="https://www.ukcisa.org.uk/">UKCISA</a>&nbsp;website for&nbsp;information regarding Fee Status for Non-UK Nationals.</p>

<p>For further information about this project, please contact Dr Mantas Mikaitis: <a href="mailto:M.Mikaitis@leeds.ac.uk">M.Mikaitis@leeds.ac.uk</a>.</p> <p>For further information about your application, please contact Doctoral College Admissions:&nbsp;<a href="mailto:phd@engineering.leeds.ac.uk">phd@engineering.leeds.ac.uk</a></p>