Designing and Implementing a Resilient Deep Learning Framework

PGR-P-1846

Key facts

Type of research degree: PhD
Application deadline: Ongoing deadline
Project start date: Tuesday 1 October 2024
Country eligibility: International (open to all nationalities, including the UK)
Funding: Competition funded
Source of funding: Doctoral training partnership
Supervisors: Professor Jie Xu
Additional supervisors: Dr Syed Ali Zaidi (Electronic & Electrical Engineering), Dr Zheng Wang and Professor Arshad Jhumka (Computing)
Schools: School of Computing, School of Electronic and Electrical Engineering

<h2 class="heading hide-accessible">Summary</h2>

Foundation models across various domains are experiencing rapid growth, necessitating continuous expansion to enhance performance. However, training these Large Language Models (LLMs) not only demands significant resources but also relies on a robust and dependable system to ensure an effective training process.<br /> <br /> Algorithm engineers face numerous challenges when training realistic LLMs, including server crashes, hardware failures, software compatibility issues, network communication errors, and unknown hangs. These failures result in the loss of training output and necessitate multiple restarts, consuming extra time and resources. For instance, launching the training process for a 175B model in a distributed environment requires several hours, occupying a substantial portion of the total training stage, which many researchers find financially burdensome.<br /> <br /> Therefore, establishing a robust and dependable platform to support the entire lifecycle of LLM development is not only complex and challenging but also urgently required.<br /> The project aims to explore and develop a resilient deep learning framework, investigating its scientific foundation, to enhance the LLM development lifecycle, with a specific focus on failover perspectives. The system is designed to tolerate any worker's crash or failure without impacting its overall execution. The automatic failover process, transparent to upper-level users, efficiently restarts and re-initializes failed workers based on soft or hard states. Given the novelty of this research, students are encouraged and supported to publish ground-breaking papers at top-tier conferences and even explore technical patents for potential startups.<br />

<h2 class="heading">How to apply</h2>

<p style="margin-bottom:11px">Formal applications for research degree study should be made online through the <a href="https://www.leeds.ac.uk/research-applying/doc/applying-research-degrees">University's website</a>. Please state clearly in the Planned Course of Study section that you are applying for <strong><em>PHD Computing FT,</em></strong> in the research information section that the research degree you wish to be considered for is <em><strong>Designing and Implementing a Resilient Deep Learning Framework</strong></em> as well as <a href="https://eps.leeds.ac.uk/computing/staff/331/professor-jie-xu">Professor Jie Xu</a> as your proposed supervisor and in the finance section, please state clearly <em><strong>the funding that you are applying for, if you are self-funding or externally sponsored</strong></em>.</p> <p>If English is not your first language, you must provide evidence that you meet the University's minimum English language requirements (below).</p> <p><em>As an international research-intensive university, we welcome students from all walks of life and from across the world. We foster an inclusive environment where all can flourish and prosper, and we are proud of our strong commitment to student education. Across all Faculties we are dedicated to diversifying our community and we welcome the unique contributions that individuals can bring, and particularly encourage applications from, but not limited to Black, Asian, people who belong to a minority ethnic community, people who identify as LGBT+ and people with disabilities. Applicants will always be selected based on merit and ability.</em></p> <p>Applications will be considered on an ongoing basis.  Potential applicants are strongly encouraged to contact the supervisors for an informal discussion before making a formal application.  We also advise that you apply at the earliest opportunity as the application and selection process may close early, should we receive a sufficient number of applications or that a suitable candidate is appointed.</p> <p>Please note that you must provide the following documents in support of your application by the closing date of 3 April 2024 for Leeds Opportunity Research Scholarship or 8 April 2024 for Leeds Doctoral Scholarship:</p> <ul> <li>Full Transcripts of all degree study or if in final year of study, full transcripts to date</li> <li>Personal Statement outlining your interest in the project</li> <li>CV</li> </ul>

<h2 class="heading heading--sm">Entry requirements</h2>

Applicants to research degree programmes should normally have at least a first class or an upper second class British Bachelors Honours degree (or equivalent) in an appropriate discipline. The criteria for entry for some research degrees may be higher, for example, several faculties, also require a Masters degree. Applicants are advised to check with the relevant School prior to making an application. Applicants who are uncertain about the requirements for a particular research degree are advised to contact the School or Graduate School prior to making an application.

<h2 class="heading heading--sm">English language requirements</h2>

The minimum English language entry requirement for research postgraduate research study is an IELTS of 6.5 overall with at least 6.5 in writing and at least 6.00 in reading, listening and speaking) or equivalent. The test must be dated within two years of the start date of the course in order to be valid. Some schools and faculties have a higher requirement.

<h2 class="heading">Funding on offer</h2>

<p><strong>Self-Funded or externally sponsored students are welcome to apply.</strong></p> <p><strong>UK</strong> – The <a href="https://phd.leeds.ac.uk/funding/209-leeds-doctoral-scholarships-2022">Leeds Doctoral Scholarships</a> and <a href="https://phd.leeds.ac.uk/funding/234-leeds-opportunity-research-scholarship-2022">Leeds Opportunity Research Scholarship</a> (open from October 2023) are available to UK applicants. <a href="https://phd.leeds.ac.uk/funding/60-alumni-bursary">Alumni Bursary</a> is available to graduates of the University of Leeds.</p> <p><strong>Non-UK</strong> – The <a href="https://phd.leeds.ac.uk/funding/48-china-scholarship-council-university-of-leeds-scholarships-2021">China Scholarship Council - University of Leeds Scholarship</a> is available to nationals of China (now closed for 2024/25 entry). The <a href="https://phd.leeds.ac.uk/funding/73-leeds-marshall-scholarship">Leeds Marshall Scholarship</a> is available to support US citizens. <a href="https://phd.leeds.ac.uk/funding/60-alumni-bursary">Alumni Bursary</a> is available to graduates of the University of Leeds.</p> <p><strong>Important:</strong>  Any costs associated with your arrival at the University of Leeds to start your PhD including flights, immigration health surcharge/medical insurance and Visa costs are not covered under this studentship.</p> <p>Please refer to the <a href="https://www.ukcisa.org.uk/">UKCISA</a> website for information regarding Fee Status for Non-UK Nationals.</p>

<h2 class="heading">Contact details</h2>

<p style="margin-bottom:11px">For further information about this project, please contact Professor Jie Xu by email to <a href="mailto:J.Xu@leeds.ac.uk">J.Xu@leeds.ac.uk</a></p> <p>For further information about your application, please contact Doctoral College Admissions by email to <a href="mailto:phd@engineering.leeds.ac.uk">phd@engineering.leeds.ac.uk</a></p>