EPSRC DLA: Multimodal Large Language Models and Data Fusion for Driver Monitoring in Automated Vehicles

One full scholarship is available in the Institute for Transport Studies for 2025/26 entry for a Home fee rated applicant. This is a highly competitive EPSRC Doctoral Landscape Award Studentship offering the award of UK fees, together with a tax-free maintenance grant (currently £19,237 for academic session 2024/25) for 3.5 years. This award is only available to Home fee rated applicants. Training and support will also be provided. Please note that whilst you may be successful in securing an academic offer for this project, this does not mean that you have been successful in securing an offer of funding. Funding is awarded on a competitive basis.

<p style="margin-bottom:11px">The Institute for Transport Studies (ITS) at the University of Leeds is the UK’s leading centre for transport research and hosts one of the most advanced academic driving simulator facilities in the world. With over 60 researchers and PhD students across disciplines such as Computer Vision, Artificial Intelligence, Machine Learning, Behaviour Modelling, Human Factors, Psychology, Micromobility and Transport Safety, ITS is committed to pioneering impactful research.</p> <p style="margin-bottom:11px">As automated vehicles (AVs) become increasingly integrated into real-world transport systems, robust driver monitoring systems (DMS) are essential, particularly during control transitions between vehicle autonomy and manual driving (also known as take-over requests or TORs). Accurate, real-time interpretation of multimodal signals (e.g. vision, audio, physiological data) is critical to assessing driver readiness and engagement.</p> <p style="margin-bottom:11px">Recent advances in <strong>Multimodal Large Language Models (MLLMs)</strong>, capable of integrating and reasoning across text, vision, and other data types, offer an exciting opportunity to revolutionise driver state monitoring in AVs. </p> <p style="margin-bottom:11px">This PhD project aims to develop a next-generation DMS framework using MLLMs and advanced multimodal fusion techniques to deliver precise, context-aware assessments of driver attention, cognitive state, and readiness to take control. </p> <p style="margin-bottom:11px"><strong>Research Objectives and Expected Outcomes:</strong></p> <ul> <li style="margin-bottom: 11px;"><strong>Develop Multimodal Fusion Models: </strong>Integrate diverse data streams from vision (e.g. gaze, posture, head movement), audio (speech, ambient noise), and physiological sensors (e.g. heart rate, skin conductance) to comprehensively model driver state.</li> <li style="margin-bottom: 11px;"><strong>Adapt and Fine-Tune MLLMs for Driver State Analysis:</strong> Extend MLLM capabilities to process multimodal data ‎and contextualise driver behaviour using pre-trained and task-specific fine-tuned models.‎</li> <li style="margin-bottom: 11px;"><strong>Enhance System Explainability and Trust:</strong><span style="font-size:11.0pt"><span style="line-height:107%"><span style="font-family:"Aptos",sans-serif"> </span></span></span>Apply explainable AI (XAI) techniques to improve system ‎transparency and user trust in driver engagement decisions.‎</li> <li style="margin-bottom: 11px;"><strong>Disseminate research:</strong> Publish findings in top-tier journals and conferences in collaboration with local and international research partners. ‎</li> </ul> <p style="margin-bottom:11px"><strong>Data Resources:</strong></p> <p style="margin-bottom:11px">The Human Factors & Safety group at ITS provides access to rich, multimodal datasets from our driving simulator, including in-cabin vision, eye-tracking, and physiological measures. Engagement with real-world datasets is also encouraged to enhance model generalisability. </p>

<p>Formal applications for research degree study should be made online through the <a href="https://www.leeds.ac.uk/research-applying/doc/applying-research-degrees">University's website</a>. You will need to create a login ID with a username and PIN. <strong>Please follow these instructions carefully or your application may not be considered.</strong></p> <ul> <li>For ‘Application type’ please select ‘Research Degrees – Research Postgraduate’.</li> <li>The admission year for this project is 2025/26 Academic Year.</li> <li>You will need to select your ‘Planned Course of Study’ from a drop-down menu. For this project, you must scroll down and select ‘<strong>EPSRC DLA Environment</strong>’. If you do not apply under this programme code, your application cannot be considered.</li> <li>Please state the funding you wish to be considered for is <strong>EPSRC Doctoral Landscape Award 2025/26: Environment</strong>.</li> <li>The project start date for this project is October 2025, please use this as your Proposed Start Date of Research.</li> <li>Please state clearly in the research information section that the research degree you wish to be considered for is <strong>EPSRC DLA: Multimodal Large Language Models and Data Fusion for Driver Monitoring in Automated Vehicles</strong> as well as <a href="https://environment.leeds.ac.uk/transport/staff/9408/dr-mahdi-rezaei">Dr Mahdi Rezaei</a> as your proposed supervisor.</li> <li>You must provide the following documents in your application: <ul> <li>Full transcripts of all degree study, or if in final year of study, full transcripts to date</li> <li>Personal statement outlining your interest in the project</li> <li>CV</li> </ul> </li> </ul> <p>If English is not your first language, you must provide evidence that you meet the University's minimum English language requirements (below).</p> <p><em>As an international research-intensive university, we welcome students from all walks of life and from across the world. We foster an inclusive environment where all can flourish and prosper, and we are proud of our strong commitment to student education. Across all Faculties we are dedicated to diversifying our community and we welcome the unique contributions that individuals can bring, and particularly encourage applications from, but not limited to Black, Asian, people who belong to a minority ethnic community, people who identify as LGBT+ and people with disabilities. Applicants will always be selected based on merit and ability.</em></p>

Applicants should have a first class or upper second class British Bachelors Honours degree (or equivalent) in an appropriate discipline. For this project, you should also have a Master's degree in . Computer Science, Engineering, AI, Data Analytics, Transport Studies, or a related field. You are also expected to have a strong background in machine learning and the integration of complex multimodal data (e.g. numeric, image, video). You should have a solid foundation in mathematics and statistics, and have demonstrable expertise in Large Language Models (LLMs) and MLLMs, including hands-on experience with their architecture, training, and application. You should also be proficient in Python, with extensive experience in PyTorch, OpenCV and relevant MLLM libraries/APIs. The ideal candidate will have: prior publications in the relevant field; experience working within multidisciplinary teams and a clear vision for contributing to intelligent transport systems, human factors research and broader societal impact. Applicants who are uncertain about the requirements for a particular research degree are advised to contact the PGR Admissions team prior to making an application.

The minimum English language entry requirement for research postgraduate research study is an IELTS of 6.0 overall with at least 5.5 in each component (reading, writing, listening and speaking) or equivalent. The test must be dated within two years of the start date of the course in order to be valid. Some schools and faculties have a higher requirement.

<p>This competitive EPSRC Doctoral Landscape Award offers the award of tuition fees at the Home rate, together with a tax-free maintenance grant (currently £19,237 for academic session 2024/25) for 3.5 years. Training and support will also be provided. This award is only available to Home fee rated applicants.</p> <p>Please note that whilst you may be successful in securing an academic offer for any project linked to this funding advertised project or own research proposal, this does not mean that you have been successful in securing an offer of funding. Funding is awarded on a competitive basis.</p> <p>Please refer to the <a href="https://www.ukcisa.org.uk/">UKCISA</a> website for information regarding fee status for Non-UK Nationals.</p>

<p>For further information please contact the Postgraduate Research Admissions team: <a href="mailto:env-pgr@leeds.ac.uk?subject=EPSRC%20DLA%3A%20Promoting%20Healthy%20Ageing%20through%20Diverse%20Mobility%20Options">env-pgr@leeds.ac.uk</a>. </p>