03-18, 16:00–16:25 (US/Eastern), Belfer Case Study Room (CGIS S020)
Open OnDemand (OOD) is ideally placed to bridge the divide between traditional shell-based batch HPC and emerging large language model (LLM) workflows. Our team deployed OOD internally at MERLIN, a small HPC research cluster located inside Perth Children’s Hospital to provide HPC resources to a range of technical and non-technical users on sensitive healthcare data. We adapted existing LLM web applications for OOD to provide a low-code playground for non-technical researchers such as medical doctors and research assistants to engage with LLM resources in a healthcare environment. This presentation will discuss the rationale, and implementation of OOD LLM web applications in a highly restricted environment and motivate future improved support for these types of workflows within OOD.
We are a team of multidisciplinary paediatric anaesthesia researchers at Perth Children’s Hospital in Western Australia. Real-world deployments of HPC in healthcare settings have been limited by the challenges of working with sensitive healthcare data within restricted network and operational environments. In Australia, health data cannot be processed outside of the country and in most cases cannot leave the hospital data centre due to privacy requirements. The large Australian public supercomputers NCI GADI and Pawsey Supercomputing Centre do not permit the storage and analysis of healthcare data. Accordingly, Australian medical HPC research has focused on genomic analysis, drug discovery and corporate AI model development. Recently, we deployed a small on-premises HPC research cluster with Open OnDemand (OOD) inside Perth Children’s Hospital primarily to facilitate the use of large language models (LLMs) on sensitive hospital data.
Our team comprises software engineers, data analysts and end users i.e. research assistants, anaesthetists and other healthcare staff who use AI models as part of everyday clinical work. We deployed OOD with the intention of enabling a variety of interactive and batch computing jobs to suit the needs of our users. We realised that there are many containerised low-code LLM web applications that would be suitable to use by non-technical users and that can be contextualised with locally developed language models and techniques such as retrieval augmented generation (RAG).
We implemented a containerised workflow using Podman Compose to orchestrate container services and to expose web services over the OOD reverse proxy. Our environment constraints mean that most users share the same node, so we additionally restrict applications to only expose ports over an authenticated Caddy reverse proxy. The hospital environment is strictly locked down and as such some web applications required patching to render client-side URLs prefixed with the OOD reverse proxy path. Future work to simplify this process is planned.
This enables an end-to-end workflow where technical researchers are able to use batch jobs to finetune and deploy models locally for use by non-technical end users in clinical situations using open-source user-friendly web applications over OOD. We believe this workflow democratises access to LLM and HPC resources and allows our team to be more agile in developing and validating LLMs in clinical settings.
Harry is a junior medical doctor in Western Australia working in internal medicine and a PhD candidate at the University of Western Australia investigating the use of artificial intelligence in paediatric anaesthesia. His Honours thesis was in using Computer Vision for myeloproliferative neoplasms. He has previously been a student at Pawsey Supercomputing Centre and is a sysadmin of MERLIN, a small GPU research cluster located within Perth Children's Hospital for research on sensitive healthcare data. His research interests encompass large language models, digital health and computer vision.
Dr Thomas Drake-Brockman (they/them) is a medical doctor and clinician-technologist working across Perth Children's Hospital, the University of Western Australia, and The Kids Research Institute Australia.