Securing $6,000 in UREAP Funding for Privacy-Preserving LLM Research
Introduction & Motivation
I am thrilled to share that my research proposal was selected for funding through the Thompson Rivers University (TRU) Undergraduate Research Experience Award Program (UREAP). This grant provides a fantastic opportunity to delve into a topic I'm passionate about: leveraging Artificial Intelligence, specifically Large Language Models (LLMs), to enhance the student experience while rigorously prioritizing data privacy.
Navigating university resources, academic requirements, and support services can often be challenging. My project aims to develop a secure, locally-hosted AI assistant tailored to the needs of TRU students. By utilizing open-source LLMs, Retrieval-Augmented Generation (RAG), and intuitive interfaces like voice commands, the goal is to provide timely, accurate, and context-aware support without compromising sensitive student data. This research stems from my own experiences and a desire to create tools that genuinely help students succeed. You can learn more about the UREAP program itself on the official TRU UREAP Award page.
Key Project Objectives
- Secure UREAP Funding: Successfully obtained an operating grant to support the research activities, including potential hardware needs for local model hosting and software licenses.
- Develop a Privacy-Centric RAG Pipeline: Implement a robust RAG system using vector databases and embedding models to index TRU-specific documents (program guides, policies, web content) securely on local infrastructure. This ensures sensitive queries and institutional data remain within TRU's control.
- Benchmark and Fine-tune Local LLMs: Evaluate various open-source LLMs (e.g., Llama 3, Phi-3, Mistral variants) against proprietary models (like GPT-4o, Claude 3) on metrics including accuracy, latency, resource consumption, and adherence to privacy constraints using TRU-specific evaluation sets. Explore fine-tuning techniques (like LoRA) to optimize performance for university-related tasks.
- Build Integrated AI Assistant Features: Create user-facing applications leveraging the RAG pipeline, including a dynamic degree planner, a voice-activated query system, and a confidential mental health resource navigator.
- Promote Open Science: Share findings, methodologies, and potentially anonymized code or evaluation frameworks with the academic community to foster further research in educational AI.
Dive Deeper: Project Poster
For a visual overview of the research framework, core technologies, implementation strategy, and the four key application pillars, please see the detailed project poster. View the project poster here.
Project Approach: The Four Pillars
Our framework addresses key student challenges through four integrated AI-powered applications, each leveraging locally-deployed LLMs and secure retrieval systems for personalized, private assistance.
University Data RAG Assistant
This component focuses on information retrieval. Public TRU documents (course calendars, policies, FAQs) are chunked, converted into vector embeddings using models like Sentence-BERT, and stored in a local vector database (e.g., ChromaDB). When a student asks a question ("What are the prerequisites for COMP 3XXX?"), the system retrieves relevant document chunks based on semantic similarity and feeds them, along with the query, to the locally hosted LLM. The LLM then synthesizes an answer grounded in the provided TRU context, ensuring accuracy and relevance.
Dynamic Degree Pathway Designer
Moving beyond simple Q&A, this tool aims to provide personalized academic planning. By interfacing with (anonymized or securely accessed) student progression data and program requirements, the LLM can suggest optimal course sequences, flag missing prerequisites, estimate time-to-completion, and align pathways with declared majors or career interests. This involves constraint checking and potentially basic planning algorithms guided by the LLM's understanding of course descriptions and dependencies.
Voice-Enabled AI Advisor
To enhance accessibility, this pillar integrates speech-to-text (e.g., using Whisper variants) and text-to-speech (e.g., Piper, Coqui TTS) technologies. Students can ask questions conversationally ("How do I apply for a scholarship?") and receive spoken responses. This module acts as a frontend to the RAG Assistant, processing voice input, querying the backend, and vocalizing the LLM's generated response.
Mental Health Resource Navigator
Handling sensitive queries requires extreme care. This module is designed *not* to provide therapy but to confidentially connect students with *existing* TRU mental health resources. Using keyword detection and potentially sentiment analysis, it identifies queries related to stress, anxiety, or counseling needs. It then directs students to appropriate official resources like TRU Counseling Services, peer support groups, or emergency contact information, ensuring responses are safe, supportive, and strictly informational, pointing towards human support systems. Privacy is paramount here.
Technology Stack & Benchmarking
The project relies heavily on open-source tools to maintain transparency and control. Key technologies include:
- LLMs: Primarily exploring models like Llama 3 (8B/70B), Phi-3 (Mini/Medium), Mistral 7B, and potentially others runnable on local hardware (GPU required for optimal performance).
- RAG Framework: Leveraging libraries like LangChain or LlamaIndex for orchestrating the retrieval and generation process.
- Embedding Models: Sentence Transformers (e.g., `all-MiniLM-L6-v2`, `bge-large-en`) for creating text embeddings.
- Vector Database: ChromaDB, FAISS, or similar for efficient similarity search.
- Voice Interface: Exploring Whisper (or faster variants like Whisper.cpp) for STT and Piper/Coqui TTS for TTS.
- Backend/Frontend: Likely Python (FastAPI/Flask) for the backend and React/Next.js for any potential web interface/demonstrator.
Benchmarking Approach
A critical part of the research involves rigorous benchmarking. Local models will be compared against each other and against API-based proprietary models (GPT-4o, Gemini Pro, Claude 3 Sonnet/Opus) using:
- Accuracy: Evaluating response quality on TRU-specific question-answering datasets, potentially using metrics like ROUGE, BLEU, or human evaluation. Standard benchmarks (e.g., MMLU subsets) may also be used for general capability assessment.
- Latency: Measuring end-to-end response time for user queries on local hardware.
- Resource Usage: Monitoring VRAM, RAM, and CPU consumption during inference.
- Privacy Compliance: Auditing data flow to ensure no sensitive information leaves the local environment during processing by local models.
Acknowledgements
I am profoundly grateful to my faculty supervisors, Dr. Ghazanfar Latif and Dr. Jaspreet Kaur. Their mentorship, expertise, and encouragement have been invaluable. They've not only guided the technical aspects but also fostered an environment where I feel empowered to pursue research in an area I genuinely care about – improving the student journey through thoughtful technology. This project addresses challenges I've observed and personally experienced, and their support in allowing me to tackle this is deeply appreciated. I also extend my sincere thanks to the UREAP committee for recognizing the potential of this project and providing the funding to make it possible.
Next Steps & Potential Impact
The UREAP funding marks the beginning of an exciting research phase. The plan outlined above represents the core focus for my work during **Summer 2025**. The immediate next steps involve setting up the development environment, refining the data ingestion pipeline for TRU documents, and conducting initial benchmarks of selected LLMs.
Refine & Fine-tune Models
Systematically benchmark local LLMs. Implement fine-tuning strategies (e.g., Parameter-Efficient Fine-Tuning methods like LoRA) using curated TRU-specific datasets (e.g., question-answer pairs based on FAQs) to enhance performance on relevant tasks.
Prototype & User Feedback
Develop functional prototypes for each of the four pillars. Conduct usability testing and gather feedback from a small group of TRU students and staff (following ethics protocols) to iterate on the design, functionality, and accuracy.
Strengthen Privacy & Security
Continuously audit and enhance data privacy measures. Implement robust access controls and data handling protocols. Explore techniques like differential privacy or homomorphic encryption if handling potentially sensitive (even anonymized) student data becomes necessary for features like the pathway designer.
Disseminate Findings
Document the research process, methodologies, benchmarking results, and challenges encountered. Plan to present findings at student research conferences, publish results if appropriate, and potentially release code components or evaluation datasets to benefit the wider educational technology community.
The ultimate impact of this project could be a blueprint for other institutions seeking to implement privacy-preserving AI student support systems. By demonstrating the viability of local, open-source solutions, this research aims to empower universities to leverage AI responsibly.
Conclusion & Feedback
Embarking on this UREAP project is an incredible learning opportunity. I am eager to contribute to the development of AI tools that are not only intelligent but also ethical and secure, directly benefiting the TRU community here in Kamloops. The work detailed here is my focus for Summer 2025, and I look forward to sharing progress and insights as the research unfolds.
I am always open to discussing ideas and learning from others in the field. If you have any thoughts, insights, or suggestions regarding this project, please feel free to reach out. You can connect with me on LinkedIn, send me an email, or visit my contact page. Thank you again to the UREAP program and my supervisors for their trust and support.