Project Goal

Build a working prototype of our local, privacy-first AI system that includes both software and hardware components — enabling document processing (e.g., summarization or classification) using AI models running entirely offline on edge devices like a NUC, Raspberry Pi, or Jetson.

Main Objective / Tasks & Activities

Learners will collaborate to integrate AI software with real hardware environments to demonstrate a complete, working edge-AI system.

Software Tasks:

• Set up a lightweight local backend (e.g., FastAPI) to run AI workflows
• Integrate with quantized open-source language models (e.g., via llama.cpp)
• Develop a simple UI or CLI to upload documents and display AI outputs
• Ensure compatibility with low-power, local hardware environments

Hardware Tasks:

• Set up and configure edge devices (e.g., Jetson Nano, Raspberry Pi, NUC)
• Test local model performance on different hardware setups
• Optimize system resource usage (RAM, CPU/GPU)
• Assist with packaging the software for reliable offline use

Deliverables:

• A working MVP that runs a full AI document processing workflow offline on local hardware
• Source code and system setup documentation
• Performance benchmarks (inference speed, memory usage) for each hardware type
• A short demo video of the full system in action

Project Outcomes and Deliverables

Learners are expected to complete the following:

Software Deliverables

• A lightweight local API (e.g., FastAPI or Flask) to handle document uploads and return AI-generated outputs (summaries or tags)
• Integration with a quantized open-source LLM (e.g., via llama.cpp or GGUF)
• A simple user interface (web-based or CLI) for testing document workflows
• Support for common file types (.txt, PDF, .docx)
• Written documentation for setup and usage

Hardware Deliverables

• Local setup of the software stack on edge hardware (e.g., Raspberry Pi, Jetson Nano, Intel NUC)
• Performance benchmarking (inference time, memory usage) across 1–2 hardware platforms
• Troubleshooting and optimization recommendations for running models on low-power devices
• Power and thermal usage notes (if relevant to the platform)

Final Deliverables

• A fully working MVP that performs AI tasks entirely offline on edge hardware
• Source code, install scripts, and documentation
• A short recorded demo of the system running end-to-end