PROJECT RECORD / active
Dedicated AI Server
Local AI Compute
Build and maintain a Linux LLM inference server using AMD Instinct MI60 and NVIDIA RTX 3060 GPUs with ROCm, including installation of a community-designed 3D-printed cooling shroud and Arctic P12 fan.
What I built
Build and maintain a dedicated local LLM inference server for experimenting with modern open-source language models while learning GPU computing, Linux administration, and hardware optimization.
Why I built it
Built to support local LLM inference, model experimentation, software development assistance, and general-purpose language model workloads.
Architecture
- Authorized user→
- Linux on Dell Precision T5810→
- Qwen-35B-A3B Q4_K_M GGUF→
- AMD Instinct MI60 / NVIDIA RTX 3060
The workstation has 32 GB of RAM and supports local inference and model experimentation across its confirmed GPU hardware.
Hardware and software
Hardware
- Dell Precision T5810
- AMD Instinct MI60
- NVIDIA RTX 3060
- 32 GB RAM
- Arctic P12 fan
- Community-designed 3D-printed cooling shroud
Software
- Linux
- ROCm
- Qwen-35B-A3B
- Q4_K_M GGUF
Uses
- Local LLM inference
- Model experimentation
- Software development assistance
- General-purpose LLM workloads
Engineering challenge
Problem
The AMD Instinct MI60 generated significantly more heat than the workstation’s stock cooling configuration could effectively remove during extended inference workloads.
Root cause
The workstation chassis and original airflow were not designed to provide sufficient directed airflow across the passive-cooled accelerator.
Solution
Installed a community-designed 3D-printed cooling shroud and an Arctic P12 fan to provide direct airflow across the GPU heatsink.
Validation
Verified stable operating temperatures during extended inference sessions and confirmed reliable operation without thermal issues.
Screenshots
Original project media is being prepared for public release.
Original project media is being prepared for public release.
Original project media is being prepared for public release.
What I’d improve next
- Increase system memory.
- Experiment with additional open-source models.
- Improve storage performance for larger model libraries.