Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save linuxtek-canada/115043e27ed3849a2b3d0f0b4b2a07d4 to your computer and use it in GitHub Desktop.
Save linuxtek-canada/115043e27ed3849a2b3d0f0b4b2a07d4 to your computer and use it in GitHub Desktop.
LocalAI docker-compose configuration
version: '3.8'
services:
localai:
image: localai/localai:latest-aio-gpu-hipblas
deploy:
resources:
limits:
cpus: 8.0
memory: 32G
environment:
- DEBUG=true
- REBUILD=true
- BUILD_TYPE=hipblas
- BUILD_GRPC_FOR_BACKEND_LLAMA=ON
- GPU_TARGETS=gfx1100
- HSA_OVERRIDE_GFX_VERSION=11.0.0
- CMAKE_BUILD_PARALLEL_LEVEL=16
ports:
- "8080:8080"
volumes:
- ./models:/build/models:cached
devices:
- /dev/dri
- /dev/kfd
security_opt:
- seccomp:unconfined
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment