Free and fully local AI integration for ODC based on Llama. Run GGUF models directly within your infrastructure for basic LLM tasks, eliminating external API costs and ensuring zero data exposure.
Built using LLamaSharp library (https://github.com/SciSharp/LLamaSharp).
Try now: link
(or install "PrivateLLM Demo" from Forge)
This utility operates within the CPU and memory limits of the ODC External Logic environment (0.56 of a physical core and 1GB of RAM). While highly efficient, it is designed for specialized tasks like text translation and classification. Do not expect the broad reasoning performance or speed of multi-billion parameter public LLMs (like GPT-4 or Gemini), as PrivateLLM is optimized for privacy and local execution within your infrastructure's available resource boundaries.
Fixed error when leaving the ModelFileURL parameter empty.
BSD-3 license (https://opensource.org/licenses/BSD-3-Clause)