Meta Nvidia AI Service

What is Meta Llama 3.1 model?

• NVIDIA uses the meta / llama-3.1-8b-instruct model which can run anywhere.

  • Advanced state-of-the-art model with language understanding, superior reasoning, 
    and text generation.

    • More information can be found here 

      • https://docs.api.nvidia.com/nim/reference/meta-llama-3_1-8b
        

In this Service via a Server action named ChatCompletion, this action calls the NVIDIA ChatCompletion API which, 

if given a list of messages comprising a conversation, the model will return a response.

• Message: Text to send to the meta llama 3.1 model.
• Response: PostCompletionsResponse structure.

Here are a few tips you help to with the integration and testing.

1) You need a Bearer Token which you can get from NVIDIA.

Can sign up via this link https://build.nvidia.com/meta/llama-3_1-8b-instruct?signin=true&integrate_nim=true

2) If you wish to play around first with this model via NVIDIA.

Check out this link https://build.nvidia.com/meta/llama-3_1-8b-instruct

3) More information about the Meta llama model and the NVIDIA API:

See https://docs.api.nvidia.com/nim/reference/meta-llama-3_1-8b-infer

Once you get your API Key from NVIDIA, you can follow the following steps:

• You can either configure it like as a site property (or save it in an Entity)

  • Go to the Data tab 

    • Right-click on the site property folder

      • 

    • Create a Site Property and call it API_Key in the component, 
      you need to access the module Meta_Nvidia_AI, you can add a default value here
      if you wish.

    • Go to the OnBeforeRequest of the NVIDIA API method and click on 
      ListAppend_Header_Auth_API_Key

      • Afterwards, set the Value of your Authorization header to 

        • “Bearer “+Site.API_Key

          • And yes, there is a space after the word Bearer. Be aware of this.

          • 
            

      • The best practice would be to use a site property, 
        set as a secret and edit the site properties value in the service center.