We are trying to use a Bedrock Inference Profile instead of specifying the foundational model ID directly. Our goal is to track Bedrock model usage via AWS cost allocation tags.
However, when we use an inference profile ARN instead of a model ID, we get the following error:
Couldn't establish a connection to the provider Endpoint 'arn:aws:bedrock:us-east-1::application-inference-profile/' failed to establish the connection with the provider. Error code: 404 Error message: BedrockException - Bedrock Invoke HTTPX: Unknown provider=None, model=arn:aws:bedrock:us-east-1::application-inference-profile/. Try calling via converse route - `bedrock/converse/`.. (OS-ABPS-40009)
We ran into a similar issue with LangChain, which we resolved by explicitly passing a provider parameter. In LangChain, provider is optional because it can be inferred from the model ID or ARN, but when using an inference profile ARN, the provider information cannot be derived from the ARN alone, so it must be supplied explicitly.
We suspect the same root cause applies in ODC, but ODC does not appear to expose an option to specify the model provider. Is there a workaround to use inference profiles in ODC?
It looks like ODC currently expects a direct foundation model ID and may not fully support Bedrock Inference Profile ARNs. The error suggests that ODC is unable to determine the provider from the inference profile ARN, similar to the issue seen in LangChain. Since ODC does not expose a parameter to explicitly pass the provider, there does not appear to be a direct workaround at the moment. As a temporary solution, you may need to continue using the foundation model ID directly or create a custom integration/API layer outside ODC where the provider can be explicitly specified before invoking Bedrock. It may also be worth raising this as a feature request or support ticket with OutSystems, as inference profile support seems limited in the current implementation.