API company Kong is introducing its open source AI Gateway, an expansion of its existing API gateway designed to facilitate the integration of applications with one or more large language models (LLMs) and enable access to them through a unified API. Additionally, Kong is unveiling various AI-specific features, including prompt engineering, credential management, and more.
Embracing AI in API Use Cases
Marco Palladino, co-founder and CTO of Kong, highlights AI as an additional use case for APIs, emphasizing the pivotal role of APIs across various domains, including mobile, microservices, and now AI. Palladino underscores the interconnectedness of APIs and AI, both in consuming and fine-tuning AI models. Recognizing the growing interest in AI adoption among organizations, Palladino emphasizes the need for effective management of data privacy and security concerns.
Seamless Integration with LLM Providers
The Kong API gateway aims to streamline AI development by enabling developers to leverage their gateway to interact with one or more LLM providers without code modifications. Currently, the gateway supports leading LLM providers such as Anthropic, Azure, Cohere, Meta’s LLaMA models, Mistral, and OpenAI.
Enhancing API Capabilities for AI Workloads
Kong introduces AI-specific features to its API gateway, empowering developers to manipulate prompts and responses dynamically. With AI response and request transformers, developers can modify prompts and results in real-time, facilitating tasks like translation and data anonymization. The gateway also incorporates prompt engineering, allowing businesses to enforce guidelines and manage prompts centrally.
Integration and Adoption
Having evolved from the Kong API Management Platform, the Kong Gateway serves as the foundation for the new AI Gateway. Existing Kong users can seamlessly upgrade to access the latest AI features. While these features are currently available for free, Kong plans to introduce premium paid features in the future, with a focus on enhancing user experience and functionality.
As organizations continue to explore AI integration, Kong’s AI Gateway emerges as a versatile solution, empowering developers to harness the power of AI seamlessly within their applications while ensuring robust security and compliance standards.
