In today’s enterprise environment, Generative AI technology has become key to enhancing business efficiency and innovation. However, with the diversification and decentralization of AI services, how to uniformly manage and call these services has become a challenge. This article will introduce how to use Golang to develop a unified API Gateway for cross-cloud AI services, achieving a secure and scalable enterprise-level solution.
This was my second public talk at the 2025 iThome CloudSummit Taiwan. Below is the outline of the talk.
- Authentication and Authorization
- Multi-Cloud AI Backend Integration (Azure OpenAI, AWS Bedrock, Google Gemini AI, etc.)
- Traffic Control and Resource Management
- Monitoring and Metrics Aggregation
Slides
The slides for this talk have been uploaded to Speaker Deck, and you can view them here. In the 90-minute presentation, we delved into how to use Golang to develop a unified API Gateway and achieve integration of cross-cloud AI services.
Talk Content
In 2022, the emergence of ChatGPT rapidly popularized Generative AI technology. With tech giants like OpenAI, Google, and Microsoft joining the fray, the ecosystem of AI services has become increasingly complex. At that time, there was no available API Gateway to uniformly manage these services. Developers had to deal with different APIs, authentication, and authorization mechanisms, which was a huge challenge for enterprises. Therefore, the team spent a lot of time developing a unified API Gateway. This Gateway not only integrates multiple AI services but also provides a secure and scalable solution. I am also very grateful to the team members at the time, Rueian, and other colleagues who assisted.
Behind the scenes, it uses the rueidis package developed by Rueian, which is a high-performance Redis client that supports features like Pipeline, and Transaction. This package has been officially recognized by Redis and is widely used.
The purpose of this talk is to share our experiences and lessons learned during the development of this API Gateway and to demonstrate how to use Golang to implement an efficient and scalable solution. It includes the following items:
- API Gateway & Routing Management
- Authentication & Authorization
- Quota & Billing Management
- Multi AI/ML Service Integration
- Monitoring & Logging
We will have the opportunity to discuss the details later, and iThome will also release related videos. We will post them for everyone’s reference. This API Gateway service only uses Redis for caching and does not use any database. This design makes the system more lightweight and able to respond to requests quickly. For details, please refer to the slides.
Photos from the Event
On that day, I wore the 2024 GopherCon Taiwan T-shirt. Thanks to the iThome team for the arrangements. The number of attendees was around 150. The venue was very spacious, and lunch was provided.