AI Gateway 已正式上市:用於管理和擴展生成式 AI 工作負載的統一介面
2024-05-22
AI Gateway 是一個 AI 操作平台,可為您的 AI 應用程式提供速度、可靠性和可觀察性。只需一行程式碼,您就可以解鎖強大的功能,包括限速、自訂快取、即時記錄和跨多個提供者的聚合分析...
繼續閱讀 »
\n \n
AI Gateway 是一個 AI 操作平台,提供統一的介面來管理和擴展生成型 AI 工作負載。從本質上講,它充當您的服務和推理提供程式之間的代理,無論您的模型在哪裡執行。只需一行程式碼,您就可以解鎖一組專注於效能、安全性、可靠性和可觀察性的強大功能,您可以將其視為 AI 操作的控制平面。這僅僅只是一個開始——我們制定了藍圖,將在不久的將來推出一系列令人興奮的功能,任何想要從 AI 工作負載中獲得更多收益的組織都會使用 AI Gateway。
\nAI 領域發展迅速,似乎每天都有新的模型、提供者或框架。鑒於如此高的變化率,很難對 AI 進行追蹤,當您使用多個模型或提供者時則尤為如此。這是推出 AI Gateway 的驅動因素之一——我們希望為您的所有模型和工具提供一個一致的控制平面,即使它們每天都在發生變化,也可從同一個控制平面進行控制。
我們已經與許多構建 AI 應用程式的開發人員和組織進行了交談,有一點很明確:他們希望圍繞 AI 操作提供更多的可觀察性、控制力和工具。這是許多 AI 提供者所缺乏的,因為他們非常關注模型開發,而不是平台功能。
為什麼選擇 Cloudflare 作為您的 AI Gateway?從某些方面來說,這是天作之合。在過去 10 多年裡,我們透過執行全球最大的網路之一來協助構建更好的網際網路,為世界各地的客戶提供效能、可靠性和安全性——Cloudflare 被近 20% 的網站用作反向代理。憑藉我們的專業知識,這感覺就像是一個自然的進展:只需變更一行程式碼,我們就可以協助您提高 AI 應用程式的可觀察性、可靠性和控制能力,所有這些都在一個控制平面中,而您則可以重新專注於構建。
下面是使用 OpenAI JS SDK 時的一行程式碼變更。歡迎查看我們的文件,瞭解其他提供者、SDK 和語言的情況。
\nimport OpenAI from 'openai';\n\nconst openai = new OpenAI({\napiKey: 'my api key', // defaults to process.env["OPENAI_API_KEY"]\n\tbaseURL: "https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_slug}/openai"\n});
\n 與客戶交談後,我們明確了一點,即我們需要先關注一些基本功能,然後再轉向一些更進階的功能。雖然我們對將要推出的功能非常期待,但以下是 GA 版目前提供的主要功能:
**分析:**聚合來自多個提供者的指標。查看流量模式和使用情況,包括一段時間內的請求數、權杖數和成本。
**即時記錄:**在構建過程中深入瞭解請求和錯誤。
\n**快取:**啟用自訂快取規則並使用 Cloudflare 的快取來處理重複請求,而不是存取原始模型提供者 API,從而幫助您節省成本並減少延遲。
\n**限速:**透過限制應用程式接收的請求數量來控制應用程式的擴展,以控制成本或防止濫用。
\n**支援您最喜歡的提供者:**截至 2024 年 5 月中旬,AI Gateway 現在原生支援 Workers AI 以及 10 個最受歡迎的提供者,包括 Groq 和 Cohere。
\n**通用端點:**如果出現錯誤,可以透過定義到另一個模型或推理提供程式的請求回退來提高復原能力。
\ncurl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_slug} -X POST \\\n --header 'Content-Type: application/json' \\\n --data '[\n {\n "provider": "workers-ai",\n "endpoint": "@cf/meta/llama-2-7b-chat-int8",\n "headers": {\n "Authorization": "Bearer {cloudflare_token}",\n "Content-Type": "application/json"\n },\n "query": {\n "messages": [\n {\n "role": "system",\n "content": "You are a friendly assistant"\n },\n {\n "role": "user",\n "content": "What is Cloudflare?"\n }\n ]\n }\n },\n {\n "provider": "openai",\n "endpoint": "chat/completions",\n "headers": {\n "Authorization": "Bearer {open_ai_token}",\n "Content-Type": "application/json"\n },\n "query": {\n "model": "gpt-3.5-turbo",\n "stream": true,\n "messages": [\n {\n "role": "user",\n "content": "What is Cloudflare?"\n }\n ]\n }\n }\n]'
\n 我們從開發人員那裡得到了很多意見反應,一些顯而易見的功能即將推出,例如持久記錄和自訂中繼資料——這些基礎功能將有助於在未來釋放真正的魔力。
但讓我們退後一步,分享一下我們的願景。在 Cloudflare,我們相信我們的平台作為一個統一的整體比作為各個部分的集合更強大。將這種理念套用到我們的 AI 產品,則意味著它們應該易於使用、組合和協調執行。
讓我們想像一下接下來的旅程。您最初加入 Workers AI 是為了使用最新的開放原始碼模型進行推理。接下來,您啟用 AI Gateway,以獲得更好的可見性和控制,並開始儲存持久記錄。然後您想要開始調整推理結果,於是您利用持久記錄、我們的提示管理工具和內建評估功能。現在您正在做出分析決策以改進推理結果。隨著每一次資料驅動的改進,您都想要獲得更多。因此,您實施我們的意見反應 API,幫助注釋輸入/輸出,本質上是構建結構化資料集。此時,您離可以立即部署到我們全球網路的一鍵微調只有一步之遙,而且還不止於此。隨著您繼續收集記錄和意見反應,您可以不斷重建微調配接器,以便為終端使用者提供最佳結果。
目前這只是一個理想的故事,但這是我們對 AI Gateway 和整個 AI 套件的未來的設想。您應該能夠從最基本的設定開始,逐步進入更進階的工作流程,全程無需離開 Cloudflare 的 AI 平台。最後,它看起來可能與上面描述的不完全一樣,但您可以確信我們致力於提供最好的 AI 操作工具,將 Cloudflare 打造成 AI 的最佳場所。
\nAI Gateway 即日起可在所有方案中使用。如果您尚未使用 AI Gateway,請查看我們的開發人員文件並立即開始使用。AI Gateway 現已免費提供核心功能,只需一個 Cloudflare 帳戶和一行程式碼即可開始使用。將來,將提供更多進階功能,例如持久記錄和祕密管理,但需付費使用。如果您有任何疑問,請造訪我們的 Discord 頻道。
"],"published_at":[0,"2024-05-22T14:00:17.000+01:00"],"updated_at":[0,"2024-10-10T00:21:59.074Z"],"feature_image":[0,"https://cf-assets.www.cloudflare.com/zkvhlag99gkb/6dIZ63ejPd9T3lR66GvbGk/b475e0d94aa62b1394b0046f5c386647/ai-gateway-is-generally-available.png"],"tags":[1,[[0,{"id":[0,"3JAY3z7p7An94s6ScuSQPf"],"name":[0,"開發人員平臺"],"slug":[0,"developer-platform"]}],[0,{"id":[0,"4HIPcb68qM0e26fIxyfzwQ"],"name":[0,"開發人員"],"slug":[0,"developers"]}],[0,{"id":[0,"3txfsA7N73yBL9g3VPBLL0"],"name":[0,"Open Source"],"slug":[0,"open-source"]}],[0,{"id":[0,"1Wf1Dpb2AFicG44jpRT29y"],"name":[0,"Workers AI"],"slug":[0,"workers-ai"]}],[0,{"id":[0,"5OywGP63AdM9Umyvaku8OP"],"name":[0,"Connectivity Cloud"],"slug":[0,"connectivity-cloud"]}],[0,{"id":[0,"1GyUhE8o287lrdNSpdRUIe"],"name":[0,"AI Gateway"],"slug":[0,"ai-gateway"]}],[0,{"id":[0,"6Foe3R8of95cWVnQwe5Toi"],"name":[0,"AI"],"slug":[0,"ai"]}]]],"relatedTags":[0],"authors":[1,[[0,{"name":[0,"Kathy Liao"],"slug":[0,"kathy"],"bio":[0,null],"profile_image":[0,"https://cf-assets.www.cloudflare.com/zkvhlag99gkb/2XeJHmfHmhCUmRwC7aeCWR/fb2194fd1e4bed0667242d081354f5f2/kathy.png"],"location":[0,"Seattle"],"website":[0,null],"twitter":[0,"@kathyyliao"],"facebook":[0,null]}],[0,{"name":[0,"Michelle Chen"],"slug":[0,"michelle"],"bio":[0,null],"profile_image":[0,"https://cf-assets.www.cloudflare.com/zkvhlag99gkb/1hrcl3aVtUbBuCMeuXETWy/93dbfbc7d41c09ba35d863312dbde89d/michelle.jpg"],"location":[0,null],"website":[0,null],"twitter":[0,"@_mchenco"],"facebook":[0,null]}],[0,{"name":[0,"Phil Wittig"],"slug":[0,"phil"],"bio":[0,null],"profile_image":[0,"https://cf-assets.www.cloudflare.com/zkvhlag99gkb/2FbDE6kgoEtV8l8hu6W85e/f31d42ea6b3cf65cfb08fb9fca5d0010/phil.jpeg"],"location":[0,null],"website":[0,null],"twitter":[0,"@pdwittig"],"facebook":[0,null]}]]],"meta_description":[0,null],"primary_author":[0,{}],"localeList":[0,{"name":[0,"AI Gateway is generally available: a unified interface for managing and scaling your generative AI workloads Config"],"enUS":[0,"English for Locale"],"zhCN":[0,"Translated for Locale"],"zhHansCN":[0,"No Page for Locale"],"zhTW":[0,"Translated for Locale"],"frFR":[0,"Translated for Locale"],"deDE":[0,"Translated for Locale"],"itIT":[0,"No Page for Locale"],"jaJP":[0,"Translated for Locale"],"koKR":[0,"Translated for Locale"],"ptBR":[0,"No Page for Locale"],"esLA":[0,"No Page for Locale"],"esES":[0,"Translated for Locale"],"enAU":[0,"No Page for Locale"],"enCA":[0,"No Page for Locale"],"enIN":[0,"No Page for Locale"],"enGB":[0,"No Page for Locale"],"idID":[0,"No Page for Locale"],"ruRU":[0,"No Page for Locale"],"svSE":[0,"No Page for Locale"],"viVN":[0,"No Page for Locale"],"plPL":[0,"No Page for Locale"],"arAR":[0,"No Page for Locale"],"nlNL":[0,"No Page for Locale"],"thTH":[0,"No Page for Locale"],"trTR":[0,"No Page for Locale"],"heIL":[0,"No Page for Locale"],"lvLV":[0,"No Page for Locale"],"etEE":[0,"No Page for Locale"],"ltLT":[0,"No Page for Locale"]}],"url":[0,"https://blog.cloudflare.com/ai-gateway-is-generally-available"],"metadata":[0,{"title":[0,"AI Gateway 已正式上市:用於管理和擴展生成式 AI 工作負載的統一介面"],"description":[0,null],"imgPreview":[0,"https://cf-assets.www.cloudflare.com/zkvhlag99gkb/1EWXM9QqB7UbqLFl1AtujG/3cec4ff7139977cf14349d91cff78278/ai-gateway-is-generally-available-BHx2Qt.png"]}]}],"locale":[0,"zh-tw"],"translations":[0,{"posts.by":[0,"作者:"],"footer.gdpr":[0,"GDPR"],"lang_blurb1":[0,"本貼文還提供以下語言版本:{lang1}。"],"lang_blurb2":[0,"本貼文還提供以下語言版本:{lang1} 和{lang2}。"],"lang_blurb3":[0,"本貼文還提供以下語言版本:{lang1},{lang2} 和{lang3}。"],"footer.press":[0,"新聞"],"header.title":[0,"Cloudflare 部落格"],"search.clear":[0,"清除"],"search.filter":[0,"篩選"],"search.source":[0,"來源"],"footer.careers":[0,"人才招募"],"footer.company":[0,"公司"],"footer.support":[0,"支援"],"footer.the_net":[0,"theNet"],"search.filters":[0,"篩選器"],"footer.our_team":[0,"我們的團隊"],"footer.webinars":[0,"網路研討會"],"page.more_posts":[0,"更多貼文"],"posts.time_read":[0,"閱讀時間:{time} 分鐘"],"search.language":[0,"語言"],"footer.community":[0,"社群"],"footer.resources":[0,"資源"],"footer.solutions":[0,"解決方案"],"footer.trademark":[0,"商標"],"header.subscribe":[0,"訂閱"],"footer.compliance":[0,"合規性"],"footer.free_plans":[0,"免費方案"],"footer.impact_ESG":[0,"影響力/ESG"],"posts.follow_on_X":[0,"在 X 上進行關注"],"footer.help_center":[0,"幫助中心"],"footer.network_map":[0,"網路分佈圖"],"header.please_wait":[0,"請稍候"],"page.related_posts":[0,"相關貼文"],"search.result_stat":[0,"針對 {search_keyword} 的第 {search_range} 個搜尋結果(共 {search_total} 個結果)"],"footer.case_studies":[0,"案例研究"],"footer.connect_2024":[0,"Connect 2024"],"footer.terms_of_use":[0,"服務條款"],"footer.white_papers":[0,"白皮書"],"footer.cloudflare_tv":[0,"Cloudflare TV"],"footer.community_hub":[0,"社群中心"],"footer.compare_plans":[0,"比較各項方案"],"footer.contact_sales":[0,"連絡銷售團隊"],"header.contact_sales":[0,"連絡銷售團隊"],"header.email_address":[0,"電子郵件地址"],"page.error.not_found":[0,"找不到頁面"],"footer.developer_docs":[0,"開發人員文件"],"footer.privacy_policy":[0,"隱私權原則"],"footer.request_a_demo":[0,"請求示範"],"page.continue_reading":[0,"繼續閱讀"],"footer.analysts_report":[0,"分析報告"],"footer.for_enterprises":[0,"企業適用"],"footer.getting_started":[0,"開始使用"],"footer.learning_center":[0,"學習中心"],"footer.project_galileo":[0,"Galileo 專案"],"pagination.newer_posts":[0,"較新貼文"],"pagination.older_posts":[0,"較舊貼文"],"posts.social_buttons.x":[0,"在 X 上進行討論"],"search.icon_aria_label":[0,"搜尋"],"search.source_location":[0,"來源/地點"],"footer.about_cloudflare":[0,"關於 Cloudflare"],"footer.athenian_project":[0,"Athenian 專案"],"footer.become_a_partner":[0,"成為合作夥伴"],"footer.cloudflare_radar":[0,"Cloudflare Radar"],"footer.network_services":[0,"網路服務"],"footer.trust_and_safety":[0,"信任和安全"],"header.get_started_free":[0,"免費開始使用"],"page.search.placeholder":[0,"搜尋 Cloudflare"],"footer.cloudflare_status":[0,"Cloudflare 狀態"],"footer.cookie_preference":[0,"Cookie 喜好設定"],"header.valid_email_error":[0,"必須是有效電子郵件。"],"search.result_stat_empty":[0,"第 {search_range} 筆搜尋結果(共 {search_total} 筆)"],"footer.connectivity_cloud":[0,"全球連通雲"],"footer.developer_services":[0,"開發人員服務"],"footer.investor_relations":[0,"投資人關係"],"page.not_found.error_code":[0,"錯誤代碼:404"],"search.autocomplete_title":[0,"插入查詢。按下 Enter 鍵即可傳送"],"footer.logos_and_press_kit":[0,"標誌與新聞資料包"],"footer.application_services":[0,"應用程式服務"],"footer.get_a_recommendation":[0,"取得建議"],"posts.social_buttons.reddit":[0,"在 Reddit 上進行討論"],"footer.sse_and_sase_services":[0,"SSE 和 SASE 服務"],"page.not_found.outdated_link":[0,"您可能使用了過時的連結,或者可能輸入了錯誤的位址。"],"footer.report_security_issues":[0,"報告網路安全問題"],"page.error.error_message_page":[0,"抱歉,我們找不到您想要的頁面。"],"header.subscribe_notifications":[0,"訂閱以接收新文章的通知:"],"footer.cloudflare_for_campaigns":[0,"Cloudflare for Campaigns"],"header.subscription_confimation":[0,"訂閱已確認。感謝訂閱!"],"posts.social_buttons.hackernews":[0,"在 Hacker News 上進行討論"],"footer.diversity_equity_inclusion":[0,"多樣性、公平性和包容性"],"footer.critical_infrastructure_defense_project":[0,"關鍵基礎架構防禦專案"]}]}" ssr="" client="load" opts="{"name":"PostCard","value":true}" await-children="">2024-05-22
AI Gateway 是一個 AI 操作平台,可為您的 AI 應用程式提供速度、可靠性和可觀察性。只需一行程式碼,您就可以解鎖強大的功能,包括限速、自訂快取、即時記錄和跨多個提供者的聚合分析...
繼續閱讀 »2024-03-14
最近,Workers AI 和 AI Gateway 團隊與本·古里安大學的網路安全研究人員就我們「公開漏洞懸賞」活動中收到的一份報告展開了密切合作。透過此程序,我們發現並完全修補了一個影響所有 LLM 提供者的漏洞。下面是具體情況...