An AI developer has hacked his iPhone to become the eyes of GPT-4, turning his iPhone into an all-seeing AI-powered assistant. 視頻里，小哥介紹道，自己采用了一些軟件。比如，他用視覺模型YOLOv8來檢測(cè)和識(shí)別物體；用OpenAI公司的語音自動(dòng)識(shí)別系統(tǒng)Whisper來檢測(cè)自己說的話，并把語音轉(zhuǎn)換成文字，輸入給GPT-4。而GPT-4寫出來的文字，會(huì)由AI語音生成器自動(dòng)讀出來。此外，他還使用了谷歌搜索引擎及Python編程。 就這樣，GPT-4既能說話，也能看見了。
YOLOv8官網(wǎng)截圖
OpenAI介紹語音自動(dòng)識(shí)別系統(tǒng)Whisper的網(wǎng)頁(yè)截圖 Mckay Wrigley took to Twitter last week to reveal how he had created an AI assistant that can see using his iPhone.
Wrigley used a wild combination of software, YoloV8 for object detection, a vision model, GPT-4 for the AI, OpenAI Whisper for voice, Google Custom Search Engine, and Python, hacked together 'in a fun way.'
“Keep in mind that this AI, it has no prior knowledge of anything. It will only get information from the camera, see all these boxes doing object detection, and from the information it can search on the internet.” 接下來，小哥按下運(yùn)行按鈕，GPT-4立即用女聲說：“我有什么能幫您的嗎？（What can I help you with? ）”
小哥問：“什么是生酮飲食（keto diet）？” GPT-4回答說：“生酮飲食是一種低碳水化合物、高脂肪的飲食。它可以降低血糖和胰島素水平，并使身體的新陳代謝從碳水化合物轉(zhuǎn)向脂肪和酮類?！?/section> “The keto diet is a low carb, high fat diet. It lowers blood sugar and insulin levels, and shifts the body's metabolism away from carbs and towards fat and ketones. ” 小哥接著說：“好的。我想知道，如果我給你看一些食物，你能告訴我哪些屬于生酮食物嗎？” “All right, I want to know if I show you food that I have, can you identify which food is keto?” GPT-4答道：“當(dāng)然。給我看看你的食物，我能告訴你哪些是生酮食物。” “Sure. Show me your food, and I'll tell you what is a keto food. ” 之后，小哥打開冰箱，用很快的速度拍攝了冰箱里的牛油果、牛排、橙子、檸檬、醬料等食物。
視頻可見，即便小哥拍得很飛速，這些物品都被檢測(cè)到，并框了起來。
之后，小哥關(guān)上冰箱，對(duì)大家解釋說：“當(dāng)我關(guān)上冰箱的時(shí)候，它就明白我不想繼續(xù)展示了，然后它應(yīng)該就能開始處理信息了?！?/section> “And when I turn away from the fridge, it's gonna know that we're not looking at the fridge anymore, and it should be able to process this.” 過了幾秒鐘，GPT-4說：“看起來，你家里有牛排、菠菜、黃油、牛油果和檸檬。這些都是很棒的生酮食物?！?/section>
“It looks like you have steak, spinach, butter, avocados, and lemons. Those are all great keto foods.” 小哥接著指示GPT-4：“我想讓你在網(wǎng)上搜索一個(gè)使用這些食材的菜譜，這樣我就可以做晚餐了?！?/section> “I want you to search the internet for a recipe that uses those ingredients so that I can make dinner. ” GPT-4連網(wǎng)搜索了一會(huì)兒后回答：“我搜到一個(gè)叫《15分鐘快手檸檬大蒜黃油牛排配菠菜》的菜譜，用到了你有的食材。” 并在屏幕上給出了菜譜的網(wǎng)頁(yè)鏈接。 之后，小哥點(diǎn)進(jìn)GPT-4給的菜譜網(wǎng)頁(yè)，發(fā)現(xiàn)不是很完美，因?yàn)槔锩嬉灿玫搅怂淅餂]有的食材。
