OpenAI has launched its new flagship model, GPT-4o, which seamlessly integrates text, audio, and visual inputs and outputs, promising to enhance the naturalness of machine interactions.
GPT-4o, where the o stands for omni, is designed to cater to a broader spectrum of input and output modalities. It accepts as input any combination of text, audio, and image and generates any combination of text, audio, and image outputs, OpenAI announced.
Users can expect a response time as quick as 232 milliseconds, mirroring human conversational speed, with an impressive average response time of 320 milliseconds.
The introduction of GPT-4o marks a leap from its predecessors by processing all inputs and outputs through a single neural network. This approach enables the model to retain critical information and context that were previously lost in the separate model pipeline used in earlier versions.
Prior to GPT-4o, Voice Mode could handle audio interactions with latencies of 2.8 seconds for GPT-3.5 and 5.4 seconds for GPT-4. The previous setup involved three distinct models: one for transcribing audio to text, another for textual responses, and a third for converting text back to audio. This segmentation led to loss of nuances such as tone, multiple speakers, and background noise.
As an integrated solution, GPT-4o boasts notable improvements in vision and audio understanding. It can perform more complex tasks such as harmonising songs, providing real-time translations, and even generating outputs with expressive elements like laughter and singing. Examples of its broad capabilities include preparing for interviews, translating languages on the fly, and generating customer service responses.
Nathaniel Whittemore, Founder and CEO of Superintelligent, commented: Product announcements are going to inherently be more divisive than technology announcements because its harder to tell if a product is going to be truly different until you actually interact with it. And especially when it comes to a different mode of human-computer interaction, there is even more room for diverse beliefs about how useful its going to be.
That said, the fact that there wasnt a GPT-4.5 or GPT-5 announced is also distracting people from the technological advancement that this is a natively multimodal model. Its not a text model with a voice or image addition; it is a multimodal token in, multimodal token out. This opens up a huge array of use cases that are going to take some time to filter into the consciousness.
GPT-4o matches GPT-4 Turbo performance levels in English text and coding tasks but outshines significantly in non-English languages, making it a more inclusive and versatile model. It sets a new benchmark in reasoning with a high score of 88.7% on 0-shot COT MMLU (general knowledge questions) and 87.2% on the 5-shot no-CoT MMLU.
The model also excels in audio and translation benchmarks, surpassing previous state-of-the-art models like Whisper-v3. In multilingual and vision evaluations, it demonstrates superior performance, enhancing OpenAIs multilingual, audio, and vision capabilities.
OpenAI has incorporated robust safety measures into GPT-4o by design, incorporating techniques to filter training data and refining behaviour through post-training safeguards. The model has been assessed through a Preparedness Framework and complies with OpenAIs voluntary commitments. Evaluations in areas like cybersecurity, persuasion, and model autonomy indicate that GPT-4o does not exceed a Medium risk level across any category.
Further safety assessments involved extensive external red teaming with over 70 experts in various domains, including social psychology, bias, fairness, and misinformation. This comprehensive scrutiny aims to mitigate risks introduced by the new modalities of GPT-4o.
Starting today, GPT-4os text and image capabilities are available in ChatGPTincluding a free tier and extended features for Plus users. A new Voice Mode powered by GPT-4o will enter alpha testing within ChatGPT Plus in the coming weeks.
Developers can access GPT-4o through the API for text and vision tasks, benefiting from its doubled speed, halved price, and enhanced rate limits compared to GPT-4 Turbo.
OpenAI plans to expand GPT-4os audio and video functionalities to a select group of trusted partners via the API, with broader rollout expected in the near future. This phased release strategy aims to ensure thorough safety and usability testing before making the full range of capabilities publicly available.
Its hugely significant that theyve made this model available for free to everyone, as well as making the API 50% cheaper. That is a massive increase in accessibility, explained Whittemore.
OpenAI invites community feedback to continuously refine GPT-4o, emphasising the importance of user input in identifying and closing gaps where GPT-4 Turbo might still outperform.
(Image Credit: OpenAI)
See also: OpenAI takes steps to boost AI-generated content transparency
Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is co-located with other leading events including Intelligent Automation Conference, BlockX, Digital Transformation Week, and Cyber Security & Cloud Expo.
Explore other upcoming enterprise technology events and webinars powered by TechForge here.
Tags: ai, api, artificial intelligence, benchmarks, chatgpt, coding, developers, development, gpt-4o, Model, multimodal, openai, performance, programming
Original post:
GPT-4o delivers human-like AI interaction with text, audio, and vision integration - AI News
- European parliament prepares tough measures over use of AI - Financial Times [Last Updated On: August 18th, 2024] [Originally Added On: April 20th, 2023]
- Nvidia stock surges on dominant A.I. market position, buy recommendation from HSBC - Fox Business [Last Updated On: August 18th, 2024] [Originally Added On: April 20th, 2023]
- Bloomberg plans to integrate GPT-style A.I. into its terminal - CNBC [Last Updated On: August 18th, 2024] [Originally Added On: April 20th, 2023]
- Deepfake porn could be a growing problem amid AI race - The Associated Press [Last Updated On: August 18th, 2024] [Originally Added On: April 20th, 2023]
- Workforce ecosystems and AI - Brookings Institution [Last Updated On: August 18th, 2024] [Originally Added On: April 20th, 2023]
- Adobe Lightroom AI Feature Tackles a Massive Problem With Photos - CNET [Last Updated On: August 18th, 2024] [Originally Added On: April 20th, 2023]
- How artificial intelligence is matching drugs to patients - BBC [Last Updated On: August 18th, 2024] [Originally Added On: April 20th, 2023]
- These are the tech jobs most threatened by ChatGPT and A.I. - CNBC [Last Updated On: August 18th, 2024] [Originally Added On: April 20th, 2023]
- Elon Musk Launches X.AI To Fight ChatGPT Woke AI, Says Twitter Is Breakeven - Forbes [Last Updated On: August 18th, 2024] [Originally Added On: April 20th, 2023]
- Two late iconic Israeli singers have been resurrected via AI for a ... - JTA News - Jewish Telegraphic Agency [Last Updated On: August 18th, 2024] [Originally Added On: April 20th, 2023]
- AI anxiety: The workers who fear losing their jobs to artificial ... - BBC [Last Updated On: August 18th, 2024] [Originally Added On: April 20th, 2023]
- Grandma exploit tricks Discords AI chatbot into breaking its rules - Polygon [Last Updated On: August 18th, 2024] [Originally Added On: April 20th, 2023]
- Commonwealth joins forces with global tech organisations to ... - Commonwealth [Last Updated On: August 18th, 2024] [Originally Added On: April 20th, 2023]
- The power players of retail transformation: IoT, 5G, and AI/ML on Microsoft Cloud - CIO [Last Updated On: August 18th, 2024] [Originally Added On: April 20th, 2023]
- AI is the word as Alphabet and Meta get ready for earnings - MarketWatch [Last Updated On: August 18th, 2024] [Originally Added On: April 20th, 2023]
- Purdue launches nation's first Institute of Physical AI (IPAI), recruiting ... - Purdue University [Last Updated On: August 18th, 2024] [Originally Added On: April 20th, 2023]
- Will AI ever reach human-level intelligence? We asked 5 experts - The Conversation [Last Updated On: August 18th, 2024] [Originally Added On: April 20th, 2023]
- The next arms race: China leverages AI for edge in future wars - The Japan Times [Last Updated On: August 18th, 2024] [Originally Added On: April 20th, 2023]
- Amazon Unleashes Bedrock: The Game-Changing AI Cloud Service Powering the Future of Tech - Yahoo Finance [Last Updated On: August 18th, 2024] [Originally Added On: April 20th, 2023]
- Atlassian taps OpenAI to make its collaboration software smarter - CNBC [Last Updated On: August 18th, 2024] [Originally Added On: April 20th, 2023]
- Dating an AI? Artificial Intelligence dating app founder predicts the future of AI relationships - Fox News [Last Updated On: August 18th, 2024] [Originally Added On: April 20th, 2023]
- Military Tech Execs Tell Congress an AI Pause Is 'Close to Impossible' - Gizmodo [Last Updated On: August 18th, 2024] [Originally Added On: April 20th, 2023]
- Philips Future Health Index shows providers plan to invest in AI - Healthcare Finance News [Last Updated On: August 18th, 2024] [Originally Added On: April 20th, 2023]
- Reddit Wants to Get Paid for Helping to Teach Big A.I. Systems - The New York Times [Last Updated On: August 18th, 2024] [Originally Added On: April 20th, 2023]
- OpenAIs CEO Says the Age of Giant AI Models Is Already Over - WIRED [Last Updated On: August 18th, 2024] [Originally Added On: April 20th, 2023]
- 9 Resources to Make the Most of Generative AI - WIRED [Last Updated On: August 18th, 2024] [Originally Added On: April 20th, 2023]
- Impact of AI on higher education panel event May 3 - Boise State University [Last Updated On: August 18th, 2024] [Originally Added On: April 20th, 2023]
- Microsoft reportedly working on its own AI chips that may rival Nvidia's - The Verge [Last Updated On: April 20th, 2023] [Originally Added On: April 20th, 2023]
- Deepfake porn could be a growing problem amid AI race - The Associated Press [Last Updated On: April 20th, 2023] [Originally Added On: April 20th, 2023]
- AI cameras: More than 2 on two-wheelers, even if children, will invite fine - Onmanorama [Last Updated On: April 20th, 2023] [Originally Added On: April 20th, 2023]
- How artificial intelligence is matching drugs to patients - BBC [Last Updated On: April 20th, 2023] [Originally Added On: April 20th, 2023]
- These are the tech jobs most threatened by ChatGPT and A.I. - CNBC [Last Updated On: April 20th, 2023] [Originally Added On: April 20th, 2023]
- Will Generative AI Supplant or Supplement Hollywoods Workforce? - Variety [Last Updated On: April 20th, 2023] [Originally Added On: April 20th, 2023]
- Marrying Human Interaction and AI with Navid Alipour - Healio [Last Updated On: April 20th, 2023] [Originally Added On: April 20th, 2023]
- Competition authorities need to move fast and break up AI - Financial Times [Last Updated On: April 20th, 2023] [Originally Added On: April 20th, 2023]
- 5 AI Projects to Try Right Now - IGN [Last Updated On: April 20th, 2023] [Originally Added On: April 20th, 2023]
- Financial Services Will Embrace Generative AI Faster Than You Think - Andreessen Horowitz [Last Updated On: April 20th, 2023] [Originally Added On: April 20th, 2023]
- Grandma exploit tricks Discords AI chatbot into breaking its rules - Polygon [Last Updated On: April 20th, 2023] [Originally Added On: April 20th, 2023]
- US FTC leaders will target AI that violates civil rights or is deceptive - Reuters [Last Updated On: April 20th, 2023] [Originally Added On: April 20th, 2023]
- Why open-source generative AI models are an ethical way forward ... - Nature.com [Last Updated On: April 20th, 2023] [Originally Added On: April 20th, 2023]
- Religion against the machine: Pope Francis takes on AI - Euronews [Last Updated On: April 20th, 2023] [Originally Added On: April 20th, 2023]
- Fujitsu launches AI platform Fujitsu Kozuchi, streamlining access to ... - Fujitsu [Last Updated On: April 20th, 2023] [Originally Added On: April 20th, 2023]
- Commonwealth joins forces with global tech organisations to ... - Commonwealth [Last Updated On: April 20th, 2023] [Originally Added On: April 20th, 2023]
- In this era of AI photography, I no longer believe my eyes - The Guardian [Last Updated On: April 20th, 2023] [Originally Added On: April 20th, 2023]
- AI is the word as Alphabet and Meta get ready for earnings - MarketWatch [Last Updated On: April 20th, 2023] [Originally Added On: April 20th, 2023]
- Google CEO Sundar Pichai warns society to brace for impact of A.I. acceleration, says its not for a company to decide' - CNBC [Last Updated On: April 20th, 2023] [Originally Added On: April 20th, 2023]
- Purdue launches nation's first Institute of Physical AI (IPAI), recruiting ... - Purdue University [Last Updated On: April 20th, 2023] [Originally Added On: April 20th, 2023]
- We soon wont tell the difference between AI and human music so can pop survive? - The Guardian [Last Updated On: April 20th, 2023] [Originally Added On: April 20th, 2023]
- Atlassian brings an AI assistant to Jira and Confluence - TechCrunch [Last Updated On: April 20th, 2023] [Originally Added On: April 20th, 2023]
- How DARPA wants to rethink the fundamentals of AI to include trust - The Register [Last Updated On: April 20th, 2023] [Originally Added On: April 20th, 2023]
- Dating an AI? Artificial Intelligence dating app founder predicts the future of AI relationships - Fox News [Last Updated On: April 20th, 2023] [Originally Added On: April 20th, 2023]
- Snapchat expands chatbot powered by ChatGPT to all users, creates AI-generated images - Fox Business [Last Updated On: April 20th, 2023] [Originally Added On: April 20th, 2023]
- ChatGPT sparks AI investment bonanza - DW (English) [Last Updated On: April 20th, 2023] [Originally Added On: April 20th, 2023]
- AI-generated spam may soon be flooding your inbox -- and it will be personalized to be especially persuasive - The Conversation [Last Updated On: April 20th, 2023] [Originally Added On: April 20th, 2023]
- AI predictions for the new year - POLITICO - POLITICO [Last Updated On: August 18th, 2024] [Originally Added On: January 4th, 2024]
- Intel Hires HPE's Justin Hotard To Lead Data Center And AI Group - CRN [Last Updated On: August 18th, 2024] [Originally Added On: January 4th, 2024]
- At Morgan State, seeking AI that is both smart and fair - Baltimore Sun [Last Updated On: August 18th, 2024] [Originally Added On: January 4th, 2024]
- Opinion | A.I. Use by Law Enforcement Must Be Strictly Regulated - The New York Times [Last Updated On: August 18th, 2024] [Originally Added On: January 4th, 2024]
- UBS boosts AI revenue forecast by 40%, calls industry the 'tech theme of the decade' - CNBC [Last Updated On: August 18th, 2024] [Originally Added On: January 4th, 2024]
- AI is here and everywhere: 3 AI researchers look to the challenges ahead in 2024 - The Conversation Indonesia [Last Updated On: August 18th, 2024] [Originally Added On: January 4th, 2024]
- What software developers using ChatGPT can tell us about how it's changing work - Quartz [Last Updated On: August 18th, 2024] [Originally Added On: January 4th, 2024]
- AI and satellite data helped uncover the ocean's 'dark vessels' - Popular Science [Last Updated On: August 18th, 2024] [Originally Added On: January 4th, 2024]
- 2024 health tech budgets to be driven by AI tools, automation - STAT [Last Updated On: August 18th, 2024] [Originally Added On: January 4th, 2024]
- Samsung's new phones replace Google AI with Baidu in China - The Verge [Last Updated On: August 18th, 2024] [Originally Added On: January 28th, 2024]
- Researchers Say the Deepfake Biden Robocall Was Likely Made With Tools From AI Startup ElevenLabs - WIRED [Last Updated On: August 18th, 2024] [Originally Added On: January 28th, 2024]
- Satya Nadella says the explicit Taylor Swift AI fakes are 'alarming and terrible' - The Verge [Last Updated On: August 18th, 2024] [Originally Added On: January 28th, 2024]
- One month with Microsoft's AI vision of the future: Copilot Pro - The Verge [Last Updated On: August 18th, 2024] [Originally Added On: February 22nd, 2024]
- Nvidia's Q4 Earnings Blow Past Expectations as Company Benefits From AI Boom - Investopedia [Last Updated On: August 18th, 2024] [Originally Added On: February 22nd, 2024]
- HOUSE LAUNCHES BIPARTISAN TASK FORCE ON ARTIFICIAL INTELLIGENCE - Congressman Ted Lieu [Last Updated On: August 18th, 2024] [Originally Added On: February 22nd, 2024]
- What is AI governance? - Cointelegraph [Last Updated On: August 18th, 2024] [Originally Added On: February 22nd, 2024]
- Scale AI to set the Pentagon's path for testing and evaluating large language models - DefenseScoop [Last Updated On: August 18th, 2024] [Originally Added On: February 22nd, 2024]
- Can AI help us forecast extreme weather? - Vox.com [Last Updated On: August 18th, 2024] [Originally Added On: February 22nd, 2024]
- Google launches Gemini Business AI, adds $20 to the $6 Workspace bill - Ars Technica [Last Updated On: August 18th, 2024] [Originally Added On: February 22nd, 2024]
- AI and You: OpenAI's Sora Previews Text-to-Video Future, First Ivy League AI Degree - CNET [Last Updated On: August 18th, 2024] [Originally Added On: February 22nd, 2024]
- Tor Books Criticized for Use of AI-Generated Art in 'Gothikana' Cover Design - Publishers Weekly [Last Updated On: August 18th, 2024] [Originally Added On: February 22nd, 2024]
- Generative AI's environmental costs are soaring and mostly secret - Nature.com [Last Updated On: August 18th, 2024] [Originally Added On: February 22nd, 2024]
- Energy companies tap AI to detect defects in an aging grid - E&E News by POLITICO [Last Updated On: August 18th, 2024] [Originally Added On: February 22nd, 2024]
- Intel Launches World's First Systems Foundry Designed for the AI Era - Investor Relations :: Intel Corporation (INTC) [Last Updated On: August 18th, 2024] [Originally Added On: February 22nd, 2024]
- Google Just Released Two Open AI Models That Can Run on Laptops - Singularity Hub [Last Updated On: August 18th, 2024] [Originally Added On: February 22nd, 2024]
- AI agents like Rabbit aim to book your vacation and order your Uber - NPR [Last Updated On: August 18th, 2024] [Originally Added On: February 22nd, 2024]