What is a GPT?
The introduction of generative pre-trained transformers (GPTs) marked a significant milestone in the adoption and utility of artificial intelligence in the real world.
The technology was created by the then fledgling research lab OpenAI, based on previous research done on transformers in 2017 by Google Labs.
It was Google's white paper "Attention is all you need", which laid the foundation for OpenAI's work on the GPT concept.
As seen in
> Model matchup surprise
> ChatGPT announcements
> Goodbye ChatGPT-4
> Why ChatGPT 4.1 is a big deal
Transformers provided AI scientists with an innovative method of taking user input, and converting it to something that could be used by the neural network using an attention mechanism to identify important parts of the data.
This architecture also allows for the information to be processed in parallel rather than sequentially as with traditional neural networks. This provides a huge and critical improvement in speed and efficiency of AI processing.
OpenAI's GPT architecture was released in 2018 with GPT-1. By significantly refining Google's transformer ideas, the GPT model demonstrated that large-scale unsupervised learning could produce an extremely capable text generation model which operated at vastly improved speeds.
GPT's also uprated the neural networks' understanding of context which improved accuracy and provided human-like coherence.
Before GPT, AI language models relied on rule-based systems or simpler neural networks like recurrent neural networks (RNNs), which struggled with long-range dependencies and contextual understanding.
The story of the GPT architecture is one of constant incremental improvements ever year since launch. GPT-2 in 2019 introduced a model with 1.5 billion parameters, which started to provide the kind of fluent text responses where AI users are now familiar with.
However it was the introduction of GPT-3 (and subsequently 3.5) in 2020 which was the real game-changer. It featured 175 billion parameters, and suddenly a single AI model could cope with a vast array of applications from creative writing to code generation.
GPT technology went viral in November of 2022 with the launch of ChatGPT. Based on GPT 3.5 and later GPT-4, this astonishing technology instantly propelled AI into public consciousness in a massive way. Unlike previous GPT models, ChatGPT was fine-tuned for conversational interaction.
Suddenly business users and ordinary citizens could use an AI for things like customer service, online tutoring or technical support. So powerful was this idea, that the product attracted a 100 million users in a mere 60 days.
Today GPT is one of the top two AI system architectures in the world (along with Google's Gemini).
Recent improvements have included multimodal capabilities, i.e. the ability to process not just text but also images, video and audio.
OpenAI has also updated the platform to improve pattern recognition and enhance unsupervised learning, as well as adding agentic functionality via semi-autonomous tasks.
On the commercial front, GPT powered applications are now deeply embedded in many different business and industry enterprises.
Salesforce has Einstein GPT to deliver CRM functionality, Microsoft's Copilot is an AI assisted coding tool which incorporates Office suite automation, and there are multiple healthcare AI models which are fine-tuned to provide GPT powered diagnosis, patient interaction and medical research.
At the time of writing the only two significant rivals to the GPT architecture are Google's Gemini system and the work being done by DeepSeek, Anthropic's Claude and Meta with its Llama models.
The latter products also use transformers, but in a subtly different way to GPT. Google however is a dark horse in the race, as it's becoming clear that the Gemini platform has the potential to dominate the global AI arena within a few short years.
Despite the competition, OpenAI remains firmly at the top of many leaderboards in terms of AI performance and benchmarks. Its growing range of reasoning models such as o1 and o3, and its superlative image generation product, GPT Image-1 which uses the technology, continue to demonstrate that there is significant life left in the architecture, waiting to be exploited.
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Yahoo
23 minutes ago
- Yahoo
Australia's WiseTech to cut some jobs in AI-driven efficiency push
(Reuters) -Australia's WiseTech Global confirmed on Wednesday it was cutting some roles as part of a workforce review to focus on "maximizing efficiency via automation and use of artificial intelligence". The software firm did not specify the number of jobs to be impacted in an emailed response to a Reuters query. The Australian Financial Review reported earlier in the day that the Sydney-headquartered logistics software provider has told employees it is increasing the use of AI across the business as part of a broad restructure, citing an email from WiseTech's chief of staff, Zubin Appoo. WiseTech, known for its flagship CargoWise platform, has a team of around 3,500 people across 38 countries, as of June 30, 2024, according to its 2024 annual report. A spokesperson said the firm is "supporting all impacted team members through this transition, including access to professional outplacement services." Wisetech's move mirrors broader industry trends, with technology companies worldwide reducing headcount to fund heavy investments in AI infrastructure. Earlier this month, Microsoft announced plans to lay off nearly 4% of its workforce, while big tech peers including Amazon, Facebook parent Meta and Alphabet's Google have all trimmed their labour forces in recent years.
Tom's Guide
25 minutes ago
- Tom's Guide
Sam Altman gives stern warning on AI, fraud and passwords — 'That is a crazy thing to still be doing'
Sam Altman, CEO of OpenAI, has quickly become one of the biggest names in AI. Like some of his competitors, Altman's position in AI hasn't stopped him speaking out on concerns in the industry. Most recently, that's around the AI fraud crisis. With AI and deepfakes, it is easier than ever for people to impersonate others and commit fraud using these advanced tools. 'A thing that terrifies me is apparently there are still some financial institutions that will accept a voice print as authentication for you to move a lot of money or do something else — you say a challenge phrase, and they just do it,' Altman said at the Federal Reserve conference in Washington. That is a crazy thing to still be doing… AI has fully defeated most of the ways that people authenticate currently, other than passwords 'That is a crazy thing to still be doing… AI has fully defeated most of the ways that people authenticate currently, other than passwords.' These comments from Altman come as AI shows its ability to replicate voices more efficiently than ever. While safeguards are often put in place by some of the bigger companies, it is still a simple process to replicate anyone's voice with a few short recordings. The technology is also growing rapidly in the video world. There are concerns that, in the near future, we'll be able to easily produce a video and audio combination that can fool anyone into believing they are having a full conversation with a real person. This speech comes as the White House ramps up to release its 'AI action plan'. This would be a policy document that outlines its approach to AI regulation. This is something other countries have looked to do as well, but few have produced very detailed approaches. Get instant access to breaking news, the hottest reviews, great deals and helpful tips. OpenAI is gearing up to be ahead of the AI movement in terms of policy. The company has confirmed that it will soon open its first Washington, DC office with a small workforce. This would be used to host policymakers and provide AI training to those making decisions on the technology. The company also recently announced an agreement with the U.K. government to find ways to use AI in government decisions, as well as helping explain the benefits and concerns of AI. Altman isn't alone in concern from the higher-ups of AI. Anthropic's CEO Dario Amodei has recently warned about the effects of AI on job security, and the Future of Life institute has recently claimed that most AI companies are not planning enough for the future. However, where AI can be a risk, it can also be a benefit. While some AI tools look to create scams, others look to detect it. As the technology gets better, it becomes more useful on both sides of the spectrum.
Android Authority
25 minutes ago
- Android Authority
The Pixel 10 Pro is copying one of the worst things about the Galaxy S25 Ultra
Joe Maring / Android Authority Just a day after official Google Pixel 10 renders leaked (showing the phone in some really striking new colors), we now have the same treatment for the higher-end Pixel 10 Pro. While the Pixel 10 Pro's design hasn't been a secret up until now, these latest renders do provide us with the best look yet at Google's upcoming flagship. And, perhaps most importantly, we now know what colors the Pixel 10 Pro will be available in. So, what are we getting? Unfortunately, the majority of the Pixel 10 Pro's colors are quite drab, featuring your choice of black/dark gray, blueish gray, and white. It's a rather depressing lineup, resembling colors that were ripped straight from Samsung's Galaxy S25 Ultra playbook. And I hate it. Which Pixel 10 Pro color is your favorite? 0 votes Obsidian NaN % Porcelain NaN % Moonstone NaN % Jade NaN % Fifty shades of gray C. Scott Brown / Android Authority As technically impressive as the Galaxy S25 Ultra may be, one of the things that annoys me most about the phone is Samsung's color lineup. One of the most powerful Android phones on the market deserves colors that are as exciting as the S25 Ultra is powerful and expensive. Instead, Samsung thought it would be better to adorn the phone in its apparent favorite color: gray. If you go to buy the Galaxy S25 Ultra, you have your choice of four colors: Titanium Black (dark gray) Titanium Gray (really gray) Titanium Silverblue (blue gray) Titanium Whitesilver (light gray) What a lineup it is. No matter what shade of gray you prefer, Samsung has you covered — whether you like a classic gray or enjoy living dangerously with a blue-tinted gray. It's so utterly boring and bland, even by modern flagship smartphone color standards. Looking at the leaked Pixel 10 Pro colors, it's hard not to draw parallels between Samsung's gray-ified Galaxy S25 Ultra and the colors Google is reportedly using for the Pixel 10 Pro. Obsidian is a decidedly dark gray color, the new Moonstone shade throws a blue-gray color into the mix, and Porcelain (while not technically gray) is a neutral and boring white option. Google can do better than this The exception here is that Jade, with its Pistachio-green backside and gold accents, is really lovely. It's not overly colorful, but it's at least trying to be something other than an off-gray. It's nice! But colors like that shouldn't be the exception; they should be the norm. And for years, that was the norm for Google. It wasn't that long ago when Google's flagship Pixels came in colors like Oh So Orange, Really Blue, and Sorta Sage. You used to be able to buy the best Pixel on the market in properly exciting colors. But fast forward to 2025, and they're being overrun by gray, more gray, and washed-out versions of other colors. Google Not only do Google's previous Pixel colors remind us of what we used to have, but it's all the more frustrating considering that the cheaper Pixel 10 is due to get much brighter and more vibrant paint jobs. The Pixel 10's Indigo color is a deep, rich blue that looks like its saturation levels were cranked up to 11, while the Limoncello finish is a brighter, more colorful version of the Pixel 10 Pro's Jade. Google is very much still capable of crafting good smartphone colors; the Pixel 10 is proof of that. So then, why does the Pixel 10 Pro look like it does? Why did Google copy Samsung's gray obsession? Why is every smartphone company under this bizarre assumption that people who buy 'Pro' and 'Ultra' phones have the color palette preference of the average House Hunters couple? Google was one of the last shining beacons of regularly exciting flagship smartphone colors, and if the Pixel 10 Pro is any indication, it, too, has finally fallen victim to the gray abyss.
