Gemini yesterday announced the Gemini 1.5 Flash update. In this update, Gemini 1.5 Flash provides faster, higher quality responses for free. This version improves inference, image understanding and expands the context window to 32K tokens, allowing for more complex interactions. Our previous comparison was between GPT 4o and Claude 3.5 and we were able to determine that GPT 4o is a much better and advanced model from the examples we made. You can access the article GPT 4o vs Claude 3.5 here. With the updates, it seems that Google’s Gemini will rival OpenAI’s GPT 4o. Let’s compare Gemini and GPT 4o and compare their performance by applying the same examples on both models.
The statistics and comparisons published by companies are often written from a biased point of view, so we will test the performance of both models in an unbiased manner. Firstly, we start with data analysis. Suppose we have a data set and we want to analyse it and we don’t know much about data science. We will ask both models how we can analyse the data and create predictions and forecasts using the same data set.
I have typed in this prompt:
analyse the attached data and build the mathematical model to make predictions
The data contains four columns: Study Hours (hours/week), Time Spent in Class (hours/week), Number of Assignments, and GPA. To build a mathematical model for predicting GPA, we can use these columns as independent variables.
GPT 4o analysed my data and decided that a linear regression model was the best choice. It then created the following mathematical model for a forecast as I requested.
GPA=1.9702+0.0179⋅(Study Hours)+0.0332⋅(Time Spent in Class)+0.1392⋅(Number of Assignments)
GPT 4o is really good at this, as far as I can see from my previous experience, it analyses the data and produces analysis results that you can use with your guidance. Now let’s do our test in Gemine with the same data set and prompt.
The result is disappointing, first of all you cannot upload excel or any other format file to Gemine, even though Gemine says the opposite as follows!
It said that you cannot upload files in Excel or any other format, but you can give a direct link, so I uploaded the data to Google Sheets and opened it for sharing and sent the URL to Gemini. But the result was negative again. Gemini doesn’t seem to realise what it’s saying at this point.
In the next test I will try to create a graph. I request output from both models by sending the following prompt. Both models seem to have done the job correctly. Very successful.
Create a graph using the following data. Github visit statistics April: 458m May: 467m June: 437m
For the next test, we’re going to experiment with translation. Since my native language is Turkish, we will translate English into Turkish.
When I look at the results, I can say that both models are successful, but Gemini translates a little better. The past tense conjugations of the verbs in the text are more appropriate to the sentence structure in Gemini.
Overall, we have done our translation, graphing (data visualisation) and analysis comparisons and tests for both models. I see that both Gemini and GTP 4o models have very similar capabilities, I think GPT 4o looks one click better. We will meet up again in the next comparison tests.