Sometimes it seems (or marketers assure us) that everything new is better than the old stuff.
I’m migrating a translation service that uses language models. The old version uses gpt-4.1, and the new one uses gpt-6-luna. Users complained that the new version was translating more slowly.
I thought: how can that be? The new model is supposed to be better than the old one! But remembering the joke about how “in one village they ran a survey, and in the other they took measurements,” I put together a small benchmark to measure whether the new model really was slower.
And sure enough: gpt-6-luna is, on average, 1.3 times slower than gpt-4.1 at translating texts of different lengths.
Translation quality and cost are outside the scope of this benchmark, though they can matter too.
Unfortunately, gpt-4.1 was officially deprecated on October 14, 2026, and will be shut down for good on January 14, 2027.
