> ## Content Index
> Fetch the complete content index at: https://nordisk-ai.ghost.io/llms.txt
> Use this file to discover other available public pages before exploring further.

# Gecko: A New Standard for Testing AI Image Generators
- URL: https://nordisk-ai.ghost.io/gecko-a-new-standard-for-testing-ai-image-generators/
- Published: 2024-06-03T21:35:02.000Z
- Updated: 2026-08-27T17:21:45.000Z
- Description: DeepMind takes AI image generation to the next level with Gecko, a new standard that promises more reliable and comprehensive evaluations of how AI models turn text into visual images.
- Author: Anders Hagen
- Tags: AI Services and Tools, AI, DALL-E, DeepMind, Gecko, Midjourney, Stable Diffusion, standard, #Migrated-1787841634512, #wp, #wp-post, #Import 2026-08-27 14:41, #Import 2026-09-07 13:41

Google's DeepMind has introduced a new method for evaluating AI-based image generators (e.g., Midjourney, Stable Diffusion), a system called "Gecko." This initiative highlights the shortcomings in how we currently assess these models' ability to turn text into images. At a time when images generated by artificial intelligence, like an astronaut on a horse or an avocado in a therapist's chair, are captivating the public, the question arises: Are these systems really as good at understanding and translating our instructions (prompts) as we think?

In their research, published on arXiv, among other places, the DeepMind team criticizes the current methods for evaluating such models, like DALL-E, Midjourney, and Stable Diffusion. They argue that the datasets and automatic metrics most commonly used today don't fully capture how well the models perform at generating relevant images based on given prompts.

[See the Research Paper Here](https://arxiv.org/pdf/2404.16820?ref=nordisk-ai.ghost.io)

### Gecko – A New Direction for Reliable Evaluation

The Gecko proposal from DeepMind aims to introduce a more comprehensive and reliable standard for benchmarking the rapidly growing technology behind AI-generated images. The research, titled "Revisiting Text-to-Image Evaluation with Gecko: On Metrics, Prompts, and Human Ratings," points out that small-scale human evaluations offer limited insight, while automatic metrics often miss important nuances and sometimes don't even align with human raters.

DeepMind's new approach aims to create a more robust framework for systematically testing and comparing the performance of AI image generators, with a stronger connection to human evaluation standards. By using both human judgments and refined metric systems, they hope to provide a more holistic picture of how AI systems handle complex, creative tasks.

### Excerpt for WordPress blog