Using generative AI to write a map description

[TomBaker]

I recently upgraded my free ChatGPT account to a paid version.  Paid versions include beta features like plugins and the Code Interpreter.  Using the GPT-4 model, I added a plugin designed to describe images in text.  It is a bit like “reverse engineering” an AI prompt that might be used in an AI image generator.  The plugin I used for this post is called SceneXplain. At the time of this post, the plugin was free to use.

Prompt:

In ChatGPT 4, I provided this prompt:

explain this image: http://images.mapsofworld.com/usa/thematic-maps/usa-population-map.jpg

Results:

ChatGPT and SceneXplain returned the following response:

"The image shows a map of the United States, which is displaying the population estimate for each state. At the top of the map, there is a caption that reads "USA POPULATION ESTIMATE BY STATE" in large letters. Below this caption, there is additional text that says "(As of July 1, 2016)".

The map is divided into various sections, each representing a different state. The states and their corresponding abbreviations are labeled with text positioned within their boundaries. Some of the state abbreviations include "WA" for Washington, "MT" for Montana, "ME" for Maine, "ND" for North Dakota, and "OR" for Oregon. These abbreviations are scattered across the map, with each one appearing near its respective state's boundary.

The map also includes labels for population ranges, symbolized by different colors. These labels are placed at the bottom of the map. They indicate the population ranges from "10,000,001-40,000,000" to "1,000,001-5,000,000" to "100,000-1,000,000". The corresponding scale for the range values is displayed as "1 82.5 125" on the right side."

Reactions?

1. At this time, the tools don’t describe the pattern in the map.
2. It’s not clear the tools understand the linkage between the legend and map
3. The tools do a reasonably decent job at describing the content of the image.
4. The narrative could be helpful as context or for framing a map that a student is to interpret. This may be especially useful for students with visual or cognitive impairments.
5. The tools don’t work well today for  complex JPG/PNG maps nor does this tool work on interactive maps.

This is a first generation technology.  What could this look like in 1 year, 5 years, 10 years?

[AyanPalit]

This article certainly demonstrates potential use of generative AI. Seems like it is a literal read from the image and not much interpretation.   

@TomBaker curious if the tools can answer questions as: 

• Which state has the highest population? Lowest?
• Is Utah’s population more than Texas? Can the tools correlate UT = Utah and TX = Texas?
• Is the population of North Dakota same as that of South Dakota?
• Is Alaska south of Louisiana?  

[DionLiddellv2]

@AyanPalit  and @TomBaker - I'm using ChatGPT4 with image analysis (edit: recently released) for the first time today, less than 3 months after your original post, Tom. I uploaded Tom's map and prompted ChatGPT with Ayan's questions. I removed "Can the tools correlate UT = Utah and TX = Texas?" from the prompt so it had no hints. Here's the verbatim answer from ChatGPT4:

Based on the map you provided:

1. The state with the highest population is California, as it's shaded with the darkest color indicating a population range of 10,000,001-40,000,000.
2. The state with the lowest population appears to be Wyoming, as it's shaded with the lightest color indicating a population range of 100,000-1,000,000.
3. Utah's population is in the range of 1,000,001-5,000,000, while Texas's population is in the range of 10,000,001-40,000,000. So, Utah's population is less than Texas.
4. Both North Dakota and South Dakota have the same color shading, indicating they are in the same population range. So, their populations are within the same range, but it doesn't specify if they are exactly the same.
5. Geographically, Alaska is not south of Louisiana. On the map, Alaska is placed in an inset box for visual clarity, but in actual geographic terms, Alaska is northwest of the contiguous United States and Louisiana.

[AyanPalit]

@DionLiddellv2 Thanks for the analysis and sharing the results. The maturity/interpretation by ChatGPT4 is impressive based on this sampling. 

[TomBaker]

@DionLiddellv2 and @AyanPalit - Impressive development over such a short period.  This is already a useful tool for learning to read a map and for supporting those with visual impairments.  I wonder what it will look like by the end of the school year?  :). It would be great to continue to occasionally post findings here to monitor development. Thank you both!

As a shameless plug, I have a workshop for Generative AI and GIS teachers next week.  It's free but requires registration at: https://t3ghub-esrit3g.hub.arcgis.com/pages/webinars

Also, you ay find this academic article on GPT models and geo of interest: https://www.tandfonline.com/doi/epdf/10.1080/13658816.2023.2266495?needAccess=true 

[_MarkusKamp_]

Especially in the context of accessibility, the evaluation and textual description of a map by generative AI can be a game changer for people with disabilities. Currently, the problem is to make the presentation of thematic maps easily and intuitively accessible to blind people. A reliable description of a thematic map with the help of a screen reader can solve this problem. I am curious to see whether Esri is already working on solutions in its laboratories and when these will be available in the map product.