Astronomy and ChatGPT: Do Your Own Homework

Posted on by onespaceman2000
Credit: OpenAI

While I recently used the new AI software ChatGPT (created by OpenAI, which was founded by Elon Musk and others) to draft up a poem, I did not use it to learn more about astronomy. Others have already attempted this and came back with mixed results.

Last month, the website Astronomy reported back on its results testing ChatGPT on a few questions. Some of these questsions included:

  • On average, how many stars are formed each year in the Milky Way?
  • What would happen if two supermassive black holes merged?
  • Why do the giant planets rotate so much faster than the terrestrial planets in our solar system?

The verdict? Astronomy was not impressed, reporting:

…once you start asking questions whose answers are not so obvious and dig into the subtleties of astrophysics — the types of informed, discerning questions that our readers regularly send us — ChatGPT becomes prone to giving answers that seem plausible, but are very wrong. To make things worse, ChatGPT is also very bad at knowing when it is making things up, and it can’t cite any sources.

Cloudy Nights users found ChatGPT to be a little more useful when listing out clear facts when answering questions such as “What are the best objects to observe in the night sky during January from the Northern Hemisphere?” It was also helpful answering this question – “What is the forum on the site Cloudy Nights about?”:

Cloudy Nights is a forum-based website that is focused on the hobby of astronomy. The site features a wide range of forums where members can discuss various topics related to astronomy, including telescopes, astrophotography, observing techniques, and more. In addition to the forums, the site also features reviews, articles, and other resources to help people learn more about astronomy and enjoy the hobby. Whether you are a beginner or an experienced astronomer, there is likely to be something of interest on the Cloudy Nights forum.

As with using Wikipedia as your main source, use ChatGPT at your own risk. I have more faith in Wikipedia because it represents the interaction of humans and therefore is based on more context for the answers. It also provides source material, unlike ChatGPT. If you use them as tools you should be okay. If you use them as oracles (in the original sense), then you are on your own.