My brief experience with copilot - it tended to hallucinate! #2
Replies: 4 comments 3 replies
-
Ha ha, yes I have! I've seen this sort of behaviour with ChatGPT myself.
The AI tries to "fill in" something for which it doesn't actually have
documentation. In my case, I asked for it to run the radiative transfer
code Hyperion (which is very poorly documented) for an AGB star with a
carbon-rich dust shell. While some of the import statements were grounded
in reality, the most important part (setting up the dust species) was
indeed accomplished by inventing a function set_carbon_dust that doesn't
exist. Each time I pointed out a function didn't exist, it apologised and
used a different guess for the same (set_mineralogy('carbon'), for example).
The AI still requires the programmer to review its code, especially in the
case of poorly-documented packages. One can also ensure better results by
having the prompts be as specific as possible. I tried a few variations of
your prompt, even going so far as to say "identify lines using
astroquery.AtomicLineList". It looks like it always fails at finding the
query_object method that is part of this package, and thus the invented
functions :D
…On Thu, Mar 23, 2023 at 4:00 PM William Henney ***@***.***> wrote:
Hi Sundar. Many thanks for your talk yesterday. So, I tried it out last
night and had mixed results. Some things worked fine, but it had real
difficulties when I asked it:
# Find a line identification from AtomicLineList
and it suggested that I do
from astroquery.atomic import Atomicline = Atomic.get_spectral_line_list(ion='H I', wavelength_min=6562.8, wavelength_max=6563.2)
which is wrong in a few respects: there is no astroquery.atomic.Atomic
and there is no function get_spectral_line_list().
I am not even sure where it was getting that from. I first thought it
might be an old version of the API, but google gives zero hits for that
function...
Have you seen any similar flights of fancy?
—
Reply to this email directly, view it on GitHub
<#1>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADXFO52YKH3SBQLWHNVUDB3W5TBX7ANCNFSM6AAAAAAWFZC7OI>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
Looking on the positive side, you could argue that |
Beta Was this translation helpful? Give feedback.
-
P.S. What do you think about turning on Discussions for the dawg-at-irya organization? Then this thread could be migrated to a Discussion, since it isn't really an Issue as such. Also, we could encourage more people to join the organization. You could send invitations to the 4 people who are following |
Beta Was this translation helpful? Give feedback.
-
Apparently Copilot also understands comments with TeX-style markup, and suggests code based on these as well! I tried the following:
subsequently, if I type |
Beta Was this translation helpful? Give feedback.
-
Hi Sundar. Many thanks for your talk yesterday. So, I tried it out last night and had mixed results. Some things worked fine, but it had real difficulties when I asked it:
# Find a line identification from AtomicLineList
and it suggested that I do
which is wrong in a few respects: there is no
astroquery.atomic.Atomic
and there is no functionget_spectral_line_list()
.I am not even sure where it was getting that from. I first thought it might be an old version of the API, but google gives zero hits for that function...
Have you seen any similar flights of fancy?
Beta Was this translation helpful? Give feedback.
All reactions