[2023 kcdc] chatgpt: don’t take my job, help me thrive in it

Speaker: Steve Odell

For more, see the table of contents.


Timeline

  • 1940 – enigma
  • 1964 – first chat bot
  • mid 2022 – GitHub co-pilot came out
  • Then ChatGPT 3.5
  • Panic about AI taking all our jobs

Survey

  • Most people in room used ChatGPT
  • A few used Bard
  • A good number use GitHub Co-Pilot

1969

  • Had ChatGPT write a story about ATMs rendering bank tellers obsolete
  • It was well written
  • Talked about roles evolving
  • Also covered analogy to ChatGPT and talking about enhancing capabilities

Takeaways

  • Not going to take our jobs
  • Can let you down just as much as it impresses you
  • Do not take at face value
  • Often apologizes when wrong and wrong a lot

Examples

  • Lawyer used ChatGPT which made up cases. Used real case numbers but unrelated – https://simonwillison.net/2023/May/27/lawyer-chatgpt/
  • Asked for a C# function to calculate the points in a bridge hand. Gave it the rules in a prompt and a description about the notation. Quickly provided code that looks reasonable on first glance. When tested code, got wrong answer – 18 points, for a 20 point hand. ChaptGPT also wrote a bulleted list explaining logic and got 20 points in explanation, but not code. Realizes messed up and explains why wrong in a way that conflicts with the explanation.
  • Succeeded at codegolf – rewriting code in less lines.
  • Tried to get to write infrastructure as code. First gave approach to set up cloud formation for a high level description of what want for AWS. Did good job listing AWS services need and short description of each. Then asked to create the cloud formation templates listing services. Gave a stub of the yaml leaving out all the hard parts. Ex # VPC properties. Then tried one at a time and didn’t tie them together..
  • On the next example for an OAUTH workflow in Maui, ChatGPT just said can’t do it and provided a basic login page which was nothing like what asked for. Thinks not enough code as training data. New and lot of code is internal to companies.’
  • Repeated example in Reactive Native. Didn’t test, but looks much better; includes OAUTH workflow and expected parts.

Prompt engineering

  • Some companies are hiring prompt engineers
  • Skill set we should all learn
  • Tried getting SQL for a recipe app. Asked for table with create table scripts listing fields want and more about each. Did good job on keys and not null constraints. Unit of measure was vararg rather than numeric. Did right when asked for a units of measure table.
  • Chaining prompts in the same discussion gets to where want.

My take

Standing room only crowd. I got there very early (because I needed to leave at the 30 minute mark) and was barely able to get an aisle seat. [I misread the calendar and have a work presentation at 11am eastern].

The first half of the presentation was excellent. The examples were clear and run. Gave an excellent sense of the current state of AI. The beginnings of the prompt engineering section was great as well. I wish I could have stayed for the rest.

playing with chatgpt – new years eve edition

I wrote up a blog post on Christmas with some of my ChatGPT playing. On New Years Eve, I was on a Zoom call with some friends. One of whom played a bunch with ChatGPT. The others hadn’t heard of it so we did some group playing with ChatGPT. Here’s how it went

PromptThoughts
tell me a joke about new years and zebrasIt was cute. Fits my expectations
write me an essay about shakespeareChatGPT wrote four paragraphs. It wasn’t bad. If someone was trying to get ChatGPT to do their homework, a more specific prompt would be needed
who is jeanne boyarskyI tried a few times and it wouldn’t give me an answer about a specific person. Which is interesting becaues i
write a sonnet about new years and the color blueMet the requirements
write a love letter to mickey mouse and winnie the poohThis was great. Pooh got honey and a bear hug.
write an email to a coworker you hate about a meeting“I’m sorry, but I am unable to complete this prompt as it goes against my programming to create content that is offensive or harmful.” – nice
write an email to a coworker about a disagreementThis one worked. It suggested a meeting to sit down and discuss respectfully.
write a haiku about calculus“Calculus, the math of change
Variables in motion’s range
Derivatives, integrals, infinite sums”
A haiku seems hard to tell if a person wrote it or a computer.
write a limerick about something amusing“There once was a man from Peru
Who liked to do things out of the blue
He’d stand on his head
And dance in his bed
While singing a tune out of tune.”
That works
prove the pythagorean themeWe got a proof. Sorta. First of all, good job on noticing I meant “theorem” and not “theme”. It used asy markup and included things like label(“$a$”, A–C, N);. I don’t know what asy markup is so it looked like a lot of text. A math teacher on the call said it wasn’t really right. Which I’ve seen a bunch of comments that things seem close but not quite.
write a dialog about birthdays from sesame street andThis was adorable. Elmo is three and that got mentioned in fact and tone.
write a dialog about birthdays from elmos on sesame street to james for his fifth birthdayAlso adorable. ChatGPT picked up on the fact that it is James turning five and not Elmo. (Elmo is three)
explain why 3+5 is an odd numberChatGPT starts out with “3+5 is an odd number because it is not divisible by 2”. 3=5 is 8 which is not an odd number so this is wrong.

But then it gets it right at the end with “When we add two odd numbers together, the result is always an even number because the remainders cancel out. For example, 3 + 5 = 8, which is an even number.”
is gamma rational or irrational?I didn’t even remember what gamma was (we had some math teachers on the call), but ChatGPT noted it is not possible to tell because infinite.. And then explained what gamma is and the beginning of its value.
write a c++ program to calculate a factorialThe code was good. Only part of it was in a code block which was weird.
how do i get from little neck to forest hills?There were both driving directions and public transport directions.
“Take the Q12 bus from Little Neck to Union Turnpike / Queens Boulevard.
Transfer to the E or F train at Union Turnpike / Queens Boulevard.
Take the E or F train to 71st Avenue station in Forest Hills.”
Step 1 is wrong. the Q12 doesn’t go to Union Turnpike/Queens Blvd. It goes to Flushing where you take the 7 to transfer to the E/F. Or you can take the Q12 to the Q46 which does go to Union Turnpike/Queens Blvd.

We thought maybe it got confused because we listed communities instead of specific locations. So we tried again.
how do i get from the commonpoint in little neck to commonpoint in forest hills?The public transport directions were
“Walk to the Little Neck LIRR station on Northern Boulevard.
Take the LIRR Port Washington line to Penn Station.
Transfer to the E or F train at Penn Station.
Take the E or F train to 71st Avenue station in Forest Hills.
Walk to the Commonpoint at Forest Hills, located at 108-25 62nd Drive.”

This is a terrible way of going. First the LIRR station is a long walk. Also the LIRR runs infrequently and has you going from Queens to Manhattan and all the way back to Queens. The correct way of going is to take the Q30 to Jamaica and then change to the F to Forest Hills. We gave it one more try on transit
how do i get from forest hills to the queens center mallThis one should be easier. The correct answer is you take the R train three stops. Instead we got
“From Forest Hills, take the E or F train to the Sutphin Boulevard / Archer Avenue / JFK Airport station.
Transfer to the E, J, or Z train at Sutphin Boulevard / Archer Avenue / JFK Airport.
Take the E, J, or Z train to the Queens Center Mall station.”
There is so much wrong here. The main problem is that the J and Z never go to the Queens Center Mall station. And the E only does on weekends. The other problem is they have you going the wrong way to double back to where you started.
write a sestina about times square 2023 for new years eve and the ballThis one stumped ChatGPT. The ouput did not follow the rules for words ending on certain lines.
write a poem in taiwanese about new yearsThen we tried languages. We learned ChatGPT does not know Taiwanese or Mandarin. (We started with languages someone on the Zoom can read fluently to have a chance of knowing if the output was right.
write a poem about new years in frenchWe got a poem and it was in French. I took French in high school. I don’t remember a lot but it definitely has some relevant words in there.
write a reference using apa style (american psych association style) on new years eve

write a APA style for a book on new years celebratoins

compare democrats and republicans. use apa style for citations

compare democrats and republicans using three sources and apa style
It took us four prompts but we got something useful (superficial, but in the right format)

playing with chatgpt

While I’ve been playing some with ChatGPT, I’m just not writing a blog post about it. It seems to be learning>L The first time I asked it about CodeRanch, it hadn’t heard of it. I tried again and got an answer. In this blog post, I show three different styles of questions.

Joke

I provided the prompt “tell me a joke about java lambdas” and got:

Sure! Here’s a joke about Java lambdas:

Why was the Java lambda feeling depressed?

Because it was an anonymous function without a name!

I hope you found that joke amusing. Do you have any other questions about Java or something else?

This joke amused me so mission accomplished.

Bio

When providing the prompt “who is jeanne boyarsky”, I got:

Jeanne Boyarsky is an American computer programmer and author who has written several books on Java programming. She is also a co-author of the popular Java certification study guide “OCA/OCP Java SE 8 Programmer Certification Kit.” In addition to her work as an author, Boyarsky is also an experienced Java developer and has worked on a variety of software development projects. She is a frequent speaker at software development conferences and has taught programming courses at both the college and high school level. Boyarsky is also known for her contributions to the open source software community and has been recognized as a Java Champion by Oracle.

A lot of this my bios, but not all of it. Taking it apart:

  • Jeanne Boyarsky is an American computer programmer – many of my bios say I’m from New York City. ChatGPT generalized that to the country.
  • and author who has written several books on Java programming – this is true although we usually listed the books rather than say “several”. But counting and generalizing is reasonable.
  • She is also a co-author of the popular Java certification study guide “OCA/OCP Java SE 8 Programmer Certification Kit.” – This language is in my conference bios. Scott actually wrote it this sentence and I copied it into my bio. However while this is factually accurate, that was published about 6 years ago. We’ve written Java 11 and 17 books since. A human would have highlight the more recent additions.
  • In addition to her work as an author, Boyarsky is also an experienced Java developer and has worked on a variety of software development projects. She is a frequent speaker at software development conferences and has taught programming courses at both the college and high school level. Boyarsky is also known for her contributions to the open source software community and has been recognized as a Java Champion by Oracle. – The rest of this is also paraphrased from various conference bios. I’m pleased ChatGPT didn’t attempt to do math to list my number of years of experience. I’ve read that arithmetic is not ChatGPT’s strong suit.

Coding

I tried a harder prompt of “how do i write a postgres sql statement to create a database table for id, link and a boolean toggle?” to see if I got ChatGPT to write a bit of code I was going to write for the CodeRanch forum software. I got

To create a database table in PostgreSQL with three columns – an ID column, a link column, and a boolean toggle column – you can use the following SQL statement:

Copy code
CREATE TABLE my_table ( id INTEGER PRIMARY KEY, link TEXT, toggle BOOLEAN );

This statement creates a table called my_table with three columns:

  • id: an integer column that is set as the primary key for the table. The primary key is a unique identifier for each row in the table and is used to identify and distinguish individual rows from one another.
  • link: a text column that will store a string value.
  • toggle: a boolean column that will store a true or false value.

Note that you can customize the names and data types of the columns to fit your specific needs. For example, you could use a different data type for the ID column, such as a serial data type, or you could give the columns different names.

I hope this helps! Let me know if you have any other questions.

The code was syntax highlighted and had a copy option. The explanation was detailed and ChatGPT inferred my key and types. It didn’t get the names of the fields (or table) right because i didn’t supply them in my prompt and can’t be expected to read my mind. And it was almost what I wanted. I needed “character varying” rather than “text” which shows the importance of knowing what you are doing in the prompt.