Pilot testing the big four (?) genAI platforms

[ShiroKuro]

I was accepted into a pilot testing program for generative AI at my uni (fewer than 40% of applicants were accepted, which means it's nice that I got a slot, but it's also interesting, it means they're not offering it to everyone, probably because it's expensive, and there was a lot of demand).

The pilot gives me full access for a year to the paid/pro and/or edu versions of ChatGPT, Copilot, Gemini, NotebookLM (and maybe Claude? I have to check that). Over the summer, we have a series of "challenges" that we complete and then give feedback on each platform and which performed better etc. I believe the end goal, of course, is to help the university make decisions about whether to offer fuller access university-wide or not. Like most unis, all faculty (and students, I think) have access to Copilot, but it's not the same (or as fully featured) as the one we have access to through this pilot.

Today I had ChaGPT, Copilot and NotebookLM write a quiz for me. Gemini wasn't able to compete the task because it can't read text from a pdf, but instead requires you enter text directly. Because my pdf was of a PPT presentation, it would be very time consuming to C&P all of that into text format. So Gemini failed this round.

Content-wise, NotebookLM was best, ChatGPT wins for ease of use, and Copilot was fine but kind of meh.

I asked it to create more questions than needed, on the assumption that some questions would be bad, and in fact, that was the case. But some questions I could easily use for an in-class quiz. But not for a final exam, because it lacks depth.

For an upcoming challenge, I will try to repeat the activity in Japanese.