[HN Gopher] Coping with Copilot
___________________________________________________________________
Coping with Copilot
Author : lameda
Score : 209 points
Date : 2022-08-20 14:49 UTC (8 hours ago)
(HTM) web link (www.sigarch.org)
(TXT) w3m dump (www.sigarch.org)
| WoodenChair wrote:
| I wrote an essay "Introductory Programming Assessment Must
| Accommodate Copilot-like Assistants" about this same topic last
| month and came to the option 3 conclusion (use more exams or in-
| class live coding assignments):
| http://www.observationalhazard.com/2022/07/introductory-prog...
| adrian17 wrote:
| Extra nitpick regarding generated code: the
| `quicksort_random_pivot` implementation isn't what people
| commonly mean by "quicksort", as in: it's not in-place. This
| means that some properties like space complexity are also going
| to be different than what's usually expected of quicksort.
| pydry wrote:
| A professor could put up an almost correct but slightly wrong
| answer on github or even a right answer with a copyright trap.
|
| So it might be more like bringing a tennis racket to a grenade
| fight.
| highwaylights wrote:
| If what you're asking of these students can easily be automated
| by a tool then the problem is in the assignment. Either don't
| request it, knowing it can be automated with ease, or accept that
| bubble sorts are a long solved problem and move on to teaching
| other things.
|
| Lest anyone think I'm saying don't teach important concepts or
| that such things no longer matter: there's a whole world of
| principles and fundamentals that can be taught and assessed
| without fizzbuzzing 100 students who already had access to Google
| and StackOverflow before Copilot came along.
|
| Yes, directly assessing hands-on coding would be much better, but
| these tiny snippets aren't really indicative of real-world tasks
| that will be needed anyway (given that almost anything so
| foundational will almost certainly be in a standard library), it
| doesn't really prepare students for real-world development work,
| and there's a general expectation that new graduates are going to
| be _very very green_ and therefore need a lot of handholding
| regardless. Even more than that, you've no way of detecting
| plagiarism already in these assignments given that a couple of
| basic rename refactors is going to stop you from proving the code
| was lifted from elsewhere to begin with.
| Merad wrote:
| > If what you're asking of these students can easily be
| automated by a tool then the problem is in the assignment.
| Either don't request it, knowing it can be automated with ease,
| or accept that bubble sorts are a long solved problem and move
| on to teaching other things.
|
| If assignments that are easily automated are a waste of time,
| do you have any suggestions on how instructors should help
| students move from "I know literally nothing about writing
| code" to "I know the basics well enough to solve problems that
| aren't trivially generated by Copilot"? In my anecdotal
| experience (albeit years ago) as a graduate assistant, for
| students who are totally new to programming that phase often
| lasts throughout the typical intro to CS course and it's not
| unusual for it to stretch into the second level course.
| Arch-TK wrote:
| While copilot does great on short simple fundamental problems, it
| sucks (and requires good attention and experience to find all the
| bugs in generated solutions) when solving anything more
| difficult. Although I can totally see how this would make
| teaching the basics difficult.
|
| That being said, I think Microsoft should be fined $100B every
| time they give one of their products away for free to students.
| Hopefully that will be sufficient incentive for them to stop so
| being so overtly anti-competitive.
| Jamie9912 wrote:
| Had to disable Javascript on your site because of the awful
| hijacked scroll smoothing
| meltyness wrote:
| My proudest moment was in an embedded systems course, the teacher
| asked for a function to generate fibonacci linearly in assembly,
| there was a reaalllllly long pause, and I barked the answer
| across the classroom. I vividly remember some stunned, disgusted
| glances.
| [deleted]
| copperx wrote:
| GitHub/Microsoft know what's up, because they make Copilot free
| for students, but teachers have to pay for it. The best thing for
| Copilot is to keep teachers as oblivious as possible.
| cothrow_away wrote:
| Euclid said that there is no royal road to geometry. People who
| pick the royal road end up elsewhere. In a society that grades
| them and acts on these grades, it may very well end up on a
| king's throne. But not geometry.
| shakow wrote:
| Could you just use another teaching language? Does copilot work
| as well with e.g. Racket, Raku or Pascal, which all seem like
| decent CS101 languages to me?
| SoftTalker wrote:
| Students don't like learning languages that they are not going
| to use professionally. They want to learn Python, Javascript,
| maybe Java or C#.
|
| This is another clear indicator that most students today see
| college as a higher class trade school, not as a place of
| higher learning.
| infinityio wrote:
| To quote the article:
|
| > Here's an approach that'll work for sure: use some, let's
| call them alternative, programming languages that Copilot
| doesn't really know. (...) Sadly, I have news: Copilot's love
| for programming languages knows no bounds! Racket! Haskell! ML!
| (...) Copilot is a ravenous beast: if any code in any language
| found its way into a GitHub repo, it's already swallowed it up
| and is hungry for more, nom nom nom.
|
| Not sure how true this is in practice - I only used Copilot
| once, with Python - but if it is that might invalidate this
| concept
| dragonwriter wrote:
| If it doesn't now, it will quickly if that solution is adopted.
| emeryberger wrote:
| Hi all - article author here.
|
| A reader of my article sent me this note, pointing me to some
| relevant hot-off-the-presses work (reproduced with permission):
|
| -----
|
| A group recently evaluated the performance of Codex (and
| another model) on 18 programming languages:
|
| https://nuprl.github.io/MultiPL-E/
|
| The high-order bits are:
|
| 1. The latest and greatest Codex is now twice as good on its
| own benchmark suite than the original version published a year
| ago.
|
| 2. It's just as good on Python as it is on JS, Scala, C++,
| Swift, TypeScript... and other languages are not too far
| behind. It's not bad at _bash_ of all things.
|
| Paper here: https://arxiv.org/abs/2208.08227
| [deleted]
| yosito wrote:
| As someone who's used machine translation as a tool to learn 2
| human languages, which I eventually gained fluency in, I'm really
| tired of AI being considered "cheating". Yes, you can use it to
| approximate answers to questions and be fairly certain those
| answers are correct. But you can also use it as a tool in your
| learning process to help you fill in the gaps until you're all
| the way there. In language, it's quite obvious that _using_ the
| language will increase your abilities over time. In programming,
| that may not be so intuitive, but it 's also true. The more
| software you build, the more you will understand about building
| software well, even if you use Copilot to help write your
| algorithms. I think the "problem" here is that tools like this
| make it difficult to enforce that _others_ learn something. But
| for self-directed people, who are committed to learning and
| understanding what they are doing, AI is more of a useful tool
| than a cheat.
| superb-owl wrote:
| While this will definitely be an issue in the short term, it's
| part of a long standing series of panics where a new technology
| develops, and everyone worries that we'll lose the skills it
| abstracts away.
|
| And to some degree we will! I certainly can't do long division
| anymore, and I struggle to navigate my own neighborhood without
| GPS. This is a bit sad, but I think something of an
| inevitability.
|
| If history is any guide, academics should embrace the new tech
| and try to teach students bigger things that copilot can't yet
| handle, with the assumption that they'll use copilot to fill in
| some details--the same way we now teach students to use graphing
| calculators to solve problems that would have been out of reach
| for their grandparents.
|
| There are so many things I never learned as an undergrad because
| we were too focused on algorithms. Database design, testing
| strategies, different architectures, programming language design,
| etc. Maybe these things can come to the forefront of the
| curriculum if we don't need to drill basic algorithms.
| svnpenn wrote:
| > I certainly can't do long division anymore, and I struggle to
| navigate my own neighborhood without GPS
|
| These are not the same thing at all. Yes calculators and such
| have largely made long division pointless, but it's still quite
| useful being able to get around without GPS. especially around
| construction, GPS is awful.
|
| You need to know the limits of the tools you're using. If you
| just say "I'm always going to use GPS, because it's always
| better" you are really doing yourself a disservice.
| tommyage wrote:
| Who should fix the bugs in the future? Nerds? Where are they
| coming from 60 years from now?
|
| E(very) S(ingle) W(ord) S(hould) be (no [ACCEPTED] completion
| here) W(ritten) W(ith) C(onscience). You should be responsible
| for your code submitted. Are your feeling confident?
|
| Won't touch Co-Pilot. Probably you are spending equal time proof
| reading than _understanding what needs to be written_ in advance.
| At least that is what I am hoping about. Of course you don't have
| to proof read, and most things can probably be grasp in a glance.
| But you rely on a black-box. From a corporation. To me that
| sounds like agreeing with a dictator.
|
| Wouldn't promise my craftmanship for things I didn't grasp in
| advance. But maybe that's a culture hindrance. I like it though.
| I want to know the code I am corrected about. I want to archive
| mastery.
| muhehe wrote:
| I've read the title as "copilot with copilot" and thought that
| copilot leaked some parts of itself :)
| BiteCode_dev wrote:
| While it is true code pilot is quite amazing and can produce some
| very decent snippets, if the exercice you give is specific
| enought (using a special database schema, file format, or API),
| with objectives that are not common (ask the user specific
| questions, and draw conclusions out of it or react to events),
| then it cannot produce a fully working program. I use code pilot
| regularly now, and it does speed up my work, but I can't do
| anything with it without understanding the problem at hand.
|
| There is nothing wrong with the student assemblings and fixing
| parts of code that exist. Doing that requires understanding those
| parts, how they interacts and what you can do with them. It's not
| killing tests. Plus as professionals, we all do that anyway.
| Heck, I learned doing that. I'm paid doing that.
|
| However, there is something wrong with academic exercices that I
| see in the wild. Things that are very abstracts, requiring way
| more than programming knowledge, or stuff that are asked out of
| context completely. Students are both understimulated, and given
| too hard bites to chew. It's a terrible way to learn, and to be
| tested IMO.
|
| Tests nowaday don't tests for knowledge, they test either for
| compliance, or act as a filter instead of a feedback loop. This
| is not education.
|
| In fact, I've yet to see school IT course that I didn't find
| deeply flawed, and I completely understand the students for
| cheating at them to that they can suffer as little as possible.
| They are not given a chance to prove themself or progress, they
| are not respected. They are forced fed junk and asked to spit it
| back all shiny.
|
| I remember when I was a student, and I went to 11 different
| schools because of my twisted life. They all sucked. And I say
| that as I spent half my student life at the top of my class grade
| ladder.
|
| I regret having cheated so rarely now.
| NikolaNovak wrote:
| As I moved as well, I experienced three different Canadian
| universities' Computer Science Program.
|
| There was only one professor and course that I still remember
| 20 years later: CS408, Software Engineering, Professor Wortman,
| UofToronto.
|
| Class project was in four phases, cumulative (basic
| functionality for the application in phase 1, progressively
| additional functionality in other phases, frequently strongly
| interacting with previous phase code).
|
| Here's the kicker: After each phase, you had to _swap your
| code_ with another team. So you had to pick up somebody else 's
| code, figure it out, and then build up on it.
|
| Few of us that had real-world working experience loved the
| course and flourished in it. This is what we are training for!
| This is what programming is like! You are taking real code and
| building a real thing with it!
|
| About 250 other students signed a petition to the Dean on how
| this is unfair and awful and they will _not_ put up with it.
| They were just too used to / spoiled by 16 years of 5
| assignments per semester with abstract, entirely separate
| questions of 1.a), 1.b), 1.c), etc.
|
| All I could think of - if you did not like this course, you are
| about to not enjoy next 4-5 decades of your life :D
|
| Other than this one course, I can say that I'm a prolific,
| enthusiastic, life-long learner, and my university experience
| was the absolute dumps - it was far less about learning, and
| far more about bureaucratic hoops and artificial constraints
| and restrictions. I was top of the class in some hard courses
| (generative compilers etc), mid-pack in the some of meh
| courses, but in retrospective, my life opened up when I was
| done with academia and could work and learn in the 'real
| world'.
| x86x87 wrote:
| > Here's the kicker: After each phase, you had to swap your
| code with another team. So you had to pick up somebody else's
| code, figure it out, and then build up on it.
|
| This is hilarious and is a good example of how sw engineering
| works in the real world. Love it.
| bobwaycott wrote:
| Hats off to your prof. This is such a brilliant idea for how
| to teach, anchored in a solid reflection of what "doing the
| job" is like day-to-day outside the academic context.
| johndough wrote:
| > After each phase, you had to swap your code with another
| team.
|
| I'd expect that most students would just "rewrite" the code
| they received from the other team and replace it with their
| own.
| DangitBobby wrote:
| I'm sure that various strategies for gaming the system were
| anticipated and prohibited.
| yosito wrote:
| > you had to swap your code with another team. So you had to
| pick up somebody else's code, figure it out, and then build
| up on it
|
| Genius idea! Though if my professors did this, I may have
| been scared away from a career in programming
| jcranmer wrote:
| There are a few issues with this kind of assignment.
|
| The first issue is that it's really hell on the TAs with all
| of the administrative work needed to do the swapping between
| phases. Software engineering is already one of the more
| annoying classes to TA, and this makes it even less
| palatable.
|
| The bigger issue is that, on the basis of fairness, you grade
| in later phases is very heavily dependent on factors that are
| outside of your control. If you get someone's code who just
| didn't complete the assignment--that is probably going to
| happen once a semester--you are going to be at a
| disadvantage. Even if there is an option to appeal to the TAs
| and get the unacceptable code base replaced with an
| acceptable one, that's still likely to be a few days' worth
| of work lost learning the codebase, discovering its
| incompleteness, and now you have to start over _again_.
|
| I'll also point out that it's not exactly real-world
| experience. It is extremely rare that someone will be dumped
| on a codebase without a prior author remaining on to help
| them ramp up on the code.
| haskellandchill wrote:
| > It is extremely rare that someone will be dumped on a
| codebase without a prior author remaining on to help them
| ramp up on the code.
|
| what? I must be living a rare life.
| ticviking wrote:
| Most of my major professional achievements involve
| jumping into code like this, and solving some problem
| anyhow.
| x86x87 wrote:
| Hahahaha. Most of the times the author has moved on a
| long time ago.
| filoleg wrote:
| Yeah, that statement was the only thing that stuck out to
| me in an, otherwise, pretty good writeup that I mostly
| agree with.
|
| The only times I can think of where the previous author
| was able to help was either during onboarding (where you
| are expected to be given smaller-scope tasks with the
| context of that code being predefined for you) or during
| work on certain specific singular projects as a team.
|
| But at any large big tech company, you are probably gonna
| be jumping around more than a few different codebases
| (more often than not, large and being worked on for
| years), even if you work on a singular team/product. And
| most of the time, you are expected to figure as much as
| possible on your own, only reaching out to the people
| when you get stuck or encounter a specific issue. Other
| than that, you are expected to be able to build at least
| a janky proof of concept with as little help as possible.
|
| Most of the time, the help I sharply needed was rather
| targeted and not really related to the code itself
| overall. It would be something that is kind of extremely
| difficult to guess on your own without it being
| explicitly told (aside from docs/code comments), things
| like "everything looks fine, but I get stuck at this auth
| step, what's wrong there?" - "oh yeah, you gotta auth as
| a part of this specific group, so you need to be added to
| this security group in this config file".
|
| I am not trying to say that i am some genius and can get
| the structure and overall mental model of how a codebase
| works all on my own easily, only with the specific
| "gotchas" (like the ones mentioned in a paragraph above)
| giving me issues. Not at all. The only way I can mentally
| get a structure of the codebase and how the whole system
| works and operates (even as a rough/simplified model) is
| by looking at the design docs/documentation (very
| helpful, but far from sufficient on its own, more like a
| supplementary material of variable quality) and, most
| importantly, making small code changes and debugging the
| code using breakpoints to see how it all flows and where.
| Imo, understanding code through debugging and reading a
| lot into the code is an irreplaceable step for
| understanding a section of any significantly-sized
| codebase.
| ticviking wrote:
| I had a similar software engineering course experience to
| the one above with a lot of things out of my control.
|
| I got an A. By being humble reflective and proactive,
| showing the instructor that I was looking for solutions to
| the problem and giving an appropriate amount of time and
| effort to the class.
| x86x87 wrote:
| I would expect that code you get from Stage N-1 is
| functional across a range of tests that are agreed upon
| upfront.
|
| For the fairness aspect, I would also let students pick any
| Stage N-1 that is not their previous N-1. That would also
| teach em that not all is created equal (and probably
| identify approaches / ideas that they can incorporate in
| their work)
| ericbarrett wrote:
| I love this idea. Maybe some extra credit if your Stage
| N-1 is the most popular choice?
| JackFr wrote:
| I think you could remove a lot of your fairness and
| logistical concerns by at stage N choosing one solution and
| everyone group has to start from there.
|
| Now for pedagogical purposes it might be that the professor
| chooses the best solution or a substantially worse
| solution.
|
| There might be some concern that this unfairly advantages
| one group over the others but it's not completely clear to
| me that it does.
| jefftk wrote:
| You could also choose two, which then ensures no team
| starts with their own work from the previous phase
| corbin wrote:
| > About 250 other students signed a petition to the Dean on
| how this is unfair and awful and they will not put up with
| it.
|
| That behaviour by the students disgusts me.
| bogdanu wrote:
| Wow, your professor had an amazing idea! I wonder why isn't
| this strategy more common in college classes.
| bcrosby95 wrote:
| My software engineering teacher did something similar. We had
| a giant group project that had to be worked on by the whole
| class. He split the class into 5 groups of 5, and we divided
| the work by group, and groups had to work together to
| integrate with each other.
| unnouinceput wrote:
| Quote: "Oh, have I mentioned that Copilot is free for students?
| Yep, COPILOT IS FREE FOR STUDENTS. It integrates helpfully right
| in their favorite IDEs."
|
| Does it integrate with Lazarus?
| debbiedowner wrote:
| Just don't grade HW and have handwritten quizzes and tests in
| class? (Yes, the grad students will suffer when grading... unless
| you pay them and then they will be lining up)
| hospadar wrote:
| I've always thought that the ability to communicate to other
| humans about programs and to be able to synthesize a conceptual
| understanding from looking at code are highly valuable and
| sometimes-overlooked in the workplace. Assignments to write a
| function don't really demonstrate either skill.
|
| Maybe more assignments like:
|
| - "Here is a function that is supposed to generate a fibonacci
| sequence, but it is not correct, explain why"
|
| - Here is a sort function. What is the name of the algorithm
| implemented by this sort function?
|
| The reality of any career in CS is going to involve a lot of
| maintenance/understanding of existing code any systems. Copilot's
| failure to replace all human programmers isn't because of it's
| inability to cough up complex code, it's because it can't
| communicate with the product owner, debug a program amidst a
| time-sensitive incident, or explain its work.
| dwohnitmok wrote:
| Those two questions are definitely within the reach of current
| AI. Not within 100%, but probably better than 70%.
| emeryberger wrote:
| Agreed. I just tried this by writing three sort functions
| named "some_sort", "another_sort", and "yet_another_sort". I
| started comments after each with "# The algorithm used in
| some_sort is known as: " and Copilot correctly completed them
| in all three cases (bubble, insertion, and merge).
| MarcScott wrote:
| And old colleague of mine used to teach CS, when students had to
| send off code they'd written to Cambridge University for it to be
| compiled and run, and the results were then returned, all by
| post.
|
| Then you were able to compile and run code locally on your
| machine. Then the IDEs came along, with syntax highlighting and
| then linting.
|
| Copilot is just the next evolution in writing code, and saving
| you the effort. I bet they railed against Timsort and it's
| inclusion in the Python standard library, but now who honestly
| writes their own sorting algorithms?
|
| This kind of generational gap reminds me so much of the "Kids are
| spending too much time reading" > "Kids are spending too much
| time watching TV" > "Kids are spending too much time on their
| phones" rants.
| social_quotient wrote:
| How does medical school, law school, and non-CS engineering
| resolve this?
|
| Seems like they would have had to solve the "how do I know what
| you know you know issue".
| tzs wrote:
| In law school there generally isn't any homework in classes
| about the law. The only classes with homework will usually be
| classes in legal research or in classes that cover non-legal
| subjects from a legal perspective. As an example of the later,
| at my school there was a "Quantitative Methods in Law" class
| that basically was an introductory statistics class focused on
| applications involving law.
|
| For the classes that are about the law (contracts, torts,
| criminal procedure, etc) it pretty much all comes down to
| exams. I only had one take-home exam [1]. All the others were
| in-class.
|
| The exam questions were mostly essay questions. There would be
| a couple paragraphs or so describing some situation, and you
| would be asked what the legal outcome should be. Your essay
| needed to identify the legal issues involved, cite the relevant
| cases and/or statutes, and argue how those applied to the facts
| of the case to support the result you were trying to argue.
|
| [1] Note: I am not a lawyer. I went to law school when I got
| burned out with programming. By the end of law school I was no
| longer burned out, and decided I'd rather be a programmer who
| knows a good bit about law than a lawyer who knows a good bit
| about programming.
| GrumpyYoungMan wrote:
| Straight up assembly language, preferably Motorola MC680x0 or
| some other contemporary that was meant to be coded by hand; it's
| the only way, man! Plus students will gain some appreciation for
| how microprocessors actually work.
|
| In the same vein, just have them write code out on paper during
| exams and coding competitions like the way we did it back in my
| day. It's not as if they're not going to need that skill anyway
| when they get whiteboarded during interview loops when even the
| most lenient employers get exasperated at "CoPilot-only"
| graduates.
|
| </tongue-in-cheek>
| somenameforme wrote:
| Do people no longer generally have exams on paper?
|
| For lower level courses we had online assignments, and pen and
| paper exams - which is where the majority of the course grade
| came from. If you didn't [intimately] know what you were doing on
| the assignments, you'd fail the classes due to the tests.
| brundolf wrote:
| Eh, it's always been super easy to cheat on programming
| assignments. Often they're variants on toy problems that have
| been used in the past, or worst-case, you could crowdsource an
| answer on stackoverflow or Reddit
|
| Just make sure Copilot is explicitly against the rules, there's
| not much more you can do than that
| twodave wrote:
| Programming and, even more so, software engineering are _trade
| skills_. There will ALWAYS be "cheaters" but you will know them
| by the end product of their work. Yes, it's important to
| understand the fundamentals. An unwillingness to learn these
| things is a great indicator that you will be a failure as a
| professional. As another commenter mentioned, the focus on grades
| at all makes the problem worse.
|
| More important than even fundamentals, though, is modeling a
| successful software development mindset and problem solving
| technique. Too many professors have not _had_ any success in
| software development, which is sad. I can count dozens of
| examples of my own personal growth that could have been handled
| in college, that I instead ended up inflicting on my first
| employer.
| lfkdev wrote:
| With that logic using a calculator is also cheating.
| tgv wrote:
| On an arithmetic test, yes, it is.
| pavelboyko wrote:
| This reminds me of a story that an old professor of theoretical
| physics told me. In the early nineties, he left the former Soviet
| Union for the United States to teach physics at one of the top
| universities. There he encountered the fact that American
| students were fantastically good at solving all his standard
| problems for integrals. It quickly became clear that the students
| were using the then-new program for symbolic calculations,
| Mathematica. As a result, our professor also mastered Mathematica
| and spent half the night finding such integrals that it still
| could not calculate for assignments.
|
| I use Copilot every day and I can assure you it makes a lot of
| mistakes. I think that, at least in the short term, CS teachers
| will still find assignments where it makes mistakes.
| jeffbee wrote:
| I guess CS programs are going to need to grow up and teach real
| topics like ethics, technical communications, effective testing,
| and large-scale systems design instead of how to write simple
| loops, which is something I learned at the age of 4 from "Gortek
| and the Microchips".
| eru wrote:
| What makes you think GPT-3 (or similar) won't write your ethics
| essay for you?
| jeffbee wrote:
| That's very easy to deal with. Oral exams or blue-book exams.
| eru wrote:
| What's a blue-book exam?
|
| You can test loop-writing in an oral-exam, too. Thus your
| proposed solution for GPT-3 writing ethics essays is so
| good, that it fixes the original problem.
| shagie wrote:
| A blue book exam is where you go into the lecture hall
| and write out, long hand, in a blue book the material
| being tested on.
|
| The blue book is a standard.
| https://www.amazon.com/BookFactory-Exam-Blue-Ruled-
| Format/dp...
|
| https://www.thoughtco.com/blue-book-1856928
|
| In the classes that I had that used blue books for exams,
| the exam book was free and given out by the proctor.
| jeffbee wrote:
| You're right, that a pretty good approach anyway. Teach
| your whole course in a fake language or on a fake machine
| like MMIX.
| youssefabdelm wrote:
| Ever heard of just-in-time learning? Copilot (if it gets good
| enough) is the damn teaching tool! All they need to do is come up
| with a programming goal/project thats extremely
| important/relevant/exciting to them, and break the problem down
| into sub-problems for copilot. This time they'll actually give a
| shit to understand why copilot wrote a bug or why the code works.
| No more boring pointless teacher assignments that hold no
| relevance whatsoever for the students most immediate and
| important goals and concerns in life. Finally life is fucking
| good for students for once and they dont need to google every
| answer for an hour.
| norwalkbear wrote:
| So algos take a back seat to imperative code
| captainmuon wrote:
| I don't know, I tutored an introductory course on data science in
| Python last semester, and yeah copilot would have been able to
| answer most of the problems. But students would have been able to
| copypaste them from somewhere else anyway. If you want to test
| them, have them _explain_ what they did, and why they wrote it
| this way and not another way. But personally, I believe that
| people are at the university to learn learning, and to learn
| independence, and if they really want to cheat then it is their
| fault. You spend a lot of time (and in some places money) to be
| here, so you might as well be honest and learn. The degree itself
| is not as helpful as one might think. It is the stuff you learn
| for the degree that helps you later on.
|
| But on the other hand, I found copilot useful as a tutor. When
| somebody asked about an API, I didn't have to akwardly search the
| documentation, I fired up VS code and have copilot suggest
| something (without the students seeing it of course ;-)).
| ctoth wrote:
| Setting aside the academic implications for a moment, if you
| think that Copilot-like models won't be a significant part of
| programming in the future I'd like to point out this research
| from Google:
|
| > We compare the hybrid semantic ML code completion of 10k+
| Googlers (over three months across eight programming languages)
| to a control group and see a 6% reduction in coding iteration
| time (time between builds and tests) and a 7% reduction in
| context switches (i.e., leaving the IDE) when exposed to single-
| line ML completion. These results demonstrate that the
| combination of ML and SEs can improve developer productivity.
| Currently, 3% of new code (measured in characters) is now
| generated from accepting ML completion suggestions.
|
| So 3% of all code at Google is now written by AI autocomplete,
| and Google developers who use this technology are 6% more
| efficient than those who don't[0].
|
| Is it bad that students are cheating on homework with this? As
| bad as it was when students used Stackoverflow to cheat, I
| reckon. But will working with AI models be an essential skill to
| learn for new (and existing) programmers? About as much as
| getting comfortable with Stackoverflow was, I reckon.
|
| [0]: https://ai.googleblog.com/2022/07/ml-enhanced-code-
| completio...
| Lev1a wrote:
| In computer class back in 9th grade there were one or two exams
| about Excel, _where we were to use excel_ to compute some of the
| answers for questions. The fact that "memorization questions"
| (if that's the right English term?) about several functions and
| concepts were on the same exam sheet and that the Excel help
| function was right there was something that seemingly slipped the
| teacher's mind.
|
| Although he was an older physics teacher who only really did
| computer class ("Informatik") because there was nobody else
| available or (more) qualified to do the job. When later on there
| were classes on programming, he made us use TurboPascal instead
| of something less cluttered and actually useful in modern economy
| like Python.
|
| That same teacher later on performed some show experiments about
| induction and electromagnetic forces in physics class where he
| let a magnet fall through a glass tube with several copper coils
| around it. So far so good, but he then tried to readjust one of
| the coils one the glass tube, which predictably shattered in his
| hands, nearly severing some fingers and actually severing some
| tendons, which IIRC were later replaced with other tendons from
| his legs or something. Needless to say, not the sharpest tool in
| the shed.
|
| My other physics teacher during 10th-12th grade physics classes
| was a severely esoteric nutjob, preferring to hold long rants
| about "science" being able to diagnose diseases in people by
| somehow shining a laser at a drop of their blood and interpreting
| the reflection/refraction because there was a "connection"
| between the person and their blood (or something like that, I
| honestly tuned out his ravings after a while). Apparently his
| wife is/was some kind of homeopathic "healer".
|
| All that is to say, computer and physics class teachers for me
| were "very fun" and "useful", since one did not have a clue that
| their students were "cheating" and both were not very in tune
| with modern and/or scientific methods.
| zitterbewegung wrote:
| Just add a requirement where the student has to explain how the
| code works.
|
| To be honest there should be a class called language model
| querying because for simple tasks it does well but eventually you
| will have to debug the output (not just copilot but GPT3 and
| dall-e).
| [deleted]
| wiradikusuma wrote:
| > Just add a requirement where the student has to explain how
| the code works.
|
| There's an AI for that!
| jefftk wrote:
| _> there should be a class called language model querying_
|
| Do you think the skills taught in that class would be relevant
| for very long?
| zitterbewegung wrote:
| Hypothetically if they became irrelevant wouldn't that imply
| that we have models that could just understand natural
| language?
| jefftk wrote:
| Not necessarily. The skills could become irrelevant just
| because the models advance and you need new and different
| skills to query them effectively.
| zitterbewegung wrote:
| But then you would have a foundation to figure out the
| new models. This is the essential problem when teaching
| any field because the field itself advances faster than
| one can effectively teach you.
|
| That's why getting an Internship is always a good idea.
| [deleted]
| GrumpyYoungMan wrote:
| Much like reverse engineering, it's easier to come up with an
| explanation when you have someone else's working solution to
| examine than it is to come up with a working solution from
| scratch on your own. So that doesn't really help.
| vinodkd wrote:
| but maybe, they'll end up learning anyway? one thing i
| realized when i went back to studying online is that i was no
| longer the student who needed to prove something to someone
| else, and the typical test annoyed me. I get that this
| article is about a teacher trying to find out how much their
| students have learned a subject, but maybe that needs to
| change?
| niek_pas wrote:
| Agreed, and, in addition, having every single student explain
| their program would be a significant burden on teaching
| resources.
| t_mann wrote:
| > we teach in programming languages that don't even exist
|
| I remember reading about a framework that actually sort of does
| that - create randomized programming languages. It was meant as a
| security feature against code injections iirc, but technically it
| should work as a solution to OP's problem.
|
| Although I guess some of the other suggestions are better and
| easier to implement - ask students to explain their code (select
| one person randomly per problem, it's quite common anyway) and
| give more weight to tests.
| O__________O wrote:
| Only matter of time before CoPilot like systems (if they do not
| already) just reverse engineer the compilers to produce and
| link a dynamic syntax to know instances in plain-text source
| code of the "foreign" programming languages and render them
| within a language the user already understands; yes, there
| might be gaps in idioms, assuming a natively written language,
| but using example you provided, there in theory would not;
| ...and if need, return a final version in the target "foreign"
| programming language.
| t_mann wrote:
| such systems might spring up, but the market is far more
| limited than for generalist CoPilot-like systems, and they
| will thus likely be less sophisticated and thus easier spot
| (by other automated solutions)
| O__________O wrote:
| Might? This is the future of systems like CoPilot, reading
| existing source code only goes so far and it is only matter
| of time before the state of the art systems dynamically map
| compilers, even foreign ones, to source code. Once mapped,
| system will use its universal transcoder to remap the code
| to a know target language and even dynamically rewrite the
| compiler of the foreign language if appropriate to fill
| idiomatic gaps.
|
| Ultimate end user for systems like these are not humans.
| niek_pas wrote:
| The scrolljacking on this site is awful. Why do people keep doing
| this?
| tazjin wrote:
| Only way to use the web is worth uBlock disabling all JS by
| default. Works on mobile, too.
| stevebmark wrote:
| This is why I love the disable javascript plugin for Chrome.
| One click and terrible sites like these are tamed. uBlock
| origin is also useful to block sticky headers.
| foepys wrote:
| I'm using reader mode in Firefox more and more often because
| websites just cannot keep their hands off my inputs nowadays.
| It's infuriating.
| mediumcat wrote:
| Copilot still works quite poorly with problems that are not
| "Stackoverflowable". Perhaps the best way to fight this is to
| create assignments that cannot be googled in 5 seconds.
| fancyfredbot wrote:
| I get that copilot is AI and pretty cool, but students could look
| up a Fibonacci program on Google before it existed. What's more
| these algorithms are written in books. If students wanted to
| cheat on your "write depth first search" assignment then they
| already were.
| SoftTalker wrote:
| Even implementing an algorithm from a book teaches you
| _something._
|
| Most book algorithms are in pseudocode so you need to know how
| to write a loop or a function in the language you are using.
|
| And just the act of typing it line-by-line will focus attention
| and help cement concepts in the brain (assuming one really
| wants to learn).
|
| Hitting TAB in Copilot doesn't do any of those things.
| throwaway675309 wrote:
| I understand what you're getting at, but feel like you're
| splitting hairs here since somebody who goes to look it up on
| Stack overflow is just going to copy paste... not type it out
| line by line like it's a page of BASIC from BYTE magazine.
| jefftk wrote:
| I think the big change here is how much easier it is? Students,
| like everyone else, are lazy. Previously just doing the
| assignment was typically less work than cheating, and Copilot
| makes that much less common.
| moffkalast wrote:
| Type the assignment into google and press enter vs type it as
| a comment and press tab?
|
| Seems roughly the same amount of effort, and since Copilot is
| not free anymore most students won't bother paying for access
| on any kind of scale that matters. Whoever was going to cheat
| will cheat regardless.
|
| Besides it's one of those "you won't have a calculator with
| you everyday" fallacies. If you can solve problems with
| Copilot in class you can also do it at your job later on.
| sliken wrote:
| Not really, for small snippets, sure. But jobs require
| getting something done. Sure you might be able to tab
| complete 10 pieces, but at some point you have to tie them
| together to get a working program and those 10 pieces will
| have different assumptions and require some real
| understanding to integrate.
| sodality2 wrote:
| > Copilot is not free anymore most students won't bother
| paying for access
|
| Copilot is free for students:
| https://docs.github.com/en/billing/managing-billing-for-
| gith...
| niek_pas wrote:
| > Previously just doing the assignment was typically less
| work than cheating
|
| I don't think this is generally true, especially for
| algorithms and smaller programs: Googling 'fibonacci python'
| takes 3 seconds.
| omginternets wrote:
| This seems like a problem for educational institutions, not for
| industry. We're already inundated with mediocre programmers, many
| of whom have CS degrees, which is precisely why the industry
| looks at job experience more than credentials [0]. I don't see
| how this is going to further reduce the supply of competent
| candidates.
|
| What it _will_ do, however, is make it difficult for CS
| departments to evaluate students based on homework, and it 's not
| clear to me that this is such a bad thing. My experience with
| university CS courses has been that assignments are largely auto-
| graded, with systems akin to unit tests. In the common case, an
| overworked TA quickly spot-checks the source code to look for
| obvious signs of academic dishonesty, but that's it. So,
| universities are left with the following choice: give everyone an
| A, or grade differently. In the optimistic case, this might even
| produce a grading strategy that improves the value of a CS degree
| as a predictor of programming competence. It likely won't, in
| which case we have a _status quo_.
|
| [0] This is a problem in its own right. I recently referred a
| grad student as a job candidate in my current company. I'm the
| most senior backend engineer, and this grad student was a
| _highly_ competent contributor to my open-source project, which
| currently plays a strategic role in our backend. Said grad
| student is unusually bright and productive in a complex specialty
| (distributed systems), and I had to push our VP Eng very hard to
| hire him. His initial reaction was along the lines of "he
| doesn't have any industry experience, so we don't know if he's
| good". It gives me nausea to think about how many outstanding,
| 2-sigma engineers are rejected because companies don't know how
| to evaluate their talents. In the end, we ended up hiring him,
| but only because I stated that I would take full responsibility
| for his productivity. I regret nothing. </rant>
| marmada wrote:
| I think Copilot will make cheating easier. A benefit of this is
| that employers may look at side projects much more, which are
| harder to cheat
| [deleted]
| charcircuit wrote:
| Copilot is limited to short snippets. You can't just ask it to
| write a HTTP server. Also I don't see how having copilot generate
| the code for listening on a server socket is any better than when
| I copied how to do it out of a man page. Copilot doesn't add
| anything new, you've always been able to look up the
| documentation or look up how other people approach the problem.
|
| The fibonacci example doesn't make much sense either due to how
| trivial it is. Without copilot you are just copying from the
| definition of the function and with it copilot copies it for you.
| The other algorithms that follow are both online and probably in
| their textbook.
| johndough wrote:
| Copilot can almost write an HTTP server. Here is how I did it:
|
| 1. Start a new python file with the line "# This is an
| implementation of an HTTP server"
|
| 2. Press autocomplete a bunch of times and stop somewhere when
| "enough" libraries are imported
|
| 3. Press "#" and let it autocomplete the comment about what
| comes next (global variables for PORT and BUFFER_SIZE in my
| case)
|
| 4. Again, Press "#" and let it autocomplete a bunch of comments
| and functions ("get_file_extension", "get_file_content_type",
| "get_file_size" in my case)
|
| 5. At this step, I had to cheat a bit since it was generating
| too many "get_*" functions, so I started a comment with "#
| Initialize" and let it autocomplete again to get
| "init_server_socket"
|
| 6. From here on, it generated handle_request, parse_request,
| get_file_path and send_response automatically.
|
| 7. Lastly, I wrote "def main" and let it autocomplete again.
|
| This produced an HTTP server which almost worked. I just had to
| fix a small issue with "file_path", since it expected that
| files were stored in the root directory, but I wanted it to
| load files from the local directory.
|
| The code is not great since it does not handle most errors
| gracefully and is vulnerable to directory traversal, but I
| didn't even have to think about the HTTP protocol while writing
| it, so it is still quite impressive.
|
| A few good points:
|
| It generated a huge selection of MIME types which I probably
| would have had to look up by hand.
|
| The generated server is multi-threaded!
|
| It automatically reuses the socket, so I do not have to wait a
| minute every time I restart the server.
|
| Here is the final code: https://bpa.st/CTRA
| muzani wrote:
| This reminds me of when my school teacher insisted that real
| engineers didn't use calculators. They'd calculate their sines
| and cosines from a table.
|
| 3 years later, the education ministry decided that was BS and let
| everyone bring calculators to math exams. Surprisingly, math
| scores didn't go up much.
| eru wrote:
| We never did much calculation in math exams. That's what
| physics class was for.
| muzani wrote:
| It was 8th or 9th grade for me when the switch happened. We
| were still doing areas and perimeters.
|
| A calculator actually helped me understand physics a lot
| better, especially things like slopes.
| eru wrote:
| Makes sense!
| TakeBlaster16 wrote:
| I had to deal with this too. "You're not going to go through
| life with a calculator in your pocket!"
| Invictus0 wrote:
| I've seen people take this to the extreme and literally be
| unable to do basic fractions and percentages either in their
| head or on paper.
| lancebeet wrote:
| I wasn't allowed to have a calculator until I was around 10 or
| 12 years old. I didn't like it at the time but now I'm quite
| happy about it.
| Brian_K_White wrote:
| It sounds like copilot could ultimately be just another
| progression in the definition of normal or baseline, like using
| calculators in math classes.
|
| Whatever the current state of the art and tools of the trade are,
| that's what they are, and probably the bulk of courses will just
| adapt to reflect that state of reality and incorporate that into
| the courses.
|
| Outside of maybe some specialist courses that may still exist but
| most students wouldn't need to take, it would just be expected
| that most students use the current tools to accomplish the tasks.
| Even going so far as to provide free student access to any paid
| tools to avoid giving the rich kids an unfair advantage.
|
| The nature of posed problems, and their teaching purpose (why
| they are posed and what you get out of them) would just change
| from what they are now.
|
| Whatever the "calculator" can do, simply becomes uninteresting
| and not required for most people to worry about.
|
| My first couple years of school I was taking mechanical
| engineering, and it was right during a transition where we were
| doing both manual and cad drafting. There is a lot of geometry
| tricks both physical drawing and math required to generate
| accurate views of any shaped objects intersecting with any other
| shaped objects, viewed from any angle, sliced at any depth.
|
| We spent a lot of time and effort on that. That is all done by
| magic now inside the cad engine.
|
| Someone still needs to know it, but 99% of people whos job is to
| design or otherwise manipulate models of objects do not need to
| know it.
|
| I, who did learn that stuff, don't think an engineer who didn't
| have to know how to generate an accurate view with nothing but a
| pencil and a calculator is a less capable less insightful less
| quality engineer.
|
| If anything they may be better for not having to spend as much of
| their intellect on mechanics & implementation vs ideas. The
| knowledge & skills I'm talking about don't contribute to their
| understanding of the real job, they aren't foundations that the
| later stages build on. It's just labor of no value at all except
| to the cad engine designer. It had a value in the past because it
| was a form of literacy. It was simply the only way to document
| the ideas you were fabricating.
|
| Perhaps copilot is like that.
|
| I don't know and I'm not saying it is, but it could be.
| spcebar wrote:
| Cheating is getting easier by the day, but the bottom line is
| that if the student is cheating, they're probably not absorbing
| the coursework (or if they are cheating and absorbing the
| coursework, there might be something wrong with the coursework).
|
| The most effective policy might not be to try and mechanically
| prevent cheating, but to explain that, this is information that
| you're going to need if you actually want to go into the field
| and if you're using copilot here you are damaging yourself in the
| long run.
|
| Maybe it's not fair to give as good grades to students who are
| cheating on assignments as those who are actually doing the
| assignments, but at the end of the day is the purpose of the
| course to measure the student or to teach the student?
| ren_engineer wrote:
| yeah Copilot really changes nothing here, all these basic
| questions can be found with a quick Google search. The reason
| Copilot is so good at these types of questions is lazy
| professors assigning the same basic stuff so there is a ton of
| training data.
|
| instead of writing an article crying about it, the professor
| could try making some unique questions to test knowledge of the
| underlying concepts
| duxup wrote:
| At that point should we count grades at all, other than a basic
| evaluation for the student's personal use?
| chiefalchemist wrote:
| True story. I got my CS degree pre-internet. I did well with
| programming. Some of my friends / peers were less talented. In
| one of my 300 level courses the assignment was more difficult
| than usual. I naturally figued it out. They struggled.
|
| Evetually I gave a copy of my code / solution to Friends Group
| A and also to another Group B. They didn't know each other. I
| said, "Be careful! If you copy, disguise it."* Deadline comes.
| Everyone hands in their work.
|
| The following week, we go to lecture. Prof walks in and writes
| a list of names on the board. I knew each name. I knew what was
| happening. I waited for the shoe to drop (i.e., my name on the
| list). The shoe never dropped**
|
| I believe they were all given Ds. Not sure why they didn't fail
| (F).
|
| * In retrospect, this was a stupid on my part. If they knew how
| to alter enough it's likely they'd be able to write it
| themselves. That is, I all but suggested they walk on water.
|
| ** Also in retrospect, the TAs + prof had to realize I was the
| source. The class was big but not that big. And if only some
| completed the assignment correctly, the source had to be not
| difficult to identify. I'm not sure why I was never pulled
| aside and spoken to. Thank gawd.
| chrismarlow9 wrote:
| If someone is just looking to fill a course requirement and has
| no interest in being a programmer, my response is "meh,
| whatever".
|
| If someone is cheating and actually expects to land a decent CS
| job, my response is "good luck on the leetcode questions and
| the 5-7 hours of questions you'll have to answer between the
| intro call, panels, and an offer".
|
| And if someone magically manages to get all that way without
| actually learning the material they will get absolutely crushed
| at work. Meetings where you have to give your professional
| opinion will induce fits of anxiety. Making changes to your
| code based on feedback and not having any idea what the
| feedback means. Even just the amount of work alone that needs
| to be done at most tech places crushes great engineers. Without
| the knowledge it will just stack up on you even further.
| oxfordmale wrote:
| You underestimate the human ability to get away with
| incompetence. Unfortunately many of those cheaters manage to
| get in roles where they can hide their inadequacies, either
| by slacking of their colleagues that do actual work or by
| sucking up to the right managers.
| chrismarlow9 wrote:
| Oh it definitely happens, I'm not saying it doesn't. But in
| those examples you mention I would say there are more
| people who are bad at their job than just the IC who "faked
| it".
| GrumpyYoungMan wrote:
| > " _...this is information that you 're going to need if you
| actually want to go into the field..._"
|
| That has never, ever stopped anyone motivated to cheat. Even
| here on highly-educated HN, you get posters with open disdain
| for what they learned in college.
|
| > " _... at the end of the day is the purpose of the course to
| measure the student or to teach the student?_ "
|
| Both. Otherwise, the college diploma really does become the
| meaningless scrap of sheepskin that its detractors claim it is.
| tux1968 wrote:
| > is the purpose of the course to measure the student or to
| teach the student?
|
| The very basis of the scientific method is measurement. If you
| can not measure your learning progress, how are you to know if
| your training methods are adequate or failing?
|
| I really don't understand this idea that testing and measuring
| students is a problem. Even the dreaded "teaching to the test",
| sounds like a good idea. If doing so would somehow exclude
| important learning, that just identifies an area where the
| testing needs to be improved.
| croes wrote:
| At the end if the it is to get a job. And cheating students
| often get better grades as most other students who don't cheat.
|
| They may lose in the long run, but the non cheaters aren't even
| allowed to run
| zeta0134 wrote:
| There's more to this: it used to be that a college degree
| carried with it something of a soft guarantee that the student
| could achieve a baseline level of work on their own. This acted
| as a useful filter for employers. If cheating runs rampant that
| filter becomes meaningless, and once the student has
| successfully cheated their way into the work force, their
| incompetence can (depending on field) do very real damage to
| the lives of others.
|
| Because of this, there will always be considerable pressure to
| detect cheating and remove offending parties from the program,
| be that the students copying answers from the internet, or the
| institutions failing to detect the problem before handing out a
| degree.
| buscoquadnary wrote:
| A college degree used to signal competence. Now it signals
| you have the ability to show up and avoid doing something to
| get kicked out for 4 years.
|
| I feel we'd probably get the same value to society if we just
| made grades 9-12 optional. It would remove the people that
| don't want to be there, improve the quality of instruction,
| and college would actually be a meaningful achievement so we
| no longer have requirements for entry level jobs to require a
| Master's degree.
| sarchertech wrote:
| In many states school is already optional after age 16.
| busyant wrote:
| > The most effective policy might not be to try and
| mechanically prevent cheating, but to explain that, this is
| information that you're going to need if you actually want to
| go into the field and if you're using copilot here you are
| damaging yourself in the long run.
|
| I don't teach CS, but I teach an online course in a STEM field.
| I give out a similar message ("you're only hurting yourself in
| the long run").
|
| My anecdotal evidence is that my warning does nothing--I have
| trap questions on various quizzes/exams and the frequency of
| cheating hasn't changed with or without the warning.
|
| My _suspicion_ is that the students who cheat feel this way: _"
| This is just some bullshit hoop that I have to jump through, so
| it's okay if I cheat. I'll figure out the important stuff when
| the time comes."_
|
| I mean, they're delusional, but I understand the mindset for
| cheating.
|
| edit: amusing story.
|
| Faculty members in my department are required to take an on-
| line course on how to handle hazardous waste. There are a few
| hours worth of videos to watch and a test that you need to pass
| at the end.
|
| Last year, one of the faculty members took the test, compiled
| the answers and emailed them around to everyone else (to save
| them the time for this "bullshit task.").
|
| I said to him, "Isn't this precisely the kind of shit that
| makes every faculty member angry when the students do it?"
|
| That being said, it was a bullshit test.
| randomdata wrote:
| _> I mean, they 're delusional, but I understand the mindset
| for cheating._
|
| Parental (other similar) pressure can indeed be quite strong.
| Absent of that pressure they wouldn't be there in the first
| place, so perhaps it is not so much being delusional as a
| rational response to their environment? If they cheat their
| way through the parents are happy and then they can return to
| the life they otherwise would have lived. The time lost is
| unfortunate in some respects, but at the same time if you're
| cheating, the time investment likely isn't that great and an
| acceptable cost given the circumstances.
|
| _> the kind of shit that makes every faculty member angry_
|
| I find it curious that the customer using the product in an
| unintended way would be upsetting to the vendor. What drives
| such emotions? In my businesses, I couldn't care less how the
| customer uses my product. If they're happy, I'm happy.
| bo1024 wrote:
| Famous professor quote/reply about this: "in teaching,
| students are not the customer. They are the product."
| randomdata wrote:
| That certainly would be true within the public education
| (primary, secondary) system, where the government is the
| customer. Hence why attendance is mandated. However,
| typically college level students are initiating and
| fulfilling the transaction, thus they are the customer.
| They offer up money in exchange for keeping arbitrary
| social pressures at bay. In rare cases they offer money
| in exchange for learning things.
|
| It is possible for the customer to also be the product.
| Especially in the age of salable data, that is becoming
| more and more common. However, when that is the case
| there is incentives given to the customers to shape them
| into what selling them as a product requires. In context
| here, that would mean something like giving discounts to
| those who don't cheat, which I am not familiar with any
| college doing, so... That brings us back to why would a
| vendor get emotional about the customer not using the
| product as intended?
| pclmulqdq wrote:
| Some courses really are required bullshit to some students. A
| lot of professors don't realize this, but if you teach a
| required course, it is simple math to say that _some_ of your
| students just don 't need to learn what you are teaching.
| Sure, a lot of students _do_ need it but don 't _think_ they
| do, but a good fraction of the class does not need to learn
| the content you are teaching.
|
| The only remaining question is how to engage these people. I
| have seen good approaches to that problem and really, really
| terrible approaches, and it seems that most professors go for
| the terrible approaches. Unfortunately, engaging people who
| don't want to be there is a lot harder than just having a
| bunch of required tests.
|
| These students are not all wrong that some courses are BS
| hoops to jump through. Meet them in the middle. Teach them
| something.
| busyant wrote:
| I know a lot of what I teach is not really relevant to
| students who aren't chemistry majors--I even say this UP
| FRONT at the beginning of the course. I tell the students
| that they should think of the course as a way of assessing
| whether they can think logically and critically about
| "weird things." Because life is full of weird things that
| need logical and critical analysis.
|
| In general, they appreciate my candor.
|
| That being said, in my experience, the students who cheat
| often delude themselves into thinking that they understand
| the material--and they really don't.*
|
| Then they seem shocked when they take a certification exam
| (where they have to leave phones at the door and are
| closely monitored) and they bomb the exam.*
|
| * I'm speaking in generalities here. I'm sure there are
| some cheaters who are being "smart" about their cheating.
| pclmulqdq wrote:
| In the time I was at school, I heard of a lot more
| cheating than would be justified solely by people
| conserving time. I have recently seen studies that argue
| that 50-80% of students cheat at some point, and
| something like 10-20% cheat in every course they take.
| Personally, when I conserved time, I just accepted the
| bad grades - it was easier than trying to cheat and not
| get caught. The OP was talking about students cheating
| because they are not engaged with the material. I think
| that's only part of the problem with cheating: there are
| lots of other factors, like pressure to get a high GPA.
| Kamq wrote:
| > this is information that you're going to need if you actually
| want to go into the field and if you're using copilot here you
| are damaging yourself in the long run
|
| When you say into the field, do you mean academia or industry?
|
| Because the things copilot does for you are absolutely not the
| things you need to know yourself to be in industry. They're,
| for the most part, things we tried to put in libraries (or more
| ideally language standard libraries for a lot of things).
|
| The fact that is solves a lot of interview questions just means
| our interview process was absolutely garbage.
|
| And I'm skeptical of the academic side of this as well. It
| sounds like professors from the early 60s being annoyed that
| students have compilers (another tool that saves a ton of work
| and repetition). Y'all are forgetting that this is the ultimate
| lazy man's field. This isn't the first time the basics have
| been swept away and replaced with something easier to work with
| (and hopefully won't be the last).
| mannykannot wrote:
| > They're, for the most part, things we tried to put in
| libraries.
|
| We need some people who can do things like making those
| libraries. It also seems plausible that the people who have
| at least some of the knowledge and judgement to do that
| effectively will, on average, be more effective on more
| mundane tasks as well.
|
| Larry Wall is one who wrote (somewhat drolly) about the
| virtues of laziness, but there was nothing lazy about what he
| did.
| Kamq wrote:
| > We need some people who can do things like making those
| libraries.
|
| Do we? For things as simple as copilot tends to put out?
| Why?
|
| Do you also believe we need to keep people around that do
| other automated things? Plowing fields by hand? Hand
| compiling higher level languages (as the first lisp
| compiler was bootstrapped)?
|
| I mean, keep the information around. Don't go burning
| textbooks on subjects just because we automated something.
| But, what exactly is the value proposition of having
| students do these things?
|
| > It also seems plausible that the people who have at least
| some of the knowledge and judgement to do that effectively
| will, on average, be more effective on more mundane tasks
| as well.
|
| And your claim is also that this is the only way to get the
| requisite knowledge and judgement? Have students take in a
| string from stdin with a format that changes every
| semester, munge it around, and do things with it instead.
|
| > Larry Wall is one who wrote (somewhat drolly) about the
| virtues of laziness, but there was nothing lazy about what
| he did.
|
| As someone who has done string munging in C, I'm not
| entirely convinced that creating perl isn't an effort
| saving defense mechanism (only half joking).
| mannykannot wrote:
| I find this a very puzzling reply, and it may be that I
| misunderstood to what you are referring with the
| "they're" in "They're, for the most part, things we tried
| to put in libraries (or more ideally language standard
| libraries for a lot of things.)" It might refer to "the
| things Copilot does for you" or alternatively "the things
| you need to know yourself to be in industry."
|
| The thing is, regardless of which way you meant it, we
| need some people who can make the sort of libraries we
| need in part precisely because automation such as Copilot
| is no substitute (at least not yet.)
|
| This observation does not (and is not intended to)
| endorse current methods of instruction or hiring; on the
| contrary, it supports spcebar's view that riding your way
| to a degree, certificate or entry-level position on the
| back of Copilot is not doing yourself any favors.
|
| The point about Larry Wall is that we don't get labor-
| saving tools without someone making an effort.
| Kamq wrote:
| > I find this a very puzzling reply, and it may be that I
| misunderstood to what you are referring with the
| "they're" in "They're, for the most part, things we tried
| to put in libraries (or more ideally language standard
| libraries for a lot of things.)" It might refer to "the
| things Copilot does for you" or alternatively "the things
| you need to know yourself to be in industry."
|
| I do see how that could be ambiguous. That's on me. I was
| referring to "the things Copilot does for you". Generally
| speaking trivial (or near trivial) algorithms.
|
| > The thing is, regardless of which way you meant it, we
| need some people who can make the sort of libraries we
| need in part precisely because automation such as Copilot
| is no substitute
|
| Fine. Some people might need to be able to implement and
| maintain libraries filled with generic algorithms,
| especially language maintainers. That's still a very
| different claim from the original "this is information
| that you're going to need if you actually want to go into
| the field". That claim implies that it's a universal
| requirement, whereas the reality is the vast majority
| probably don't need that knowledge.
|
| Implementing these things is tedious, and results in a
| bunch of code that has to be maintained vs using
| something out of the standard library. For example, sort
| or max functions, which are at the intersection of what I
| see copilot generate a lot and intro to CS classes. Even
| without copilot, that's not really a skill that the
| average practitioner needs to have ready at all times, in
| fact I'd probably block a PR that implemented either of
| those things in my projects, it results in extra code
| that needs to be maintained and can be broken by a typo
| or something silly.
|
| > The point about Larry Wall is that we don't get labor-
| saving tools without someone making an effort.
|
| My comment about perl was sarcastic, and probably not
| helpful. That, also, is my bad.
| [deleted]
| russfink wrote:
| Adjunct instructor here. Before I came into the program,
| cheating was on the rise. When given programming assignments,
| students would go to Geeks for geeks and download solutions,
| some of them mostly correct, and submit those for credit. One
| remedy was to try to make programming assignments harder and
| harder. Eventually, they got so hard that we were asking people
| new to computer science to implement a full arbitrary length
| integer calculator using nothing but a single tape Turing
| machine (in Java, not BF).
|
| Eventually, some of us came to the realization that you can
| never prove that somebody is cheating. People have been known
| to hire tutors to do their assignments for them. There is just
| no end.
|
| As a result, we evaluate students on four dimensions.
| Programming assignments,* homework assignments, but also class
| discussions and group participation. Those last two count for a
| small, but non-trivial percentage of the grade, and are usually
| enough to separate and identify those who understand what they
| are doing, from those who are just "following along" solutions
| that they find on the internet.
|
| * one addition to programming assignments includes an analysis
| write up. Tell me in human words what is happening. Why do you
| see that effect. What is the running time. And separately,
| comment your code to tell me how it works. Those two parts
| count significantly towards the grade.
|
| Any other professors here, what have you found that works?
| scott00 wrote:
| Not a professor, but my undergrad classes were generally
| graded as 10-15% homework and 85-90% exams.
|
| With grading like that cheating on the homework is basically
| pointless, so few did it.
| emeryberger wrote:
| I think grading on multiple dimensions makes sense, though
| scaling is a challenge.
|
| As for analysis write-up: turns out, language models can also
| explain what the code does really well, in convincing
| vernacular. They generate reasonable comments.
|
| https://twitter.com/emeryberger/status/1560618825208320001?s.
| ..
|
| As an illustrative example, I just typed in a little test
| into Copilot. I only typed the first line and the text that
| starts with "a paragraph explaining...", and typed a "#" for
| each comment. Copilot filled in the rest. def
| compute_sum_recursively(n): """A paragraph
| explaining what this function does and why, and explaining
| its running time: This function computes the sum of
| the first n natural numbers. The running time of
| this function is O(n). """ # Base case: n
| = 0 if n == 0: # The sum of the first
| 0 natural numbers is 0 return 0 else:
| # Recursive case: n > 0 # The sum of the first
| n natural numbers is n + the sum of the first n-1 natural
| numbers return n + compute_sum_recursively(n-1)
| sireat wrote:
| I teach data structures and algorithms among other subjects.
|
| My assignments are similar to everyone else's (CLRS, JeffE,
| Leetcode) yet different enough that simply copy and pasting
| will not work.
|
| All I ask is that they comment their code, analyze complexity
| and cite sources.
|
| I've been using copilot for the last year in my live coding
| sessions. It saves so much time (especially in comments). If
| students want to use Copilot I do not see a problem. It is
| just another tool.
|
| Copilot is near useless without decent programming knowledge.
| 80% of time it is fantastic. About 10% of time Copilot gives
| you subpar snippets. O(n2) instead of O(n) etc. Then 10% of
| time it just gives you wrong snippets.
|
| Just like GPT-3, once you go beyond a paragraph it loses
| context.
| leeoniya wrote:
| > one addition to programming assignments includes an
| analysis write up.
|
| when i was interviewing candidates this was the fastest way
| to filter those who understood and those who didnt.
|
| ask someone to talk you through how to solve some real-life
| problem. to look at a stack trace and describe what they see,
| or to run a profile and trace through the cause of a
| hotspot/contention. show them an issue and have them live
| debug to root-cause and fix. it's okay if they use the
| internet, SO, etc. -- that's how we all do it. see what they
| have to look up! just listening to the amount of depth
| someone can verbally communicate during novel problem solving
| (including additional questions they ask you) turns a 60min
| interview that wastes time into a 10min one that tells you if
| you can move on.
|
| a favorite one of mine was to ask the candidate to describe
| in as much detail as possible what happens between a
| keystroke typed into a google search box and the search
| results appearing. the diversity of replies is facinating.
| some will tell you "google returns results from its
| database", others will ask you if you want them to first
| describe how the keyboard works at an electrical level
| through the USB or bluetooth driver stack.
|
| "Never memorize something that you can look up." --someone
| smart
| matheusmoreira wrote:
| That's a great way to evaluate someone! Individualized and
| personal, uses concrete problems, no artificial
| restrictions.
|
| It's a shame our mass education systems cannot use it. They
| simply cannot apply such a humane method to hundreds or
| thousands of students. They are reduced to applying their
| bullshit test questions because it's the best they can come
| up with at their scale.
| nkingsy wrote:
| Ugh I had someone ask me to debug a verbally communicated
| error message in an interview.
|
| Like they picked some random port configuration issue that
| had stumped them for days in the past and thought cool
| let's remove the internet, the command line, and ask people
| to solve it on the spot.
|
| I offered plenty of plans for how I would go about
| debugging, but I didn't know the one simple trick.
|
| I agree with trying to use challenges that are closer to
| the real work, but it's really hard to do that without
| over-testing domain knowledge.
|
| For instance, I ask candidates to do some asynchronous
| control flow. These are all candidates with js listed as
| their best language and I offer to let them look things up
| or to show them the promise apis they need, but a certain
| percent just refuse to engage with the problem because they
| don't have the domain knowledge and seem to feel they're
| being tested unfairly.
|
| The problem does a really good job of showing a candidates
| grasp of all the tricky parts of js, so I keep using it
| despite the drawbacks.
| leeoniya wrote:
| > Ugh I had someone ask me to debug a verbally
| communicated error message in an interview.
|
| i dont mean that the problem is only verbally
| communicated. i mean that the debugging process the
| candidate does is verbally communicated.
|
| in the scenario you're describing you'd be sitting at
| that machine with access to the internet on another
| machine. like, you know, in real life.
|
| > The problem does a really good job of showing a
| candidates grasp of all the tricky parts of js, so I keep
| using it despite the drawbacks.
|
| async stuff in js is pretty good, but it's also easy to
| go too deep on it with some bizzarely poorly architected
| code. many js devs still fail to grasp all the
| implications of closures, or how to avoid memory leaks,
| or how to work with the GC rather than against it.
| [deleted]
| mdaniel wrote:
| > but also class discussions and group participation
|
| I really, really hated those parts of school, second only to
| group projects. I do appreciate why teachers want that, and I
| hear you about how it can surface those who are cheating, but
| I do wonder if it's causing the class to suffer for the
| alleged misdeeds of a few
| ethbr0 wrote:
| I disliked them in school, for the usual reasons, but found
| them the most useful exercises for developing career
| skills.
|
| No person an island.
|
| And if they are, I don't want to be on a work team with
| them.
| jahewson wrote:
| Is doing other people's homework for them a career skill?
| qsort wrote:
| School assignments are so different from the dynamics of
| a real workplace that I find this point very moot. Always
| hated them with a passion, but otoh I always enjoyed team
| math and programming competitions, situations where you
| have to do stuff that's actually hard, work with people
| you actually respect and _trust_ them to do their job.
| NikolaNovak wrote:
| That's the thing.
|
| For me at least, the dynamics of "School Group Project"
| are basically completely different and separate from
| "Work team". I happen to do well in both, but I do not
| enjoy both: the motivations and structure and dynamics
| and goals and timelines of "School Group Project" are so
| much more artificial and ultimately pointless, and
| obviously to all from the start.
|
| Same things with discussions - I happen to be an engaged
| student, usually front row, hand always up, discussing
| with instructor and team and colleagues and everybody. I
| like to be engaged and figure things out together. But
| I'll never be half the developer than my colleague who
| barely speaks a word unless asked. He's friendly, meek,
| polite, and excellent team player and developer - he just
| does not initiate conversations, _especially_ in group
| settings. I can imagine he 'd have a nice big 0 in that
| category if that was a grade criteria.
|
| Which is not to criticize the professors who try to use
| group discussions; just to point out, this is not a
| solved problem at scale any more than
| interviewing/hiring. There's too much humanity and too
| few absolutes :)
| clusterhacks wrote:
| Hi, sometime adjunct here.
|
| Cheating on programming assignments has been rampant forever
| at every undergrad institution I have experience with and
| somewhat present among graduate students. My experience spans
| about 25 years in that space now.
|
| When I taught an introductory class, I gave open-book exams
| with no laptops or phones allowed. About 1/3 of the class was
| unable to write a syntactically correct for loop in Python
| despite our textbook being an introductory Python-based book
| chock full of examples. It was pretty clear that a subset of
| students were either working together on project assignments
| or out-and-out having someone else do the assignments for
| them. I mainly compensated for this by having a large part of
| the grade being a 1-1 meeting with me to talk me through the
| code. That and the exam had the effect of actually making
| cheating somewhat less worthwhile. But this approach simply
| doesn't scale these days - my max class was around 32-33
| students and the time I spent meeting with students was
| _insane_. I haven 't taught in a few years and understand
| that class sizes of 150+ are not uncommon. I could have never
| used the same approach with that many students. I probably
| would have doubled-down on exams and made exam length darn
| near impossible to finish without actually knowing the
| material well enough to do without referring to the textbook.
|
| I was teaching more from a practice-based viewpoint so mostly
| I came up with "weird" projects that mirrored problems I
| spend time on (data cleaning, using existing libraries to do
| neat little things, and also having students pick a personal
| project to implement that I helped them scope appropriately).
|
| We bitch and moan about interview whiteboarding but given
| grade inflation its kind of hard to trust university
| credentials. Grade inflation was kind of disheartening in
| that the worst students didn't really get a poor grade. But I
| also didn't have any problems with the top 1/3 of the class
| getting a very high grade - these students were motivated,
| understood the material, and often impressed me with where
| they got in a single semester.
| SoftTalker wrote:
| I recently worked at a large state university. I remember a
| conversation with an instructor in a master-level operating
| systems course. One of the assignments was to implement a
| simple filesystem. The amount of cheating was insane. He
| started academic dishonesty proceedings against many of the
| students but the department pressured him to "work it out."
|
| He had to basically interview each student individually and
| ask for an explanation of the code. Most could not explain
| what their code was doing (because they didn't write it).
|
| They were all given an opportunity to resubmit original
| work. Many could not do it, because they simply didn't
| understand the basics of programming, or the language they
| were using, or how computers even worked.
|
| These were Masters students.
| clusterhacks wrote:
| That is just awful - I have similar stories. To me, it
| seems so unfortunate that students are just in this
| desperate grind to get all A's.
|
| But I'm on the other side and don't have the stress of
| "my whole life depends on looking like an all-star" at
| university. Like, what do you do if you leave school with
| such a severe lack of actual skill?
|
| OTOH, the last year I taught, _every_ student in the
| program I was an adjunct for graduated with a job offer
| in hand. Maybe it all works out fine.
| sarchertech wrote:
| The majority of CS Masters students in the US don't have
| a BS in CS, or have one from a non top tier overseas
| university. Even more so than undergrad, Masters students
| in CS are there for the credentials, so I would expect
| cheating to be worse.
| [deleted]
| philipswood wrote:
| When playing with GPT-3, I've found it to be impressively
| good at explaining what code does in English...
| drran wrote:
| When teachers require recording of timelapses, we will have
| AI which will be able to generate text and a video of a
| user typing it. IMHO, they should start to require
| timelapses right now.
| NoraCodes wrote:
| That requires either trusting that students haven't
| edited out the evidence of cheating, which is pointless,
| or forcing them to use specific, proprietary software to
| record the video, which is unacceptable.
| aaomidi wrote:
| I do want to say that, those criteria exclude neurodivergent
| folk (who for many reasons may have never even been
| diagnosed).
|
| Honestly I think moving away from grades is going to be the
| way forward. Part of this is also making university, etc
| cheaper so you're not financially incentivized to not take a
| class one, or multiple more times.
| DiggyJohnson wrote:
| Do you also believe that we should move away from
| evaluation entirely? Genuinely asking.
| matheusmoreira wrote:
| Evaluation is vital but the current model used in
| education today is not evaluation, it is punishment.
| Failed to get the answers right? You are punished. You
| lose points. Your GPA plummets. You could even fail the
| class and have to take it again which means the
| punishment is not only social but economical.
|
| There's real life consequences to this kind of
| evaluation. Huge consequences. Students cannot afford to
| make mistakes. There's huge pressure and anxiety before
| and during a test because the stakes are so high. Failing
| at this stuff can cost someone their future: future jobs,
| future career, even the student loans that enable them to
| study in the first place may only be afforded to students
| who get good grades.
|
| Is it any wonder people cheat?
| johndough wrote:
| If you define negative evaluations as "punishment", then
| evaluation without punishment is useless.
| aaomidi wrote:
| I think during education yes, during execution of the
| skills learned in education - not really but also
| depends.
|
| A lot of people hate how tech interviews are done, right?
| Cause they have nothing to do with the job the majority
| of the time. Those are places where I can see
| improvements.
|
| However, some jobs do genuinely need you to interact with
| a diverse set of people all the time, for those you need
| to make sure for the sake of your company & the candidate
| that they match the criteria. Other jobs, you need only
| maintain strong relations with small groups, and the
| requirements for those are a lot easier to hit - even the
| most introverted people can do well in smaller groups
| that don't change a lot.
|
| I do really think a lot of evaluations that happen during
| education though makes a confrontational relationship
| with learning. You're constantly judged about your
| _ability to learn_and that has far reaching impacts
| beyond your first twoish decades of your life. You get
| scared of making mistakes, you get scared of taking
| risks, etc.
|
| So, evaluate when it's important for the function &
| safety of the job, and the candidate. Skip evaluation
| when its evaluation for the sake of evaluation.
| matheusmoreira wrote:
| > Part of this is also making university, etc cheaper so
| you're not financially incentivized to not take a class
| one, or multiple more times.
|
| I'd rather people had better opportunities in life that
| didn't depend on education. There's a lot of people who
| hate school and are just going through the motions because
| they believe it's the only path to success in life.
| nyanpasu64 wrote:
| Mass-produced education is like a train, once you fall off
| you'll get hurt trying to get back on. I've come to the
| realization that given the inhumane pace, difficult and
| poorly-taught curriculum, and ambiguous/out-of-scope
| homework assignments in a top-tier university's CS program,
| the majority of my university classes were either not worth
| taking, or I'd be better off self-studying if I found an
| intrinsically rewarding project (definitely less panic
| attacks and psychological trauma, sometimes even similar or
| deeper learning). Perhaps I'd have a better time in a less
| prestigious university that doesn't see its purpose as
| weeding out and breaking lesser students.
| jstanley wrote:
| > One remedy was to try to make programming assignments
| harder and harder.
|
| How is that going to stop cheating? Surely that will just
| make people _more_ inclined to cheat, since they have less
| chance of solving it on their own.
| gfaster wrote:
| It would prevent people from simply going and downloading
| solutions from the internet. A lot of easier assignments
| are fairly well treaded and already have solutions floating
| around.
| yummybear wrote:
| Doing algorithms and functions are the least of my worries as a
| web developer. I need a copilot that helps me determine which
| kind of ui elements i need to help with task. I need it to
| automate standard issues like make login functionality invludkng
| two factor or to determine data structures for a novel problem.
| That'd be great.
| criloz2 wrote:
| I have been using copilot for two months and I find it more
| annoying than useful, am the only one that feel like this?, I
| also used tabnine, and it gives a lot of syntax errors in rust,
| so I could barely use it for a week. I definitely can agree that
| copilot is better than tabnine, but I feel like writing code is
| the easy part of programming, and copilot doesn't offer a crazy
| help to justify the monthly payment and that my private code is
| uploaded to Microsoft servers. Not only that, but I find that my
| IDE suggestion are good enough most of the time.
| thefourthchime wrote:
| I use it with python and vscode, it's great there. Sure, half
| the stuff it makes isn't correct, or nearly correct, but it's
| very close, and I just need to change a little here or there.
|
| It also does a fantastic job of filling in method parameters
| from my local variables, again not 100% of the time, but 90% of
| the time it nails it.
| stew-j wrote:
| I'm not sure how it would be done, there might not even be
| enough existing code to train it on, but I would like to try
| out Copilot for code-based CAD models in something like
| OpenSCAD. I've never tried it at all, but thanks for sharing
| your experience, all.
| [deleted]
| nprateem wrote:
| Yeah I was going to post this. I did some coding the other day
| and it kept spitting out code that looked right but was calling
| functions that didn't exist. It also competes with IntelliJ's
| auto-complete making things doubly irritating.
|
| I leave it disabled more often than it's enabled, unless I'm
| doing something simple like wanting to copy a file, which in Go
| is stupidly obtuse.
| coding123 wrote:
| No big deal. If students have access to this, then instead of
| trying to teach them and grade them on fibonacci - let the next
| generation have larger assignments. Like building a 3d rendered
| first person shooter (a basic one with no AI)
| ___j___ wrote:
| bb
| analog31 wrote:
| I wonder if copilot could be used against itself, i.e., to find
| problems that it is absolutely guaranteed to get wrong.
| i_am_toaster wrote:
| "Coping with google... students can just search for the answers
| by typing in 'depth first search code python'. Buckle up and
| hunker down for one slippery slope of an article!"
| stew-j wrote:
| Guess how I learned query/replace in Emacs in 1985?
|
| My friend who copied my Pascal programming assignment is now a
| director at a $1B hedge fund. I'm the founder of a series of
| scrappy, ramen noodle startups. I guess it's the "Senator
| Blutarsky" effect:
|
| https://en.wikipedia.org/wiki/John_%22Bluto%22_Blutarsky
| SoftTalker wrote:
| The skill of getting others to do work for you is highly valued
| in the Director and C-suites.
| Spivak wrote:
| I love their take at the end which is to not expand the already
| crazy anti-cheating apparatus and just give students more
| interesting problems and let them use all the information at
| their disposal. I wish more professors took this attitude. Having
| every student implement the exact same famous algorithms is
| really silly when you take a step back. You don't gain any
| appreciation for them because they're unmotivated and you lack
| all the foundational knowledge. The proofs/explanations are all
| totally bespoke and don't generalize.
|
| You really win as a professor when you design a project that
| naturally causes you to reach for the knowledge you're trying to
| teach. One of the best was my networking class where the project
| had us secretly rediscover tcp by trying to invent reliable
| transmission over UDP and then once we sufficiently cut our teeth
| the second half of the course was implementing a real working
| version on top of the professor provided userspace ip stack.
| withinboredom wrote:
| Here's the thing. When you learn to be a carpenter, they teach
| you how and when to use certain tools. The same can be for any
| profession. This is a new tool. They shouldn't be telling
| students not to use it, but when to use it and what it is suited
| for.
| bfung wrote:
| On tests, make students write short programs with pen and paper.
|
| Good practice for white boarding interviews.
| chiefalchemist wrote:
| The good students will take the time to learn and understand
| (even if they didn't write the code). Other students, as they
| _always_ have, will look for short cuts.
|
| Not everyone who take CS1* courses are CS (or similar eng)
| majors. We keep crying "Everyone should learn to code" and now
| that's made easier than ever some of continue to complain. Huh?
|
| Full disclosure: I started to use CoPilot a couple of months ago.
| It does not remove me from the process. It does allow me to shift
| more brain cycles from the mundane to the more difficult. My
| general feeling is, I have yet to get close to taking full
| advantage of what it can offer me.
| dvh1990 wrote:
| It is not the lecturers job to police their students. People that
| cheat only end up cheating themselves, and nobody but the cheater
| should do anything about it.
| refulgentis wrote:
| I love the choice of an AI-generated photo for the header, and
| the hidden message of its poor quality if you look at it longer
| than a glance.
|
| There's an emerging divide between those with experience with AI
| tools and people who tried them a few times and got hyped and
| blogged a maximalist view of consequences.
| brimstedt wrote:
| I don't think using copilot is much worse than autocomplete.
|
| It's an improvement of our toolbox, students SHOULD learn to use
| it.
|
| It's not much worse than using c instead of assembler, or java
| instead of c, or ...
|
| Throughout time people have invented new tools that makes their
| predecessors more efficient in their proffesion.
|
| Too many programmers these days behaves like the workers did when
| automatic looms were invented. Or when the chainsaw was
| invented.. or the computers.. or the...
| tgv wrote:
| > Throughout time people have invented new tools that makes
| their predecessors more efficient in their proffesion.
|
| And yet, here we are, learning reading and writing like it's
| 500BC.
| hbn wrote:
| I don't know if the idea of future generations of programmers
| squeezing by on having a robot make best guesses to get their
| work done without understanding it makes me feel confident in
| terms of job security, or fills me with dread knowing that I'll
| surely be relying on the software they produce to access my bank,
| have my cars controls hijacked because it thinks it knows better
| than me, keep my life support running when I'm old, etc.
| Aeolun wrote:
| This is no different from finding the solutions on wikipedia, or
| any of a thousand other websites. People that want to cheat will
| do so, and they might even be correct in a completely selfish
| sense.
| driverdan wrote:
| The solution doesn't seem that hard to me. Put less weight on
| homework grades and more weight on projects and tests. If they
| cheat the homework they'll bomb on tasks that can't be solved
| with Copilot.
| caust1c wrote:
| Maybe AI will know what to do with our burning planet...
| [deleted]
| joak wrote:
| Grading students is probably an archaic way of teaching. You want
| them engaged not cheating. But ok, institutions want grades...
| What about this solution?:
|
| https://theconversation.com/i-no-longer-grade-my-students-wo...
| joak wrote:
| Previously discussed here:
| https://news.ycombinator.com/item?id=30842587
| cmrdporcupine wrote:
| I understand the problem here and I sympathize with professors in
| these circumstances. Learning the fundamentals will remain
| important. But as others have pointed out: if students are
| unwilling to learn, or are taking shortcuts, it will ultimately
| hurt them in the long run.
|
| One thing tho is that things like Copilot put the lie to the
| hypothesis (propagated mostly by the Google-style job interview)
| that intensely coding clever for-loops to perform algorithmic
| magic (for things usually already in the standard library) is the
| best measure of the competence of a software developer. If it can
| literally be done _by a machine_ , maybe we should be measuring
| based on something else. Especially since this kind of thing is
| best done on the job _by looking in a good textbook_ or _using
| the standard library_. Probably grading at the university level
| needs to consider this also.
|
| I haven't used Copilot. I doubt I'm its intended audience. After
| 30 years, actually writing _code_ is perhaps the easiest part of
| my job. The mechanics is the easy part. The big picture thinking
| and figuring how to get it all together into a _system_ is the
| hard part (and honestly there are plenty of people far better at
| it than I am). Now, if we get a Copilot for _that_ ... then our
| profession is in trouble.
| greymalik wrote:
| > the hypothesis (propagated mostly by the Google-style job
| interview) that intensely coding clever for-loops to perform
| algorithmic magic (for things usually already in the standard
| library) is the best measure of the competence of a software
| developer
|
| I think the hypothesis is, rather, that people who can
| successfully run that gauntlet are smart and motivated. The
| details of the problems they're solving along the way are
| almost incidental.
| thefourthchime wrote:
| I've been writing code for nearly as long and man, you have to
| try it. It's more like autocomplete on roids than something
| that generates Fibonacci methods automagically.
| ALittleLight wrote:
| I've used Copilot for a little while. I'm away from my
| computer at the moment but I'm very confident it would
| generate Fibonacci methods.
| glacials wrote:
| I don't think they mean that it can't, just that the better
| way to think about the advantages it gives an experienced
| engineer is more along the lines of "autocomplete v2", i.e.
| a keystroke-saver.
| andrepd wrote:
| Is it a keystroke saver if you need to review every line
| it proposes to you?
| throwaway675309 wrote:
| Yes because often times while I'm unable to recall the
| exact idiosyncratic keywords incantation that I need,
| copilot will retrieve them automatically. This saves me a
| context switch to MSDN docs or stack overflow.
|
| It automates a great deal of boilerplate crap that you
| have to do especially in web frameworks such as angular.
| matthewmacleod wrote:
| Yes. For my part, it's a very obvious boost in exactly
| that sort of keystroke-saving way.
|
| Here's an example I just now tried in the codebase I had
| open. I literally typed in the following Go in a
| function: mean := geom.Vec3D{} for
|
| Copilot then autocompleted this, me pressing tab to
| accept each line, to: mean :=
| geom.Vec3D{} for _, detection := range filtered {
| mean.X += detection.X mean.Y += detection.Y
| mean.Z += detection.Z } mean.X /=
| float64(len(filtered)) mean.Y /=
| float64(len(filtered)) mean.Z /=
| float64(len(filtered))
|
| It's not a perfect platform by any means, but it's also
| pretty useful for this sort of autocompletion.
| gopher_space wrote:
| I can picture autogen boilerplate being collected and
| distributed in versioned "community expansion packs" to
| popular languages, and I'm not looking for fragmentation
| like that in my tools. I _really_ don 't want my IDE
| involved like this.
|
| I'd rather see Copilot used to expand existing libraries.
| Pulling potential additions to your own library off of
| Copilot would be an interesting twist on the situation. A
| hacktoberfest-alike based off this would be weird.
| julian37 wrote:
| This is a good example of why I dread Copilot: even if Go
| specifically couldn't express this any more concisely,
| there is a language that can and Copilot's very existence
| makes it less likely for that other language to be used
| as much as it deserves.
|
| Besides, the generated example seems to be missing code
| to gracefully handle the case where len(filtered) is
| zero. Maybe there's a precondition that prevents that
| from happening or maybe a division by zero is exactly
| what you'd want, but at face value it looks like the bot
| did a rush job.
| morelisp wrote:
| Zero is gracefully handled; the mean of a zero-sized set
| is best represented by NaN, and this would be idiomatic
| in most languages' IEEE754-style handling.
|
| Saturation is not. This is what really bugs me: If I'm
| going to drag in a billion GPUs of external computation
| (or a dependency, which is basically the same thing but
| with human brains), I want it to provide the hard
| algorithm I can't write, not the easy one I can. I am not
| limited by typing speed.
|
| I also wonder what a "detection" is.
| Beltalowda wrote:
| I found carefully reviewing the suggestions it gave me more
| work than actually writing the code myself. Granted, I only
| used it for a day, but many of the suggestions were subtly
| wrong, needless inefficient, or used outdated/deprecated
| paradigms/standard library stuff.
|
| I only used it for a language I'm very familiar with. I'd be
| a lot more hesitant using it for a language I'm less familiar
| with because I won't be able to spot the problems so easily.
| fassssst wrote:
| RTFA :)
| throwaway6977 wrote:
| Yup exactly, and the ability to basically be fluent in any
| language.
| cmrdporcupine wrote:
| I will try it at some point, but I have... feelings... about
| its abuse of open source licenses software as a learning set.
|
| Also I work in Rust these days, and use CLion. Not sure if it
| would be much help in that environment.
| bayindirh wrote:
| I left GitHub on the spot, and moved to Source Hut the day
| Copilot released.
| MichaelMoser123 wrote:
| someone can still mirror your stuff on github. I wonder
| if they should make a special open source license, that
| disallows use of the source code for the purpose of
| training something like Copilot.
| gopher_space wrote:
| If it's your creation you can make your own license. Copy
| and paste a license you like and add a few lines.
|
| This fact solves many "we need this lib but it's too
| open" problems. The author can make a special license
| just for you if they want.
| bayindirh wrote:
| As in this example: https://web.archive.org/web/201302031
| 12329/http://dev.hasenj...
| bayindirh wrote:
| > Someone can still mirror your stuff on github.
|
| It is the nature of the beast, however I make my stance
| clear and stand by my principles. It is equally important
| for me.
| dragonwriter wrote:
| > I wonder if they should make a special open source
| license, that disallows use of the source code for the
| purpose of training something like Copilot.
|
| Since Microsoft uses material for copilot outside of
| licensing on the basis that it is Fair Use, that would
| probably have no effect in practice on whether or not the
| material is used in training something like Copilot. For
| that to matter, you'd first have to win a lawsuit on the
| basis that training something like Copilot requires
| permission of the copyright owner of the training
| material, to invalidate the premise of Microsoft's
| action.
| MichaelMoser123 wrote:
| > Since Microsoft uses material for copilot outside of
| licensing on the basis that it is Fair Use, that would
| probably have no effect in practice
|
| i think that any formal violation of an open source
| license would be very bad in terms of public relations -
| even if they can claim fair use.
| bayindirh wrote:
| They got it covered. Passed the responsibility to user in
| their official documentation.
| cmrdporcupine wrote:
| I don't care about all the handwaving here, it comes down
| to this.
|
| The moment a fragment of code from a GPL'd or AGPL'd
| project shows up almost verbatim in someone's closed
| source or non-copylefted, etc. project, and someone
| proves it, sparks are going to fly.
|
| And it's probably already happening, just people haven't
| discovered it yet.
|
| How many years did the Oracle/Google lawsuit go on for?
| And in the end about a handful of a lines of code only
| tangentially related to the issue at hand?
|
| Part of the lesson from that must be: employers should be
| telling their workers to say TF away from Copilot or
| things like it. And be careful in general when browsing
| source. License literacy is critical.
|
| I don't touch it because I need to feed my kids. I don't
| need my career exploded. Overcautious? Maybe. I'll let
| someone else find out. I make a living in and around open
| source software.
| bayindirh wrote:
| > The moment a fragment of code from a GPL'd or AGPL'd
| project shows up almost verbatim in someone's closed
| source or non-copylefted, etc. project, and someone
| proves it, sparks are going to fly.
|
| I'm hoarding popcorn and can't wait for that, honestly.
|
| > License literacy is critical.
|
| It's beyond critical, but most people I have talked says
| that they see the right to copy and use any code they see
| online. They don't care.
|
| Building this open source corpus was not easy, and we
| need to defend it too. This is a culture.
| sidlls wrote:
| How is it abusive?
| Guest42 wrote:
| It was trained using source code whose licenses don't
| permit such usage.
| skybrian wrote:
| This is unclear. I have never seen an open source license
| that was explicit about this. Seems like a grey area.
|
| It's not even clear how often training machine learning
| algorithms on code results in copyright violations.
| CoPilot does have a setting to detect and disallow direct
| copying, but how well does it work?
|
| This legal uncertainty is enough that I wouldn't advise
| using it, but maybe people who use it will be fine?
| sidlls wrote:
| In what sense? Copilot isn't a derivative work in the
| sense these licenses usually are understood to mean. And
| given that they're open source code bases I expect
| licenses to explicitly disallow things, and consider
| anything not explicitly disallowed as permitted.
| teddyh wrote:
| > _Copilot isn't a derivative work in the sense these
| licenses usually are understood to mean_
|
| The phrase "derived work" is, IIUC, a phrase from
| copyright law. And you'd have a hard time convincing me
| that Copilot-generated code is not a derived work from
| its training data.
|
| > _And given that they're open source code bases I expect
| licenses to explicitly disallow things, and consider
| anything not explicitly disallowed as permitted._
|
| That is very much _not_ how copyright and licences work.
| Copyright law permits the copyright holder the exclusive
| right to make copies of the work, making derived works,
| (and to do some other related things, like making a
| public performance of it, etc.), so to do any of those
| things, you need explicit permission, i.e. a _license_
| from the copyright holder to do it. A license is not a
| list of things you are forbidden to do; on the contrary
| it is a list of things you are _permitted_ to do, which
| you would not otherwise be legally allowed to do
| according to copyright law.
| scoot wrote:
| I guess any "abuse" is the fact that any code reproduced
| verbatim doesn't carry the license under which it was
| originally published. But the reality is that what Copilot
| is most useful for is (semi) intelligent autocomplete, and
| generating functions that almost certainly exist as
| StackOverflow answers. It won't do any thinking for you
| when it comes to the bigger picture.
| TheRealPomax wrote:
| Not until there's a setting that can guarantee the
| automcomplete is based on verifiably license-unencumbered
| source that is automatically tracked in some kind of
| sourcemap that tells me which parts of "the code I didn't
| write" comes from which other project and file inside that
| project.
|
| Until then, copilot is a giant liability that ensures I can't
| use it for code that my company ends up owning, nor can I
| contribute code I write with it to literally any open source
| project because in a very real sense: I didn't write it. I
| just assembled it from parts unknown, and those parts may end
| up being lawsuits.
| honkdaddy wrote:
| As a hypothetical, what would that case actually look like?
| I'm suing you because I have a strong belief that part of
| the codebase of your personal project was assembled from
| code I wrote and didn't license permissively, so now I'm
| claiming ownership?
|
| Obviously IANAL so this is largely conjecture, but until we
| _actually_ see how this would play out in court, I'm
| leaning towards this being less of a legal issue than folks
| here act like. For personal projects, I'd say the
| likelihood of some other engineer reading your code,
| noticing a similarity or duplication, and dragging you to
| court for it is near 0.
| thaeli wrote:
| I would also be comfortable with a service that was willing
| to broadly indemnify me as the customer from copyright and
| patent claims arising from code generated by their service.
| I doubt that will ever happen either.
| alain94040 wrote:
| Copilot doesn't change the fact that trivial questions (such as
| 'write a Fibonacci function') can already be googled.
|
| What I don't know (I can't try the 'free' Copilot without
| providing a credit card), is how well it understands unusual
| constraints and separate pieces of C++ classes that make the
| real programs we write.
|
| For instance, a simple test would be: "write a fibonacci
| function but skip the number 5". Does it do the right thing?
|
| Another challenge would be a C++ class with several fields.
| That's what our real programs look like: composite data
| objects, mixing several different algorithms together to
| implement complex behavior.
|
| Maybe my data structure has a hash table of items, as well as a
| direct link to the largest item. When I say: "write the
| function to insert a new item in the list, and remember to
| update the largest item if it is larger than the current one",
| would Copilot do the right thing? Each step is easy in itself
| (adding an element to a hash, comparing an item to another
| one).
| wilg wrote:
| I (surprisingly) could not get it to generate a Fibonacci
| function that skips 5.
| thewebcount wrote:
| As a human, I'm having trouble understanding the problem
| description. What does it mean to "skip 5"? I've always
| understood the definition of a Fibonacci sequence to be,
| starting with 2 numbers (usually 1 and 1) the next number
| in the sequence is the sum of the previous 2.
|
| So starting with 1 and 1, we get 2. 2 and 1 is three. 3 and
| 2 is five. What do we do from here? Do I not add 5 and 3?
| If not, then what do I add? 3 and 2 again? Do I repeat 3
| and get 6? What does it mean to "skip 5"?
| hibbelig wrote:
| You compute the Fibonacci sequence first. Then you use
| that as the input for another function that produces the
| modified sequence by dropping 5.
| Applejinx wrote:
| I code for a living and Copilot can't do the least part of my
| job, though it is possible it could find uses here and there
| around the fringes (perhaps not even then).
|
| I'm coding audio DSP along the lines (real example here) of,
| "Given that the Dolby noise reduction system is deeply part of
| the sound of classic seventies and eighties recording, what
| parts of it can be generalized to make DSP processing producing
| the same general effects on the sound, and having done so, how
| should this be tweaked and adjusted to optimize for pleasurable
| results?"
|
| This doesn't exist on Copilot. Damn near nothing I do for work
| exists on Copilot, and even if it (being an MIT-licensed public
| code library) gets slurped up into Copilot, Copilot has
| absolutely no way to determine whether a 0.8273 in the
| algorithm should be leaning towards 0.83 to brighten the sound
| of something happening, that it would do this, or why you'd
| want brighter rather than darker, or whether you are right in
| wanting this or would do better to go darker knowing that other
| things will happen to the listening experience.
|
| Copilot's inability to do these things is much like Dall-E's
| inability to do art. There is much visualization and a striking
| absence of purpose or intention. With Copilot, it will do all
| the boring or common stuff for you. If that's your job, maybe
| you're not aiming at the strengths of humans, rather you are
| aiming at things too easy to automate...
| baby wrote:
| If you're writing code you are the audience for copilot. Just
| try it
| Brian_K_White wrote:
| This sounds kind of creepy culty. I think it's the pushing
| "just try it" vs merely reporting "I like it".
| netr0ute wrote:
| If you're into being hydrated, water is great. Just try it
| t_mann wrote:
| Leetcode interviews have maybe just become an example for
| https://en.wikipedia.org/wiki/Goodhart%27s_law - as long as
| people aren't aware, it's a great signal, but after people
| catch onto it, it becomes a measure of something else
| tgv wrote:
| > if students are unwilling to learn, or are taking shortcuts,
| it will ultimately hurt them in the long run
|
| That's a very limited, and dismissive view. If you allow
| students to use copilot, you're handing out certificates or
| diplomas to people who can't code. That will very quickly erode
| the value of your institute's certification, and with it, that
| of the other students. Otherwise, why not give everyone an MSc
| in CS? "It'll only hurt themselves in the long run".
|
| > coding clever for-loops to perform algorithmic magic
|
| A for loop for searching or summation is not algorithmic magic.
|
| > maybe we should be measuring based on something else
|
| No, why? Students have to understand the fundamentals. What
| those fundamentals are depends on the expertise. Once you've
| shown mastery, you can use more advanced tools. Teaching has
| always been like this, and for a good reason.
| nightpool wrote:
| I'll be honest: has anyone here ever looked at a resume with
| an MSc in CS and said "Yep, I bet this person knows how to
| write code well"? Surely there are some research level jobs
| where a PhD in CS is necessary, but for most coding jobs, an
| MSc is just 2 years of missed experience where you're
| learning bad habits and antipatterns by never having your
| code used by other people in any strenuous way.
|
| Certainly there are a lot of pros and cons to getting a BA in
| CS. But "learning how to write code" is not the major
| advantage of a university education, nor should it be.
| morelisp wrote:
| I'll generally rate an MSc lower than BA/BSc. (The
| exceptions would be if you needed the MSc to meet some visa
| requirement, or if you had an BSc in some weakly-related
| field beforehand.)
|
| From downvoters: I would genuinely love to hear a defense
| of an MSc per se. What university has their shit together
| enough to actually accelerate/compress 2-4y of work
| experience into 1-2y years of academic project, which is
| what's always the claim? If all you're doing is the
| courses, those are accessible to 4th year undergraduates in
| every university I know.
|
| To a person, everyone I've interviewed who did an MSc for
| the coursework had to because they finished their BSc with
| minimum effort and no one was interested in hiring them.
| Even worse is when you see 4y BSc, 2y work, 1y MSc.
| vladtheimpala wrote:
| One does not typically _return to undergraduate study_
| after gaining a first BS degree. If you have a BS (or in
| my case a MS as well) in a not-directly related technical
| field and wish to study CS, you get a graduate degree.
| This often requires extensive additional coursework, in
| my case about 12 additional semester hours. Thanks for
| ranking people like me lower than someone with less
| education; I doubt I would enjoy working for you anyway.
| morelisp wrote:
| I described precisely this case even before my edit and
| said it was an exception.
|
| There are a _lot_ of people coming with 4+1y or 4+2y
| purely CS /SE. That's who I'm talking about.
| vladtheimpala wrote:
| I'd quibble with the meaning of weakly related vs. not
| directly related, but fair enough.
| throwaway675309 wrote:
| A masters program is generally going to cover deeper
| theoretical subjects and will also give you stronger
| mathematical underpinnings in those areas.
|
| If you're just looking for a code monkey to do your
| e-commerce react website then obviously work experience
| will always trump academia.
| morelisp wrote:
| Those aspects of a MSc are easily accessible to any
| motivated undergraduate as well.
| bcrosby95 wrote:
| I've thought of going back for an MSc because it sounds
| kinda fun. Although probably in math rather than CS.
|
| I never thought it would be held against me though. Seems
| weird.
| herval wrote:
| Depends on the school and depends on the person. The same
| way many companies in the US assume any grad from MIT is a
| good engineer, there are companies who look at resumes w/
| an MSc and bump them up. From personal experience, MScs are
| particularly well-regarded in Brazil, for instance.
| hedora wrote:
| I've interviewed lots of new grads, and also observed my own
| classmates.
|
| Most institutions have been granting CS diplomas to people
| that cannot code for decades.
|
| Those people tend not to find long-lasting programming jobs.
| Karellen wrote:
| Computer science is to programming what materials science
| is to construction. It produces really interesting new
| building blocks that actual practitioners can put to
| amazing uses.
|
| But research scientists aren't necessarily great engineers,
| and engineers aren't necessarily great research scientists.
| That's OK though, it's why we have separate disciplines.
| squeaky-clean wrote:
| The CS program at my university, and most people I've
| spoken too, wasn't research oriented. At least not until
| the graduate level. Maybe at really respected places like
| MIT. Yes there were algorithms classes and such, but it
| was more about memorization than actually studying how
| they were made or how you would make a novel one. Final
| projects were always about building a real-world-ish
| application on a team.
|
| But I'd say about 20% of the students that graduated
| still didn't understand things like references very well.
| They'd write their code with "foo.bar" and when it didn't
| compile they'd randomly change "." to "->" or add random
| asterisks around until it compiled.
|
| (Classes were primarily in C++. We did some classes Java
| as well, and in my 3rd year Microsoft came in with a big
| sponsorship to change all the Java classes to C#
| classes.)
| cmrdporcupine wrote:
| I mean, it's more like "woodworker" vs "carpenter." We
| don't ask people framing houses to plane their own wood.
| Most practitioners in our industry don't apply much
| fundamental CS knowledge.
|
| After graduating most CS grads seem to forget
| foundational CS stuff anyways.
|
| I never got a CS degree. So I trained myself. While
| working and also a lot of a year when I was unemployed
| during the .com crash.
|
| Which only made me more frustrated to find that most jobs
| are just "you nail these two boards together" and most
| programmers are clueless about things like what the
| "relation" in "relational DBMS" means.
| cpuguy83 wrote:
| I don't think copilot is to the point where this is true.
|
| Copilot mostly helps write boiler plate. It can also do a
| good job with adding correct comments to that boiler plate.
|
| It's a tool that saves time and repetition. It's not a tool
| that's writing algorithms or solving problems for you.
| jrumbut wrote:
| A lot of CS education is limited by students' ability to
| code.
|
| If everyone was great at coding after a year or two, you
| could do a lot more algorithm analysis,theory, engineering
| principles, mathematics and other sciences, etc.
| PragmaticPulp wrote:
| > If you allow students to use copilot, you're handing out
| certificates or diplomas to people who can't code. That will
| very quickly erode the value of your institute's
| certification, and with it, that of the other students.
|
| Very true. Like it or not, the reputation of educational
| institutions plays a significant role in candidate selection
| processes at many companies. If a company brings in 10
| graduates from a local college and discovers that 7/10 of
| them can't actually write basic code, they're going to become
| more hesitant to bring in other graduates from that college.
| Meanwhile, someone who has Stanford or Harvard on their
| resume is virtually guaranteed an interview. Reputation
| matters far more than I ever thought it would.
| geraldwhen wrote:
| Most people I meet with CS degrees cannot code. Not anything.
| Some can't declare variables or get a file to compile.
| Brian_K_White wrote:
| "you're handing out certificates or diplomas to people who
| can't code"
|
| The diploma is not issued after only the 101 course.
| varjag wrote:
| So people who couldn't find a max value in an array will
| somehow excel at advanced concepts later?
| Brian_K_White wrote:
| You seem to have missed the point of this thread.
|
| The problem an earlier comment expressed is that the
| student will sabotage themselves.
|
| What else did you think that means besides exactly what
| you just said?
|
| They sail through the beginning and splat in the middle,
| because they didn't have to learn anything in the
| beginning.
|
| My comment is reponding to one that says the problem is
| that students get diplomas witbout learning.
|
| I am saying that diplomas and certs are not a problem,
| not that there is no problem. The problem is what the
| earlier comment said in the first place.
|
| Actually I'm not even sure that's a problem either in the
| long term, but it is the immediate problem. It may just
| be during a transition period while schools have not yet
| adapted to reality and are still worried about
| calculators in math classes. But at this moment, it's a
| problem.
| throwaway675309 wrote:
| While it would likely be prohibitively expensive, I really
| wish there was an independent board that would certify
| software engineering as a certification and required an oral
| defense in the same style as a PhD.
|
| I recognize that it's kind of gatekeeping but it feels like
| more and more diploma mills are just hanging out degrees, not
| to mention all the coding camps out there.
| x86x87 wrote:
| The grind you have to go through in a "google style" interview
| has nothing to do with the actual work you normally do. It's
| done to show 1) you will do large amounts of boring work
| without questioning the need for it 2) you got the fundamentals
| and you know what's there / when using something you can grok
| what's going on under the covers. Nobody needs to implement a
| sorting algorithm until they do - the risk of NIH and
| reinventing the wheel is high. When you do you need to have a
| good understanding of why
| [deleted]
| renewiltord wrote:
| Yeah, I totally get it. The purpose of education is credentialing
| and so cheating undermines that. If the purpose of education were
| solely instruction you would advise students not to use Copilot
| to finish assignments but those who do would only harm
| themselves.
| wiseowise wrote:
| Are comments supposed to be funny?
| rwky wrote:
| When I did my BSc in computer science we had to do exams which
| involved writing code using pen and paper without a computer in
| sight. Bringing that back would solve this problem ;)
| throwaway4good wrote:
| What is this? Computer science Luddites?!
|
| Just pose better questions and more open ended assignments.
| tazjin wrote:
| These examples of producing known algorithms in response to
| function signatures make it seem to me like Copilot is an
| intelligent, vendoring "package manager", not something that
| solves problems by itself.
|
| The perceived productivity boost that people get from it could be
| interpreted as a condemnation of how verbose a lot of coding
| still is even in known areas. Lots of brain cycles wasted on
| doing things suboptimally that we can already do optimally.
| vbezhenar wrote:
| The difference is that package management hides packages
| somewhere deep and you're not supposed to care about code
| quality. If it popular and works, good enough. Copilot puts
| terrible code snippets right before your eyes and you'll spend
| more time making it better and fixing bugs rather than writing
| that code from scratch. I tried copilot and for me it's not
| ready yet. I guess my issue is that I don't like average code
| and copilot was trained on that code, so it's as good as
| average coder and that bar is not high. Or I'm too picky and
| high-minded, doesn't matter, the result is the same.
| [deleted]
| O__________O wrote:
| Curious, has anyone has seen a "cheaters" guide to programming --
| that basically teaches a predictable barebones process that
| assumes unmoderated access to internet, computer, etc - and using
| Google, StackOverflow, CoPilot, etc. to solve 80% of programming
| issues average programmer faces?
| T3RMINATED wrote:
| emeryberger wrote:
| Hi everyone, article author here. I was also interviewed by The
| Register about this topic:
| https://www.theregister.com/2022/08/19/copilot_github_studen....
| stew-j wrote:
| I might have missed it today in your articles or comments here
| --it's been a hectic day--but has there been some study of just
| how different code would be given that the students are using
| the same text from questions? Is there randomization intrinsic
| to Copilot, or is it just because minor variations in textual
| input causes code to be so different?
|
| My wife taught CS, she did catch cheaters pre-Copilot, and my
| first thought it that she probably would enter test questions
| and print out a reference sheet for Copilot generated results.
| Barrin92 wrote:
| >"Copilot is different, he said, "It actually generates novel
| solutions. Not like they're super-crazy, sophisticated, genius
| solutions. But it makes new solutions that are superficially
| different enough that they plausibly could have come from a
| student.""
|
| virtually every cheating student has always slightly altered
| their solutions. At best copilot saves you the five minutes of
| Googling and refactoring. For the committed cheater who wants
| to avoid a week's worth of homework and learning that's hardly
| meaningful. And in fact probably not even that because you
| still need to actually read what copilot outputs because it
| makes obvious bot-like mistakes quite often.
___________________________________________________________________
(page generated 2022-08-20 23:00 UTC)