[HN Gopher] Kite is saying farewell and open-sourcing its code
___________________________________________________________________
Kite is saying farewell and open-sourcing its code
Author : dynamicwebpaige
Score : 264 points
Date : 2022-11-20 20:57 UTC (2 hours ago)
(HTM) web link (www.kite.com)
(TXT) w3m dump (www.kite.com)
| EGreg wrote:
| Netscape open-sourcing their code is what led to Firefox and an
| open Web, as a counterweight to closed source browsers. Safari
| took WebKit from Konqueror
|
| I wish more projects would do this
| johannes1234321 wrote:
| Mind that they continued working on it and built Netscape
| versions on top of Mozilla/Phoenix/Firebird/Firefox. This
| people with knowledge of the code base and domain pushed it
| along.
| synergy20 wrote:
| copilot might impact kite's future, it's hard to compete against
| microsoft.
|
| copilot: "Get code suggestions in more than a dozen coding
| languages including Python, JavaScript, TypeScript, Go, and
| Ruby", how about c, c++ even lua here? if they cover c and c++ I
| can pay $10 per month right away.
| mkoubaa wrote:
| The thesis that helping developers write code has value is flat
| wrong. We spend so much more time reading, reviewing, designing,
| arguing/bitching about code than we do writing it. Orders of
| magnitude more.
|
| Any developer tooling company must understand this basic fact.
| dgudkov wrote:
| >Our 500k developers would not pay to use it.
|
| You need to have 500 users to understand that, not 500K. A well-
| written postmortem otherwise.
| sanguy wrote:
| These guys were a complete joke; and a good example of fleecing
| the VC community.
|
| Good riddance to bad rubbish
| satvikpendem wrote:
| Can you explain why they're a joke?
| malwrar wrote:
| "Our 500k developers would not pay to use it. Our diagnosis is
| that individual developers do not pay for tools."
|
| I don't like depending on something I could lose in a month or
| tethers me to the internet. I consider that more a service than a
| tool. I'd prefer to just buy something once that just works, but
| that business model might be dead too since people will pirate
| things that aren't tethered to some serverside component.
|
| I guess what I'm saying is that I want to buy tools, but people
| are only renting. Personally I'm largely holding out hope this
| becomes someone's open source passion project and I can truly own
| my tools.
| bdg wrote:
| Automating software is a really hard problem. I think I can
| imagine a possible roadmap to it, but it's so hard to explain it
| in under an hour, it would require several sequential new
| technologies, and some of it hinges on parts of information
| theory I don't know enough about, and statistical ML isn't part
| of the core.
| satvikpendem wrote:
| I've used Kite, it simply wasn't as good as Copilot. I'm not sure
| why they say that Copilot still doesn't work well, it works well
| enough for me and I presume everyone else who pays for it.
|
| That being said, glad to see a lack of Our Incredible Journey
| type language here and more of a true postmortem of their
| business and technical decisions. It is rare to see a company go
| into so much detail when shutting down.
| MisterSandman wrote:
| What an honest, transparent message. Kudos.
| gibsonf1 wrote:
| The key issue is that ml/dl is pure statistics - there is no
| intelligence or learning or conceptual awareness of space-time,
| so that technology can never do so many things people try to do
| with it.
| perlgeek wrote:
| Some ml/dl tools (think gpt3) seem to be able to answer
| questions that we previously thought you needed gintelligence
| for. I think the line between "pure statistics" and
| "intelligence" are much more blurry than they used to be, and
| might go away entirely.
| gibsonf1 wrote:
| Actually, I think the industry is finally realizing that
| there is no intelligence there, especially with gpt3 which
| can figure out with great precision what statistically comes
| next, but there is zero understanding of the space-time
| conceptual meaning for gpt3 in that answer - its not designed
| to do anything but figure out statistically what is most
| likely to come next.
|
| Gary Marcus has been doing a good job of exposing this:
| https://garymarcus.substack.com/
|
| Another key piece of evidence, the failure of all FSD
| attempts trying to use ml/dl thinking its more than just
| statistics.
| hobofan wrote:
| > Another key piece of evidence, the failure of all FSD
| attempts trying to use ml/dl thinking its more than just
| statistics.
|
| I would almost put FSD as a good example here. Yes, some
| attempts here are very naive and try to use ML as a magic
| black box tries to covers a long stretch of the system from
| vision to turning the steering wheel. However the best
| performers just utilize ML for small well-defined parts of
| the system and in a "statistics on steroids" way with most
| of the other parts utilizing much more traditional methods.
| hobofan wrote:
| Some of the things GPT3 can achieve are very impressive, but
| once you work you work with it a little bit, you definitely
| feel that it just regurgitates the masses of text it has been
| trained on and tries to piece it together in the most
| cohesive ways. And for production usage, it (and similar
| models) have huge problems with halucination where it will
| confidently spit out "facts" that are just plain wrong.
| Mikeb85 wrote:
| Individual developers pay for tools, they just have to be worth
| it. JetBrains' whole existence is a testament to that.
|
| From what I remember, people got super annoyed at Kite for
| placing ads in open source projects and they just never caught
| on.
| m00dy wrote:
| >> It may cost over $100 million to build a production-quality
| tool capable of synthesizing code reliably, and nobody has tried
| that quite yet.
|
| $100 million is nothing tbh.
| svnt wrote:
| $100 million on zero product-market fit is a lot.
| samspenc wrote:
| While this maybe true for FAANG and other tech giants, it is
| unfortunately still a lot of money for most startups that are
| working their way to IPO.
|
| Tbh even at tech giants that make tens of billions of dollars,
| a $100 million investment is likely a lot, I'm guessing this
| sort of investment will require sign-off by CEO or at least VP
| level along with a solid business plan.
| uJustsaidit wrote:
| azhenley wrote:
| Kite rejected me for a position years ago which motivated me to
| go raise $1M from the NSF to research AI-based dev tools before I
| moved on to Microsoft.
|
| They seemed like a really cool team, I wish them the best.
| codetrotter wrote:
| > it fell short of the 10x improvement required to break through
| because today's state of the art for ML on code is not good
| enough. You can see this in Github Copilot, which is built by
| Github in collaboration with Open AI. As of late 2022, Copilot
| has a number of issues preventing it from being widely adopted.
|
| True, AI assisted coding does not deliver 10x. But as a user of
| another AI assistant, I feel that it gives me ~1.25x to ~2x
| improvement for the keyboard typing when I code. And that is
| respectable too :) AI for me currently allows me to tab complete
| some things that previously an IDE on its own was not able to.
| lawxls wrote:
| Which one do you use?
| lolinder wrote:
| > The largest issue is that state-of-the-art models don't
| understand the structure of code, such as non-local context.
|
| Depending on how local he's talking, this isn't really true of
| Copilot. In my experience it will use context all the way up to
| the top of the file, even in very long files. And at least the
| Rust version even seems to look at the imports--if you have a use
| declaration it will actually correctly build and use structs in
| other files regardless of whether you've yet used them in the
| current file.
| ShamelessC wrote:
| I had a similar thought. I think in this instance, they mean
| something like a labeled/supervised training approach where the
| model is given not just the tokenized code, but also perhaps is
| grounded in the possible structure (indentation vs. semicolons,
| function-scope, etc.).
|
| My understanding is that copilot is largely a self-supervised
| approach. They feed a massive body of (somewhat noisy) code
| into the model. The model really does learn a lot of structure
| on its own and this is a testament to deep learning on noisy
| datasets.
|
| I'm guessing the "hooks" that they have already from IDE's,
| language-servers, etc. _are_ quite "structure-aware" - so they
| want the predicted structure as well as the code, so they can
| improve the typing experience beyond line-completion.
|
| I think the estimation of 100 million for such a task is maybe
| too high? I don't know - it feels like you could actually get
| quite close to such a system by simply using thousands of
| custom prompt engineering tricks that prepend structure
| examples to the prompt?
| adamsmith wrote:
| Yes, this was precisely what I was referring to. In small-
| enough programs (e.g. one file) Copilot has all the context.
| The other extreme would be something like the Chromium
| codebase. Because of this, Copilot looks better in quick demos
| than real-world use. (Though of course it is very impressive
| and this tech will get there, hopefully very soon!)
| lolinder wrote:
| But what I'm saying is that it does use imports, at least in
| Rust. I'm assuming that somehow behind the scenes they're
| concatenating the contents of the imports into the prompt.
|
| I can imagine this is easier in a language like Rust that has
| a really strict module system, and to be fair the project
| that I've been using it on is a side project that isn't over
| 10,000 lines of code yet. If I were up to 30 imports per file
| I can imagine concatenating would become much less effective.
| adamsmith wrote:
| Does it seem to only understand imports of public
| libraries? If so, it's likely that, rather than
| understanding the contents of those libraries, it's
| learning from others' use of those library APIs. If not, it
| is likely just understanding the words in the API at a
| shallow depth.
| vessenes wrote:
| Condolences to the Kite team. But, congratulations, too - you
| have some of the highest value engineering experience in the
| world. I'm sure you'll land somewhere great; try and take some
| time off if you can afford it!
|
| Mulling over business models, and noticing the 'devs won't pay'
| narrative in the blog post, it's interesting to see the existing
| business models in AI; basically they seem to be:
|
| * API-driven cloud calls (this is a way to get high value out of
| your existing cluster if you're AWS, MS, etc.)
|
| * Platform play + possible eventual lock-in: OpenAI/Microsoft
|
| * Subscription service for very specific needs (Grammarly,
| writing support)
|
| I wonder if engineers would pay $9.99/month (or even
| $49.99/month) for a 'grammar checker for PRs' - essentially:
| "Avoid embarrassing bugs before you commit". That is, I wonder if
| Kite could have been successfully sold as the third tier - sub
| service for something very specific.
|
| I guess if it's a good idea, someone could pull the Kite repos
| and launch it -- but my guess is there may be a market in there.
| lbhdc wrote:
| I am not convinced there is a market there. This is a feature
| in existing ides, and the grammar suggestions are often wrong.
| candiddevmike wrote:
| Devs almost always lack any kind of purchase authority. Any
| tool that appeals to devs needs to appeal to their management
| more, either showing some kind of cost savings over existing
| tools, increased dev productivity, or the new fangled "dev
| experience" where this tool, by shear awesomeness, will let
| devs put aside the low salary, process hell, and keep them
| employed.
| vessenes wrote:
| I understand that's Kite's perspective (and yours --
| "purchasing won't pay for this"), but devs are _not_ paid
| meager salaries in general, and definitely might care about
| their code quality when it 's put out in 'public' whether
| that be internal repos, or github.
|
| Payscale estimates average engineering salary as having
| between $3,000 and $7,000 a month more in disposable income
| over writers -- and I would guess almost every professional
| writer pays for grammarly.
|
| But, I agree that this is a new concept, and just spitballing
| -- right now, these sorts of linters and code formatting
| tools are mostly open source, so it would be some product
| marketing work to see if the market would actually pay.
| candiddevmike wrote:
| Like everything in most companies, it's never about money,
| it's about control/power. It's inevitably someone's job to
| manage all the SaaS shit at a company, and damnit we just
| bought a JetBrains license why do you need Copilot?
| neilv wrote:
| Opensourcing code when you shut down has the nice effect of
| making it available to the world.
|
| It also has the nice effect of keeping the code available to the
| people most familiar with it, as they move on to other ventures.
| imiric wrote:
| Open sourcing when you're shutting down is only marginally
| better to not releasing anything.
|
| It means that you didn't believe in open source while you were
| in business, and are only doing so now to score some points
| with your customers. There's no guarantee that someone will
| step up and maintain the project for you.
| happytiger wrote:
| Thank you for open sourcing your startup. I'm sorry it didn't
| work out. I think you deserve a big congratulations for being the
| first to really go after this problem. It's a correct problem --
| it's a big market and the solution will come eventually -- I'm
| just sorry it turned out to be too gnarly to solve for you right
| now! I would have loved for it to have worked out better.
|
| I agree that Kite didn't deliver the 10x. I was an early user and
| tried hard to use it but didn't find the benefit compelling
| enough to drop into my workflows, but it was very exciting.
|
| I'm sure I speak for all of HackerNews when I wish you the best
| for whatever is next for the team.
|
| Also, what are you good folks doing next?
| jawns wrote:
| > We failed to build a business because our product did not
| monetize, and it took too long to figure that out.
|
| This is the one-sentence summary about why the business failed,
| but it's kind of a strange way of putting it.
|
| I am dead sure that there were plenty of advisers along the way
| who told the company's executives that its monetization plan was
| weak and unlikely to succeed. But everyone assumes that they'll
| be the exceptional case.
|
| "It took too long to figure that out" makes it seem like the most
| likely scenario wasn't staring them in the face the whole time.
| oxfordmale wrote:
| I disagree with their statement that individual developers do not
| pay for tools. I have paid for tools out of my own pocket on many
| occasions. However, being able to deliver code 18% faster isn't
| enough to fork out $9.99 a month. First of all it is relatively
| expensive. For that amount I can get a personal license for
| PyCharm. Secondly coding speed never tends to be a bottle neck
| for delivering a feature or a product on time. I can see why
| Engineering Managers are not willing to pay for this.
|
| I do wish the Kite team all the best, and I hope they can re-use
| their skills in products that are commercially viable.
| [deleted]
| randomdata wrote:
| Most importantly, coding is the fun part of the job. This seems
| like trying to sell a DALL-E-esq product to a visual artist
| promising 18% faster deliverables. Even if it is true, who is
| going to be in a rush to give way in that aspect of the job and
| sell their manager on it to spend more time doing the less fun
| things?
|
| On the other hand, create an AI that can stand in during
| pointless meetings and the blank checks will shower down.
| selimnairb wrote:
| This metric seems silly on its face. 10 bucks to get 18% more
| productivity out of a $10k per month developer? If this was
| indeed the case, everyone who employs software engineers would
| instantly pay this. Maybe they should have marketed more? Or
| maybe there are other problems with the technology (e.g., fears
| over copyright infringement?).
| oxfordmale wrote:
| Apparently it wasn't a very good product, even before Github
| Copilot came out:
|
| https://medium.com/swlh/kite-vs-tabnine-which-ai-code-
| autoco...
| IshKebab wrote:
| Yeah the problem is the "if" part. It may well be true but
| productivity is notoriously hard to measure and anyone making
| any claims about exact productivity increases is clearly
| pulling a number out of the air. People know this.
|
| We have plenty of techniques that we know improve
| productivity (e.g. static types) but some people still don't
| believe it because it's really hard to _prove_ productivity
| increases.
| nonbirithm wrote:
| One last reminder that they once hijacked several open-source
| repos to inject advertisements for their service into the
| codebases.
|
| https://news.ycombinator.com/item?id=14836653
| [deleted]
| lzooz wrote:
| Buying something and then changing it is not "hijacking" that
| thing.
| BoorishBears wrote:
| If you have a nuanced understanding of the language, yes, it
| is.
|
| The common definitions have to do with stealing, but an
| equally valid definition of the word hijack is to:
|
| > take over (something) and use it for a different purpose.
|
| Taking over a project so you can have it to advertise your
| service is exactly that.
| joecool1029 wrote:
| Hey if we're going to talk language, maybe you should just
| use 'Kife', it looks like Kite and means to steal.
| (Allegedly derived from Old English word 'kip', net says
| it's British slang, but I've heard it a few times in
| northeast US.
| _cs2017_ wrote:
| > take over (something) and use it for a different purpose.
|
| You are misleading readers in order to promote your agenda.
| You clearly speak perfect English, so you know what hijack
| means. "take over (something) and use it for a different
| purpose." is not found as a definition of "hijack" in any
| dictionary. "Hijack" implies "unlawfully" or "without
| having a right to do so".
|
| Of course, every word can be used in a slightly different
| meaning; for example, in software can (harmlessly) hijack
| an entity (circumventing the usual API for expediency or
| performance). Such broadened semantics is perfectly fine
| when there's no confusion about the meaning. Very clearly
| in the case of OP, there was a clear intention to imply
| "unlawful" or "without having a right", so this exception
| doesn't apply.
|
| The sad thing is that I actually _support_ your agenda. I
| just don 't support promoting it through misleading
| statements.
| BoorishBears wrote:
| You should let Cambridge know: https://dictionary.cambrid
| ge.org/us/dictionary/english/hijac...
|
| > to take control of or use something that does not
| belong to you for your own advantage:
|
| And Encyclopaedia Britannica:
| https://www.britannica.com/dictionary/hijack
|
| >: to take or take control of (something) for your own
| purposes
|
| And Merriam Webster: https://www.merriam-
| webster.com/dictionary/hijack
|
| > : to take or take control of (something) as if by
| hijacking > often, specifically : to change the topic or
| focus of (something, such as a conversation) : REDIRECT
|
| It wasn't my statement by the way, I just figure if
| you're going to nitpick you should at least be correct
| about the nit.
| nielsole wrote:
| The claim in the referenced article is maybe more fitting:
|
| > many programmers would consider [this] a violation of the
| open-source spirit.
| _cs2017_ wrote:
| I encourage you and everyone else to follow ethical rules in
| fighting unethical behavior of corporations.
|
| Instead of making the untrue statement above, just say
|
| "They used, in my opinion, an unethical way to advertise their
| product; specifically, they bought OSS products and put their
| ads in there."
| perlgeek wrote:
| I've tried Kite once, and wasn't really impressed. For example,
| back when I tried it, it wouldn't offer _any_ kind of
| autocompletion within a string. Even vim 's built-in autocomplete
| tries to complete words for you there, based on other words
| you've used before.
|
| Kite did sometimes offer some good suggestions in regular code,
| but it tried _really_ hard to understand your code, and went
| belly-up when it didn 't.
|
| At that time, I tried some other ML-based autocompletion tool
| which wasn't specific to python, and which usually worked much
| better, except that it used far too much memory and caused
| regular crashes.
|
| Maybe they improved kite since I tried it, or maybe "individuals
| don't pay for dev tools" isn't the whole story. Or maybe both.
|
| Anyway, kudos for both trying and for open-sourcing the code at
| the end!
| RandyRanderson wrote:
| Would've thought Adam Smith would be able to monetize something
| if anyone could.
| rch wrote:
| > the Kite Engine, which performs all the code analysis and
| machine learning 100% locally on your computer (no code is sent
| to a cloud server).
|
| I was never aware they changed the architecture to keep code
| analysis entirely local. I would have purchased a subscription,
| had I known.
| ElKrist wrote:
| "Their manager might, but engineering managers only want to pay
| for discrete new capabilities, i.e. making their developers 18%
| faster when writing code did not resonate strongly enough."
|
| Are there a lot of businesses where individual developer
| productivity, with a narrow definition of LOC per hour, is the
| bottleneck?
|
| I've worked for 10 years as a web dev and the bottleneck is very
| often at the product management level (tickets not ready, goals
| changing, haven't got the credentials for the 3rd party API
| yet..) and a minority of the time it's my brain (yes sometimes I
| need to think before I write code). It's rarely how fast I can
| write a function. So if you make me 18% faster at something I do
| 1% of the time... good luck making money out of me
| onlyrealcuzzo wrote:
| > I've worked for 10 years as a web dev and the bottleneck is
| very often at the product management level
|
| Anecdotally, this was my experience at many companies before
| working at FAANG. But it's not my experience now.
| jll29 wrote:
| Things I've payed for that I'm still using today:
|
| - Sublime
|
| - GitHub.com
|
| - ACM Digital Library
|
| (The latter two are subscriptions.)
|
| Things I've payed for in the past that I no longer use:
|
| - MS Visual C++
|
| - Omicron Pascal
|
| - Application Systems Modula-2
|
| - Atari ST GFA BASIC 2.0
|
| - Berkeley YACC and FLEX port to TOS/GEM
|
| - ...
|
| Overall, many dev tools are free nowadays, which creates an
| expectation, perhaps, that it should all be free (I disagree in
| principle, but of course it is nice to see this trend
| progressing).
|
| I appreciate that Kite is posting a post mortem for others to
| learn, and I wish they had been able to find a niche where people
| pay for their work. I love software tools as a work product, but
| have been told by many experienced people it's not a good area
| for making money.
| janoc wrote:
| I don't think people have problem paying for tools that are
| genuinely useful for them, regardless of whether or not there
| are free tools around.
|
| The problem with Kite seems that their engineered first ("This
| machine learning AI is so cool, what can we do with it?" "I am
| a VC, are you doing AI? TAKE MY $$$$!") and only after burning
| through millions started to look at how to actually make money
| out of it.
|
| And discovered that:
|
| a) Hobbyist/individual developers rarely want to pay yet
| another subscription (can't justify it if you aren't making
| money with it & even $10/month subscriptions do add up!)
|
| b) Corporate developers don't have purchasing authority.
| Everything must get approved, by both accountants and
| legal/compliance. Expecting a large company to pay a huge
| monthly/annual subscription fee for what is essentially a
| better autocompleter? Good luck with that.
|
| That "Oh but your developers will be 18% faster!" argument is
| BS. 90% of the corporate developer's time isn't spent on typing
| code but on debugging, design, maintenance and meetings. Kite
| (or Copilot) don't help with that.
|
| c) What about copyright/compliance issues? This has been
| trained on Github repositories - i.e. the same as Github's
| Copilot. How do I know where does the completed code come from?
| What about licenses on that code? Can I filter only for
| permissive/non-contagious (i.e. non-GPL) licenses? What about
| my code/whatever I type? Does it get sent to your servers? That
| alone is a complete no-go for companies.
|
| In other words, a classic case where one shouldn't ask whether
| something could be done but whether it should. Someone outside
| of their engineering bubble and with a bit of business acumen
| would have told them that. Or at least told them to do a market
| research first, _before_ spending all that time and money.
|
| But hey, they had a good ride for the VC's money and are
| winding it down in an organized manner, without leaving a ton
| of shattered lives and a mountain of debt behind. So that's a
| plus.
| Aeolun wrote:
| I think I've tried all of the code completion tools and Kite is
| the only one I didn't end up paying for. It just wasn't useful
| enough.
| tommica wrote:
| I'm sorry that it did not pan out, but thanks for sharing the
| code!
|
| Hopefully the next project goes well!
| dibt wrote:
| Good riddance. I still remember how they were phoning-home
| without being 100% transparent about it, and the injection of
| ads.
|
| > We failed to build a business because our product did not
| monetize, and it took too long to figure that out.
|
| Yet people always defend telemetry in software, saying it's how
| they improve their product. 7 years of telemetry, and they
| couldn't figure it out?!
| sqs wrote:
| Sourcegraph CEO here. I respect what you and your team built.
| It's tough to build a brand new kind of product, and I heard from
| many people who loved Kite over the last several years.
| jrpt wrote:
| "Our diagnosis is that individual developers do not pay for
| tools. Their manager might, but engineering managers only want to
| pay for discrete new capabilities, i.e. making their developers
| 18% faster when writing code did not resonate strongly enough."
|
| I never used Kite, but I've tried Github Copilot twice, and found
| it marginal at best (and distracting at worst - which is why I
| turned it off both times). If Kite was similar, the reason I'm
| not paying is that coder AIs are not providing any value.
|
| Developers are somewhat reluctant to pay for tools but I think
| you can get them to pay for things that are worth it. I've been
| paying for code editors for years.
| glenngillen wrote:
| I'd recommend anybody thinking about building a devtool to read
| Neil Davidson's "Don't roll the dice". It's a pretty old book,
| but Neil has also made it available for free now and the
| general lessons still hold true today.
|
| Some IC developers will pay for tools, it's very hard to have
| that happen at a price point that supports the scale required.
| So feature discriminate on the things their boss needs, and
| charge for that. And then the next set of features for their
| bosses' boss, and so on until you're selling into the C-suite.
| deforciant wrote:
| Paying for copilot :) at least in go it's great to write tests
| and sometimes some smaller functions :) totally worth paying
| for it, even from your own pocket if the company wouldn't allow
| expensing it
| janoc wrote:
| If the company wouldn't pay for it then better think twice
| because you could get in hot water with legal. That's not a
| tool one's job or even company's business is worth risking
| over.
|
| Copilot has a ton of still unresolved legal and compliance
| issues (copyright violation problems, sending proprietary
| code to Microsoft as you are writing it, etc.) and most
| larger businesses won't touch it with a 10 foot pole for that
| reason. There is even a class action lawsuit against
| Microsoft over Copilot already.
| joshvm wrote:
| "Our diagnosis is that individual developers do not pay for
| tools."
|
| I think this is probably true. If you need a tool for your day
| job, your company ought to be paying for it. Some companies
| have slush funds for small purchases like books, but
| subscription costs for services would normally need to be
| approved. If you're a solo consultant then perhaps you'd pay
| for tools that make you more productive. But for personal
| projects the value-add would have to be pretty high to be
| paying another O($10-20) a month on top of other subscriptions.
|
| The big group of "hobbyist" coders are students, and they get
| copilot for free via Github's very generous edu package (and so
| does anyone with an edu email address I think). The bigger
| problem is that this is a very expensive project. It's better
| suited to a big company with money to burn and deep pockets to
| give it away to junior devs who will evanglise for it at their
| new companies (e.g. students) for nothing. See Matlab.
| dijit wrote:
| If you'll allow me to go on a tangent here;
|
| The sheer volume of subscription services I've signed up for
| as the CTO for a startup is mind-boggling. $8 here, $19
| there, $49 for something important, $99 for something
| essential.
|
| Some tools are easily worth it, especially when you see what
| is charged for other (less valuable) tools.
|
| Gitlab, Confluence, Jira, Asana, 1Password, co-pilot,
| codepen, sentry, jetbrains, gitlab plugins for jetbrains,
| Visual Studio, Docker Desktop, Perforce, Slack,
| etc;etc;etc;etc
|
| Then there's things like Spacelift ($250!)!
|
| The most frustrating thing is that:
|
| 1) I need to justify these expenses each for what value they
| bring, some things are nice to have but bring so little value
| on paper.
|
| 2) You can't just enable tools for _some_ people, there 's
| huge overlap and that overlap gets greater
|
| I get that people need to be paid, but these things very
| quickly add up. I'm paying about 7-13% of peoples salaries
| already in these subscriptions, and I feel like a total dick
| for saying no to people or trying to consolidate these.
| jnwatson wrote:
| The weird thing is that 13% seems high. It is hard to
| imagine they are less than 13% more efficient with those
| tools.
|
| It is weird that software engineers are the only engineer-
| types that are supposed to be able to do their job with
| just a computer and a built-in editor.
| einpoklum wrote:
| > Some tools are easily worth it, especially when you see
| what is charged for other (less valuable) tools.
|
| Maybe, but most of the tools you listed are not in that
| category IMNSHO.
| throwaway675309 wrote:
| When I tried kite over a year ago I was relatively unimpressed
| with it. Even though it ran as a plug-in to jet brains IDE it
| required a windows installed service and two separate
| executables running in the background (kite.exe, kited.exe),
| and that stuff continue to run after exiting my IDE which was
| unacceptable for me.
|
| Kite may have been the first to market but copilot blew them
| out of the water in terms of overall functionality.
| ilrwbwrkhv wrote:
| I think the real reason is that developers are maybe some of
| the hardest to fool customers on the planet.
|
| Since we literally build all of this our B.S. detection meter
| is really high.
|
| Kite thought it can go after the up and coming new developers
| by doing slightly shady things.
|
| However, developers also have an incredible allergy to such
| tactics and it forever taints your brand.
|
| So overall, developers do pay for tools, just not useless ones
| with shady growth tactics.
| esperent wrote:
| > Kite thought it can go after the up and coming new
| developers by doing slightly shady things
|
| I briefly tried Kite a few years ago. I didn't notice
| anything shady although maybe I just didn't stick around long
| enough.
|
| What shady tactics are you referring to?
| stanislavb wrote:
| On the contrary, I find Github Copilot extremely helpful and
| saving me heaps of time. Yes, it's not writing the logic
| instead of me, but it acts like the best companion I could have
| in most of the cases.
| dgacmu wrote:
| I pay for copilot. It saves me a modest number of minutes of
| time per week. That's worth a small fee.
|
| And before someone jumps in: I and my other co-founder who also
| uses copilot (We are the only two in the company who do, I
| think, without checking) _are_ the compliance team. We 're both
| very senior and use copilot basically a line or three at a time
| as a smart autocomplete. It's still worth it.
| hanselot wrote:
| It's really just perfect for remembering obscure things and
| can easily be prompted to generate the boilerplate. If you
| surround it with your style you will see it try to use the
| same techniques, however if you work on large code bases it
| gets annoying when it starts copying the bad habits you are
| trying to get rid of. In those cases it's actually kind of
| good for bringing to your attention that the building next
| door is still on fire.
| yarg wrote:
| I paid for intellij - damned near the entire architecture team
| where I worked had a copy, and the company sure as shit wasn't
| the one paying for it.
|
| (I eventually stopped subscribing, in part because they were
| too slow distancing themselves from Russia, in part because of
| their movement away from open source with their newer tooling.)
|
| Developers will pay for software, if the value proposition is
| there.
| lolinder wrote:
| > in part because they were too slow distancing themselves
| from Russia
|
| I'd cut them some slack here. They had to get their team out
| of there first--with the way Putin is running things, they
| sure as hell couldn't announce they were leaving Russia until
| everyone who was going to follow them was out of there.
|
| On the day of the invasion they tweeted a statement
| condemning the attack, and within two weeks announced they
| were leaving Russia.
|
| https://twitter.com/jetbrains/status/1496786254494670851?lan.
| ..
|
| https://blog.jetbrains.com/blog/2022/03/11/jetbrains-
| stateme...
| vinyl7 wrote:
| > Developers are somewhat reluctant to pay for tools but I
| think you can get them to pay for things that are worth it.
|
| Indeed, I payed for a debugger because MSVC is pretty terrible
| eloff wrote:
| My experience with copilot has been very different. It easily
| pays for itself, and getting my employer (seed stage startup)
| to spring for it for the entire team was an easy sell.
|
| Yeah it's pretty dumb most of the time. But I know that, and I
| don't use code from it without carefully checking it out and
| modifying it. But it's still a huge help. Just the time saved
| writing tests alone pays for it. And I've had a few spooky
| experiences where it feels like it knows the bug fix before I
| do. Think of it as a smarter auto-complete.
|
| The technology has a long way to go, but I completely disagree
| with Kite here. It's already good enough to pay for. If my
| company didn't pay for it, I would. I already pay for
| JetBrains, and it costs more than Copilot. I would give up
| JetBrains before I give up Copilot.
|
| My guess here is Kite positioned themselves as a free
| alternative to Copilot and then couldn't monetize. There very
| likely is more to it though.
| jrsj wrote:
| Kite has been around for a lot longer, if anything Copilot
| was Github copying them
| esperent wrote:
| I don't think it's reasonable to say either was copying. AI
| assisted tooling is obvious and people have been waiting
| decades for the tech to reach a point where they can build
| these tools. Kite tried to get in early - too early
| probably - but even if they were the very first they didn't
| invent the idea.
| morelisp wrote:
| How are you validating the quality of its tests? Are you
| trying any mutations, checking branch coverage, etc.?
| lolinder wrote:
| > Just the time saved writing tests alone pays for it.
|
| This, so much. My code since using Copilot is easily ten
| times better tested than it was before, and I wasn't
| especially lazy when it comes to testing.
|
| Given 1-2 hand-written unit tests, Copilot can start filling
| in test bodies that correctly test what's described in the
| function name. When I can't think of any more edge cases,
| I'll go prompt it with one more @Test annotation (or
| equivalent in another language) and it will frequently come
| up with edge cases that I didn't even think of and write a
| test that tests that edge case.
|
| (One great part about this use case for those who are a
| little antsy about the copyright question is that you can be
| pretty darn confident that you're not running a risk of
| accidental copyright violation. I write the actual business
| logic by hand, which means copilot is generating tests that
| only interact with an API that _I_ wrote.)
| jascination wrote:
| Out of interest, how are you using it to write tests? Do you
| just write "make a test for functionX" or something?
|
| (Don't have much experience with it)
| mattwad wrote:
| The best part about it for me is just the Intellisense (in
| Typescript). I'm using it on probably 3/5 lines that I
| write as a smarter version, but I rarely use it to do more
| than finish the current line I am writing.
| lolinder wrote:
| You just write the name of the test. I'm doing this a lot
| and Rust right now for a toy language, and I can write
| something like: #[test] fn
| adds_two_numbers_correctly()
|
| Copilot will produce a test body that feeds an addition
| expression into my interpreter and validates that the
| output is what is expected.
| satvikpendem wrote:
| Yes, if you show an example, or even have the test file
| open, it will make the other tests for you.
| dboreham wrote:
| I wonder if this says something about the nature of test
| code?
| djbusby wrote:
| Sure, it's been loads of boilerplate since forever.
| premun wrote:
| It is amazing for typing out mock data. Say you're testing
| parsing of XML - it can easily suggest the the assertions
| over the data parsed from the XML. Example test that was
| 95% coming out of Copilot:
| https://github.com/dotnet/arcade-
| services/blob/61babf31dc63c...
|
| It also predicts comments and logging messages amazingly
| well (you type "logger." add 7/10 times get what you want,
| sometimes even better), incorporating variables from the
| context around. This speeds up the tedious parts of
| programming when you are finalizing the code (adding docs +
| tracing).
|
| Honestly, Copilot saves me so much time every week while
| turning chores into a really fun time.
| didibus wrote:
| The tool just has to be very well integrated and easy to use.
| That's why copilot is seeing adoption, because Microsoft owns
| VSCode and has built a very simple integration of Copilot into
| VSCode.
|
| That said, I'm not even sure VSCode or Copilot is lucrative, if
| it wasn't owned by Microsoft, could they both be sustainable
| businesses?
| grepLeigh wrote:
| I'd be curious to hear about services/tools developers _do_ pay
| for. The diagnosis that developers do not pay for tools seems
| off to me.
|
| A few tools that I put on the company card when I worked at a
| Big Tech Co as an IC:
|
| * DataGrip (Jet brains)
|
| * Colab Pro (Google)
|
| * Postman Pro
|
| These were all small $ enough where I didn't need to justify
| the expense. It was just assumed that if I thought the tool was
| worth the $, it was.
|
| For more expensive purchasing decisions, there was a longer
| purchasing/approval process. But the expense would have to be
| 5-6 figures per year before hitting this barrier.
| make3 wrote:
| Copilot is really great. Kite is garbage, & they have
| absolutely zero consumer trust from all the bullshit they did
| luckylion wrote:
| > Developers are somewhat reluctant to pay for tools but I
| think you can get them to pay for things that are worth it.
| I've been paying for code editors for years.
|
| Especially when you don't market to developers in general, but
| freelancers/contractors specifically. It might be hard to sell
| to salaried developers (they'll buy because it's nicer to work
| with good tools), but it's easy to sell tooling to anyone who
| makes more money when they get more done.
| arcturus17 wrote:
| Loads of devs I've spoken to, from junior to principal level,
| absolutely love Github Copilot though. Don't know who is paying
| it for them, nor if Kite was significantly worse, but I think
| that at least Copilot has a brilliant future ahead of it.
| candiddevmike wrote:
| Until Microsoft grants Copilot users blanket protection over
| copyright claims from Copilot generated code, I wouldn't even
| think of touching it.
| epolanski wrote:
| Same.
|
| Never ever I'm risking breaking copyright, and I also don't
| like Microsoft not including their own code in the model.
| Jenk wrote:
| We may well find out the answer to that when this
| lawsuit[0] concludes.
|
| [0]: https://www.theverge.com/2022/11/8/23446821/microsoft-
| openai...
| candiddevmike wrote:
| If Copilot generates the exact same code as the source, I
| don't see how that process could be exempt, it's like
| using the clipboard on your PC with extra steps.
| urthor wrote:
| It's interesting, the ones I've spoken to are _extremely_
| suspicious.
|
| "GitHub Copilot blocks your ability to learn." Is a common
| refrain.
|
| I don't see ANY industry-wide consensus on whether GitHub
| Copilot truly helps developers right now.
|
| The only scenario I can get anyone to agree on is generating
| templates. Aka, JSON or CSS files that you then edit.
| hanselot wrote:
| throwaway675309 wrote:
| It doesn't block your ability to learn any more than any
| auto suggestion systems. But I guess it depends on what you
| value in terms of learning, for me, copilot allows me to
| focus on the larger architectural problems while not having
| to worry about the exact syntax of certain things (DSL
| query language, middleware express, typescript def
| annotations, etc).
|
| Every time I don't have to context switch to look up some
| technical errata in my browser is a complete win for me.
| satvikpendem wrote:
| I use it. It's good for boilerplate code or basically
| anything where you can avoid looking at the docs. For
| example, I was writing an ML training loop and it correctly
| filled in the rest of the function after I wrote the first
| few lines. The code is basically what's in the pytorch
| docs, just fit to my model and scenario.
| dreamyfigment wrote:
| I _love_ Copilot but the only reason I use it is because I
| qualify under their open source developers program, I just
| can't justify paying $10/month for it.
| robertlagrant wrote:
| How much do you love it, then? What would you pay, if it's
| less than $10/mo?
| didibus wrote:
| It saves a bit of time, but doesn't seem to make a
| difference on time to market of features, products,
| improvements or bug fixes.
|
| In my experience, it's a quality of life improvement, but
| the things that dictate actual time to market is
| bottlenecked by things that aren't solved by copilot,
| such as overall design, decision making, requirements
| gathering, code structure/architecture, solution
| ideation, user acceptance, infrastructure setup, etc.
|
| I think if it eventually could help with those other
| tasks, you'd see time to market gains, and that would
| start to make it really valuable.
| [deleted]
| ShamelessC wrote:
| Your anecdote is trivially rebutted with another. I tried
| Github Copilot _more than twice_ (gasp), and now pay 10$/month
| for it. Happily.
| candiddevmike wrote:
| How does your workplace/compliance officer feel about you
| using it?
| ShamelessC wrote:
| Workplace? I think I remember those...
| jrpt wrote:
| That's why I tried it twice. I've been hearing people say
| they liked it. But I haven't found it very helpful, and often
| distracting, so I ended up turning it off. I'll probably try
| again next year when the models are improved to see if I feel
| any differently then.
| ShamelessC wrote:
| Yeah, I understand. I just see a lot of people on here who
| seem to be deliberately looking for reasons not to like
| Copilot.
|
| You don't fit that stereotype, of course. So feel free to
| ignore the following.
|
| Developer tools have learning curves. One doesn't simply
| open vim/emacs for the first time with a full understanding
| of how to use it (or why it's a good tool to use, even).
| Historically, we have had _no_ problem with the steepness
| of this curve. But, when it comes to Copilot, there's a lot
| of "tried it and it output an obvious bug! how did this
| make it past quality assurance?? such a liability!" Just
| very reactionary and all-that.
|
| Anyway, sorry for the toxic response.
| epolanski wrote:
| Risking breaking copyright and not supporting a model of
| code laundering isn't exactly looking for reasons.
| ivalm wrote:
| I pay for github co-pilot. It seems surprisingly bad for
| typescript and excellent for python.
| cowmix wrote:
| I tried it. Ironically, it was pretty good for Powershell and
| so so for Python (in my case at least).
| refulgentis wrote:
| This is a very self-serving recap: "we were too early and we're
| still too early and Copilot proves it because its not 10x": it is
| 10x, sorry.
| throwaway675309 wrote:
| Yeah having tried kite years ago, copilot absolutely destroys
| them in terms of helpful suggestions both at a line level and
| at a code block level. Its contextual awareness of surrounding
| code is also fantastic.
|
| Now whether or not that's due to the fact that copilot had the
| financial resources to train a significantly superior ML model
| is another question, but throwing shade at copilot is a fairly
| transparent move.
| vagab0nd wrote:
| Yeah, I was very confused by that paragraph. Copilot is not
| perfect, but it's "good enough" that I'm happily paying $10/mo.
| nnoitra wrote:
| rockzom wrote:
| > Our 500k developers would not pay to use it.
|
| lol
| dopeboy wrote:
| From one founder (of a much smaller startup) to another: respect
| for writing this. It probably wasn't easy but the fact that you
| took the time to do it and share learnings so that the next
| startup in the space can benefits speaks volumes about y'all.
| dmarlow wrote:
| I'm confused.
|
| "we were 10+ years too early to market, i.e. the tech is not
| ready yet."
|
| "Our diagnosis is that individual developers do not pay for
| tools."
|
| "We built the most-advanced AI for helping developers, but it
| fell short of the 10x improvement required to break through
| because today's state of the art for ML on code is not good
| enough."
|
| Sounds like you know why people didn't pay for it. If it truly
| did make people as productive as you claim, it would have sold
| like hot cross buns on a cold day.
| [deleted]
| make3 wrote:
| plus, again, they had zero developer trust because of all the
| ultra shady stuff they did
| victorvosk wrote:
| I find co-pilot useful when I am working with a language I am not
| familiar with but I imagine that isn't the case for most
| developers working their day to days. I see ML and AI in dev as
| more of a code generation tool. Describe something large in a
| prompt, get a bunch of code. Then a dev can run through it like a
| code-review, making changes and tweaking it to suite the need of
| the client/business.
| tomcam wrote:
| Mad props for facing the truth and unsparingly admitting
| responsibility. So so rare.
|
| Whoever wrote this will go far.
| oofbey wrote:
| I think they're spot on to say they were too early. But their
| analysis of the current state is pretty tainted by their personal
| situation.
|
| Many people I know find copilot extremely helpful. I think tools
| like it are about to become extremely important to the
| productivity of everyday developers. I seriously doubt it will
| take $100M to develop. The company Kite might have needed $100M
| to get there, but I bet you a few smart people working evenings
| in their garages can get there too.
|
| Also the "nobody pays for dev tools" line is pretty obviously a
| weak excuse. Github is a developer tool that was worth $7B+. The
| truth was they just didn't provide _enough_ value to get people
| to pay for it. That's clearly true, and goes along with their
| idea that they were too early. Not that the problem is
| impossible.
| ACV001 wrote:
| It failed because they did it exactly in reverse of how it should
| have been done. First they assembled the team, then they outlined
| the product then marketing and then only then they realized
| nobody would pay for that. You're supposed to first sell your
| product and then build it! I wonder whether anyone raised this
| issue in the early stage...
| yieldcrv wrote:
| correction: sell your product first and then hire another firm
| to build it to your specifications
| renewiltord wrote:
| I pay for Copilot. Integrates with my neovim and my Jetbrains
| IDEs. I love it. Great stuff honestly.
|
| My favourite use is at the command line. It's great!
|
| I pay for it myself and use it in all sorts of contexts.
| inglor wrote:
| > As of late 2022, Copilot has a number of issues preventing it
| from being widely adopted.
|
| I see CoPilot all around me and it's generally well regarded and
| pretty widely adopted given how new it is.
|
| Is there any data for this statement you can share?
|
| (Thanks for working on kite and good luck!)
| solarkraft wrote:
| I never cared a lot about Kite. But oh boy, suddenly it's the
| only product in a category I do care about! Thank you!
| ShamelessC wrote:
| What category is that? Open source code generation?
| satvikpendem wrote:
| If so, there are others too, like Fauxpilot, and the
| Salesforce one, both are open source I believe.
| jokethrowaway wrote:
| As a fellow failed startupper, this blog post reads like any
| other failed startup goodbye post.
|
| Sure, your people were great but they didn't innovate enough to
| make an attractive product (granted, AI code autocompletion is
| hard - I doubt we'll get something I'd be happy to pay before we
| reach GAI and we'll be all out of a job by then).
|
| Oh and the "It's not the tech fault which is amazing, it's just a
| sales pipeline issue!"
|
| Look, I understand caring about your employees and I said the
| same BS when my company failed trying to shift all the blame on
| me and not on my team. When you are in a startup it's everyone's
| job to say "hey, btw, what we want to do will suck because the
| tech is not there".
|
| If you see something raise it and try to pivot, or you'll be out
| of a job with worthless grades.ss in
|
| Maybe you could have cut your losses earlier on.
| throwthere wrote:
| > Our diagnosis is that individual developers do not pay for
| tools.
|
| Throwing salt on the wound here but that's just false. I mean,
| there's copilot and it's alternative that I can't think of the
| name right now. more broadly there's Jet brains ides, visual
| studio, Productivity apps, etc. look at product hunt or appsumo
| or popular show hns. Devs pay for tools, just not Kite.
|
| Edit: I should clarify, enough devs pay for tools to make the
| market sustainable. Not all devs pay for tools.
| svnt wrote:
| You think copilot is self-sustaining/profitable?
| throwthere wrote:
| > You think copilot is self-sustaining/profitable?
|
| Yes.
| dustingetz wrote:
| copilot hit $40M ARR in the first month: 400k subscribers *
| $100/yr
|
| https://www.ciodive.com/news/github-copilot-microsoft-
| softwa...
| marcinzm wrote:
| How much of that is paid for by devs and how much is paid for
| by their employers?
| vkou wrote:
| Getting my manager to spring for an IntelliJ license instead
| of a free Eclipse was the easiest thing in the world.
|
| (That manager getting the purchase order approved through
| corporate took months and months, but that's neither here nor
| there.)
| jmnicolas wrote:
| Frankly I don't pay for tools. Money is tight at work and at
| home, so if it's not free it ain't happening.
| legerdemain wrote:
| Kite made me a very good (for a startup) job offer a few years
| back. They had a very friendly and welcoming bunch of people, and
| even Adam, the founder, came off as a typical human being in
| conversation. Easily the best job I've ever turned down, even
| knowing that Copilot would eat their lunch a year or two later.
| didip wrote:
| This is the first time I heard of Kite and I frequent HN a lot.
|
| Maybe they should have spent more budgets on marketing.
|
| That said, I agree that no one wants to pay for developer
| productivity. The only exceptions are IDE and databases.
| peter_d_sherman wrote:
| >"First, we failed to deliver our vision of AI-assisted
| programming because we were 10+ years too early to market, i.e.
| the tech is not ready yet."
|
| I highly doubt that you failed! You blazed a trail forward for
| people in the future to follow. Financial success is not the same
| thing as taking a super tough problem to solve and then making
| inroads solving or starting to solve the many sub-problems (and
| their sub-problems) that invariably show up as a result of taking
| that path.
|
| >"Then we grew our user base. We executed very well here, and
| grew our user base to 500,000 monthly-active developers, with
| almost zero marketing spend."
|
| That's extremely impressive in my book! (By comparison, I failed
| to get 2 users -- for one of the apps I built -- and that was
| _with_ marketing spend! <g>)
|
| >"Then, our product failed to generate revenue. Our 500k
| developers would not pay to use it."
|
| You might mean that there may have been an issue with
| communicating the VALUE of your product such that users would
| "see" (magical word, "see" -- "percieve", "understand", "observe
| in a way that you do") the VALUE of it -- such that they would be
| willing to equally-and-oppositely exchange their money for that
| VALUE...
|
| Finally:
|
| I do not think that you failed, and _you have no reason to
| apologize to your investors, customers, employees and others._
|
| You pushed the envelope -- and you created great value for future
| generations who will no doubt benefit from your pioneering steps
| in this gargantuan undertaking.
|
| Well done -- and I think more people should appreciate you for
| that!
| rubiquity wrote:
| > First, we failed to deliver our vision of AI-assisted
| programming because we were 10+ years too early to market, i.e.
| the tech is not ready yet.
|
| That's not the same thing as being too early to the market. That
| simply means you didn't have a solution capable of solving a
| problem.
| dynamicwebpaige wrote:
| "While we built next-generation experiences for developers, our
| business failed in two important ways.
|
| First, we failed to deliver our vision of AI-assisted programming
| because we were 10+ years too early to market, i.e. the tech is
| not ready yet.
|
| We built the most-advanced AI for helping developers, but it fell
| short of the 10x improvement required to break through because
| today's state of the art for ML on code is not good enough. You
| can see this in Github Copilot, which is built by Github in
| collaboration with Open AI. As of late 2022, Copilot has a number
| of issues preventing it from being widely adopted.
|
| The largest issue is that state-of-the-art models don't
| understand the structure of code, such as non-local context. We
| made some progress towards better models for code, but the
| problem is very engineering intensive. It may cost over $100
| million to build a production-quality tool capable of
| synthesizing code reliably, and nobody has tried that quite yet.
|
| Nonetheless, we could have built a successful business without
| 10x'ing developer productivity using AI, and we did not do that.
|
| We failed to build a business because our product did not
| monetize, and it took too long to figure that out."
| acyou wrote:
| Value generation in software doesn't equal profit generation. Is
| it a flawed business model to pursue growth first and profit
| later? No, as long as there is a good plan to get that future
| profit. If the 500k developers weren't driving business spending
| decisions enough to pay for Kite, either it isn't particularly
| useful or it's a sign of the times. I'm guessing from the rest of
| the context it's the former, no one seems to be crying out that
| this is a great product that will be widely missed. This sort of
| failure is good and a good decision by the business leaders. It
| keeps our economy healthy, you want the real winners to win, and
| not every bet works out.
| mistrial9 wrote:
| code repos
|
| https://github.com/orgs/kiteco/repositories
| 29athrowaway wrote:
| Once it's done, your product manager will push any improvements
| to the bottom of the backlog.
| netik wrote:
| Ten years too early? no.
|
| They got wiped out by microsoft, github copilot, and litigation
| issues around AI provided code.
| TheRealPomax wrote:
| I think you'll find that trying to make copilot in 2014 was
| definitely 10 years too early. Hell, even Copilot is a few
| years to early at the moment.
| selimnairb wrote:
| I would never willingly pay for AI coding tools. Why should I
| help improve a product that has a chance of putting myself or my
| fellow software developers out of work in the future?
___________________________________________________________________
(page generated 2022-11-20 23:00 UTC)