Asking whether people actually use open source coding assistants. It’s kinda funny that the author says their model acts passive aggressively, as it adds things like # this is a comment when asked to add comments to code results.
But it turned out that the first comment addressed a big issue, which is that the OP was using a base model instead of an instruction tuned model.
Some comments on the open source coding assistant space:
- The extension does a lot of heavy lifting, provides context to the model, metrics such as where in the project you want to autocomplete and other stuff… there have been many attempts to solve this, but it’s really difficult. There’s not much money in this (if at all), and a lot of teams lose focus, have to work on something else to make money, or go work for someone else. Rift was really promising (they went the language server route), but that’s been dormat for 4-5 months now. Continue is considered the best for chat-style stuff, and they have a lot of integrations, tabby is also recommended for completion.
- One of the continue.dev developers: “What models did you try and what provider(s) were you using? I’d like to get this fixed, so that no else runs into it (I am an author of Continue)”
- “Thank you! We’ll try to reproduce and address these issues. Continue automatically attempts to detect the correct prompt format based on the model value that you provide, so I am guessing something is going wrong there. We enable users to customize the chat template when it doesn’t work, but it should have worked for Llama and Mistral/Mixtral models”
- Of course, Reddit is not an official channel for bug reports. But I’m sure that they way this happens is through casual browsing of the sub, which is a much more organic way for the developers to interact with other community members.
- “Thank you! We’ll try to reproduce and address these issues. Continue automatically attempts to detect the correct prompt format based on the model value that you provide, so I am guessing something is going wrong there. We enable users to customize the chat template when it doesn’t work, but it should have worked for Llama and Mistral/Mixtral models”
- The Continue dev above also wrote their own comment: “I am one of the authors of Continue. I use DeepSeek Coder 33B for chat (using the Together API) and StarCoder 3B for tab autocomplete (using Ollama on my Mac). I find them to be quite useful As the top comment mentions, it looks like the reason you are seeing useless responses is because you are using a base model instead of an instruct model. If you can share your config.json, I could tell you if adjusting your settings also might help, though this might be easier / faster if we chat on the Continue Discord”
- Others in the threads ask the dev whether paying for a GPT API or paying for GitHub Copilot is more worth it in the end. They answer that the nice middle-ground might be to pay for an API-access model for code generation and using a local, base model for code completion.
- Another follow-up comment suggested whether letting the two modules, chat and autocompletion, be aware of each other’s outputs. This inspired another Continue dev who was following the thread:
- “Actually an awesome idea to have chat output fed into autocomplete. We’re already set up to use the copy buffer, but this is even better in many ways! (I’m another Continue author, came across this, was excited : )”
- Others share a bunch of models ranging from non-coding oriented ones to coding-specialised models.
- One other notes that they haven’t been able to find a model that is actually useful, because they work on a codebase that requires some amount of mental model around it. Others agree, saying that they mostly use models to learn about well known API endpoints that they aren’t familiar with.
Notable point in this post is that this is an artifact that requires multiple models to work correctly, and it doesn’t just depend on one type of model. The two models also vary in size, so people may be using a mixture of an open model and a closed model.