Although DeepSeek-Coder-V2 achieves impressive performance on standard benchmarks, we find that there is still a significant gap in instruction-following capabilities compared to current state-of-the-art models like GPT-4 Turbo. This gap leads to poor performance in complex scenarios and tasks such as those in SWEbench.

Feb 2, 2024 · Also, I have tried deepseek-6.7b, mistral-7b and Mixtral-8x7b in the same set of CS questions and deepseek fared much worse than general models. For short bash scripts it was okay, but other models were the same. Also, for reasoning and doing some tasks with feedback loops Mixtral is the best simply because it tends to hallucinate less.

r/DeepSeek: Subreddit for the DeepSeek Coder Language Model

DeepSeek Coder comprises a series of code language models trained on both 87% code and 13% natural language in English and Chinese, with each model pre-trained on 2T tokens. We provide various sizes of the code model, ranging from 1B to 33B versions.

Nov 9, 2023 · If you can't get it running, give the GPTQ version a try in the text-generation-webui. (TheBloke/deepseek-coder-6.7B-instruct-GPTQ for example), I believe it works without issue. Also, if you have a powerful macbook, it runs great in LM Studio on OSX. I've heard the latest llama.cpp build runs it without issue as well.

Jan 27, 2024 · Our extensive evaluations demonstrate that DeepSeek-Coder not only achieves state-of-the-art performance among open-source code models across multiple benchmarks but also surpasses existing closed-source models like Codex and GPT-3.5. Furthermore, DeepSeek-Coder models are under a permissive license that allows for both research and ...

Jan 27, 2024 · Our extensive evaluations demonstrate that DeepSeek-Coder not only achieves state-of-the-art performance among open-source code models across multiple benchmarks but also surpasses existing closed-source models like Codex and GPT-3.5. Furthermore, DeepSeek-Coder models are under a permissive license that allows for both research and ...

Jun 23, 2024 · did you try deepseek-coder-v2:236b-instruct-q3_K_S (102GB) to see that works ? that would show some large version of deepseek-coder-v2 is ok and your computer is ok. which might indicate some corruption or bad packaging of deepseek-coder-v2:236b-instruct-q2_K. also, consider deleting & re-downloading deepseek-coder-v2:236b-instruct-q2_K.

Unfortunately not on the leaderboard yet. Codestral-22B could fit the gap between Llama 3 8B and DeepSeek-Coder-V2 quite nicely, if it had a permissive license. However, with the custom non-production license it’s only available from Mistral, at $1 / $3 for a million input / output tokens. That doesn’t make it a cheap model at all.

Nov 22, 2023 · It is the instruct model. You can see underneath the prompt box that it's the deepseek-coder-1.3b-instruct_Q5_K_s model. I used the prompting template in the model, and it slightly improved answers. But if I ask if to write some code, it almost never does and says something gibberish. Does your GPU/CPU quality affect the AI's output?