Question classifier proves to be ineffective

[tigerun]

Self Checks

• This is only for bug report, if you would like to ask a quesion, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• Pleas do not modify this template :) and fill in all the required fields.

Dify version

0.6.7

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

After many times of attempts, it is evident that the 0.6.7 version of the question classifier abysmal intellect for problem categorization, frequently failing to identify issues accurately.

✔️ Expected Behavior

Urgent rectification

❌ Actual Behavior

Question classifier proves to be ineffective

[dosubot]

To address the ineffectiveness of the question classifier in Dify version 0.6.7, consider the following targeted strategies:

1. Customize Prompts for Classification: Tailor the QUESTION_CLASSIFIER_SYSTEM_PROMPT and QUESTION_CLASSIFIER_USER_PROMPT templates to better suit your classification needs. This involves refining the job description, task details, and constraints within the system prompt and accurately defining the input text, categories, and classification instructions in the user prompts.

2. Implement Keyword Extraction: Enhance your classifier's accuracy by extracting and analyzing keywords from the input text. This approach can significantly improve problem categorization.

3. Utilize Classification Instructions: Adding classification instructions can provide additional context or criteria for classification, aiding in the refinement of accuracy.

4. Iterative Testing and Refinement: Use real examples to test your classifier, comparing its performance to expected outcomes and adjusting your classification logic based on these comparisons.

5. Adjust Based on Feedback: Continuously refine your system and user prompts, as well as classification instructions, based on feedback to improve classifier performance.

Ensure your Docker environment is correctly set up and all dependencies are properly installed. If issues persist, providing specific examples of misclassified questions or any error messages could help in diagnosing and resolving the problem more effectively.

Details

Sources

• en/features/workflow/nodes/question-classifier.md
• api/core/workflow/nodes/question_classifier/template_prompts.py
• api/core/workflow/nodes/question_classifier/question_classifier_node.py
• en/getting-started/install-self-hosted/environments.md
• en/getting-started/install-self-hosted/docker-compose.md
• en/features/workflow/node/question-classifier.md
• api/core/rag/datasource/keyword/jieba/jieba_keyword_table_handler.py
• api/core/rag/datasource/keyword/jieba/stopwords.py

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

[YidaHu]

What LLM do you use? A lot of it has to do with LLM performance. You can provide more information.

[Theysua]

Can you provide more evidence for the community to further investgate this issue please?
It will be nice if you can provide an example of the problem you are encountering
Thanks!

[tigerun]

Deploying the Qwen 14b model locally on Xinference. If I use qwen-max-0403,everything looks ok

[tigerun]

Back to 0.6.6 version, the Question classifier still be ineffective

[YidaHu]

@tigerun I think it has to do with the LLM capability and the service is fine

[tigerun]

0.6.6 everything is ok，but when update to 0.6.7 ，Question classifier look like a idiot，do nothing except update.

[tigerun]

To resolve the issue with the question classifier version 0.6.7, follow these steps:

Locate services.api.image in your docker-compose.yaml file.
Replace the current image tag langgenius/dify-api:0.6.7 with langgenius/dify-api:897e07f63954d39a260d771a2d01b7932d005645
Similarly, find services.web.image within the same file and replace langgenius/dify-api:897e07f63954d39a260d771a2d01b7932d005645
After making these changes, run the command docker compose up -d to restart your services. By doing this, the issue with the question classifier should be resolved.