Are we ready to embrace generative AI for software Q&A?
Stack Overflow, the world's largest software Q&A (SQA) website, is facing a significant traffic drop due to the emergence of generative AI techniques. ChatGPT is banned by Stack Overflow after only 6 days from its release. The main reason provided by the official Stack Overflow is that the...
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| 格式: | text |
| 語言: | English |
| 出版: |
Institutional Knowledge at Singapore Management University
2023
|
| 主題: | |
| 在線閱讀: | https://ink.library.smu.edu.sg/sis_research/8569 https://ink.library.smu.edu.sg/context/sis_research/article/9572/viewcontent/2303.04942.pdf |
| 標簽: |
添加標簽
沒有標簽, 成為第一個標記此記錄!
|
| 總結: | Stack Overflow, the world's largest software Q&A (SQA) website, is facing a significant traffic drop due to the emergence of generative AI techniques. ChatGPT is banned by Stack Overflow after only 6 days from its release. The main reason provided by the official Stack Overflow is that the answers generated by ChatGPT are of low quality. To verify this, we conduct a comparative evaluation of human-written and ChatGPT-generated answers. Our methodology employs both automatic comparison and a manual study. Our results suggest that human-written and ChatGPT-generated answers are semantically similar, however, human-written answers outperform ChatGPT-generated ones consistently across multiple aspects, specifically by 10% on the overall score. We release the data, analysis scripts, and detailed results at https://github.com/maxxbw54/GAI4SQA. |
|---|