I recently built a large language model chat interface, but traditional request methods only transfer data to the frontend after the model completes its entire output, causing users to think the interface or API has frozen.

Additionally, due to stateless microservices, using WebSockets for long connections requires handling reconnect logic, which introduces extra complexity. SSE supports reconnection after disconnection by default, is based on the HTTP protocol, and is lightweight and simple for transmitting text data.

I also encountered some pitfalls during implementation; please read the code comments below.

JAVA

Issues encountered:

  • Nginx caches messages, preventing them from streaming. Nginx caching needs to be disabled to enable direct streaming.
  • new SseEmitter() disconnects after 30 seconds by default, so new SseEmitter(0L) needs to be specified to allow unlimited message push duration.
  • SSE can only transmit single-line data and cannot recognize spaces and line breaks, so spaces and carriage returns need to be replaced with placeholders.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
@ResponseBody
public SseEmitter handleSseRequest(HttpServletRequest request, HttpServletResponse response) {
  	// 关闭Nginx缓存，直接推流
    response.setHeader("Cache-Control", "no-cache");
    response.setHeader("X-Accel-Buffering", "no");
    // 流式事件流响应
    response.setContentType(MediaType.TEXT_EVENT_STREAM_VALUE);
  	// 默认30秒断连，改为0则不限制
    SseEmitter emitter = new SseEmitter(0L);

    // 每100毫秒推送一个消息，总共30个消息
    Flowable.interval(100, TimeUnit.MILLISECONDS)
            .take(30)
            .map(i -> "SSE COUNTER: " + i + "&#92n&#92n")
      			.subscribe(
              data -> {
                  try {
                      emitter.send(SseEmitter.event().data(data));
                  } catch (Exception e) {
                      emitter.completeWithError(e);
                  }
              },
              emitter::completeWithError,
              emitter::complete
    				);
    return emitter;
}

Frontend

Issues encountered:

  • Since EventSource only supports the GET method, values can be passed via fields when establishing a connection.
  • SSE can only transmit single-line data and cannot recognize spaces and line breaks, so spaces and carriage returns need to be replaced with placeholders.
  • It is recommended to accumulate received data and continuously parse the full data into HTML to achieve real-time parsing like ChatGPT, improving the user experience.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
// 由于EventSource只支持get方式，可以建立连接时通过字段传值
let eventSource = new EventSource(target + "?content=" + encodeURIComponent(content));
let sseContent = ""
eventSource.onmessage = function(event) {
  let data = event.data

  try {
    let ssedata = event.data
    // SSE只能传输单行数据，且无法识别空格和换行，因此需要将空格和回车替换为占位符
    ssedata = ssedata.replaceAll(" ", " ").replaceAll("&#92n", "\n")
    // 累加接收数据
    sseContent += ssedata
    // 将Markdown数据持续解析为HTML，实现类似ChatGPT的返回实时解析，体感很好
    document.getElementById("msg_" + msgidx).innerHTML = convertMarkdownToHtml(sseContent)
  } catch(e) {
  }

  // 锁定div滚动条到界面底端
  let objDiv = document.getElementsByClassName("lite-chatbox").item(0);
  objDiv.scrollTop = objDiv.scrollHeight;
};

eventSource.onerror = function(event) {
  // 直接关闭即可，如网路异常则自动重连
  eventSource.close()
};