As we all know, there are many AI chat software on the market, most of which use SSE to push messages, and ChatGPT is no exception. Everyone knows that WebSocket can communicate in two directions, and naturally it can also push messages from the server to the client. SSE only pushes messages in one direction, so why is it used in AI chat?

What is SSE?

SSE is Server-Send Events.

SSE description on Wikipedia:

Server-Sent Events (SSE) is a server push technology enabling a client to receive automatic updates from a server via an HTTP connection, and describes how servers can initiate data transmission towards clients once an initial client connection has been established. They are commonly used to send message updates or continuous data streams to a browser client and designed to enhance native, cross-browser streaming through a JavaScript API called EventSource, through which a client requests a particular URL in order to receive an event stream.

Why use SSE instead of WebSocket?

The decision to use SSE (Server-Sent Events) or WebSocket depends on your specific requirements and context.

When to Use SSE:

  1. One-Way Communication: SSE is suitable for one-way communication from the server to the client. The server can push data to the client at any time. This is particularly useful for scenarios where you need to send real-time events or updates from the server to the client, such as news updates, stock quotes, weather forecasts, and more.
  2. Simplicity: SSE is easier to implement and use as it operates on top of the HTTP protocol and utilizes standard browser APIs. In most cases, you can establish an SSE connection with just a few lines of JavaScript code.
  3. No Extra Protocol: SSE doesn’t require additional protocol handshakes because it directly uses HTTP. This simplicity makes it easier to deploy in certain situations.

When to Use WebSocket:

  1. Two-Way Communication: WebSocket supports two-way communication, allowing both the client and server to exchange data. This is well-suited for applications that require real-time bidirectional interaction, such as online games, instant messaging, collaborative tools, and more.
  2. High Interactivity: WebSocket allows for more interactive exchanges, enabling complex message sharing and data transfer between the client and server. SSE is better suited for one-way event notifications.
  3. Low Latency: WebSocket typically offers lower latency because they are full-duplex connections, enabling immediate data transmission when needed. SSE may have slight latency in certain cases because it operates on one-way connections.

If you only need to push events or notifications from the server to the client and don’t require complex bidirectional communication, SSE may be the simpler choice. However, if you need real-time bidirectional communication and can manage some additional complexity, WebSocket may be better suited for your application.

Why does CharGPT favor SSE?

CharGPT’s inclination towards SSE (Server-Sent Events) can be attributed to its simplicity, its suitability for real-time text generation and communication, and the absence of a need for bidirectional communication. SSE enables CharGPT to seamlessly push real-time generated text content to clients, making it ideal for the development of real-time conversation systems or text-based interaction applications. While CharGPT can certainly utilize alternative communication protocols, SSE often proves to be more convenient in meeting its specific requirements for real-time text-based communication.

How to create SSE?

To set up Server-Sent Events (SSE), you need to establish an HTTP connection between the server and the client and use the SSE protocol to achieve real-time event push. Here are the key steps for setting up SSE:

Server-Side Implementation:

  1. Set HTTP Headers:

    In the server response, you need to set specific HTTP headers to indicate to the browser that the response will contain SSE data. For example, you need to set “Content-Type” to “text/event-stream” and typically set “Cache-Control” to “no-cache” to prevent browser caching of the response.

  2. Establish Connection:

    Create an HTTP endpoint that the client can connect to in order to receive SSE events. Typically, this is a specific URL (e.g., /sse).

  3. Send Events:

    On the server, you can write code to send SSE events. This is typically done by sending data to the client connection in the format of an SSE event stream. An SSE event stream consists of lines of text, each line including an event identifier, data fields, and so on. For example:

1
data: This is an SSE message

Note that each event ends with two newline characters.

Client-Side Implementation:

  1. Establish Connection:

  2. On the client side, you can use JavaScript to establish a connection to the SSE endpoint, usually by using the EventSource object. For example:

    1
    const eventSource = new EventSource('/sse');

  3. Listen for Events:

    You can add event listeners to the EventSource object to handle SSE events received from the server. For example:

    1
    eventSource.addEventListener('message', function(event) { console.log(event.data); });

  4. Handle Events:

  5. When the server sends SSE events, the event listener on the client will capture them, and you can perform the necessary actions within the listener.

  6. Close Connection:

  7. Once you no longer need the SSE connection, make sure to close it to release resources. For example:

    1
    eventSource.close();

These steps allow you to set up a basic SSE connection. Please note that SSE is typically used for one-way communication from the server to the client. If bidirectional communication is required, WebSocket may be more suitable for your needs. Also, consider factors such as security, error handling, and compatibility to ensure SSE functions correctly in your application.

Why is fetch used for data processing?

The traditional request method mainly relies on using Ajax to initiate requests to the backend. This method has an obvious feature: a request can only receive one response data. After receiving the single response result, the request ends. However, in some scenarios, we need to continuously receive data updates from the backend. In this case, we need to abandon the traditional Ajax method and choose some methods that are more suitable for real-time data updates.

Finally, we focused on two main APIs: EventSource and Fetch, which have different advantages and uses. So we compared the two APIs respectively.

Features EventSource fetch API
Compatibility Broadly supported, including Internet Explorer 8 and later Supported in newer browsers, not fully supported in Internet Explorer
Data format Only supports text data sent by the server, automatically converted to text Various data formats including text, JSON, Blob, etc. can be obtained
Error handling Automatically try to reconnect, you can listen to the ‘error’ event to handle errors There is no built-in retry mechanism, you need to manually handle errors and may need to retry
Streaming processing Supports simple processing of streaming data sent by the server Streaming processing is not directly supported, but the body attribute of the Response object can be used to obtain the streaming interface
CORS problem Restricted by the same origin policy, cross-origin loading cannot be performed unless the server is configured with appropriate CORS headers Not restricted by the same origin policy, data can be requested across origins, but the server needs to be configured with appropriate CORS headers
Flexibility Only GET requests can be sent, and string parameters can be concatenated Any type of request can be initiated. Flexible parameter passing

Through comparison, we can find that using fetch has more advantages, so our case below is based on fetch:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
import { useEffect, useState } from "react";
import "./App.scss";

function App() {
  const [chatText, setChatText] = useState("");

  const getRes = async () => {
    try {
      const res = await fetch("http://localhost:3000/sse", {
        method: "get",
      });
      const reader = res.body?.getReader();
      let text = "";
      while (reader) {
        const { value, done } = await reader.read();
        const chars = new TextDecoder().decode(value);
        if (done) {
          break;
        }
        const dataArray = chars.trim().split("\n\n");
        const jsonObjects = dataArray.map((data) => {
          const jsonString = data.substring("data: ".length);
          return JSON.parse(jsonString);
        });
        jsonObjects.forEach((item) => {
          text += item.content;
        });
        setChatText(text);
      }
    } catch (error) {
      console.log("error", error);
    }
  };
  useEffect(() => {
    getRes();
  }, []);
  return <div>{chatText}</div>;
}

When using fetch, we also need to pay attention to the getReader method. The getReader method is a way to process the response body. It returns a ReadableStreamDefaultReader object that can be used to read response data asynchronously. This method is typically used with large responses or streaming data so that the data is processed incrementally as it arrives, rather than loading the entire response data into memory at once.

By reading the return value of the stream through the read method, we can deconstruct the two values ​​value and done from the return value, but the value value is a Uint8Array, and we need to convert it into UTF-8 characters. The done value is mainly used to distinguish whether the reading is completed. During reading, done will return false. After all content is read, it will return true. We can use the done value to determine whether the response has ended.

When processing SSE, we also need to pay attention to the data format. The backend returns a data format that is a string starting with data. The content we need is the value corresponding to content, so we need to take some methods to get its value.

  1. The content data part {“content”:”…”} can be easily extracted using regular expressions, but the later maintenance and code readability are lower.
  2. Using JSON.parse, you need to convert the string into a standard json string first, and then into an object. If processed in this way, we can operate more flexibly, but if the data does not conform to the JSON format, a parsing error will be thrown. , which requires us to implement appropriate error handling in the code to prevent the application from crashing.

After obtaining the content, we only need to splice the strings each time and then re-render the page to achieve the typewriter effect.