Socket disconnects on big payload.

[faisal-anwar825]

I am using socket.io to create a realtime chat which includes attachments as well. I am sending files in Base64 form. but the issue is that if file size is around 500 to 600 KB. it gets send successfully but when file size increases more than that the sockets disconnects as soon as i send the message, with the error

Transport close: The connection was closed

i have tried increasing the maxHttpBufferSize to 1MB and 100MB but still files > 7, 800 KBs wont upload.

Socket.IO server version: 4.7.2

This is the code from my server

`

    import {
      WebSocketGateway,
      WebSocketServer,
      SubscribeMessage,
      OnGatewayConnection,
      OnGatewayDisconnect,
    } from '@nestjs/websockets';
    import { Server, Socket } from 'socket.io';
    import { SocketsService } from '../sockets.service';
    import { WsGuard } from '@app/common';
    import { ChatDto } from '../dto/chat.dto';
    import { Injectable, UseGuards } from '@nestjs/common';
    
    @Injectable()
    @WebSocketGateway({ cors: '*:*' })
    export class ChatGateway implements OnGatewayConnection, OnGatewayDisconnect {
      @WebSocketServer() public server: Server;
      users = 0;
    
      constructor(
        private readonly socketService: SocketsService,
      ) { }
    
      async onModuleInit() {
        // Call the configureWebSocketServer method when the module has been initialized
        this.configureWebSocketServer();
      }
    
      async handleConnection(client: Socket) {
        // A client has connected
        this.users++;
    
        client.removeAllListeners('disconnect');
    
        client.on('disconnect', () => this.handleDisconnect(client));
        // Notify connected clients of current users
        this.server.emit('users', this.users);
      }
    
      afterInit(server: Server) {
        this.socketService.socket = server;
      }
    
      async handleDisconnect(client: Socket) {
        // A client has disconnected
        this.users--;
        client.removeAllListeners('disconnect');
    
        client.on('disconnect', () => this.handleDisconnect(client));
        // Notify connected clients of current users
        this.server.emit('users', this.users);
      }
    
      @SubscribeMessage('chat')
      @UseGuards(WsGuard)
      async onChat(client: Socket, payload: ChatDto) {
        // expect to receive the file here
      }
    
      async configureWebSocketServer() {
    
        this.server.engine.opts.maxHttpBufferSize = 4e8;
      }
    }

`

[Malikrehman00107]

@rauchg

please !

[Malikrehman00107]

@dantehemerson

[darrachequesne]

Hi! I see you are using NestJS, did you increase the body parser size limit?

Reference: https://docs.nestjs.com/faq/raw-body#body-parser-size-limit

[faisal-anwar825]

Yes i have set bodyparser limit as well. this is my main.ts file

`

  import { NestFactory } from '@nestjs/core';
  import { SocketsModule } from './sockets.module';
  import * as cors from 'cors';
  import * as bodyParser from 'body-parser';
  
  async function bootstrap() {
    const corsOptions = {
      origin: [
        // cors option
      ],
    };
    const app = await NestFactory.create(SocketsModule, { cors: true });
    app.enableCors(corsOptions);
    app.use(cors());
    app.use(bodyParser.json({ limit: '50mb' }));
    app.use(bodyParser.raw({ limit: '50mb' }));
  
  
      await app.listen(3000);
  }
  bootstrap();

`

[Malikrehman00107]

@darrachequesne

[llaakso]

Large packets will cause timeout so you need to split the data to multiple packets. This approach is also good for not to block event loop which often happens with large data. I believe this should be addressed in socket.io level as devs are still wondering this issue. If nothing else, just console warning when the packet is likely too big.

[faisal-anwar825]

That's understandable for maybe files that are huge in size but a 700KB or 800KB shouldn't need breaking down. no?

[Malikrehman00107]

@llaakso are you ref like this?

[darrachequesne]

@faisal-anwar825 @Malikrehman00107 unfortunately I was not able to reproduce the issue: https://github.com/socketio/socket.io/tree/main/examples/nestjs-example

src/views/index.hbs

socket.on("connect", () => {
  // [...]
  socket.emit("big file", "a".repeat(1e7)); // 10 MB
});

src/events/events.gateway.ts

@WebSocketGateway({
  maxHttpBufferSize: 1e8
})
export class EventsGateway {
  // [...]
  @SubscribeMessage('big file')
  handleBigFile(@MessageBody() data: string): void {
    console.log("received", data.length);
  }
}

Same with the onModuleInit hook:

@WebSocketGateway({})
export class EventsGateway {
  // [...]
  async onModuleInit() {
    this.server.engine.opts.maxHttpBufferSize = 1e8;
  }

  @SubscribeMessage('big file')
  handleBigFile(@MessageBody() data: string): void {
    console.log("received", data.length);
  }
}

I believe this should be addressed in socket.io level as devs are still wondering this issue.

@llaakso I guess we could indeed split big payloads into several chunks. Maybe as a custom parser? What do you think?

Reference: https://socket.io/docs/v4/custom-parser/

[Malikrehman00107]

@darrachequesne we are still facing the issue so I belive the idea from @llaakso is a good one!

we are going to try and will surely update in case it works.

best