Failed to produce to topic default: Broker: Message size too large.

[kartikzheng]

Hello everyone, I reported the following error when using fluent-bit log collection: “Failed to produce to topic default: Broker: Message size too large.”

Kafka sets the maximum size of a message to 10 MB.
Message.max. bytes=10585760

The main configurations of Fluent bit are:

input-kubernetes.conf: |
    [INPUT]
        Name              tail
        Tag               kube.*
        Path              /var/log/containers/*.log
        Docker_Mode        On
        Docker_Mode_Flush  5
        Docker_Mode_Parser read_firstline
        Parser            docker
        DB                /var/log/flb_kube.db
        Mem_Buf_Limit     80MB
        #Buffer_Chunk_Size 1MB
        #Buffer_Max_Size   5MB
        Skip_Long_Lines   On
        Refresh_Interval  10

Both Buffer_Chunk_Size and Buffer_Max_Size have tried to set different sizes. The fluent bit version also tried to update to the latest version, but the above error was reported. May I ask you how to solve it? thank you.

[quanghungb]

hey @kartikzheng did you manage to resolve it? I do have the same issue with the 3.2.4 with different settings :(