Skip to content

Encoder buffer grows exponentially #177

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
pnemtsev opened this issue Sep 21, 2017 · 1 comment
Closed

Encoder buffer grows exponentially #177

pnemtsev opened this issue Sep 21, 2017 · 1 comment

Comments

@pnemtsev
Copy link

package main

import (
	"fmt"

	jsoniter "github.com/json-iterator/go"
)

type NopWriter struct {
	total int
}

func (w *NopWriter) Write(p []byte) (n int, err error) {
	fmt.Println("len", len(p))
	fmt.Println("cap", cap(p))
	w.total += len(p)
	fmt.Println("total", w.total)
	fmt.Println()
	return len(p), nil
}

func main() {
	jsoniter.NewEncoder(new(NopWriter)).Encode(make([]int, 10000000))
}

This code produces following output:

len 493
cap 512
total 493

len 1006
cap 1024
total 1499

len 2030
cap 2048
total 3529

len 4078
cap 4096
total 7607

len 8174
cap 8192
total 15781

len 16366
cap 16384
total 32147

len 32750
cap 32768
total 64897

len 65518
cap 65536
total 130415

len 131054
cap 131072
total 261469

len 262126
cap 262144
total 523595

len 524270
cap 524288
total 1047865

len 1048558
cap 1048576
total 2096423

len 2097134
cap 2097152
total 4193557

len 4194286
cap 4194304
total 8387843

len 8388590
cap 8388608
total 16776433

len 3223568
cap 16777216
total 20000001

As you can see, the inner Encoder buffer grows exponentially on each Write. I want to set a fixed capacity of the inner buffer to limit the memory allocated by Encoder.

@taowen
Copy link
Contributor

taowen commented Sep 21, 2017

should be fixed now

@taowen taowen closed this as completed Sep 21, 2017
zhenzou pushed a commit to zhenzou/jsoniter that referenced this issue Feb 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants