Reduce memory use for PULUMI_OPTIMIZED_CHECKPOINT_PATCH #11666
Conversation
Changelog[uncommitted] (2022-12-15)Bug Fixes
|
| cp.UntypedDeployment = sentinel | ||
| pattern, err := json.Marshal(cp) | ||
| if err != nil { | ||
| var jsonIterConfig = jsoniter.Config{SortMapKeys: true}.Froze() |
There was a problem hiding this comment.
I could go either way on this.
My gut says unless jsonIterConfig is mutable (Froze() indicates to me it isn't), I'd throw it in a function so no one tries to mutate it.
On the other hand, Froze() will make it apparent the minute anyone tries to mutate it (as long as they test their change), and putting this in a function will create 1 unnecessary allocation each time Write is called. Probably better to leave it as is.
There was a problem hiding this comment.
It's a strange lib. Froze() allocates state/cache/pools. This needs to be a module-level var I think for best results (reuse of the pools).
There was a problem hiding this comment.
SGTM! IMO module level variables is typically an anti-pattern, except when it isn't. Which is Golang in a nutshell... :\
This is a good example of when it isn't.
| return err | ||
| } | ||
|
|
||
| func (c *marshalUntypedDeployment) writeToStream(stream *jsoniter.Stream) error { |
There was a problem hiding this comment.
I have no comprehension of the incantation order here 😅 For example, what does WriteMore() do? Why is it necessary? Maybe a single line comment somewhere at the top of the method would help? Not a big deal either way, I feel like someone coming across this is going to have to read the API docs anyway.
There was a problem hiding this comment.
WriteMore writes a "," .. this is a pretty low-level API.
|
bors merge |
|
Build succeeded: |
Description
Fixes #11653
Inject fewer newlines, and per @blampe suggestion do not inject more than 1024 newlines even if we have more than 1024 resources. This limits Myers matrix to 1024x1024 max size which should be 8MB or so.
Promotes jsoniter from indirect to direct dependency. It's easier to work with for preserving indents and doing newline injection.
Checklist
make changelogand committed thechangelog/pending/<file>documenting my change