Skip to content
This repository has been archived by the owner on Feb 13, 2024. It is now read-only.

flush: guarantee that all inflight messages are sent #352

Merged
merged 5 commits into from
Nov 16, 2022

Conversation

horacio-penya
Copy link
Contributor

Simple fix so flush waits for previous inflight messages to be sent.

Resolves #309

@edsonjab
Copy link
Contributor

edsonjab commented Nov 4, 2022

Hi @horacio-penya could you help us run this command please? standard --fix to organize the code and pass the test on circleci

@edsonjab
Copy link
Contributor

edsonjab commented Nov 7, 2022

Hi @horacio-penya now some tests are failing

yarn test
yarn run v1.22.15
$ standard && nyc ava --timeout=20s&& .buildscript/e2e.sh

Browserslist: caniuse-lite is outdated. Please run:
npx browserslist@latest --update-db

Why you should do it regularly:
https://github.com/browserslist/browserslist#browsers-data-updating
✔ expose a constructor
✔ require a write key
✔ create a queue
✔ default options
✔ remove trailing slashes from host
✔ overwrite defaults with options
✔ keep the flushAt option above zero
✔ enqueue - add a message to the queue
✔ enqueue - stringify userId
✔ enqueue - stringify anonymousId
✔ enqueue - stringify ids handles strings
✔ enqueue - don't modify the original message
✖ enqueue - flush on first message
✔ enqueue - flush the queue if it hits the max length
✔ enqueue - extend context
✔ identify - enqueue a message
✔ identify - require a userId or anonymousId
✔ group - enqueue a message
✔ group - require a groupId and either userId or anonymousId
✔ track - enqueue a message
✔ track - require event and either userId or anonymousId
✔ page - enqueue a message
✔ page - require either userId or anonymousId
✔ screen - enqueue a message
✔ screen - require either userId or anonymousId
✔ alias - enqueue a message
✔ alias - require previousId and userId
✔ isErrorRetryable
✔ dont allow messages > 32kb
✔ flush - flush when reaches max payload size (10.3s)
✔ flush - wont flush when no flush condition has meet
✖ ensure we can pass our own axios instance
✔ flush - skip when client is disabled (10.3s)
✔ flush - don't fail when queue is empty (10.3s)
✔ enqueue - skip when client is disabled (10.3s)
✖ enqueue - prevent flushing through time interval when already flushed by flushAt
✔ enqueue - flush after a period of time (10.3s)
✔ enqueue - don't reset an existing timer (10.3s)
✔ ensure other axios clients are not impacted by axios-retry
✔ flush - send messages (10.3s)
✔ flush - respond with an error (10.3s)
✔ flush - do not throw on axios failure if errorHandler option is specified (10.3s)
✔ flush - evoke callback when errorHandler option is specified (10.3s)
✔ flush - time out if configured (10.8s)
✔ ensure that failed requests are not retried forever (1.6s)
✔ ensure that failed requests are retried (7s)

3 tests failed

enqueue - flush on first message

/home/circleci/analytics-node/test.js:229

228: client.enqueue('type', {})
229: t.true(client.flush.calledOnce)
230:

Value is not true:

false

client.flush.calledOnce
=> false

client.flush
=> Function flush {
wrappedMethod: AsyncFunction [
undefined,
],
}

client
=> Analytics {
axiosInstance: Function wrap {
create: Function create {},
defaults: {
adapter: Function httpAdapter {},
env: Object { … },
headers: Object { … },
maxBodyLength: -1,
maxContentLength: -1,
timeout: 0,
transformRequest: Array [ … ],
transformResponse: Array [ … ],
transitional: Object { … },
validateStatus: Function validateStatus {},
xsrfCookieName: 'XSRF-TOKEN',
xsrfHeaderName: 'X-XSRF-TOKEN',
},
delete: Function wrap {},
get: Function wrap {},
getUri: Function wrap {},
head: Function wrap {},
interceptors: {
request: InterceptorManager { … },
response: InterceptorManager { … },
},
options: Function wrap {},
patch: Function wrap {},
patchForm: Function wrap {},
post: Function wrap {},
postForm: Function wrap {},
put: Function wrap {},
putForm: Function wrap {},
request: Function wrap {},
},
enable: true,
errorHandler: undefined,
flushAt: 2,
flushInterval: 10000,
flushed: true,
host: 'http://localhost:4063',
maxQueueSize: 460800,
path: '/v1/batch',
pendingFlush: null,
queue: [
{
callback: Function noop {},
message: Object { … },
},
{
callback: Function noop {},
message: Object { … },
},
],
timeout: false,
writeKey: 'key',
}

ensure we can pass our own axios instance

/home/circleci/analytics-node/test.js:714

713:
714: t.true(stubAxiosPost.called)
715: t.true(stubAxiosPost.alwaysCalledWith('https://my-dummy-host.com/test…

Value is not true:

false

stubAxiosPost.called
=> false

stubAxiosPost
=> Function functionStub {
callThrough: Function {},
callsArg: Function {},
callsArgAsync: Function {},
callsArgOn: Function {},
callsArgOnAsync: Function {},
callsArgOnWith: Function {},
callsArgOnWithAsync: Function {},
callsArgWith: Function {},
callsArgWithAsync: Function {},
callsFake: Function {},
create: Function create {},
createStubInstance: Function {},
get: Function {},
onCall: Function onCall {},
onFirstCall: Function onFirstCall {},
onSecondCall: Function onSecondCall {},
onThirdCall: Function onThirdCall {},
rejects: Function {},
reset: Function reset {},
resetBehavior: Function resetBehavior {},
resolves: Function {},
resolvesArg: Function {},
resolvesThis: Function {},
returns: Function {},
returnsArg: Function {},
returnsThis: Function {},
set: Function {},
throws: Function {},
throwsArg: Function {},
throwsException: Function {},
usingPromise: Function {},
value: Function {},
wrappedMethod: Function wrap {},
yields: Function {},
yieldsAsync: Function {},
yieldsOn: Function {},
yieldsOnAsync: Function {},
yieldsRight: Function {},
yieldsRightAsync: Function {},
yieldsTo: Function {},
yieldsToAsync: Function {},
yieldsToOn: Function {},
yieldsToOnAsync: Function {},
}

enqueue - prevent flushing through time interval when already flushed by flushAt

/home/circleci/analytics-node/test.js:282

281: client.enqueue('type', {})
282: t.true(client.flush.calledTwice)
283:

Value is not true:

false

client.flush.calledTwice
=> false

client.flush
=> Function flush {
wrappedMethod: AsyncFunction [
undefined,
],
}

client
=> Analytics {
axiosInstance: Function wrap {
create: Function create {},
defaults: {
adapter: Function httpAdapter {},
env: Object { … },
headers: Object { … },
maxBodyLength: -1,
maxContentLength: -1,
timeout: 0,
transformRequest: Array [ … ],
transformResponse: Array [ … ],
transitional: Object { … },
validateStatus: Function validateStatus {},
xsrfCookieName: 'XSRF-TOKEN',
xsrfHeaderName: 'X-XSRF-TOKEN',
},
delete: Function wrap {},
get: Function wrap {},
getUri: Function wrap {},
head: Function wrap {},
interceptors: {
request: InterceptorManager { … },
response: InterceptorManager { … },
},
options: Function wrap {},
patch: Function wrap {},
patchForm: Function wrap {},
post: Function wrap {},
postForm: Function wrap {},
put: Function wrap {},
putForm: Function wrap {},
request: Function wrap {},
},
enable: true,
errorHandler: undefined,
flushAt: 2,
flushInterval: 10,
flushed: true,
host: 'http://localhost:4063',
maxQueueSize: 460800,
path: '/v1/batch',
pendingFlush: null,
queue: [
{
callback: Function noop {},
message: Object { … },
},
{
callback: Function noop {},
message: Object { … },
},
{
callback: Function noop {},
message: Object { … },
},
],
timeout: false,
writeKey: 'key',
}

@horacio-penya
Copy link
Contributor Author

I've made some changes (to prevent a different race condition and to make every test pass)

The tests pass in my dev env, on github I see synk test failing because it's not authenticated and publish, size and test-node8 in "Expected" status since 3 hours ago.

Can you give them a look? Thanks!

@edsonjab
Copy link
Contributor

edsonjab commented Nov 7, 2022

We will check why is failing on that validation and that test with node 8 is on status "Expected"

@MichaelGHSeg MichaelGHSeg merged commit 8bfe443 into segmentio:master Nov 16, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

flush does not guarantee that all inflight messages are sent
3 participants