Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds .timings array with DNC, TCP, request and response times #2452

Merged
merged 7 commits into from Feb 21, 2017
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
12 changes: 11 additions & 1 deletion README.md
Expand Up @@ -812,7 +812,17 @@ default in Linux can be anywhere from 20-120 seconds][linux-timeout]).

---

- `time` - If `true`, the request-response cycle (including all redirects) is timed at millisecond resolution, and the result provided on the response's `elapsedTime` property. The `responseStartTime` property is also available to indicate the timestamp when the response begins.
- `time` - If `true`, the request-response cycle (including all redirects) is timed at millisecond resolution, and the result provided on the response's `elapsedTime` property. The `responseStartTime` property is also available to indicate the timestamp when the response begins. In addition, there is a `.timings` object available with the following properties:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove extra space before In addition

- `start`: Timestamp when `request()` was initialized
Copy link
Contributor

@Huachao Huachao Dec 5, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the timestamp like start, end which is got from process.hrtime a useful time for user, since the result from process.hrtime is relative to an arbitrary time in the past, and not related to the time of day. So the start and end time seem not useful and user also can't utilize them

Copy link
Contributor Author

@nicjansma nicjansma Dec 6, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's useful to have start and end if you want to make any duration calculations other than the built in dns, tcp, etc. For example, you may want a duration from the start to the first byte (timings.response - timings.start) instead of firstByte (connect to response).

You may also just want to log or keep timestamps (relative to module startup) to be able to sort requests by start time, etc.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nicjansma if I want to know the start time of the request in Unix format, how can we achieve this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's available today as request.startTime

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry, I mean the start time of response, can we use responseStartTime, it is still available?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup! I didn't break any of the additional timestamps, so responseStartTime is still available.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 glad you thought about the deprecated behavior.

Can you add a comment near wherever elapsedTime and responseStartTime are set in the code to indicate that they are now deprecated in favor of the .timings property? Once this has had some time to bake we can remove any README references to the old properties and think about how we deprecate them.

- `socket` Timestamp when the [`http`](https://nodejs.org/api/http.html#http_event_socket) module's `socket` event fires. This happens when the socket is assigned to the request (after DNS has been resolved).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove extra space before This

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed -- there were 15 other occurrences of double-spaces, so if you're going for consistency I've fixed those as well.

- `connect`: Timestamp when the [`http`](https://nodejs.org/api/http.html#http_event_connect) module's `connect` event fires. This happens when the server acknowledges the TCP connection.
- `response`: Timestamp when the [`http`](https://nodejs.org/api/http.html#http_event_response) module's `response` event fires. This happens when the first bytes are received from the server.
- `end`: Timestamp when the last bytes of the response are received.
- `dns`: Duration of DNS lookup (`timings.socket` - `timings.start`)
- `tcp`: Duration of TCP connection (`timings.connect` - `timings.socket`)
- `download`: Duration HTTP fetch (`timings.end` - `timings.response`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change to Duration of HTTP?

- `total`: Duration entire HTTP round-trip (`timings.end` - `timings.start`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also add firstByte description?


- `har` - A [HAR 1.2 Request Object](http://www.softwareishard.com/blog/har-12-spec/#request), will be processed from HAR format into options overwriting matching values *(see the [HAR 1.2 section](#support-for-har-1.2) for details)*
- `callback` - alternatively pass the request's callback in the options object

Expand Down
1 change: 1 addition & 0 deletions package.json
Expand Up @@ -38,6 +38,7 @@
"mime-types": "~2.1.7",
"node-uuid": "~1.4.7",
"oauth-sign": "~0.8.1",
"performance-now": "^0.2.0",
"qs": "~6.3.0",
"stringstream": "~0.0.4",
"tough-cookie": "~2.3.0",
Expand Down
37 changes: 35 additions & 2 deletions request.js
Expand Up @@ -28,6 +28,7 @@ var http = require('http')
, Multipart = require('./lib/multipart').Multipart
, Redirect = require('./lib/redirect').Redirect
, Tunnel = require('./lib/tunnel').Tunnel
, now = require('performance-now')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we must use the high resolution watch?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was looking at cURL results for quite a few API endpoints and I see it takes a few ms (10 - 50ms) on an average for most events. Although, a high-resolution watch would be nicer; as @mikeal suggested - we can possibly skip in interest of keeping the dependencies low.


var safeStringify = helpers.safeStringify
, isReadStream = helpers.isReadStream
Expand Down Expand Up @@ -713,6 +714,10 @@ Request.prototype.start = function () {
// this is usually called on the first write(), end() or on nextTick()
var self = this

if (self.timing) {
var startTime = now()
}

if (self._aborted) {
return
}
Expand Down Expand Up @@ -749,6 +754,9 @@ Request.prototype.start = function () {

if (self.timing) {
self.startTime = new Date().getTime()

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would recommend move to Date.now() instead of 'new Date().getTime()` as the former is twice as fast. Performance test: https://jsperf.com/date-now-vs-new-date

self.timings = {
start: startTime
}
}

var timeout
Expand All @@ -766,6 +774,13 @@ Request.prototype.start = function () {
self.emit('drain')
})
self.req.on('socket', function(socket) {
if (self.timing) {
self.timings.socket = now()
socket.on('connect', function() {
self.timings.connect = now()
})
}

var setReqTimeout = function() {
// This timeout sets the amount of time to wait *between* bytes sent
// from the server once connected.
Expand Down Expand Up @@ -847,12 +862,30 @@ Request.prototype.onRequestError = function (error) {

Request.prototype.onRequestResponse = function (response) {
var self = this

if (self.timing) {
self.timings.response = now()
}

debug('onRequestResponse', self.uri.href, response.statusCode, response.headers)
response.on('end', function() {
if (self.timing) {
self.elapsedTime += (new Date().getTime() - self.startTime)
debug('elapsed time', self.elapsedTime)
self.timings.end = now()

self.timings.dns = self.timings.socket - self.timings.start
self.timings.tcp = self.timings.connect - self.timings.socket
self.timings.firstByte = self.timings.response - self.timings.connect
self.timings.download = self.timings.end - self.timings.response
self.timings.total = self.timings.end - self.timings.start

debug('elapsed time', self.timings.total)

// elapsedTime includes all redirects
self.elapsedTime += Math.round(self.timings.total)
response.elapsedTime = self.elapsedTime

// timings is just for the final fetch
response.timings = self.timings
}
debug('response end', self.uri.href, response.statusCode, response.headers)
})
Expand Down
11 changes: 11 additions & 0 deletions tests/test-timing.js
Expand Up @@ -31,8 +31,19 @@ tape('non-redirected request is timed', function(t) {
t.equal(err, null)
t.equal(typeof res.elapsedTime, 'number')
t.equal(typeof res.responseStartTime, 'number')
t.equal(typeof res.timings, 'object')
t.equal((res.elapsedTime > 0), true)
t.equal((res.responseStartTime > r.startTime), true)
t.equal((res.timings.start > 0), true)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do a t.deepEqual(res.timings, ...) here instead, so that we check that only the properties we expect are present. Just realized this will mean we can't compare the properties equally because we don't know their exact values. Hmm...

Can you think of a way to check for unexpected properties as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that's why I hadn't used .deepEqual initially. I've added a check to get all property names and compare.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome 👍

t.equal((res.timings.socket >= res.timings.start), true)
t.equal((res.timings.connect >= res.timings.socket), true)
t.equal((res.timings.response >= res.timings.connect), true)
t.equal((res.timings.end >= res.timings.response), true)
t.equal((res.timings.dns >= 0), true)
t.equal((res.timings.tcp > 0), true)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or >= ?

t.equal((res.timings.firstByte > 0), true)
t.equal((res.timings.download > 0), true)
t.equal((res.timings.total > 0), true)
t.end()
})
})
Expand Down