Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: Monotonicity in UUIDv7 #150

Merged
merged 9 commits into from Jan 11, 2024
Merged

fix: Monotonicity in UUIDv7 #150

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
960071a
Monotonicity in UUIDv7
it512 Jan 9, 2024
f27ed75
Merge branch 'master' into monotonicity
it512 Jan 9, 2024
654bc6c
fix Monotonicity
it512 Jan 10, 2024
64f9de4
Merge branch 'monotonicity' of http://47.75.211.166:5080/https://gith…
it512 Jan 10, 2024
5986a71
fix comment
it512 Jan 10, 2024
4b1aab9
Monotonicity 2
it512 Jan 11, 2024
314e206
lock
it512 Jan 11, 2024
8888abc
fix comment
it512 Jan 11, 2024
a6e034a
fix comment
it512 Jan 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
12 changes: 8 additions & 4 deletions time.go
View file
Open in desktop
Expand Up @@ -27,6 +27,8 @@ var (
lasttime uint64 // last time we returned
clockSeq uint16 // clock sequence for this run

lasttimev7 int64 // for uuid v7

timeNow = time.Now // for testing
)

Expand Down Expand Up @@ -67,12 +69,12 @@ func getTime() (Time, uint16, error) {
}

// ClockSequence returns the current clock sequence, generating one if not
// already set. The clock sequence is only used for Version 1 UUIDs.
// already set. The clock sequence is used for Version 1 and 7 UUIDs.
//
// The uuid package does not use global static storage for the clock sequence or
// the last time a UUID was generated. Unless SetClockSequence is used, a new
// random clock sequence is generated the first time a clock sequence is
// requested by ClockSequence, GetTime, or NewUUID. (section 4.2.1.1)
// requested by ClockSequence, GetTime, NewV7 or NewUUID. (section 4.2.1.1)
func ClockSequence() int {
defer timeMu.Unlock()
timeMu.Lock()
Expand All @@ -86,8 +88,9 @@ func clockSequence() int {
return int(clockSeq & 0x3fff)
}

// SetClockSequence sets the clock sequence to the lower 14 bits of seq. Setting to
// -1 causes a new sequence to be generated.
// SetClockSequence sets the clock sequence to the lower 14 bits of seq
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The period was lost here. In general, do not reformat comments unless they are incorrect.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

// uuid v1 and v6 use 14 bits seq. uuid v7 use 12 bits seq.
// Setting to -1 causes a new sequence to be generated.
func SetClockSequence(seq int) {
defer timeMu.Unlock()
timeMu.Lock()
Expand All @@ -104,6 +107,7 @@ func setClockSequence(seq int) {
clockSeq = uint16(seq&0x3fff) | 0x8000 // Set our variant
if oldSeq != clockSeq {
lasttime = 0
lasttimev7 = 0
}
}

Expand Down
23 changes: 21 additions & 2 deletions uuid_test.go
View file
Open in desktop
Expand Up @@ -825,7 +825,7 @@ func TestVersion6(t *testing.T) {
func TestVersion7(t *testing.T) {
SetRand(nil)
m := make(map[string]bool)
for x := 1; x < 32; x++ {
for x := 1; x < 128; x++ {
uuid, err := NewV7()
if err != nil {
t.Fatalf("could not create UUID: %v", err)
Expand Down Expand Up @@ -887,7 +887,26 @@ func TestVersion7FromReader(t *testing.T) {
if err != nil {
t.Errorf("failed generating UUID from a reader")
}
if uuid1 != uuid3 {
if uuid1 == uuid3 { // Montonicity
t.Errorf("expected duplicates, got %q and %q", uuid1, uuid3)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this message is no longer accurate.

}
}

func TestVersion7Monotonicity(t *testing.T) {
length := 4097 // [0x000 - 0xfff]
myString := "8059ddhdle77cb52"

SetClockSequence(0)

uuids := make([]string, length)
for i := 0; i < length; i++ {
uuidString, _ := NewV7FromReader(bytes.NewReader([]byte(myString)))
uuids[i] = uuidString.String()
}

for i := 1; i < len(uuids); i++ {
if uuids[i-1] > uuids[i] {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be >= ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

t.Errorf("expected seq got %s > %s", uuids[i-1], uuids[i])
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/expected/unexpected

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

}
}
}
28 changes: 23 additions & 5 deletions version7.go
View file
Open in desktop
Expand Up @@ -20,6 +20,7 @@ import (
// NewV7 returns a Version 7 UUID based on the current time(Unix Epoch).
// Uses the randomness pool if it was enabled with EnableRandPool.
// On error, NewV7 returns Nil and an error
// Note: this implement only has 12 bit seq, maximum of 4096 uuids are generated in 1 milliseconds
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I understand it, this is not a limitation of the implementation but rather of the standard.

// Note: Version 7 UUIDs have a 12 bit sequence number limiting the number of 
// unique UUIDs to 4096 in 1 millesecond.

func NewV7() (UUID, error) {
uuid, err := NewRandom()
if err != nil {
Expand All @@ -44,15 +45,15 @@ func NewV7FromReader(r io.Reader) (UUID, error) {

// makeV7 fill 48 bits time (uuid[0] - uuid[5]), set version b0111 (uuid[6])
// uuid[8] already has the right version number (Variant is 10)
// see function NewV7 and NewV7FromReader
// see function NewV7 and NewV7FromReader
func makeV7(uuid []byte) {
/*
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| unix_ts_ms |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| unix_ts_ms | ver | rand_a |
| unix_ts_ms | ver |rand_a (12 bit counter)|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|var| rand_b |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Expand All @@ -61,7 +62,7 @@ func makeV7(uuid []byte) {
*/
_ = uuid[15] // bounds check

t := timeNow().UnixMilli()
t, s := getTimeV7()

uuid[0] = byte(t >> 40)
uuid[1] = byte(t >> 32)
Expand All @@ -70,6 +71,23 @@ func makeV7(uuid []byte) {
uuid[4] = byte(t >> 8)
uuid[5] = byte(t)

uuid[6] = 0x70 | (uuid[6] & 0x0F)
// uuid[8] has already has right version
uuid[6] = 0x70 | (0x0F & byte(s>>8))
uuid[7] = byte(s)
}

func getTimeV7() (int64, uint16) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don 't think this is quite right. Maybe something along the lines of:

now := time.Now().UnixNano()
milli := now / 1000
seq := (now - milli * 1000) << 2 // use the upper 10 bits

There is nothing in the standard that requires Version 7 UUIDs to be monotonically increasing within a given millisecond but it is acceptable to use sub-millisecond values. It is also legitimate for the first 64 bits of two Version 7 UUIDs to be identical. The randomness in the last 62 bits is what makes them unique.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I misread part of the standard, it should be monotonic, but the above is still basically correct. What you need to do is have a protected global variable:

now := time.Now().UnixNano() << 2 // time in 1/4 nanoseconds
if now <= lastTime {
    now = lastTime + 1
}
lastTime = now
milli := now / 4000
seq := now - milli * 4000

By using 1/4 nanoseconds we can generate 4 UUIDs within a nanosecond that do not cause us to return a time that is actually in the future by a nanosecond.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

v7 using millisecond (1/1000) timestamp, we are using microsecond (1/1000000) lower 12 bit set to rand_a
1 millisecond can generate 4096 uuids

now := timeNow().UnixMicro()
 t, s := now/1000, now&4095 // 2^12-1, 12 bits
 uuid[6] = 0x70 | (0x0F & byte(s>>8))
 uuid[7] = byte(s)


defer timeMu.Unlock()
timeMu.Lock()

if clockSeq == 0 {
setClockSequence(-1)
}
now := timeNow().UnixMilli()

if now <= lasttimev7 {
clockSeq = clockSeq + 1
}
lasttimev7 = now
return now, clockSeq
}