Special characters

[Rukes]

Hello everyone,

this is not a bug report, sorry, but I need to ask. I've been checking up encoding and converters sections but it didnt helped me much.

As a feature is this library described with Accepts any character (except ; or = obviously). But I have a problem with saving characters like ě š č ř ž ý á í é etc.. If I save a cookie containing one of this chars, at its place is the string "cut" (example: poznámka 123 -> pozn)

Is there any way how to figure out of this?

thanks :)

[carhartl]

Might be an older bug, which version are you using?

Could you give the v3 rc a try?

[carhartl]

https://github.com/js-cookie/js-cookie/releases/tag/v3.0.0-rc.0

[Rukes]

I am currently using CDN, should I try to download v3 as you said?

[Rukes]

@carhartl I tested v3 (min.js file) and it still does not work

[carhartl]

Works for me without a problem (tested with v2): https://jsfiddle.net/otx4qfys/

Have you tried after deleting all cookies?

[carhartl]

If that doesn't work, would be good to see the actual code in question...

[Rukes]

IMPORTANT NOTE - this issue is happeing only using Safari browser (macOS), tested with Chrome on macOS and works fine, also Windows 10 Chrome is working too. Sorry for that, I discovered this right now.

Yes. This code is called when you fill input (called "note") and submit add-to-cart button.
You can see this: var cookie = pid + ";" + i + ";" + name + ";" + number + ";" + size + ";" + ac + ";" + note + ";" + ks;

name, number, size, ac are empty, note is where problem is coming and ks is a number

So if input contains for ex. poznámka, var cookie will be: 1;1;;;;;pozn - missing the end

    $('.add-to-cart-button').click(function () {
        var pid = $(this).data("pid");
        var count = $("#order-form-input-" + pid).val();
        if(count.length === 0 || count === 0){
            return;
        }
        var params = getParameters(pid);
        for(var i = 1; i <= count; i++){
            var name = "", number = "", size = "", ac = "", note = "", ks = "";
            for(var j = 0; j < params.length; j++) {
                var paramType = params[j];
                var input = $("#item-" + i + "-" + pid + "-" + paramType).val();
                if(input.length !== 0){
                    switch (paramType) {
                        default:
                            break;
                        case "name":
                            name = input;
                            break;
                        case "number":
                            number = input;
                            break;
                        case "size":
                            size = input;
                            break;
                        case "ac":
                            ac = input;
                            break;
                        case "note":
                            note = input;
                            break;
                        case "ks":
                            ks = input;
                            break
                    }
                }
            }
            var cookie = pid + ";" + i + ";" + name + ";" + number + ";" + size + ";" + ac + ";" + note + ";" + ks;
            Cookies.set("item_" + itemIndex, cookie);
            itemIndex++;
        }
        Cookies.set("item_lastindex", itemIndex);
    });

[carhartl]

Looks like an odd bug in Safari not even related to our library.

When trying to write a cookie value directly like so ("1;1;;;;;poznámka;" with semicolons percent-encoded => "1%3B1%3B%3B%3B%3B%3Bpoznámka%3B"):

document.cookie = "test=1%3B1%3B%3B%3B%3B%3Bpoznámka%3B; path=/"

we end up with a cookie with the value "1%3B1%3B%3B%3B%3B%3Bpozn" and the path of the current page (not '/'), so it really looks like Safari (Safari 13.1, macOS Catalina 10.15.4) would ignore everything after and including á. ASCII characters seem to work fine.

Same for

document.cookie = "test=poznámka%3B; path=/" // cookie has a value of "pozn" and path of current page
// ...
document.cookie = "test=fooá; path=/" // cookie has a value of "foo"

So it really looks like Safari just doesn't accept these characters for whatever reason.

[carhartl]

As a workaround you'd have to create a cookie api instance with converters that do a full encodeURIComponent() for writing and decodeURIComponent() on reading...

How to use converters: https://github.com/js-cookie/js-cookie/tree/latest#converters

Example:

var api = Cookies.withConverter({
  read: function (value, name) {
    return decodeURIComponent(value)
  },
  write: function (value, name) {
    return encodeURIComponent(value)
  }
})

Not sure at this point if there's anything we can do to workaround it elsehow.

[Rukes]

Okay, thank you for provided support, I'll try somehow to figure out.

Have a nice day :)

[carhartl]

Here‘s a Stackoverflow answer with some more pointers :

https://stackoverflow.com/questions/1969232/what-are-allowed-characters-in-cookies/1969339#1969339

Safari simply refuses to send any cookie containing non-ASCII characters.

@FagnerMartinsBrack Both v2 and v3 do not take care of that. Should we revisit our encoding approach yet again? 😢

[carhartl]

RFC 6265 again (which Safari seems to follow, despite Section 5.2):

 cookie-octet      = %x21 / %x23-2B / %x2D-3A / %x3C-5B / %x5D-7E
                       ; US-ASCII characters excluding CTLs,
                       ; whitespace DQUOTE, comma, semicolon,
                       ; and backslash

ASCII only!

[carhartl]

MDN suggests:

A <cookie-value> can optionally be wrapped in double quotes and include any US-ASCII characters excluding control characters, Whitespace, double quotes, comma, semicolon, and backslash.

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie

[carhartl]

Just to clarify - it works correctly in v2!

@Rukes

In order for this to work without setting up any converters please make sure to use the latest stable release (2.2.1):

<script src="https://cdn.jsdelivr.net/npm/js-cookie@2/src/js.cookie.min.js"></script>

(From https://github.com/js-cookie/js-cookie/tree/latest#readme)

Likely you've been using the cdn snippet from https://github.com/js-cookie/js-cookie (master) all along which will give you the release candidate of version 3:

<script src="https://cdn.jsdelivr.net/npm/js-cookie@rc/dist/js.cookie.min.js"></script>

The problem at hand has been introduced in v3.

I'm going to reopen this ticket as this needs fixing.

[Rukes]

I am fine with the convertor, it works good for now.

I'll also try 2.2.1 version

[FagnerMartinsBrack]

@carhartl
I knew it! Only I didn't remember which characters I tested exactly when I wrote this comment before:

When I built the encoding, I tested every single character in the latest Safari, Chrome, Firefox, Opera, and IE. Many of the ones that didn't comply with the standard couldn't be read or written using document.cookie, which would make your application break for some browsers.

Learning: Next time I should document this research somewhere so that somebody else (including myself years from then) don't go through this path again only to find the wall.

Yes, we have to go with the server-side section if we want to be robustand allow "every character to work". Unless we can get Safari to change their implementation (which I believe won't be easy).

I vote for acknowledging cookie implementations are screwed up and encode every character except characters that are allowed in the server-side part of the spec. Happy to be proven wrong though.

[FagnerMartinsBrack]

Shall we ping spec authors again for the third view on this?

[carhartl]

Shall we ping spec authors again for the third view on this?

No.

[carhartl]

So here's my plan..: I'm going to bring back the encoding implementation from v2 (i.e. revert some commits) - apparently it has been rock-solid, thanks @FagnerMartinsBrack -, but then also, for reading:

Be conservative in what you do, be liberal in what you accept from others

So when reading a cookie, say for a key foo[bar], we'd first try to find a key with the encoded key foo%5Bbar%5D, and fallback to looking for the unencoded one (the former having precedence!).

In a way, when writing a cookie we will be adhering to Section 4 of RFC 6265 (because we have to w/ Safari), when reading Section 5 applies. This will tackle #595 and similar requests/issues.

So much for the theory, let's see if it goes like planned :)

[FagnerMartinsBrack]

@carhartl

So when reading a cookie, say for a key foo[bar], we'd first try to find a key with the encoded key foo%5Bbar%5D, and fallback to looking for the unencoded one (the former having precedence!).

Isn't that how it was done before? When reading the key you would decodeURIComponent() it and then if there's no percent-encoding it would try to read it as normal?

The previous encoding tests were strictly adhering to section 4. Which means that it only encodes the characters that are required by the RFC, nothing else. We need to be careful to not encode characters that are not necessary as that might create regressions on code that is on the border fo the 4k cookie size limit.

Here's an idea to verify this regression: Maybe we can build a test that asserts on the length of a cookie as 3999 bytes (using only the possible characters that should be encoded) using v2 code and then make sure that the tests are still passing when integrated with v3?

[carhartl]

Isn't that how it was done before? When reading the key you would decodeURIComponent() it and then if there's no percent-encoding it would try to read it as normal?

I don’t think so. Otherwise we wouldn’t have #595.

[carhartl]

Reverted to fix this regression: 294247e

[carhartl]

I don’t think so. Otherwise we wouldn’t have #595.

@FagnerMartinsBrack No, you were right, it already worked like this. I couldn't find an explicit test for it, so I'm adding one.

[FagnerMartinsBrack]

it already worked like this

Like how?