Bug: [CURLRequest] body contains "HTTP/1.0 200 Connection established"

[okatse]

PHP Version

8.1

CodeIgniter4 Version

4.4.4

CodeIgniter4 Installation Method

Composer (using codeigniter4/appstarter)

Which operating systems have you tested for this bug?

Windows

Which server did you use?

apache

Database

No response

What happened?

$response = $this->curlClient->request('POST', $url);
echo $response->getBody();

Body return this

HTTP/1.0 200 Connection established
HTTP/1.1 200 OK
referrer-policy: no-referrer
x-frame-options: SAMEORIGIN
strict-transport-security: max-age=16070400; includeSubDomains
cache-control: no-store
x-content-type-options: nosniff
set-cookie: KC_RESTART=; Version=1; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Max-Age=0; Path=/a/; Secure; HttpOnlyHttpOnly;SameSite=none; Secure
pragma: no-cache
x-xss-protection: 1; mode=block
content-type: application/json
content-length: 2684
x-envoy-upstream-service-time: 38
date: Wed, 07 Feb 2024 07:06:40 GMT
server: istio-envoy

{"data":1,"expires_in":1800}

Steps to Reproduce

$response = $this->curlClient->request('POST', $url);
echo $response->getBody();

Expected Output

echo $response->getBody(); == {"data":1,"expires_in":1800}

I understand that getBody returns all the information returned by the server. But I guess it should be that $response->headers() returns the headers and $response->getBody() returns the data.

Anything else?

executing curl from the command line returns only data correctly. Postman also returns the data correctly

[kenjis]

Thank you for reporting!
But we cannot reproduce the issue.
Can you show the exact steps to reproduce the issue?

[okatse]

I changed

if (strpos($output, 'HTTP/1.1 100 Continue') === 0) {
$output = substr($output, strpos($output, $breakString) + 4);
}

    if (strpos($output, 'HTTP/1.1 200 Connection established') === 0) {
        $output = substr($output, strpos($output, $breakString) + 4);
    }

to

if (strpos($output, 'HTTP/1.0 100 Continue') === 0) {
$output = substr($output, strpos($output, $breakString) + 4);
}

    if (strpos($output, 'HTTP/1.0 200 Connection established') === 0) {
        $output = substr($output, strpos($output, $breakString) + 4);
    }

and it works properly

see what header the server returns

HTTP/1.0 200 Connection established
HTTP/1.1 200 OK

[okatse]

As part of the explanation. The protocol header can take several values https://everything.curl.dev/http/versions
HTTP/1.0 100 Continue
HTTP/1.0 200 Connection established
HTTP/1.1 100 Continue
HTTP/1.1 200 Connection established
HTTP/2 100 Continue
HTTP/2 200 Connection established
HTTP/3 100 Continue
HTTP/3 200 Connection established

and in the code we have it rigid

if (strpos($output, 'HTTP/1.1 100 Continue') === 0) {
$output = substr($output, strpos($output, $breakString) + 4);
}

	This code reuest to www.google.pl return HTTP/1.0 200 Connection established HTTP/2 200
	
   $client = \Config\Services::curlrequest();
   $options = [
    'http_errors' => false,
    'debug'       => true, 
    'proxy'       => 'ip_proxy',
    
    ];

   $adr ='https://www.google.pl';
   $response = $client->request('GET', $adr, $options);

        foreach ($response->headers() as $name => $value) {
            echo $name . ': ' . $response->getHeaderLine($name) . "\n";
        }

        echo $response->getBody();

[kenjis]

Cannot reproduce. Why do you still use HTTP/1.0?

$ docker pull ubuntu/squid
$ docker run -d --name squid-container -e TZ=UTC -p 3128:3128 ubuntu/squid

$ curl -v --proxy http://localhost:3128 https://www.google.pl
* Host localhost:3128 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:3128...
* Connected to localhost (::1) port 3128
* CONNECT tunnel: HTTP/1.1 negotiated
* allocate connect buffer
* Establish HTTP proxy tunnel to www.google.pl:443
> CONNECT www.google.pl:443 HTTP/1.1
> Host: www.google.pl:443
> User-Agent: curl/8.5.0
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 200 Connection established
< 
* CONNECT phase completed
* CONNECT tunnel established, response 200
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 / x25519 / id-ecPublicKey
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=*.google.pl
*  start date: Jan  9 06:34:13 2024 GMT
*  expire date: Apr  2 06:34:12 2024 GMT
*  subjectAltName: host "www.google.pl" matched cert's "*.google.pl"
*  issuer: C=US; O=Google Trust Services LLC; CN=GTS CA 1C3
*  SSL certificate verify ok.
*   Certificate level 0: Public key type EC/prime256v1 (256/128 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 1: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 2: Public key type RSA (4096/152 Bits/secBits), signed using sha384WithRSAEncryption
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://www.google.pl/
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: www.google.pl]
* [HTTP/2] [1] [:path: /]
* [HTTP/2] [1] [user-agent: curl/8.5.0]
* [HTTP/2] [1] [accept: */*]
> GET / HTTP/2
> Host: www.google.pl
> User-Agent: curl/8.5.0
> Accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
< HTTP/2 200 
< date: Thu, 08 Feb 2024 02:33:26 GMT
< expires: -1
< cache-control: private, max-age=0
< content-type: text/html; charset=ISO-8859-2
< content-security-policy-report-only: object-src 'none';base-uri 'self';script-src 'nonce-iP9bOGIZesCEhajHCzCyYg' 'strict-dynamic' 'report-sample' 'unsafe-eval' 'unsafe-inline' https: http:;report-uri https://csp.withgoogle.com/csp/gws/other-hp
< p3p: CP="This is not a P3P policy! See g.co/p3phelp for more info."
< server: gws
< x-xss-protection: 0
< x-frame-options: SAMEORIGIN
< set-cookie: 1P_JAR=2024-02-08-02; expires=Sat, 09-Mar-2024 02:33:26 GMT; path=/; domain=.google.pl; Secure
< set-cookie: AEC=Ae3NU9PVS9ERTsF4MBaHEHkFIpmqIbkOuLRcgNpHYHUUBRKYAD0stHx2Sok; expires=Tue, 06-Aug-2024 02:33:26 GMT; path=/; domain=.google.pl; Secure; HttpOnly; SameSite=lax
< set-cookie: NID=511=UfuEJr8IAiuabesErKhUm8XK_shijk-0oa_lw1ouTTKbhvhOVvlTSmnR_D3d_VTJwbVZu-0CIDzCYlUliNZmDphIXDndV_DCmDKc-asSmjILCQ7DB4cHQEjAE9EO9TDl9Ts17qhOiHraSA1LHHF-Fz8_E0o9cMc0SokTdcALKR8; expires=Fri, 09-Aug-2024 02:33:26 GMT; path=/; domain=.google.pl; HttpOnly
< alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
< accept-ranges: none
< vary: Accept-Encoding
< 
<!doctype html>...

[okatse]

I admit that my case is isolated. I deliberately don't use HTTP/1.0. I connect to different external services through different proxies and one of them uses an old protocol. I have no influence on it. But Codeigniter shouldn't be limited to just one HTTP/1.1
This piece of code solves my problem

    if (preg_match('/HTTP\/\d\.\d 200 Connection established/', $output)) {
        $output = substr($output, strpos($output, $breakString) + 4);
    }

[okatse]

I'm not the only one who has this problem. The problem is that the proxy returns its own header and the curl library returns too much
https://stackoverflow.com/questions/16965530/what-to-do-with-extra-http-header-from-proxy
cfug/dio#2053

[okatse]

A small update. I migrated from 4.4.4 to 4.5.1.
My proxy sends headers:

HTTP/1.0 200 Connection established

HTTP/1.1 200 OK
cache-control: no-store
pragma: no-cache
content-length: 2684
content-type: application/json

\vendor\codeigniter4\framework\system\HTTP\CURLRequest.php
Line 392 in version 4.5.1 has been changed to :

        if (str_starts_with($output, 'HTTP/1.1 200 Connection established')) {
            $output = substr($output, strpos($output, $breakString) + 4);
        }

But the problem was not solved
My solution also stopped working

Don't set it rigidly.
str_starts_with($output, 'HTTP/1.1 200 Connection established')
The protocol will change and there will be a problem