Hikvision intercom digest auth - possible issue due to pull/2463

[mateuszdraco]

I'm using httpx to get to Hikvision intercom API, which uses digest auth. It accepts the WWW-Authenticate token once, later it just spams with Unauthorized responses.

Python code I used to test it:

>>> import httpx
>>> a = httpx.DigestAuth("admin", "***")
>>> httpx.get("http://192.168.0.208/ISAPI/VideoIntercom/callStatus?format=xml", auth=a)
<Response [200 OK]>
>>> httpx.get("http://192.168.0.208/ISAPI/VideoIntercom/callStatus?format=xml", auth=a)
<Response [401 Unauthorized]>
>>> httpx.get("http://192.168.0.208/ISAPI/VideoIntercom/callStatus?format=xml", auth=a)
<Response [401 Unauthorized]>
>>> a = httpx.DigestAuth("admin", "***")
>>> httpx.get("http://192.168.0.208/ISAPI/VideoIntercom/callStatus?format=xml", auth=a)
<Response [200 OK]>
>>> httpx.get("http://192.168.0.208/ISAPI/VideoIntercom/callStatus?format=xml", auth=a)
<Response [401 Unauthorized]>
>>> httpx.get("http://192.168.0.208/ISAPI/VideoIntercom/callStatus?format=xml", auth=a)
<Response [401 Unauthorized]>
>>> httpx.get("http://192.168.0.208/ISAPI/VideoIntercom/callStatus?format=xml", auth=a)
<Response [401 Unauthorized]>
>>> httpx.get("http://192.168.0.208/ISAPI/VideoIntercom/callStatus?format=xml", auth=a)
<Response [401 Unauthorized]>
>>> a = httpx.DigestAuth("admin", "***")
>>> httpx.get("http://192.168.0.208/ISAPI/VideoIntercom/callStatus?format=xml", auth=a)
<Response [200 OK]>
>>> a = httpx.DigestAuth("admin", "***")
>>> httpx.get("http://192.168.0.208/ISAPI/VideoIntercom/callStatus?format=xml", auth=a)
<Response [200 OK]>
>>> httpx.get("http://192.168.0.208/ISAPI/VideoIntercom/callStatus?format=xml", auth=a)
<Response [401 Unauthorized]>
>>> httpx.get("http://192.168.0.208/ISAPI/VideoIntercom/callStatus?format=xml", auth=a)
<Response [401 Unauthorized]>
>>> a = httpx.DigestAuth("admin", "***")
>>> httpx.get("http://192.168.0.208/ISAPI/VideoIntercom/callStatus?format=xml", auth=a)
<Response [200 OK]>

Wireshark also shows the same, I can provide the dump but overall it's the same - httpx uses the same nonce token and it is rejected.

I observed this behavior with Hikvision intercom API but possibly this is more broad issue.

[zanieb]

Ref #2463

[mateuszdraco]

Should I raise an issue in order to proceed with this?

[mateuszdraco]

The fact of life is: it stopped working. Can we have a flag to workaround this issue?

[mateuszdraco]

And about the description from wiki, what is cnonce in this example? I believe nonce in Authorization is the same with each subsequent request.

[florimondmanca]

Thanks a lot for the Wireshark record, that’s very helpful.

After a thorough review of the output, the code, the Wikipedia overview the spec, I don’t see what could be triggering an issue on our side.

Unfortunately Wireshark has truncated the full Authorization header. We can see it ends with « q » which I assume would be the beginning of the qop field. But so we don’t get to see if cnonce is included (given the code, it should be as the challenge has a qop field set).

It could be helpful to try and get the full Authorization header sent?

Also, the spec says servers MAY accept a nonce being returned only once, in which case they should send back a 401 with a new nonce in WWW-Authenticate and optionally stale=true (for browsers).

The output confirms the API doesn’t return such a WWW-Authenticate.

This would be consistent with the server refusing our request as invalid.

The fact of life is: it stopped working. Can we have a flag to workaround this issue?

I assume this means you pinned HTTPX to a version where things still work? Which version is it? Could help pin point what change triggers this issue in your case. I assume it would be PR #2463 which makes HTTPX reuse the challenge always (until a 401 with WWW-Authenticate is received), in line with Requests.

Are you able to reproduce with Requests?

[mateuszdraco]

By any chance, do you have some experience with Wireshark and can give me some hints about how to enable it? If not I can try to look for that and provide you a better example.

So my use case is that I'm using httpx through Home Assistant :) and they update httpx version on regular basis, which is great, yet Hikvision integration stopped working for me. I managed to downgrade httpx manually on my VM and eventually it's working again, yet I don't update Home Assistant since then. I've opened another request on HA as well. And yes, it worked before PR #2463.

[florimondmanca]

By any chance, do you have some experience with Wireshark and can give me some hints about how to enable it? If not I can try to look for that and provide you a better example.

I don’t have much experience, unfortunately!

You should be able to get the full output in a Python console as in initial discussion post using HTTPX_LOG_LEVEL=trace:
https://www.python-httpx.org/environment_variables/

[mateuszdraco]

Unfortunately there is not that much info with the log level set:

HTTPX_LOG_LEVEL=trace python                                                                                                                                                  ✔  
Python 3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import httpx
>>> httpx.__version__
'0.23.3'
>>> a = httpx.DigestAuth("redacted", "redacted")
>>> httpx.get("http://192.168.0.208/ISAPI/VideoIntercom/callStatus?format=xml", auth=a)
TRACE [2023-02-05 13:21:16] httpx._config - load_ssl_context verify=True cert=None trust_env=True http2=False
TRACE [2023-02-05 13:21:16] httpx._config - load_verify_locations cafile=/home/mateusz/src/py-httpx/lib/python3.10/site-packages/certifi/cacert.pem
DEBUG [2023-02-05 13:21:16] httpx._client - HTTP Request: GET http://192.168.0.208/ISAPI/VideoIntercom/callStatus?format=xml "HTTP/1.1 401 Unauthorized"
DEBUG [2023-02-05 13:21:17] httpx._client - HTTP Request: GET http://192.168.0.208/ISAPI/VideoIntercom/callStatus?format=xml "HTTP/1.1 200 OK"
<Response [200 OK]>
>>> httpx.get("http://192.168.0.208/ISAPI/VideoIntercom/callStatus?format=xml", auth=a)
TRACE [2023-02-05 13:21:22] httpx._config - load_ssl_context verify=True cert=None trust_env=True http2=False
TRACE [2023-02-05 13:21:22] httpx._config - load_verify_locations cafile=/home/mateusz/src/py-httpx/lib/python3.10/site-packages/certifi/cacert.pem
DEBUG [2023-02-05 13:21:24] httpx._client - HTTP Request: GET http://192.168.0.208/ISAPI/VideoIntercom/callStatus?format=xml "HTTP/1.1 401 Unauthorized"
<Response [401 Unauthorized]>

I'll try to find more data with wireshark.

[mateuszdraco]

If used as in documentation:

requests.get(url, auth=HTTPDigestAuth('redacted', 'redacted'))

then it will work. If I store auth and try to reuse it - it's not working. So I think it's the same as httpx.

As of the intercom, that's the model: ds-kv8113-wme1-b. You may find it's documentation here on page 72.

[florimondmanca]

Are you able to get away with not reusing httpx.DigestAuth(), i.e. passing a new instance on each request?

[mateuszdraco]

I can submit this as a request to HomeAssistant, if that's the advised solution.

[florimondmanca]

I can submit this as a request to HomeAssistant, if that's the advised solution.

It would make the code return to its prior state — authenticating on every request.

[tomchristie]

If I store auth and try to reuse it - it's not working. So I think it's the same as httpx.

If the digest auth isn't working on either the requests or httpx implementation when using token reuse with nonce counting, then that might indicate that the issue is on the server/application side?

[pergolafabio]

I have some ISAPi documents, can share later

[pergolafabio]

Here is the link to all ISAPi docs from Hikvision that I also share on Home Assistant forum.. it's for all hardware...

https://www.dropbox.com/s/obppstvpvxja9l7/isapi.zip?dl=0

[Fauzan4869]

package com---------------------;

import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.util.EntityUtils;

import java.io.IOException;

import java.io.File;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.StandardOpenOption;
import java.util.Scanner;

public class DigestAuthISAPI {

public static void main(String[] args) {
	
    Scanner scanner = new Scanner(System.in);
    
    // MASUKKAN NAMA USER PATH ANDA
    System.out.print("MASUKKAN IP CAMERA: ");
    String ip = scanner.nextLine();
    
    // MASUKKAN NAMA USER PATH ANDA
    System.out.print("MASUKKAN NAMA USER PATH ANDA: ");
    String pathCam = scanner.nextLine();

    // MASUKKAN FOTO KE BERAPA ( SUPAYA TIDAK MEREPLACE FOTO YANG SUDAH ADA )
    System.out.print("MASUKKAN FOTO KE BERAPA: ");
    int take = scanner.nextInt();    	
	
	String url = "";
    url = "http://" + ip + "/ISAPI/Streaming/Channels/1/picture";
    String passwordCam = "username:password";
    
	String nama = "";
	String path = "";
	path = "/home/" + pathCam + "...............................";
	nama = "hasilcam_ " + take + ".JPG";

	
    // Tutup Scanner setelah selesai digunakan
    scanner.close(); 		
	
    try {
    	
        String[] credentials = passwordCam.split(":");
        String username = credentials[0];
        String password = credentials[1];			
		        	
    	
        // Setting - setting Credential Provider
        CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
        credentialsProvider.setCredentials(
                new AuthScope(AuthScope.ANY),
                new UsernamePasswordCredentials(username, password)
        ); 

        // Membuat Client HTTP Dengan Autentikasi Digest
        CloseableHttpClient httpClient = HttpClients.custom()
                .setDefaultCredentialsProvider(credentialsProvider)
                .build();

        // Membuat Permintaan HTTP GET
        HttpGet httpGet = new HttpGet(url);

        // Eksekusi Permintaan
        HttpResponse response = httpClient.execute(httpGet);

        // Memeriksa Respon Status
        int statusCode = response.getStatusLine().getStatusCode();
        if (statusCode == 200) {
            // Permintaan Berhasil, Melakukan Proses Responnya
            HttpEntity entity = response.getEntity();

            // Mengatur File Path / Direktori
            Path targetPath = (new File(path + nama)).toPath();

            // Mengecek apakah direktori sudah ada
            File directory = targetPath.getParent().toFile();
            if (!directory.exists()) {
                // Jika direktori belum ada, buat direktori baru
                directory.mkdirs();
            }

            // Menjadikan Respon sebuah File
            Files.write(targetPath, EntityUtils.toByteArray(entity), StandardOpenOption.CREATE);

            // Mengecek apakah file sudah ada
            File file = targetPath.toFile();
            if (!file.exists()) {
                // Jika file belum ada, buat file baru
                file.createNewFile();
            }
            
            System.out.println("File created at: " + targetPath);

// String responseBody = EntityUtils.toString(entity);
// System.out.println("Response Body: " + responseBody);
} else {
// Request failed, handle the error
System.err.println("Request failed with status code: " + statusCode);
}

        // Close the HTTP client
        httpClient.close();

    } catch (IOException e) {
        e.printStackTrace();
    }
}

}