yt-dlp @2022.11.11: proxy mode is broken for some reason

[aeiouaeiouaeiouaeiouaeiouaeiou]

Example output:

[debug] Command-line config: ['--verbose', '--proxy', 'socks5://184.170.248.5:4145', 'https://youtu.be/dQw4w9WgXcQ']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.11.11 [8b64402] (pip)
[debug] Python 3.10.8 (CPython x86_64 64bit) - macOS-10.8.5-x86_64-i386-64bit (OpenSSL 3.0.7 1 Nov 2022)
[debug] exe versions: ffmpeg 4.4.2 (fdk,setts), ffprobe 4.4.2, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.09.24, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {'http': 'socks5://184.170.248.5:4145', 'https': 'socks5://184.170.248.5:4145'}
[debug] Loaded 1723 extractors
[debug] [youtube] Extracting URL: https://youtu.be/dQw4w9WgXcQ
[youtube] dQw4w9WgXcQ: Downloading webpage
WARNING: [youtube] Unable to download webpage: <urlopen error [Errno 4] Host unreachable>
[youtube] dQw4w9WgXcQ: Downloading android player API JSON
WARNING: [youtube] <urlopen error [Errno 4] Host unreachable>. Retrying (1/3)...
[youtube] dQw4w9WgXcQ: Downloading android player API JSON
WARNING: [youtube] <urlopen error [Errno 4] Host unreachable>. Retrying (2/3)...
[youtube] dQw4w9WgXcQ: Downloading android player API JSON
WARNING: [youtube] <urlopen error [Errno 4] Host unreachable>. Retrying (3/3)...
[youtube] dQw4w9WgXcQ: Downloading android player API JSON
[youtube] dQw4w9WgXcQ: Downloading iframe API JS
WARNING: [youtube] Unable to download webpage: <urlopen error [Errno 4] Host unreachable>
[youtube] dQw4w9WgXcQ: Downloading web player API JSON
WARNING: [youtube] <urlopen error [Errno 4] Host unreachable>. Retrying (1/3)...
[youtube] dQw4w9WgXcQ: Downloading web player API JSON
WARNING: [youtube] <urlopen error [Errno 4] Host unreachable>. Retrying (2/3)...
[youtube] dQw4w9WgXcQ: Downloading web player API JSON
WARNING: [youtube] <urlopen error [Errno 4] Host unreachable>. Retrying (3/3)...
[youtube] dQw4w9WgXcQ: Downloading web player API JSON
WARNING: [youtube] Unable to download API page: <urlopen error [Errno 4] Host unreachable> (caused by URLError(Socks5Error(4, 'Host unreachable')))
ERROR: [youtube] dQw4w9WgXcQ: Unable to download API page: <urlopen error [Errno 4] Host unreachable> (caused by URLError(Socks5Error(4, 'Host unreachable')))
  File "/Users/user/Library/Python/3.10/lib/python/site-packages/yt_dlp/extractor/common.py", line 674, in extract
    ie_result = self._real_extract(url)
  File "/Users/user/Library/Python/3.10/lib/python/site-packages/yt_dlp/extractor/youtube.py", line 3758, in _real_extract
    webpage, master_ytcfg, player_responses, player_url = self._download_player_responses(url, smuggled_data, video_id, webpage_url)
  File "/Users/user/Library/Python/3.10/lib/python/site-packages/yt_dlp/extractor/youtube.py", line 3726, in _download_player_responses
    player_responses, player_url = self._extract_player_responses(
  File "/Users/user/Library/Python/3.10/lib/python/site-packages/yt_dlp/extractor/youtube.py", line 3464, in _extract_player_responses
    raise last_error
  File "/Users/user/Library/Python/3.10/lib/python/site-packages/yt_dlp/extractor/youtube.py", line 3435, in _extract_player_responses
    pr = initial_pr if client == 'web' and initial_pr else self._extract_player_response(
  File "/Users/user/Library/Python/3.10/lib/python/site-packages/yt_dlp/extractor/youtube.py", line 3355, in _extract_player_response
    return self._extract_response(
  File "/Users/user/Library/Python/3.10/lib/python/site-packages/yt_dlp/extractor/youtube.py", line 860, in _extract_response
    for retry in self.RetryManager():
  File "/Users/user/Library/Python/3.10/lib/python/site-packages/yt_dlp/utils.py", line 5921, in __iter__
    self.error_callback(self.error, self.attempt, self.retries)
  File "/Users/user/Library/Python/3.10/lib/python/site-packages/yt_dlp/extractor/common.py", line 3882, in _error_or_warning
    RetryManager.report_retry(
  File "/Users/user/Library/Python/3.10/lib/python/site-packages/yt_dlp/utils.py", line 5929, in report_retry
    raise e
  File "/Users/user/Library/Python/3.10/lib/python/site-packages/yt_dlp/extractor/youtube.py", line 862, in _extract_response
    response = self._call_api(
  File "/Users/user/Library/Python/3.10/lib/python/site-packages/yt_dlp/extractor/youtube.py", line 538, in _call_api
    return self._download_json(
  File "/Users/user/Library/Python/3.10/lib/python/site-packages/yt_dlp/extractor/common.py", line 1034, in download_content
    res = getattr(self, download_handle.__name__)(url_or_request, video_id, **kwargs)
  File "/Users/user/Library/Python/3.10/lib/python/site-packages/yt_dlp/extractor/common.py", line 998, in download_handle
    res = self._download_webpage_handle(
  File "/Users/user/Library/Python/3.10/lib/python/site-packages/yt_dlp/extractor/common.py", line 868, in _download_webpage_handle
    urlh = self._request_webpage(url_or_request, video_id, note, errnote, fatal, data=data, headers=headers, query=query, expected_status=expected_status)
  File "/Users/user/Library/Python/3.10/lib/python/site-packages/yt_dlp/extractor/common.py", line 825, in _request_webpage
    raise ExtractorError(errmsg, cause=err)

  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 1348, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1282, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1328, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1277, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1037, in _send_output
    self.send(msg)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 975, in send
    self.connect()
  File "/Users/user/Library/Python/3.10/lib/python/site-packages/yt_dlp/utils.py", line 1494, in connect
    self.sock.connect((self.host, self.port))
  File "/Users/user/Library/Python/3.10/lib/python/site-packages/yt_dlp/socks.py", line 264, in connect
    self._make_proxy(socket.socket.connect, address)
  File "/Users/user/Library/Python/3.10/lib/python/site-packages/yt_dlp/socks.py", line 260, in _make_proxy
    setup_funcs[self._proxy.type](address)
  File "/Users/user/Library/Python/3.10/lib/python/site-packages/yt_dlp/socks.py", line 235, in _setup_socks5
    raise Socks5Error(status)
yt_dlp.socks.Socks5Error: [Errno 4] Host unreachable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/user/Library/Python/3.10/lib/python/site-packages/yt_dlp/extractor/common.py", line 807, in _request_webpage
    return self._downloader.urlopen(self._create_request(url_or_request, data, headers, query))
  File "/Users/user/Library/Python/3.10/lib/python/site-packages/yt_dlp/YoutubeDL.py", line 3692, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 519, in open
    response = self._open(req, data)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 536, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 496, in _call_chain
    result = func(*args)
  File "/Users/user/Library/Python/3.10/lib/python/site-packages/yt_dlp/utils.py", line 1527, in https_open
    return self.do_open(
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 1351, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 4] Host unreachable>

On other sites it basically displays the following error:

Unable to download webpage: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:997)> (caused by URLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:997)')))

This happens regardless of whether an HTTP or SOCKS proxy is used. Google showed ​a similar problem, but does certifi use sometimes system certificates or am I misunderstanding something?

[ryandesign (Ryan Carsten Schmidt)]

does certifi use sometimes system certificates or am I misunderstanding something?

I don't know. I don't have experience with proxies nor with certifi. Cc'ing Andrew who maintains py-certifi.

Have you already performed the Let's Encrypt fix on your system? (I don't know if it's relevant for this but it's good to do for other reasons.)

[aeiouaeiouaeiouaeiouaeiouaeiou]

Replying to ryandesign:

Have you already performed the Let's Encrypt fix on your system?

I did it right after reinstalling the system, otherwise apps like Mail and Contacts would not work.