How to fetch a non-ascii url with urlopen?

Encode the unicode to UTF-8, then URL-encode.

Use iri2uri method of httplib2. It makes the same thing as by bobin (is he/she the author of that?)

Python 3 has libraries to handle this situation. Use
urllib.parse.urlsplit to split the URL into its components, and
urllib.parse.quote to properly quote/escape the unicode characters
and urllib.parse.urlunsplit to join it back together.

>>> import urllib.parse
>>> url = 'http://example.com/unicodè'
>>> url = urllib.parse.urlsplit(url)
>>> url = list(url)
>>> url[2] = urllib.parse.quote(url[2])
>>> url = urllib.parse.urlunsplit(url)
>>> print(url)
http://example.com/unicod%C3%A8

In python3, use the urllib.parse.quote function on the non-ascii string:

>>> from urllib.request import urlopen                                                                                                                                                            
>>> from urllib.parse import quote                                                                                                                                                                
>>> chinese_wikipedia = 'http://zh.wikipedia.org/wiki/Wikipedia:' + quote('首页')
>>> urlopen(chinese_wikipedia)

For those not depending strictly on urllib, one practical alternative is requests, which handles IRIs “out of the box”.

For example, with http://bücher.ch:

>>> import requests
>>> r = requests.get(u'http://bu00DCcher.ch')
>>> r.status_code
200

It is more complex than the accepted @bobince’s answer suggests:

• netloc should be encoded using IDNA;
• non-ascii URL path should be encoded to UTF-8 and then percent-escaped;
• non-ascii query parameters should be encoded to the encoding of a page URL was extracted from (or to the encoding server uses), then percent-escaped.

This is how all browsers work; it is specified in https://url.spec.whatwg.org/ – see this example. A Python implementation can be found in w3lib (this is the library Scrapy is using); see w3lib.url.safe_url_string:

from w3lib.url import safe_url_string
url = safe_url_string(u'http://example.org/Ñöñ-ÅŞÇİİ/', encoding="<page encoding>")

An easy way to check if a URL escaping implementation is incorrect/incomplete is to check if it provides ‘page encoding’ argument or not.

Based on @darkfeline answer:

from urllib.parse import urlsplit, urlunsplit, quote

def iri2uri(iri):
    """
    Convert an IRI to a URI (Python 3).
    """
    uri = ''
    if isinstance(iri, str):
        (scheme, netloc, path, query, fragment) = urlsplit(iri)
        scheme = quote(scheme)
        netloc = netloc.encode('idna').decode('utf-8')
        path = quote(path)
        query = quote(query)
        fragment = quote(fragment)
        uri = urlunsplit((scheme, netloc, path, query, fragment))

    return uri

works! finally

I could not avoid from this strange characters, but at the end I come through it.

import urllib.request
import os


url = "http://www.fourtourismblog.it/le-nuove-tendenze-del-marketing-tenere-docchio/"
with urllib.request.urlopen(url) as file:
    html = file.read()
with open("marketingturismo.html", "w", encoding='utf-8') as file:
    file.write(str(html.decode('utf-8')))
os.system("marketingturismo.html")

Another option to convert an IRI to an ASCII URI is to use furl package:

gruns/furl: URL parsing and manipulation made easy. – https://github.com/gruns/furl

Python’s standard urllib and urlparse modules provide a number of URL
related functions, but using these functions to perform common URL
operations proves tedious. Furl makes parsing and manipulating URLs
easy.

Examples

Non-ASCII domain

http://国立極地研究所.jp/english/ (Japanese National Institute of Polar Research website)

import furl

url = 'http://国立極地研究所.jp/english/'
furl.furl(url).tostr()

'http://xn--vcsoey76a2hh0vtuid5qa.jp/english/'

Non-ASCII path

https://ja.wikipedia.org/wiki/日本語 ("Japanese" article in Wikipedia)

import furl

url = 'https://ja.wikipedia.org/wiki/日本語'
furl.furl(url).tostr()

'https://ja.wikipedia.org/wiki/%E6%97%A5%E6%9C%AC%E8%AA%9E'