urllib.error.HTTPError: HTTP Error 403: Forbidden

[Almoullim]

When i run it with any request, i get this HTTP error:

Traceback (most recent call last):
  File "main.py", line 7, in <module>
    for torrent in t.search('neighbors'):
  File "/usr/lib/python3.5/site-packages/tpb/tpb.py", line 149, in items
    for item in super(Paginated, self).items():
  File "/usr/lib/python3.5/site-packages/tpb/tpb.py", line 59, in items
    request = urlopen(str(self.url))
  File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.5/urllib/request.py", line 472, in open
    response = meth(req, response)
  File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.5/urllib/request.py", line 510, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

This is the code i tried:

from tpb import TPB
from tpb import CATEGORIES, ORDERS

t = TPB('https://thepiratebay.org') # create a TPB object with default domain

# search for 'public domain' in 'movies' category
search = t.search('public domain', category=CATEGORIES.VIDEO.MOVIES)

# return listings from page 2 of this search
search.page(2)

# sort this search by count of seeders, and return a multipage result
search.order(ORDERS.SEEDERS.ASC).multipage()

# search, order by seeders and return page 3 results
t.search('python').order(ORDERS.SEEDERS.ASC).page(3)

# multipage beginning on page 4
t.search('recipe book').page(4).multipage()

# search, in a category and return multipage results
t.search('something').category(CATEGORIES.OTHER.OTHER).multipage()

# get page 3 of recent torrents
t.recent().page(3)

# get top torrents in Movies category
t.top().category(CATEGORIES.VIDEO.MOVIES)

# print all torrent descriptions
for torrent in t.search('public domain'):
    print(torrent.info)

# print all torrent files and their sizes
for torrent in t.search('public domain'):
    print(torrent.files)

[SIGINT]

I had this identical issue while experimenting with TPB API (and Python 3 for that matter). I'm pretty sure the user agent that is set by urlopen() method is blocked by the web server hosting thepiratebay.org. This is explained in this stackoverflow post as well. After manually setting it in 'tpb.py', I no longer encounter this issue.

[tanimislam]

@SIGINT what did you do to 'tpb.py'?

[SIGINT]

@tanimislam:

As a quick hack, I manually set the user agent in tpb.py where urllib.request.urlopen() is invoked. As per the documentation for the urllib module, urllib.request.urlopen() accepts either a string (url) or Request object. The user agent string can be set when constructing the Request as follows:

"""
Request URL and parse response. Yield a ``Torrent`` for every torrent
on page.
"""
user_agent = 'Mozilla/5.0 (iPhone; CPU iPhone OS 5_0 like Mac OS X) AppleWebKit/534.46'
request = urlopen(Request(str(self.url), data=None, headers={'User-Agent': user_agent}))
#request = urlopen(str(self.url))

It was a quick fix to get around the server blocking certain (likely robot) user agents. Hope this helps!

[tanimislam]

@SIGINT thanks for the update; I actually found a solution using SickRage. Specifically, this bit of code in (relative to top level directory) sickbeard/providers/thepiratebay.py

[ah3243]

Just as a note make sure you change all the urlopen() calls in case you're still getting 403 errors.

[Ayush-Goyal-coding]

I am getting the same issue again. Is it solved for everyone?